# P09: a torque-driven arm in MuJoCo

This CPU starter compares a programmed feedback baseline with a trained imitation
network in an actual MuJoCo simulation. It is separate from the browser's ideal
joint-velocity plant. No viewer, GPU, account, remote inference or paid API is required.
Downloading the pinned packages requires internet access; simulation is local.

## Set up and run

Use Python 3.12. Tested on macOS ARM64; other platforms remain unverified.
From the extracted `controller-simulator` folder:

```sh
python3.12 -m venv .venv
# macOS/Linux; Windows uses .venv\Scripts\python.exe instead
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python checks.py
.venv/bin/python run.py train --seed 7 --output policy-7.json
.venv/bin/python run.py evaluate --model policy-7.json --output evaluation-7.json
```

Repeat with seeds 19 and 41 and new output names. Keep every run, including
failures. Training accepts 1–2,000 updates; the supplied recipe uses 2,000.
Evaluation replays training and checks code hashes/package versions before
using weights. Hashes identify bytes, not trusted authorship. Outputs never
overwrite existing files. Files are inert JSON; imported policy arrays are
bounded and never executed as code. Package/source changes require a new freeze.

## Understand the contracts

The arm has 1 m and 0.8 m links with 0.04 m capsule radii and density 200 kg/m³.
Both hinges rotate about world z; elbow angle is relative to the upper link.
It moves horizontally with zero gravity, with real coupled inertia and joint
damping. Control is torque in Nm, not angular velocity. MuJoCo uses RK4 with a
0.005 s step. A run has at most 1,200 steps (six simulated seconds).

Observation contains four numbers: two joint angles in radians, then two
velocities in radians/second. The controller also receives two requested goal
angles. Hidden true state, contact truth and the external checker never enter
the policy. A separate checker evaluates true joint/speed/torque bounds and
whole-link distance to the obstacle at each integrated sample. Joint limit is
±2.6 rad, speed limit ±3 rad/s and torque limit ±8 Nm. Bad commands are rejected
before any integration; actual violations are recorded before termination.

Success requires 100 consecutive samples (0.5 s) within 0.03 rad of each target
and 0.03 rad/s speed. Collision and operating failure override reaching a target.
A six-second timeout is not success. Ending an episode is not a braking plan.
The disk-contact check is sampled every 5 ms, not continuous swept-volume
verification. Contact response, floating-point integration and model mismatch
prevent treating this as a physical safety certificate. Self-collision is not
an independently assessed criterion. There is no moving obstacle or real sensor.

## Baseline and learned model

The baseline requests `clip(8 × angle error − 3 × measured velocity, −8, 8)`.
For error 0.5 rad and velocity 1 rad/s it requests 1 Nm. Error is target minus
observed angle. Damping opposes observed velocity, not a hidden state.

The supplied learned policy has 114 parameters: four error/velocity inputs,
sixteen tanh hidden units and two tanh outputs scaled to ±8 Nm. It learns the
baseline from a fixed 625-row grid, by simultaneous full-batch gradients at rate
0.05. Loss is mean half squared error of normalized torque across rows, summed
over the two outputs. This is imitation, not reward learning or a claim to
outperform the teacher. Different approximation error can change closed-loop
settling. The fitting grid contains no simulator trajectories or outcomes.

Six public evaluation scenarios change target, damping, sensor noise, shoulder
bias and obstacle geometry. Their configurations are excluded from fitting, but
they are inspectable repeated-development cases, not unseen independent tests.
Do not tune on them and then call their results final evaluation. A reviewer must
supply new scenarios after freezing your implementation for independent evidence.
Noise seeds match, but feedback changes the resulting states. A biased sensor can
report near-target readings while the real arm settles at the wrong angle.

## Implement and challenge

1. Implement `torque` and `tip_position` in `learner.py`. Run
   `.venv/bin/python learner_checks.py`. The untouched exercise intentionally fails.
2. Compare your controller with the supplied reference using a local script:

```python
from simulator import rollout
from learner import torque
result = rollout(torque, 'noise')
print(result['status'], result['steps'])
```

3. Add an observation-only estimator. Keep previous commands and your own state
   in a controller object; do not read `Arm`, `MjData` or external audit truth.
   Document whether your prediction accounts for torque dynamics. The browser's
   ideal-velocity prediction cannot simply be copied here.
4. Reproduce the forward pass and training gradients in `policy.py`; use finite
   differences and same-checkpoint updates. Keep edited code and every run.
5. Add a new acceptance scenario before evaluating your frozen controller. Explain
   changes to inertia/damping/noise and the distinction between an integration
   approximation, a checker failure, and task failure. Have another person choose
   unfamiliar cases; record that review separately from these public checks.

## Deliverables and review

Provide source, environment, frozen model, all seed results, a copied receipt
showing the observation/action/true-state boundary, frame arithmetic, estimator
error, collision/limit failure, and a report of what remains unresolved. The
Course 11 project stores selected code or an artifact description inertly; keep
full source and simulator receipts locally. Its browser importer does not execute
MuJoCo or accept these results as authenticated evidence. Human artifact review,
learner pilots and broader platform profiling remain pending.

`checks.py` verifies gradients, replay, action/termination contracts, geometry,
independently derived coupled-arm inertia, copied receipts and failure retention.
These software checks do not establish independent learner proficiency.

## Sources and licence

Original teaching code/scenarios are MIT licensed; see LICENSE.txt. Dependencies
are installed from their registries, not bundled. MuJoCo is Apache-2.0; dependency
licences remain with the packages. Official bindings/API documentation:
https://mujoco.readthedocs.io/en/3.3.7/python.html
https://mujoco.readthedocs.io/en/3.3.7/XMLreference.html