DRL Labs

Continuous Control Recovery Track

Grabber: a browser-native 2D arm that learns to grab a coin and carry it home

Grabber is the live continuous-control lane for DRL: human play, PPO checkpoint playback, and a structured training workflow with learning-timeline snapshots. It keeps the Reacher lineage visible without depending on the legacy Unity runtime.

Back to Continuous Control View Archive Context
Custom 2D task PPO + timeline snapshots

Human Control

Drive the arm with continuous shoulder, elbow, and grip control

Runtime ready

Create a session to begin.

Play

Hold keys to push the three control axes continuously

Keyboard: `A/D` shoulder, `J/L` elbow, `W/S` grip close/open. Hold the keys to keep stepping.

Shoulder

Elbow

Grip

Machine Play

Inspect PPO checkpoints and learning snapshots on the same task

Train or load a checkpoint to inspect it here.

Training

Bounded environment, reward, and PPO controls

Environment

Reward

PPO

Learning Timeline

Replay saved training milestones

Choose a completed training job to inspect its timeline snapshots.

Jobs

Recent training and evaluation runs

No job selected.

Curator Guide

Why Grabber is the live continuous-control lane

The old Reacher branch is rich but too entangled with Unity, ML-Agents, and older environment assumptions to become the first live continuous-control experience. Grabber preserves the core teaching shape instead: continuous actions, a visible arm, a clear target, and a staged objective that lets users watch learning improve over time.

Live first

Why this is not a direct Unity Reacher port

Grabber keeps the arm-control intuition of Reacher but replaces the legacy simulator with a browser-native 2D task. That removes the old runtime baggage while keeping the ideas users actually need to see: continuous control, grasp timing, and return-home behavior.

Task structure

Grab, then carry the coin home

The policy does not only need to reach. It must approach the coin, close the grip inside the capture radius, keep possession, and then bring the coin back into the visible home zone for a short dwell window.

  • Approach shaping gives the policy a path toward the coin.
  • Latch and carry rewards turn the task into a sequence instead of one contact event.
  • The home-zone hold window makes success visible to humans and stable enough for checkpoint comparison.

How to read the page

Three surfaces, one control language

Human play, machine playback, and training all share the same three control axes: shoulder, elbow, and grip. The color-coded gauges make it obvious which degree of freedom is doing the work on each step.

Why PPO leads

The live lane teaches policy learning directly

The historical Reacher archive is DDPG-heavy, but Grabber uses PPO for the live page because it is a cleaner fit for the bounded worker pipeline and a more stable first public training surface. The archive still points back to DDPG as lineage, not as the first runtime dependency.

Source Trail

What this page is preserving

Continuous Control project root

project bundle

The original project branch with Reacher notebooks, reports, environment notes, and auxiliary experiments.

source-material/classwork/project-reports/p2_continuous-control

Project README

markdown

Defines the original double-jointed target-reaching task, its observation space, and the continuous action framing.

source-material/classwork/project-reports/p2_continuous-control/README.md

Single-agent DDPG

python module

The simpler historical Reacher lane that informs the live lab’s control vocabulary even though the runtime is different.

source-material/classwork/project-reports/p2_continuous-control/ddpg_single_agent

Reacher DDPG notebooks

notebook set

Actor-critic experiments, reward notes, and action-space references that sit behind the live Grabber page as archive lineage.

source-material/ddpg/reacher