Lunar Recovery Track

Lunar Lander: real play, real machine control, and a bounded DQN training playground

This page is the first serious recovery pass on the lunar experiments. It keeps the real discrete LunarLander runtime in the loop while turning the old notebook branch into three connected surfaces: human play, checkpoint playback, and a local-first training editor.

Back to Value-Based View Archive Context

Discrete LunarLander-v3 first Gymnasium + DQN recovery path

Human Control

Fly the lander yourself

Runtime ready

Create a session to begin.

Play

Use the same 4-action space the DQN sees

Keyboard: `Space` = no-op, `ArrowLeft` = left booster, `ArrowUp` = main engine, `ArrowRight` = right booster.

Machine Play

Drive the same environment with a saved controller

Controller Playback speed

Pick a controller to inspect its checkpoint summary and then play it back.

Training Playground

Edit a bounded DQN recipe, then run it locally

The editor is intentionally constrained. You can tune the training config, the network shape, the epsilon schedule, and the reward helper, but the harness owns the runtime and checkpoint layout.

Jobs

Recent training and evaluation runs

No job selected.

Curator Guide

Why Lunar Lander is the right next recovery target

The lunar branch is where your archive stops being mostly lecture material and starts looking like an active research notebook. It combines the familiar 8-value LunarLander state space, real rendered gameplay, multiple algorithm families, and enough helper code to rebuild something genuinely interactive. This page deliberately starts with the discrete DQN path because it is the shortest line from human play to learned control.

Recovered first

Why the discrete DQN path leads v1

Most of the lunar notebooks point at the 4-action discrete LunarLander environment. That makes it the best first milestone because the same action space works for keyboard play, machine playback, and DQN training without needing a second UI language for continuous throttles.

What the old branch contains

One environment, several algorithm experiments

The archive does not contain just one lunar project. It branches into DQN, prioritized replay, double DQN, PPO, DDPG, and a hybrid path toward continuous control.

The DQN notebooks are the cleanest place to recover a playable web experience quickly.
The PPO notebooks explain policy-driven control and include rollout helpers, but they are a second wave after the DQN runway is stable.
The DDPG and continuous-control paths remain important, but they add a harder action space and a heavier recovery burden.

What to watch while playing

The reward function tells the story

LunarLander rewards getting closer to the pad, moving more slowly, and staying upright. Crashes are punished heavily, and fuel-burning engine usage carries a cost. The point of DQN here is to turn those local incentives into a stable landing policy over many noisy episodes.

How the new page is structured

Play, Machine Play, and Training fit together

Human play lets you feel the action space. Machine play lets you inspect what a controller actually does with the same interface. The training editor turns the old notebook code into a bounded experiment surface that can save reproducible checkpoints and then feed them straight back into playback.

Source Trail

What this page is recovering

Discrete DQN notebook

notebook

Primary lunar DQN notebook with rendering, training, checkpoint save/load, and rollout playback.

source-material/lunar/dqn/lunar_DQN.ipynb

Discrete DQN script

python module

Notebook-export style script that bundles the environment setup, Q-network, replay loop, and checkpoint calls.

source-material/lunar/dqn/LL_DQN.py

Q-network

python module

Feed-forward network definition that anchors the first recovered checkpoint format.

source-material/lunar/dqn/Q_network.py

DQN agent

python module

Replay-buffer and learning loop reference for the discrete lunar path.

source-material/lunar/dqn/DQN_agent.py

PPO helpers

python module

Useful for later rollout and animation ideas, but not the first live recovery path.

source-material/lunar/ppo/lunar_PPO_utils.py

Continuous DDPG notebook

notebook

The future bridge into continuous LunarLander once the discrete lane is stable.

source-material/lunar/ddpg/lunar_DDPG.ipynb

Glossary

Terms to keep visible

State vector: The 8 numbers describing lander position, velocity, angle, angular velocity, and whether each leg touches the ground.
Discrete action space: A small menu of engine commands: do nothing, fire left, fire main, or fire right.
Q-value: The model's estimate of how good one action is from the current state if the rest of the future is handled well.
Replay buffer: A memory of past transitions that lets DQN learn from mixed experience instead of only the most recent episode.
Target network: A slower-moving copy of the Q-network that stabilizes learning targets.
Reward shaping: Extra reward hints layered onto the environment reward to make learning signals easier to follow.

This Page Keeps

Live now

The real Gymnasium LunarLander environment and its rendered gameplay frames.
The discrete DQN recovery path that dominates the lunar DQN notebooks.
The connection between live play, saved checkpoints, and the same 4-action control surface.

This Page Omits

Still archive-first

Continuous LunarLander and DDPG-powered throttle control.
The PPO and parallel-environment recovery paths as first-class runtime features.
Public-safe code execution hardening for the training editor.