Foundations
Foundations and warm-up labs
The cleanest path for reviewing how RL pieces fit together before deep networks dominate the picture.
Archive review + curriculum map + live lab
Turn a raw DRL archive into a navigable lab instead of a folder dump.
This app treats the repository as a curriculum spine, a paper shelf, and an archive of experiments. The goal for phase 2 is reviewability first: see what is reusable, what is legacy, and what deserves a live demo later.
Interactive Demos
Start here if you want intuition first
These pages explain the idea in plain language and let the controls do the teaching.
Finance
Finance as a DRL application branch
A rare self-contained domain application in the repo, and one of the most promising early interactive demos.
Review-first
Foundations, papers, transcripts, and finance can become useful web content immediately.
Legacy stack
Navigation and legacy Reacher remain valuable archive assets, but the live continuous-control lane now runs through Grabber instead of the old Unity stack.
Archive lane
Lunar and Grabber are the current live recovery tracks, while older experiments and rough branches stay visible as curated lab-notebook material.
Primary Arms
Catalog of reusable branches
These are the reusable branches DRL can explain and demonstrate as its own lab.
Tabular RL + Classic Control
Foundations and warm-up labs
The cleanest path for reviewing how RL pieces fit together before deep networks dominate the picture.
Ready to curate now
Mostly modernizable
These notebooks are the best first candidates for live DRL web lessons because they explain the math cleanly and usually avoid Unity-specific baggage.
DQN + Navigation + Lunar variants
Value-based methods and DQN lineage
The strongest bridge from the course tutorials into project work and later experiments.
Good base after curation
Mixed modern + legacy
It gives you both the clean reference implementation and the messier follow-on experiments that show how you were exploring beyond the coursework.
REINFORCE + PPO + Actor-Critic theory
Policy gradients and actor-critic studies
The theory-heavy branch that ties together REINFORCE, PPO, A2C/A3C, GAE, and your Pong experiments.
Strong archive; selective live reuse
Legacy Atari stack for some assets
This is the best source for explanatory pages about variance reduction, baselines, clipped objectives, and the move from vanilla policy gradients into actor-critic methods.
Grabber live lab + Reacher archive lineage
Continuous control and the Reacher / Grabber lane
A live browser-native continuous-control lab now sits beside the deeper Reacher archive, giving DRL a modern entrypoint without reviving Unity first.
Live lab + deep legacy archive
Grabber is live; old Reacher still needs isolation
This branch now connects a live, understandable continuous-control lab to the heavier historical Reacher work, so users can learn the control ideas first and then trace the lineage back into the archive.
Tennis + Soccer + MARL lecture material
Multi-agent RL and collaboration/competition
The branch for Tennis, Soccer, and the theory around cooperation, competition, and Markov games.
Great study content, thin runtime assets
Theory-first; missing env bundles in repo
It gives the future DRL arm a true multi-agent sub-tree without forcing phase 2 to solve live environment recovery first.
Almgren-Chriss execution environment
Finance as a DRL application branch
A rare self-contained domain application in the repo, and one of the most promising early interactive demos.
Best domain-specific live demo candidate
Mostly self-contained Python
Unlike the Unity branches, this section is domain-specific and mostly self-contained, which makes it ideal for a real web-native DRL application page.
Reference shelf + transcript layer
Papers, cheatsheets, and transcript assets
The reference shelf that can anchor the educational side of the DRL arm even before everything is runnable again.
Ready immediately
No special runtime
It gives the web app depth. Instead of only showing code, you can tie each section back to the source papers and class explanations that motivated it.
Rough experiments + dead ends + utility scraps
Archive and lab-notebook material
Useful for your personal review, but not something to treat as production-grade code without curation.
Archive first
Unreliable as-is
It still matters because it captures your own reasoning trail and the branches you explored after the coursework. That is valuable review material, even when the code is not clean enough to power the live app directly.
Recommended Start
What deserves attention first
Foundations
This branch is notebook-first. It walks from policy/value iteration into Monte Carlo and temporal-difference learning, then moves into discretization, tile coding, hill climbing, cross-entropy, and Taxi. The material is structured as short exercises plus matching solution notebooks.
Review this branchPolicy Gradients
The cleanest pieces are the REINFORCE notebooks and the actor-critic transcripts. Around that, you have several Pong PPO/REINFORCE experiments, support utilities, and saved policy checkpoints.
Review this branchContinuous Control
The new live lane is Grabber: a custom 2D arm task with continuous actions, machine playback, and bounded PPO training. Behind it, the archive still holds the course DDPG tutorials, multiple Reacher implementations, single-agent and twenty-agent project variants, vectorized environment experiments, PPO/REINFORCE side paths, sample transition CSVs, and the vendored ML-Agents Python package.
Review this branchMulti-Agent
This is mostly notebook and transcript material. The project notebooks show the Tennis/Soccer environments, while the subtitle archive covers motivation, applications, Markov games, cooperation/competition, and a paper walk-through.
Review this branchFinance
The finance notebook and modules model trade execution using the Almgren-Chriss framework. The code defines a market environment, plotting helpers, and actor/critic networks for optimal execution experiments.
Review this branch