Almgren-Chriss execution environment
This page turns the old Almgren-Chriss finance work into a no-code demo. Move the sliders and watch how an execution plan changes when patience, urgency, and market impact pull against each other. It shows the benchmark behavior first so the finance trade-off is legible before actor-critic training enters the picture.
What the sliders mean
This plan is moderately front-loaded: it starts assertively, then tapers as the position gets smaller.
Large trades push the execution price against you.
Waiting keeps you exposed to random price movement.
A good strategy balances those two forces instead of optimizing only one.
RL Translation
The notebook frames each decision around remaining inventory, time left to liquidate, and recent price behavior.
The policy chooses how much of the remaining position to sell now instead of waiting for later trades.
Good behavior reduces implementation shortfall while controlling the risk of adverse price moves before the order is finished.
This page is intentionally benchmark-first. The analytical Almgren-Chriss schedule is the target behavior a learned actor-critic policy would try to imitate or improve on.
Execution Schedule
Front-loaded plans appear as tall early bars.
Inventory Decay
Steeper early drops mean the strategy is exiting faster.
Risk vs. Impact
Each point is a different risk preference for the same market.
Curator Guide
The original finance notebook does more than draw a schedule. It reframes optimal liquidation as a reinforcement-learning problem with states, actions, and rewards, then compares learned behavior against the Almgren-Chriss benchmark. This page intentionally starts from the benchmark so the core trade-off is visible before any actor-critic training enters the picture.
From the notebook
The finance notebook defines the state as recent log returns plus normalized time and remaining inventory. The action is the fraction of the remaining position to sell now. The reward is based on improvement in Almgren-Chriss utility after each step.
Why this demo starts here
The archived notebook eventually moves into DDPG. For a no-code public demo, the analytical schedule in `syntheticChrissAlmgren.py` is the better first surface because it shows exactly what the agent is trying to balance before neural networks make the picture noisier.
What to notice
Each slider changes the balance between impact and uncertainty.
Bridge to the rest of the repo
Finance looks different from CartPole or Reacher, but the structure is familiar: a state, a policy, a reward signal, and a sequential trade-off under uncertainty. It is one of the cleanest examples in the archive of DRL being used as an application rather than just a toy benchmark.
Source Trail
Introduces the liquidation problem, defines states/actions/rewards, and sketches the actor-critic training loop.
source-material/finance/DRL.ipynb
Implements the linear-impact Almgren-Chriss simulator and the closed-form trade schedule used by this demo.
source-material/finance/syntheticChrissAlmgren.py
Contains plotting and table helpers from the notebook-oriented workflow.
source-material/finance/utils.py
Defines the actor and critic networks used when the notebook escalates from the benchmark to learned policies.
source-material/finance/model.py
Background material that connects the finance notebook's RL framing to the broader actor-critic branch in the repo.
source-material/classwork/ContinuousControl_Transcripts
Glossary
The gap between the portfolio's paper value at the starting price and the cash you actually realize while selling.
The systematic execution cost created by spread and price impact, even before random price motion is considered.
The uncertainty caused by prices wandering while you are still trying to finish the sale.
The weight that says how much you care about uncertainty relative to pure execution cost.
The immediate price concession paid on the shares sold right now.
The lingering price effect of having traded a large amount into the market.