GAE

High-Dimensional Continuous Control Using Generalized Advantage Estimation

John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, Pieter Abbeel · ICLR 2016

Read paper · arXiv:1506.02438

Estimate advantages from exponentially weighted TD residuals, with lambda controlling the bias–variance tradeoff.

What you will implement

Implement the backward GAE recurrence and episode boundary handling.

• 1. GAE Backward Recurrence

  Implement the backward GAE recurrence from TD residuals with correct episode boundaries.

  Implement advantages and value targets; trajectories and value estimates are supplied.