Skip to content

Clear frame history when an environment automatically resets - #377

Open
sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:masterfrom
sylvesterkaczmarek:fix/frame-stack-automatic-episode-reset-20261006
Open

sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:masterfrom
sylvesterkaczmarek:fix/frame-stack-automatic-episode-reset-20261006

Conversation

@sylvesterkaczmarek

@sylvesterkaczmarek sylvesterkaczmarek commented Oct 5, 2026 •

Copy link
Copy Markdown

Summary

Fixes #376.

Reset each frame stacker when processing a FIRST timestep, covering both explicit resets and the dm_env automatic-reset path. Continuing-episode histories remain intact. The old implementation appended a new episode's initial observation to the previous episode's frames. Default zero fill, flattening, nested structure, metadata and observation specs are unchanged.

Validation

python -m pytest -q acme/wrappers/frame_stacking_reset_test.py

20 focused tests passed, including eight new non-zero-LAST-discount variants prompted by community review. Of the original 12, six fail on unchanged upstream and six are controls. Coverage includes zero-discount terminations and non-zero-discount time-limit truncations, nested dict/tuple observations, flattened and unflattened stacks, one/multiple frames, float32/uint8, explicit resets, step-after-LAST and step on a fresh environment. Tests check observations against independently constructed stacks and validate their specs.

The local macOS launcher supplies an empty launchpad module only to satisfy Acme's unrelated top-level signal-handler import. No Launchpad function is called, and the shim is outside the patch. The wrapper, NumPy operations, dm_env classes and test environment all execute normally. The attempted existing frame-stacking suite cannot collect because acme.testing.fakes imports unavailable Reverb. Those tests, distributed execution and training are not claimed as run.

The additional time-limit cases use real dm_env.truncation timesteps. All 20 focused cases passed on Python 3.12/macOS CPU with the production Acme wrapper/spec modules, real NumPy, dm_env and dm-tree, and an isolated package-import shim (no model or distributed execution). Scoped Ruff correctness, Python syntax and git diff --check pass. No dependency or workflow changes.

Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Frame stacking carries previous-episode observations across automatic resets

1 participant