feat(scripts): memtour.py, a memory tour through the control bridge #570

Open

tonesi wants to merge 1 commit from tonesi/Longshot:feat/memtour into main

pull from: tonesi/Longshot:feat/memtour

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 2 Commits 1 Files changed 3 +540

tonesi commented 2026-11-14 20:09:16 +00:00

Contributor

• Copy link
• Copy Markdown

Type: feat · Risk: low · Tested on: Linux, RTX 3070 Ti · Needs: none

Why

"The game outgrows my RAM and the swap fills" (issue 388) can only be answered, and a fix only checked, with the same tour run the same way on each PC and build. This is that tour as one command, on top of the control bridge, so a tester needs no private harness. See #388. The tracker was searched (issues and pull requests, all states, for memory, RAM, swap, tour, sampler, control bridge): none adds a memory tour. Related, not the same: #562 shows the game's own CPU and RAM in an FPS-counter page and writes them to a file from inside the game, with no swap, no shared-memory split and no tour; this reads /proc from outside, over a scripted route. #416 is the change that was measured with this tour first.

What changed

A new script starts a game folder through the control bridge, loads a save, walks through some areas and writes down the game's memory as it goes. Its test needs no game.

• scripts/memtour.py (new): starts the game with longshot_ctl.py's start_session (test profile: headless, a copy of the save, the source only read), presses A until the save is loaded (Tour.to_play), teleports through --dests (default 13,1,2,3,0,7,9,11, or all) with a walk and camera swings in each, then quits. Sampler writes mem.csv every 2 s from /proc/<pid>: RssAnon, RssShmem, swap (anonymous and shared), game_total (resident + swapped, file pages left out), MemAvailable, swap in use. marks.csv has the tour's events; summary.md (summarize) the peaks and each area's peak, end value and lowest MemAvailable. Linux only: it reads /proc and says so elsewhere.
• tests/test_memtour.py (new): the sampler against a made-up /proc; the tour against a fake bridge (A presses until the save loads, a refused teleport retried, one refused for good skipped, a save that never loads, an error reply); the summary's numbers per area.
• docs/development.md: one row in the commands table.

Measurements

Verdict: nothing to measure: a script and its test, and no game code changes. (One real run of the script is under Testing.)

Flags and settings

None.

Testing

Tested: on Linux (RTX 3070 Ti 8 GB, 15 GB RAM), python -I -B tests/run_all.py: 483 passed, 5 failed, 84 skipped (no clang, Vulkan or SPIR-V headers here). Four of the failures also fail on a clean forge/main (test_oot_saves, test_bake_portability, test_cpm_discovery, test_walk_key); test_crash_record failed once under 8 parallel checks and passes alone. test_memtour.py, test_longshot_ctl.py and test_control.py pass (after the rebase onto 8b619764, the first two again). In the game, one run: python -I scripts/memtour.py <game folder built from main 0caf0618> --save <Zora's River save> --dests 13,1 --dwell 15 --settle 10 loaded the save, teleported to Zora's Domain and Hyrule Field, quit cleanly and wrote the three files (peak game total 9,491 MB, lowest MemAvailable 840 MB, 39 samples).

Not tested: Windows (the script refuses, by design); a first start that prepares shaders before the first frame (--launch-timeout is there for it); runs of more than a few minutes through this script (the long tours behind issue 388 used a private harness that this replaces).

Must be tested: None.

Risks

What could break: nothing the game or the build uses: a new script, its test and a docs row. The test is discovered by run_all.py and runs in under a second.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Issues

See #388

Checklist

Everyone

• One topic.
• The title is a Conventional Commit (type(scope): description) of the status line's type.
• Why gives the reason plainly and links the issues that fit (the tracker searched).
• Measurements has a verdict, or one line saying why there is nothing to measure.
• Testing names what was tested, what wasn't, and what must still be tested, where and how.
• Rebased on current main.
• python -I -B tests/run_all.py passes (any failure that also fails on main is named).
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• A game-free test added for the fix, failing on main and passing here.
• Docs updated.

🤖 Generated with Claude Code

**Type:** feat · **Risk:** low · **Tested on:** Linux, RTX 3070 Ti · **Needs:** none ## Why "The game outgrows my RAM and the swap fills" (issue 388) can only be answered, and a fix only checked, with the same tour run the same way on each PC and build. This is that tour as one command, on top of the control bridge, so a tester needs no private harness. See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/388. The tracker was searched (issues and pull requests, all states, for memory, RAM, swap, tour, sampler, control bridge): none adds a memory tour. Related, not the same: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/562 shows the game's own CPU and RAM in an FPS-counter page and writes them to a file from inside the game, with no swap, no shared-memory split and no tour; this reads `/proc` from outside, over a scripted route. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/416 is the change that was measured with this tour first. ## What changed A new script starts a game folder through the control bridge, loads a save, walks through some areas and writes down the game's memory as it goes. Its test needs no game. - `scripts/memtour.py` (new): starts the game with `longshot_ctl.py`'s `start_session` (test profile: headless, a copy of the save, the source only read), presses A until the save is loaded (`Tour.to_play`), teleports through `--dests` (default `13,1,2,3,0,7,9,11`, or `all`) with a walk and camera swings in each, then quits. `Sampler` writes `mem.csv` every 2 s from `/proc/<pid>`: RssAnon, RssShmem, swap (anonymous and shared), `game_total` (resident + swapped, file pages left out), MemAvailable, swap in use. `marks.csv` has the tour's events; `summary.md` (`summarize`) the peaks and each area's peak, end value and lowest MemAvailable. Linux only: it reads `/proc` and says so elsewhere. - `tests/test_memtour.py` (new): the sampler against a made-up `/proc`; the tour against a fake bridge (A presses until the save loads, a refused teleport retried, one refused for good skipped, a save that never loads, an error reply); the summary's numbers per area. - `docs/development.md`: one row in the commands table. ## Measurements **Verdict:** nothing to measure: a script and its test, and no game code changes. (One real run of the script is under Testing.) ### Flags and settings None. ## Testing **Tested:** on Linux (RTX 3070 Ti 8 GB, 15 GB RAM), `python -I -B tests/run_all.py`: 483 passed, 5 failed, 84 skipped (no clang, Vulkan or SPIR-V headers here). Four of the failures also fail on a clean `forge/main` (`test_oot_saves`, `test_bake_portability`, `test_cpm_discovery`, `test_walk_key`); `test_crash_record` failed once under 8 parallel checks and passes alone. `test_memtour.py`, `test_longshot_ctl.py` and `test_control.py` pass (after the rebase onto `8b619764`, the first two again). In the game, one run: `python -I scripts/memtour.py <game folder built from main 0caf0618> --save <Zora's River save> --dests 13,1 --dwell 15 --settle 10` loaded the save, teleported to Zora's Domain and Hyrule Field, quit cleanly and wrote the three files (peak game total 9,491 MB, lowest MemAvailable 840 MB, 39 samples). **Not tested:** Windows (the script refuses, by design); a first start that prepares shaders before the first frame (`--launch-timeout` is there for it); runs of more than a few minutes through this script (the long tours behind issue 388 used a private harness that this replaces). **Must be tested:** None. ## Risks **What could break:** nothing the game or the build uses: a new script, its test and a docs row. The test is discovered by `run_all.py` and runs in under a second. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Issues See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/388 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

tonesi added 1 commit 2026-11-14 20:09:16 +00:00

feat(scripts): memtour.py, a memory tour through the control bridge ... ef5809ac42

Starts a game folder through longshot_ctl.py (test profile, a copy of a save),
presses A until the save is loaded, teleports through areas with a walk in
each, and samples the game's resident + swapped memory, MemAvailable and swap
every 2 s. Writes mem.csv, marks.csv and a summary.md with the peaks and each
area's numbers. For "the game outgrows my RAM" reports (issue 388) and for A/B
runs of a change that should move it. Linux only (reads /proc).

tests/test_memtour.py covers the sampler (a made-up /proc), the tour (a fake
bridge: A presses, refused teleports, a save that never loads) and the summary,
with no game.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

tonesi commented 2026-11-14 20:18:48 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Measurements section filled in (a script: measured by its game-free checks and one real run), 'Measured' ticked, no testers needed.

Measurements section filled in (a script: measured by its game-free checks and one real run), 'Measured' ticked, no testers needed.

tonesi force-pushed feat/memtour from ef5809ac42 to 33da09baab 2026-11-14 20:19:29 +00:00 Compare

tonesi commented 2026-11-14 20:20:59 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Rebased onto 8b619764 and the description rewritten to the new template. Measured: a script, so its verdict says there is nothing to measure; no testers needed. Tracker searched; PR 562 (in-game system readings) named as related, not overlapping.

Rebased onto 8b619764 and the description rewritten to the new template. Measured: a script, so its verdict says there is nothing to measure; no testers needed. Tracker searched; PR 562 (in-game system readings) named as related, not overlapping.

tonesi changed title from WIP: feat(scripts): memtour.py, a memory tour through the control bridge to feat(scripts): memtour.py, a memory tour through the control bridge 2026-11-14 20:21:10 +00:00

gmourao referenced this pull request 2026-11-15 05:51:24 +00:00

Game: Ganondorf initially renders completely black, then recovers after a long delay on RX 9060 XT #715

mooki added the

kind

feature

platform/linux

platform/windows

priority

low

labels 2026-11-15 13:56:02 +00:00

Some checks are pending

Hide checks

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 15s

Details

beanstalk/approval Test farm not requested

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

Some workflows are waiting to be reviewed.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'feat/memtour:tonesi-feat/memtour'

git switch 'tonesi-feat/memtour'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8015c9a

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8015c9a

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!570

No description provided.