feat(scripts): memtour.py on Windows: the game's commit split into the guest window, VRAM blocks and the rest #992
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!992
Loading…
Reference in a new issue
No description provided.
Delete branch "d4nkn3ss/Longshot:feat/memtour-windows"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Windows 11, RTX 3080 · Needs: none
Why
The most reported problem (worklist row 5.6) is memory that fills up over play, ending in "Out of memory (RAM plus the page file)". Most of those reports come from Windows players, and
scripts/memtour.py(#570), the one-command memory tour, read/procand stopped on Windows ("Linux only"). The crash reports in #894, #881, #861 and #899 give 20–30 GB of "private" memory but no breakdown of it. On Windows it splits into three parts that need very different fixes:See #388 (the same question on Linux) and #939.
What changed
On Windows, memtour now samples the game's commit charge and splits it into those three parts, alongside the working set, the system's commit and its limit, and the RAM available. The summary reports the same for each area. Linux is unchanged. The new
--imageand--romfsoptions run it on a development build (work/sa/build), which has no image beside it.scripts/memtour.py:split_regions(): the split, a pure function over VirtualQueryEx's regions. The guest window is the allocation atA64_HOST_BASE; write-combined is committed private memory mappedPAGE_WRITECOMBINE.win_api()/win_sample(): the ctypes calls (OpenProcesswithPROCESS_QUERY_INFORMATIONonly,VirtualQueryEx,GetProcessMemoryInfo,GetPerformanceInfo). They read the address space's layout, never its contents.Samplertakes a row from either OS and stops once the process is gone.summarize()hands Windows rows tosummarize_windows().main()runs on Windows.tests/test_memtour.py:The Linux sampler check now asks for the Linux sampler by name.
docs/development.md's memtour row says what it samples on each OS.Why write-combined stands for the VRAM blocks: a probe of 32 unmapped 32 MB
vkAllocateMemorys per memory type on the RTX 3080 (NVIDIA 621.14, Windows 11) raised PrivateUsage by 1,026 MB for every device-local type. Each showed up as a committed private write-combined region, with the working set unchanged, and freeing gave it back. Over a 43-minute tour that part tracks the log'svk: memory heaps: heap 0 (device)blocks (7.0–7.5 GB of blocks, 7.3–7.8 GB write-combined).gpu/vk/vk_texture.cpp's commit reserve (#346) already assumes VRAM blocks are charged; this measures how much. On AMD and Intel the column may mean something else, which is why it is named for what it is, not "VRAM".Measurements
Verdict: nothing to measure: a script and its checks. What the new memtour showed on main is in a comment below: no growth on main over a second lap, and about 20 GB of commit, ~8 GB of it VRAM blocks.
Flags and settings
None
Testing
Tested:
python -I -B tests/test_memtour.py: 21 pass on Windows 11.python -I -B tests/run_all.pyfrom PowerShell with llvm-mingw from the Longshot Builder: 584 passed, 26 failed, 42 skipped on this PR @079131a9. main @fd2e3e98gives the same counts with the same 26 failures:test_appimage_player_build,test_auto_handheld,test_cpp_preflight,test_crash_dialog,test_crt_options,test_env_on,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_lift_cleanup_error,test_linux_preflight,test_make_game_cache,test_make_game_prepare,test_missing_nup,test_options_budget_once,test_pads_after_menu,test_parallel_init_order,test_pending_guest_import,test_readback_late,test_release,test_replay_write_order,test_run_all_changed,test_tested_configurations,test_vk_occlusion_queries,test_vk_pgoandtest_walk_key. Some have open PRs, among them #804, #806, #144, #954, #937 and #625.Not tested: Linux. Its sampler is unchanged apart from being chosen by name, and its checks pass here on the made-up /proc. AMD and Intel on Windows, where the write-combined column may not be the VRAM blocks.
Must be tested:
python -I scripts/memtour.py <game folder> --save <save> --dests 1,2 --dwell 30, expectingmem.csvwith the Linux columns and the summary as before (by hand:summary.md).Risks
What could break: memtour on Linux, if the sampler's choice went wrong. The game is untouched.
Turning it off: revert the commit.
Compatibility: Nothing changes for players. On Linux, memtour's files are as before.
Issues
See #388, #939
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
What memtour's split showed on main, for row 5.6. These numbers are observations of one run on each build, not a measurement of this PR.
Verdict: nothing to measure for the change itself: a script with its checks. What follows is what it showed on main.
Hardware: Windows 11 Pro 26300, RTX 3080 10 GB (NVIDIA 621.14), i9-12900K, 64 GB, system-managed page file
Scene: adult save; teleports to memtour's default destinations 13, 1, 2, 3, 0, 7, 9, 11, two laps of 150 s each with a walk; defaults, windowed
Method: the commit split sampled every 60 s from outside the game (the same split as
split_regions()), one run on each build, lap 1's end against lap 2'sHow to reproduce:
python -I scripts/memtour.py work/sa/build --image work/sa/image/image.bin --romfs <dump>\romfs --save <save folder> --laps 2 --dwell 150 --env HOS_OOT_CHEATS=1a8a30c22, lap 1a8a30c22, lap 2dc390e59(Nov 14), lap 1dc390e59, lap 2vk: imports:live at the endOn main nothing grows in lap 2, and the rest holds at 4.3 GB. A commit of ~20 GB is the fixed size of a play session on a 10 GB card here, and ~8 GB of it is VRAM that uses no RAM. On
dc390e59the rest still grew by ~440 MB a lap. My Nov 12 tour log had imports at 430 live (5.4 GB) and private memory at 27 GB, so the import give-back merged since then shows.The end-to-end run of this PR's memtour (
--dests 1,2 --dwell 30) on main @a8a30c22:Posted by an AI agent (Claude Code) for d4nkn3ss.
e66783ea2c079131a913Rebased on main @
fd2e3e98(now079131a9).run_all.pygives the same 26 failures on both, andtest_memtour.pypasses (21). The description is updated.Posted by an AI agent (Claude Code) for d4nkn3ss.
WIP: feat(scripts): memtour.py on Windows: the game's commit split into the guest window, VRAM blocks and the restto feat(scripts): memtour.py on Windows: the game's commit split into the guest window, VRAM blocks and the restView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.