WIP: perf(nvn): a call of several recordings makes room for all of them at once (HOS_NVN_CALL_RESERVE=1 to try it) #2023

Draft
siantm wants to merge 3 commits from siantm/Longshot:perf/nvn-call-reserve into main
Contributor

Summary

nvnCommandBufferCallCommands and nvnCommandBufferCopyCommands copy each called recording into the caller's
recording segment. They did it one recording at a time: each called recording that did not fit grew the segment by
doubling, a copy of everything the segment held at each step, so a call of several large recordings could grow (and
copy) the segment more than once.

With HOS_NVN_CALL_RESERVE=1, a call of 2 to 64 recordings reads the game's array of handles once, checks all of the
handles, makes room for all of their records once (cmd_reserve_rec), then copies each straight in, using the handles
and sizes it checked (inline_recordings; called_size gives a handle's record bytes, copy_records copies them).
From its check to its last copy the call counts itself in calls_copying, and free_dead_copies frees no released
recording copy while such a call is copying (the next AddControlMemory or sweep frees it), so a recording the game
releases meanwhile is still there to copy. A call whose recordings come to more than CMD_SEG_MAX (4 GiB), and any
segment growth past it, stops with the NVN diagnostic instead of wrapping a size. The bytes recorded and the command
memory counts are the same as one at a time. A handle that is not live still stops the game, now before any of the
call is copied, which the game does not see. The result expected: fewer segment growths and fewer bytes copied by them
in calls of several recordings.

Both ways, CallCommands and CopyCommands check their count and array in one place (call_handles: a count of 0 reads
nothing, a negative count or a count with no array stops with the diagnostic), read each handle once, and the
HOS_NVN_WATCH_DRAW watch's table is taken and read atomically by the recording threads. The recordings' log line
(every 50,000 recordings) now also counts the segments' growths, the bytes they copied and the calls given their room
at once, so the in-game A/B can compare them.

The switch: HOS_NVN_CALL_RESERVE (new, an attempt, bool, its row in gpu/vk/switches.def), off unless set,
since nothing of it is measured in the game: unset or =0 is main's code, one recording at a time, the segment growing
for each that does not fit; =1 (any value but 0, false, off, no or empty) turns it on. It is read at the
first call of several recordings and cached; threads making that call together may each read the variable, and all
keep the first answer stored. An empty value in longshot_env.txt on Windows removes the variable (off, as unset).

From 3e8a2abd on exp/x86-asm-sync, where it was the second half of HOS_NVN_SEG_PEAK; the first half (a grown
segment growing at once to the command buffer's recent peak) is not part of this. Prior work: #1812 (GummiNaomi)
prefetched the next handle in the same CallCommands / CopyCommands loops and started each first segment at a size
hint; its author closed it unmerged, having measured no change at 4 or 16 cores. This one does neither: it sizes the
caller's segment once for the whole call.

Read first: call_reserve_on, inline_recordings, call_handles and free_dead_copies in host/nvn_native.c, then
tests/test_nvn_call_reserve.py.

Testing

Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head eac24360 on main 7c23d2be (main has since added only contributed pipeline-state lists, #2041). Queue job fb-2023-5 of this PC's job queue, every check in a hidden console: tests/test_nvn_call_reserve.py at the test commit 2c924d1a fails on the branch's earlier code (the switch on when unset; its cases one by one there, job fb-2023-3: the pin cases and two huge cases run until killed, a count with no array crashes, the others stop with other messages); at the head, tests/run_all.py --strict --exact tests/test_nvn_call_reserve.py tests/test_switch_table.py tests/test_recording_log_handoff.py tests/test_host_alloc_wait.py: 4 passed; tests/test_nvn_call_reserve.py again with no --cc and no CC (llvm-mingw found through LONGSHOT_MINGW): passes; tests/run_all.py --strict --changed 7c23d2be: 52 checks, 40 passed, 9 failed, 3 skipped (Linux-only); the 9 failures fail on main @ 7c23d2be too, run from a base worktree in the same job: tests/test_crash_dialog.py, tests/test_dev_ab.py, tests/test_dev_packs.py, tests/test_frame_overlap.py, tests/test_host_guest_writes.py, tests/test_linux_pressure.py, tests/test_present_queue_default.py, tests/test_target_reuse_memory.py, tests/test_vk_occlusion_queries.py; host/nvn_native.c compiles with the game build's own compile command (syntax only). The whole suite (mise run test) and lint did not run here (Still to test).

It has NOT run in the game on this PC yet.

Still to test:

  • Windows 11, i7-9700, RTX 4060 (this PC): Kokiri Forest turning the camera, and Castle Town Market walking through the
    crowd, on main @ 7c23d2be, this PR unset and this PR with HOS_NVN_CALL_RESERVE=1, each warmed a minute, then at
    least three interleaved three-minute runs of each (ABC ABC ABC), expecting no not a live recording line, no FATAL,
    the same picture all three ways, fewer segment growths and MB copied by them in the recordings' log line with it set,
    and average fps, 1% lows, median and p99 frame times and peak host memory of this PR with it set at least those of
    the other two; a difference beyond the runs' spread is a gain or a loss, one within it is no difference measured (by
    hand)
  • Linux NVIDIA and AMD, Windows AMD and Intel, Steam Deck: Kokiri Forest and Castle Town Market, three minutes each on
    main, on this PR unset and on this PR with HOS_NVN_CALL_RESERVE=1, warmed and interleaved, expecting no not a live recording line, no FATAL and the same picture (the test farm: linux+nvidia, linux+amd, windows+amd, windows+intel,
    steamdeck: Kokiri Forest and Castle Town Market three minutes, main, HOS_NVN_CALL_RESERVE unset and =1, warmed and
    interleaved, the log, frame times and a screenshot)
  • Linux: build the host and run python3 -I -B tests/test_nvn_call_reserve.py --cc gcc, expecting PASS (by hand)
  • A PC with pre-commit and ruff installed: mise run test and mise run lint at this head, expecting every failure to
    fail on main too and lint to pass (by hand; neither tool is installed on this PC)
  • Windows 11, i7-9700, RTX 4060 (this PC), with HOS_NVN_CALL_RESERVE=1: Castle Town Market, ten minutes walking
    through the crowd, the peak of the recordings' log line's live copies (MB) against the same run unset (copies released
    but held for a call count in it until freed) and the longest a released copy waited past the replay; this head logs
    the first, not the second (a temporary log line or a debugger), expecting the peak within a few MB of the unset run
    and the wait under a frame; if not, the bound in Risks comes before it can be turned on (by hand)

Measurements

Verdict: not measured: no micro-benchmark of this part alone and no run in the game on this PC yet, so it is off
by default; the in-game A/B is in Still to test. On the branch, HOS_NVN_SEG_PEAK as a whole (this and the peak
growth) measured no difference in play (2cc8489c: Kokiri Forest, i7-9700, RTX 4060, interleaved runs); its
micro-benchmark (3e8a2abd) is mostly the peak growth, which is not here, so it is not quoted. The test's segment counts
come from a malloc-backed fixture and say nothing of play.

Risks

What could break: with it set, a call's records copied wrong or counted wrong would show as wrong or missing draws in
any scene, or wrong heap sizes the game computes from the command memory counts; the test compares every recording's
bytes and counts with the switch on and off over 3,000 recordings, and a call's bytes and counts against the records
as written. With it set, released recording copies are freed later while calls of several recordings copy, and this
is not bounded at this head: free_dead_copies (host/nvn_native.c:1582) frees nothing while calls_copying (:1581)
is non-zero (:1586), and that count is raised and lowered only by the calls themselves (:1889, :1904). Nothing waits for
it to reach zero, caps the held list, or brings it to zero at a point each frame: the submit and present paths do not
touch it, and several recording threads calling CallCommands or CopyCommands back to back could keep it above zero at
every AddControlMemory (:1693, :1614) and sweep (:1653). At worst every recording copy released meanwhile stays held,
so the held memory grows with the copies released (on main they are freed once the replay passes them). Each call holds
the count only for its own checks and copies, so in practice a free should find a gap, but no code guarantees it; the
in-game figure is in Still to test. The bound to add before it could be turned on: two call counts by epoch, the free
switching new calls to the other count and freeing what was released before the switch once the old count drains, so
a released copy waits at most for the calls already copying when it was released.
Unset (the default), what changes from main: a call with a negative count or a count but no array stops with the NVN
diagnostic (it read memory at address 0 before), a recording segment past 4 GiB stops with it, the watch is atomic and
the log line has three more counts. Turn it off: unset it or HOS_NVN_CALL_RESERVE=0. Host code: the runtime is
rebuilt; nothing changes for players' saves, options or caches.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary `nvnCommandBufferCallCommands` and `nvnCommandBufferCopyCommands` copy each called recording into the caller's recording segment. They did it one recording at a time: each called recording that did not fit grew the segment by doubling, a copy of everything the segment held at each step, so a call of several large recordings could grow (and copy) the segment more than once. With `HOS_NVN_CALL_RESERVE=1`, a call of 2 to 64 recordings reads the game's array of handles once, checks all of the handles, makes room for all of their records once (`cmd_reserve_rec`), then copies each straight in, using the handles and sizes it checked (`inline_recordings`; `called_size` gives a handle's record bytes, `copy_records` copies them). From its check to its last copy the call counts itself in `calls_copying`, and `free_dead_copies` frees no released recording copy while such a call is copying (the next AddControlMemory or sweep frees it), so a recording the game releases meanwhile is still there to copy. A call whose recordings come to more than `CMD_SEG_MAX` (4 GiB), and any segment growth past it, stops with the NVN diagnostic instead of wrapping a size. The bytes recorded and the command memory counts are the same as one at a time. A handle that is not live still stops the game, now before any of the call is copied, which the game does not see. The result expected: fewer segment growths and fewer bytes copied by them in calls of several recordings. Both ways, CallCommands and CopyCommands check their count and array in one place (`call_handles`: a count of 0 reads nothing, a negative count or a count with no array stops with the diagnostic), read each handle once, and the `HOS_NVN_WATCH_DRAW` watch's table is taken and read atomically by the recording threads. The recordings' log line (every 50,000 recordings) now also counts the segments' growths, the bytes they copied and the calls given their room at once, so the in-game A/B can compare them. The switch: `HOS_NVN_CALL_RESERVE` (new, an attempt, bool, its row in `gpu/vk/switches.def`), **off unless set**, since nothing of it is measured in the game: unset or `=0` is main's code, one recording at a time, the segment growing for each that does not fit; `=1` (any value but `0`, `false`, `off`, `no` or empty) turns it on. It is read at the first call of several recordings and cached; threads making that call together may each read the variable, and all keep the first answer stored. An empty value in `longshot_env.txt` on Windows removes the variable (off, as unset). From 3e8a2abd on `exp/x86-asm-sync`, where it was the second half of `HOS_NVN_SEG_PEAK`; the first half (a grown segment growing at once to the command buffer's recent peak) is not part of this. Prior work: #1812 (GummiNaomi) prefetched the next handle in the same CallCommands / CopyCommands loops and started each first segment at a size hint; its author closed it unmerged, having measured no change at 4 or 16 cores. This one does neither: it sizes the caller's segment once for the whole call. Read first: `call_reserve_on`, `inline_recordings`, `call_handles` and `free_dead_copies` in `host/nvn_native.c`, then `tests/test_nvn_call_reserve.py`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head `eac24360` on main `7c23d2be` (main has since added only contributed pipeline-state lists, #2041). Queue job `fb-2023-5` of this PC's job queue, every check in a hidden console: `tests/test_nvn_call_reserve.py` at the test commit `2c924d1a` fails on the branch's earlier code (the switch on when unset; its cases one by one there, job `fb-2023-3`: the `pin` cases and two `huge` cases run until killed, a count with no array crashes, the others stop with other messages); at the head, `tests/run_all.py --strict --exact tests/test_nvn_call_reserve.py tests/test_switch_table.py tests/test_recording_log_handoff.py tests/test_host_alloc_wait.py`: 4 passed; `tests/test_nvn_call_reserve.py` again with no `--cc` and no `CC` (llvm-mingw found through `LONGSHOT_MINGW`): passes; `tests/run_all.py --strict --changed 7c23d2be`: 52 checks, 40 passed, 9 failed, 3 skipped (Linux-only); the 9 failures fail on main @ 7c23d2be too, run from a base worktree in the same job: `tests/test_crash_dialog.py`, `tests/test_dev_ab.py`, `tests/test_dev_packs.py`, `tests/test_frame_overlap.py`, `tests/test_host_guest_writes.py`, `tests/test_linux_pressure.py`, `tests/test_present_queue_default.py`, `tests/test_target_reuse_memory.py`, `tests/test_vk_occlusion_queries.py`; `host/nvn_native.c` compiles with the game build's own compile command (syntax only). The whole suite (`mise run test`) and lint did not run here (Still to test). **It has NOT run in the game on this PC yet.** **Still to test:** - Windows 11, i7-9700, RTX 4060 (this PC): Kokiri Forest turning the camera, and Castle Town Market walking through the crowd, on main @ 7c23d2be, this PR unset and this PR with `HOS_NVN_CALL_RESERVE=1`, each warmed a minute, then at least three interleaved three-minute runs of each (ABC ABC ABC), expecting no `not a live recording` line, no FATAL, the same picture all three ways, fewer segment growths and MB copied by them in the recordings' log line with it set, and average fps, 1% lows, median and p99 frame times and peak host memory of this PR with it set at least those of the other two; a difference beyond the runs' spread is a gain or a loss, one within it is no difference measured (by hand) - Linux NVIDIA and AMD, Windows AMD and Intel, Steam Deck: Kokiri Forest and Castle Town Market, three minutes each on main, on this PR unset and on this PR with `HOS_NVN_CALL_RESERVE=1`, warmed and interleaved, expecting no `not a live recording` line, no FATAL and the same picture (the test farm: linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest and Castle Town Market three minutes, main, HOS_NVN_CALL_RESERVE unset and =1, warmed and interleaved, the log, frame times and a screenshot) - Linux: build the host and run `python3 -I -B tests/test_nvn_call_reserve.py --cc gcc`, expecting PASS (by hand) - A PC with pre-commit and ruff installed: `mise run test` and `mise run lint` at this head, expecting every failure to fail on main too and lint to pass (by hand; neither tool is installed on this PC) - Windows 11, i7-9700, RTX 4060 (this PC), with `HOS_NVN_CALL_RESERVE=1`: Castle Town Market, ten minutes walking through the crowd, the peak of the recordings' log line's live copies (MB) against the same run unset (copies released but held for a call count in it until freed) and the longest a released copy waited past the replay; this head logs the first, not the second (a temporary log line or a debugger), expecting the peak within a few MB of the unset run and the wait under a frame; if not, the bound in Risks comes before it can be turned on (by hand) ## Measurements **Verdict:** not measured: no micro-benchmark of this part alone and no run in the game on this PC yet, so it is off by default; the in-game A/B is in Still to test. On the branch, `HOS_NVN_SEG_PEAK` as a whole (this and the peak growth) measured no difference in play (2cc8489c: Kokiri Forest, i7-9700, RTX 4060, interleaved runs); its micro-benchmark (3e8a2abd) is mostly the peak growth, which is not here, so it is not quoted. The test's segment counts come from a malloc-backed fixture and say nothing of play. ## Risks What could break: with it set, a call's records copied wrong or counted wrong would show as wrong or missing draws in any scene, or wrong heap sizes the game computes from the command memory counts; the test compares every recording's bytes and counts with the switch on and off over 3,000 recordings, and a call's bytes and counts against the records as written. With it set, released recording copies are freed later while calls of several recordings copy, and this is not bounded at this head: `free_dead_copies` (host/nvn_native.c:1582) frees nothing while `calls_copying` (:1581) is non-zero (:1586), and that count is raised and lowered only by the calls themselves (:1889, :1904). Nothing waits for it to reach zero, caps the held list, or brings it to zero at a point each frame: the submit and present paths do not touch it, and several recording threads calling CallCommands or CopyCommands back to back could keep it above zero at every AddControlMemory (:1693, :1614) and sweep (:1653). At worst every recording copy released meanwhile stays held, so the held memory grows with the copies released (on main they are freed once the replay passes them). Each call holds the count only for its own checks and copies, so in practice a free should find a gap, but no code guarantees it; the in-game figure is in Still to test. The bound to add before it could be turned on: two call counts by epoch, the free switching new calls to the other count and freeing what was released before the switch once the old count drains, so a released copy waits at most for the calls already copying when it was released. Unset (the default), what changes from main: a call with a negative count or a count but no array stops with the NVN diagnostic (it read memory at address 0 before), a recording segment past 4 GiB stops with it, the watch is atomic and the log line has three more counts. Turn it off: unset it or `HOS_NVN_CALL_RESERVE=0`. Host code: the runtime is rebuilt; nothing changes for players' saves, options or caches. ## Checklist ### Ready for review - [ ] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [ ] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
perf(nvn): a call of several recordings makes room for all of them at once (HOS_NVN_CALL_RESERVE=0 to disable)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 21s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
15dbbfd830
nvnCommandBufferCallCommands and CopyCommands copy each called recording into the caller's segment. One at a time,
each called recording that did not fit grew the segment by doubling, a copy of everything it held at each step, so a
call of several large recordings could grow (and copy) the segment more than once. A call of several recordings now
checks all of the handles, makes room for all of their records once (cmd_reserve_rec), then copies each straight in
(inline_recordings; called_size is the records' size, inline_recording takes it). The bytes recorded and the command
memory counts are the same; a handle that is not live still dies before anything of it is copied, now before any of
the call is copied, which the game does not see.

HOS_NVN_CALL_RESERVE (new, kill switch, its row in gpu/vk/switches.def): on by default, as the branch shipped it;
=0 copies them one at a time, as before. Read once, published atomically (threads that come first together keep one answer).

On the branch this went with HOS_NVN_SEG_PEAK (3e8a2abd: a grown segment grows at once to the command buffer's
recent largest recording), whose micro-benchmark of the recording path (a large caller after small ones: 1000 ->
12 KB of grow copies, -21..-23 % recording time) is mostly that part; that part is not ported (a segment on main
never shrinks within a command buffer's life, so it only acts after the command buffer is initialized again, which
#1812's segment size hints tried; #1812 was closed unmerged by its author). The in-play A/B on the branch
(2cc8489c: Kokiri, i7-9700, RTX 4060) found no difference for HOS_NVN_SEG_PEAK as a whole. Not measured on its own.

Test: tests/test_nvn_call_reserve.py runs the real recording path cut from the source (slot table, cmd_reserve_rec,
cmd_alloc, BeginRecording, EndRecording up to its copy, flatten, inline_recordings, CallCommands, CopyCommands) through
3,000 recordings of twelve command buffers calling 1 to 6 earlier recordings at a time, with the switch on and off:
every recording's bytes and command memory counts the same, fewer segments asked for with it on.

#1812 (GummiNaomi, closed unmerged by its author: no change at 4 or 16 cores) prefetched the next handle in the
same CallCommands / CopyCommands loops; this makes the room for all of a call's recordings once instead.

The test sets the switch in each run's own process (unset, 0, off, No, 1, yes, 2, and 0 again) and sets it the
other way after its first read, which must not change the answer. The C compiler is --cc, $CC, LONGSHOT_MINGW's clang
on Windows, then clang, gcc or cc.

Assisted-by: Claude Code
Member

Possibly related, for context:

  • PR #1780 WIP: perf(host): a finished command recording is handed over instead of copied (HOS_NVN_REC_HANDOFF=0 to copy) — Touches NVN recording copy/segment handling in the same host file; distinct handoff optimization but nearby code may overlap.
Possibly related, for context: - PR #1780 WIP: perf(host): a finished command recording is handed over instead of copied (HOS_NVN_REC_HANDOFF=0 to copy) — Touches NVN recording copy/segment handling in the same host file; distinct handoff optimization but nearby code may overlap.
BeanBot left a comment

Recommendation: Request changes. The new path can use a different handle from the one it validated, can outlive a validated recording, and can overflow the batch capacity calculation. The reported fixture passes, but the player benefit and enabled default still need in-game evidence.

CallCommands and CopyCommands reserve space for a batch before copying records. This can avoid intermediate segment allocations and copies when a batch crosses several capacity thresholds. It also adds a handle-list and size pass to every multi-handle call, including calls whose segment already has room.

@fix will do (12)

  1. L1 host/nvn_native.c:1847: The batch copy re-reads the guest array after validation and reserve. (more on the line)
  2. L2 host/nvn_native.c:1844: A live-set hit does not pin the recording. (more on the line)
  3. L3 host/nvn_native.c:1843: The new batch total is unchecked, and cmd_reserve_rec adds used and OP_RESERVE and doubles cap unchecked. (more on the line)
  4. L4 (minor) host/nvn_native.c:1846: The extracted watch path carries forward a shared-state race. (more on the line)
  5. L5 (minor) host/nvn_native.c:1858: PN checks the reserved guest window. (more on the line)
  6. L6 (minor) gpu/vk/switches.def:121: The row promises that an empty value disables the switch. (more on the line)
  7. L7 (minor) tests/test_nvn_call_reserve.py:263: The growth assertion pools both entry points, and both sides share the copy helper. (more on the line)
  8. L8 (minor) tests/test_nvn_call_reserve.py:124: The fixture primes the switch directly before the first NVN call and only uses live handles. (more on the line)
  9. L9 (minor) tests/test_nvn_call_reserve.py:135: The fixture simulates reinitialization by zeroing the object, orphaning its slot and its potentially multi-megabyte segment. (more on the line)
  10. L10 (minor) tests/test_nvn_call_reserve.py:216: The compiler picker treats CC as a single filename. (more on the line)
  11. L11 (its part an edit can do) host/nvn_native.c:1841: Count multi-handle calls, growths, and bytes copied by growth. Reuse validated sizes and skip redundant reserve checks on the batch path.
  12. L12 (minor): The checkbox says current main, but local main is 40bfceb0 and the tested base is 75317e1f. The reported 47 checks are the changed-file selection and leave the whole-suite and lint requirements open. Keep the current-main box unticked until verified, and record the complete check and lint results for the updated head.

For the author (1) (a run, a measurement or a decision)

  • L11 (minor) host/nvn_native.c:1841: The malloc-backed fixture counts requests and cannot show whether batching helps in play. (more on the line)

Checks: Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Unproven. The author reports equivalent output and fewer segment requests in a 3,000-recording synthetic fixture, with no request totals published. The earlier combined HOS_NVN_SEG_PEAK branch showed no measurable gain in play. This head has no isolated in-game comparison with 75317e1f.

Other systems: The changed host path feeds every Vulkan vendor and both inline and threaded replay on Windows and Linux, including Steam Deck. Queue waits, deferred binding resolution and registration order are unchanged in the diff. Recording lifetime, diagnostic watches and host RAM are affected. The diff changes no save, audio, input, shader cache or renderer ABI code. No game run verifies rendering or frame pacing yet.

Switches: HOS_NVN_CALL_RESERVE is a new developer kill switch at gpu/vk/switches.def:121. Unset enables batching for calls with at least two handles. 0, false, off and no disable it, as does an empty value when the CRT preserves it. Enabled matches the branch's test configuration. Measurements still need to establish the shipping default.

Reviewed: 15dbbfd830 against 75317e1f31: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo template, the commit and all three file diffs. Read the new test in full and the changed NVN functions with their allocation, handle publication and retirement, watch, guest range and submit code. Checked the switch table insertion, switch parser, source-cut helpers and test-runner discovery. No review checklist, PR run logs, comments or reviews were available. The rest of nvn_native.c, unrelated switch rows and unrelated recorded runs were skimmed.; nothing run (a code review).

Checked and fine:

  • The new cached integer uses atomic accesses throughout. Relaxed ordering is sufficient to publish a standalone boolean, and a losing compare-exchange returns the published result.
  • For a stable list of live recordings and representable sizes, payload order, BEGIN/END exclusion, empty-record handling and command counters match the previous loop.
  • The switch row is sorted and has the correct host prefix and kill-switch kind. The fixture isolates this switch in each child process.
  • The diff contains synthetic test data and no game-derived files, build artifacts, generated edits or third-party changes. The commit has an Assisted-by trailer. No renderer API version bump applies.
  • The fixture uses the standard library and its documented compile and child execution stay within its temporary-directory workflow.

Testing: Read-only review. The author reports 38 of 47 selected Windows checks passed, including the new fixture and switch table, with the same nine failures on the base. Forgejo checks were still running. Linux build and GCC fixture results, the full suite, lint and in-game results remain outstanding.

Overlaps and merge order: No overlapping open PR found in the supplied evidence or local history. The description identifies #1812 as closed and unmerged. No merge-order dependency found.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The new path can use a different handle from the one it validated, can outlive a validated recording, and can overflow the batch capacity calculation. The reported fixture passes, but the player benefit and enabled default still need in-game evidence. CallCommands and CopyCommands reserve space for a batch before copying records. This can avoid intermediate segment allocations and copies when a batch crosses several capacity thresholds. It also adds a handle-list and size pass to every multi-handle call, including calls whose segment already has room. **`@fix` will do (12)** 1. **L1** `host/nvn_native.c:1847`: The batch copy re-reads the guest array after validation and reserve. (more on the line) 2. **L2** `host/nvn_native.c:1844`: A live-set hit does not pin the recording. (more on the line) 3. **L3** `host/nvn_native.c:1843`: The new batch total is unchecked, and cmd_reserve_rec adds used and OP_RESERVE and doubles cap unchecked. (more on the line) 4. **L4** (minor) `host/nvn_native.c:1846`: The extracted watch path carries forward a shared-state race. (more on the line) 5. **L5** (minor) `host/nvn_native.c:1858`: PN checks the reserved guest window. (more on the line) 6. **L6** (minor) `gpu/vk/switches.def:121`: The row promises that an empty value disables the switch. (more on the line) 7. **L7** (minor) `tests/test_nvn_call_reserve.py:263`: The growth assertion pools both entry points, and both sides share the copy helper. (more on the line) 8. **L8** (minor) `tests/test_nvn_call_reserve.py:124`: The fixture primes the switch directly before the first NVN call and only uses live handles. (more on the line) 9. **L9** (minor) `tests/test_nvn_call_reserve.py:135`: The fixture simulates reinitialization by zeroing the object, orphaning its slot and its potentially multi-megabyte segment. (more on the line) 10. **L10** (minor) `tests/test_nvn_call_reserve.py:216`: The compiler picker treats CC as a single filename. (more on the line) 11. **L11** (its part an edit can do) `host/nvn_native.c:1841`: Count multi-handle calls, growths, and bytes copied by growth. Reuse validated sizes and skip redundant reserve checks on the batch path. 12. **L12** (minor): The checkbox says current main, but local main is 40bfceb0 and the tested base is 75317e1f. The reported 47 checks are the changed-file selection and leave the whole-suite and lint requirements open. Keep the current-main box unticked until verified, and record the complete check and lint results for the updated head. **For the author (1)** (a run, a measurement or a decision) - **L11** (minor) `host/nvn_native.c:1841`: The malloc-backed fixture counts requests and cannot show whether batching helps in play. (more on the line) **Checks:** Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven. The author reports equivalent output and fewer segment requests in a 3,000-recording synthetic fixture, with no request totals published. The earlier combined HOS_NVN_SEG_PEAK branch showed no measurable gain in play. This head has no isolated in-game comparison with 75317e1f. **Other systems:** The changed host path feeds every Vulkan vendor and both inline and threaded replay on Windows and Linux, including Steam Deck. Queue waits, deferred binding resolution and registration order are unchanged in the diff. Recording lifetime, diagnostic watches and host RAM are affected. The diff changes no save, audio, input, shader cache or renderer ABI code. No game run verifies rendering or frame pacing yet. **Switches:** HOS_NVN_CALL_RESERVE is a new developer kill switch at gpu/vk/switches.def:121. Unset enables batching for calls with at least two handles. 0, false, off and no disable it, as does an empty value when the CRT preserves it. Enabled matches the branch's test configuration. Measurements still need to establish the shipping default. **Reviewed:** 15dbbfd8300d against 75317e1f31ff: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo template, the commit and all three file diffs. Read the new test in full and the changed NVN functions with their allocation, handle publication and retirement, watch, guest range and submit code. Checked the switch table insertion, switch parser, source-cut helpers and test-runner discovery. No review checklist, PR run logs, comments or reviews were available. The rest of nvn_native.c, unrelated switch rows and unrelated recorded runs were skimmed.; nothing run (a code review). **Checked and fine:** - The new cached integer uses atomic accesses throughout. Relaxed ordering is sufficient to publish a standalone boolean, and a losing compare-exchange returns the published result. - For a stable list of live recordings and representable sizes, payload order, BEGIN/END exclusion, empty-record handling and command counters match the previous loop. - The switch row is sorted and has the correct host prefix and kill-switch kind. The fixture isolates this switch in each child process. - The diff contains synthetic test data and no game-derived files, build artifacts, generated edits or third-party changes. The commit has an Assisted-by trailer. No renderer API version bump applies. - The fixture uses the standard library and its documented compile and child execution stay within its temporary-directory workflow. **Testing:** Read-only review. The author reports 38 of 47 selected Windows checks passed, including the new fixture and switch table, with the same nine failures on the base. Forgejo checks were still running. Linux build and GCC fixture results, the full suite, lint and in-game results remain outstanding. **Overlaps and merge order:** No overlapping open PR found in the supplied evidence or local history. The description identifies #1812 as closed and unmerged. No merge-order dependency found. </details> --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -118,6 +118,7 @@ SW(HOS_NO_PRIORITIES, kill, bool, "Windows: boosted threads get above-normal or
SW(HOS_NO_RENDERER_WAIT, kill, bool, "the game's first GPU work waits for the renderer", "Non-zero skips the wait for the renderer before the game's first GPU work, as before lighting-fix.")
SW(HOS_NO_SDL, kill, bool, "SDL controllers on", "SDL's controller thread is never started: no SDL pads, only XInput.")
SW(HOS_NO_THP, attempt, bool, "off: THP as the system sets it outside the guest window", "=1 disables transparent huge pages for the whole process (PR_SET_THP_DISABLE).")
SW(HOS_NVN_CALL_RESERVE, kill, bool, "on: a CallCommands or CopyCommands of several recordings checks them all and makes room for all of them before copying any", "=0 (or false, off, no, empty; any other value is on) copies them one at a time, the segment growing for each that does not fit, as before. Read once, at the first such call.")
Member

L6 (minor): The row promises that an empty value disables the switch. On Windows, the file loader passes NAME= to _putenv, which removes the variable, so an empty HOS_NVN_CALL_RESERVE entry in longshot_env.txt leaves batching enabled. Document that case and give =0 as the portable spelling. Also describe the first value as cached, since simultaneous first callers can each read the environment before one publishes.

**L6** (minor): The row promises that an empty value disables the switch. On Windows, the file loader passes NAME= to _putenv, which removes the variable, so an empty HOS_NVN_CALL_RESERVE entry in longshot_env.txt leaves batching enabled. Document that case and give =0 as the portable spelling. Also describe the first value as cached, since simultaneous first callers can each read the environment before one publishes.
@ -1823,2 +1855,4 @@
}
}
NV(nvnCommandBufferCallCommands) {
const uint64_t *h = (const uint64_t *)PN(2, 8ull * (uint32_t)I(1));
Member

L5 (minor): PN checks the reserved guest window. A positive count with a null array still reaches h[0], and a list crossing into an unmapped region can pass PN and fault during the scan. This is inherited at these entry points. Share argument validation between both wrappers, handle zero count before pointer translation, and reject invalid counts, missing arrays and spans outside mapped guest memory through the NVN diagnostic.

**L5** (minor): PN checks the reserved guest window. A positive count with a null array still reaches h[0], and a list crossing into an unmapped region can pass PN and fault during the scan. This is inherited at these entry points. Share argument validation between both wrappers, handle zero count before pointer translation, and reject invalid counts, missing arrays and spans outside mapped guest memory through the NVN diagnostic.
@ -1823,0 +1838,4 @@
return on;
}
static void inline_recordings(Ctx *c, nvn_cmd *o, const uint64_t *h, int n, const char *how) {
if (n > 1 && call_reserve_on()) {
Member

L11 (minor): The malloc-backed fixture counts requests and cannot show whether batching helps in play. Run the base, head with the switch off and head with it unset on the same builds and settings, with warmed interleaved repeats in Kokiri Forest and a heavy area such as Castle Town Market. Report average FPS, 1% lows, median and p99 frame times, and peak host memory. Complete the own-PC game run and Tier 1 Windows, Linux and Deck checks before merge.

**L11** (minor): The malloc-backed fixture counts requests and cannot show whether batching helps in play. Run the base, head with the switch off and head with it unset on the same builds and settings, with warmed interleaved repeats in Kokiri Forest and a heavy area such as Castle Town Market. Report average FPS, 1% lows, median and p99 frame times, and peak host memory. Complete the own-PC game run and Tier 1 Windows, Linux and Deck checks before merge.
@ -1823,0 +1840,4 @@
static void inline_recordings(Ctx *c, nvn_cmd *o, const uint64_t *h, int n, const char *how) {
if (n > 1 && call_reserve_on()) {
size_t all = 0;
for (int i = 0; i < n; i++) check_handle(h[i]), all += called_size(h[i]);
Member

L3: The new batch total is unchecked, and cmd_reserve_rec adds used and OP_RESERVE and doubles cap unchecked. A list can repeat the same live large recording, so total bytes can exceed what a host segment can represent even when the source recording and array each fit. The sum can wrap or capacity can double through zero and loop forever before allocation. Check accumulation, reserve padding and capacity growth against a representable allocation limit, then fail through the normal fatal path. Cover the limit in the fixture with synthetic sizes.

**L3**: The new batch total is unchecked, and cmd_reserve_rec adds used and OP_RESERVE and doubles cap unchecked. A list can repeat the same live large recording, so total bytes can exceed what a host segment can represent even when the source recording and array each fit. The sum can wrap or capacity can double through zero and loop forever before allocation. Check accumulation, reserve padding and capacity growth against a representable allocation limit, then fail through the normal fatal path. Cover the limit in the fixture with synthetic sizes.
@ -1823,0 +1841,4 @@
if (n > 1 && call_reserve_on()) {
size_t all = 0;
for (int i = 0; i < n; i++) check_handle(h[i]), all += called_size(h[i]);
if (all) cmd_reserve_rec(cmd_host(o), all);
Member

L2: A live-set hit does not pin the recording. AddControlMemory or the overwrite sweep can retire an early handle while this call validates the rest or waits in reserve. free_dead_copies waits for replay submits, which does not protect this CPU copy. The second pass can then read a freed or reused prefix. The old path already had a shorter check-to-copy window. Hold a lifetime reference from validation through memcpy, with retirement deferring reclamation until that reference is released. Keep allocation retries outside copy_lock.

**L2**: A live-set hit does not pin the recording. AddControlMemory or the overwrite sweep can retire an early handle while this call validates the rest or waits in reserve. free_dead_copies waits for replay submits, which does not protect this CPU copy. The second pass can then read a freed or reused prefix. The old path already had a shorter check-to-copy window. Hold a lifetime reference from validation through memcpy, with retirement deferring reclamation until that reference is released. Keep allocation retries outside copy_lock.
@ -1823,0 +1843,4 @@
for (int i = 0; i < n; i++) check_handle(h[i]), all += called_size(h[i]);
if (all) cmd_reserve_rec(cmd_host(o), all);
for (int i = 0; i < n; i++) {
if (nwatch_handles) watch_handle_called(c, h[i], how);
Member

L4 (minor): The extracted watch path carries forward a shared-state race. EndRecording increments nwatch_handles while CallCommands and CopyCommands read it and scan watch_handles. Two writers that match watch_draw_cmd can also race through the 16-entry limit. With HOS_NVN_WATCH_DRAW enabled, synchronize the pending buffer and table update, publish initialized entries before the count, and read a stable snapshot before logging.

**L4** (minor): The extracted watch path carries forward a shared-state race. EndRecording increments nwatch_handles while CallCommands and CopyCommands read it and scan watch_handles. Two writers that match watch_draw_cmd can also race through the 16-entry limit. With HOS_NVN_WATCH_DRAW enabled, synchronize the pending buffer and table update, publish initialized entries before the count, and read a stable snapshot before logging.
@ -1823,0 +1844,4 @@
if (all) cmd_reserve_rec(cmd_host(o), all);
for (int i = 0; i < n; i++) {
if (nwatch_handles) watch_handle_called(c, h[i], how);
inline_recording(o, h[i], called_size(h[i]));
Member

L1: The batch copy re-reads the guest array after validation and reserve. If a slot changes while the host allocates, called_size can follow the replacement without checking it. The fallback also reloads h[i] after check_handle, where main passed one value into inline_recording. Capture each handle once and use the captured handle and size for the copy. A bounded snapshot with the original one-at-a-time path for larger lists would keep temporary storage predictable.

**L1**: The batch copy re-reads the guest array after validation and reserve. If a slot changes while the host allocates, called_size can follow the replacement without checking it. The fallback also reloads h[i] after check_handle, where main passed one value into inline_recording. Capture each handle once and use the captured handle and size for the copy. A bounded snapshot with the original one-at-a-time path for larger lists would keep temporary storage predictable.
@ -0,0 +121,4 @@
int main(int argc, char **argv) {
if (argc != 3) return 1;
test_reserve = atoi(argv[1]);
read_once(call_reserve_on, "HOS_NVN_CALL_RESERVE", test_reserve);
Member

L8 (minor): The fixture primes the switch directly before the first NVN call and only uses live handles. It would miss a single-handle call caching the answer early or the batch validation being removed. Drive first use through both entry points in fresh children, exercise invalid-handle rejection with a bad later handle, and cover simultaneous first callers. Add the advertised false and empty cases where the runtime preserves them.

**L8** (minor): The fixture primes the switch directly before the first NVN call and only uses live handles. It would miss a single-handle call caching the answer early or the batch validation being removed. Drive first use through both entry points in fresh children, exercise invalid-handle rejection with a bad later handle, and cover simultaneous first callers. Add the advertised false and empty cases where the runtime preserves them.
@ -0,0 +132,4 @@
for (int k = 0; k < recordings; k++) {
const int which = (int)(rnd() % NCMD);
nvn_cmd *o = &cmds[which];
if (rnd() % 100 == 0) /* (initialized anew: a new slot, a new segment from 64 KiB; the old one is left) */
Member

L9 (minor): The fixture simulates reinitialization by zeroing the object, orphaning its slot and its potentially multi-megabyte segment. Real Initialize releases both. This makes the growth counts depend on leaked segments and increases memory use on small CI workers. Exercise the real release and initialization path and free surviving fixture allocations at teardown.

**L9** (minor): The fixture simulates reinitialization by zeroing the object, orphaning its slot and its potentially multi-megabyte segment. Real Initialize releases both. This makes the growth counts depend on leaked segments and increases memory use on small CI workers. Exercise the real release and initialization path and free surviving fixture allocations at teardown.
@ -0,0 +213,4 @@
def compiler(given):
"""--cc, else $CC, else the configured llvm-mingw's clang (LONGSHOT_MINGW, Windows: run_all.py sets it to the
clone's folder), else clang, gcc or cc on PATH"""
for c in (given, os.environ.get("CC")):
Member

L10 (minor): The compiler picker treats CC as a single filename. CC='ccache gcc' fails when subprocess tries to execute that literal name. A direct Windows invocation also misses the clone's llvm-mingw unless the runner has populated LONGSHOT_MINGW. Support compiler command arguments while preserving explicit executable paths with spaces, and try the clone-local toolchain as the runner does.

**L10** (minor): The compiler picker treats CC as a single filename. CC='ccache gcc' fails when subprocess tries to execute that literal name. A direct Windows invocation also misses the clone's llvm-mingw unless the runner has populated LONGSHOT_MINGW. Support compiler command arguments while preserving explicit executable paths with spaces, and try the clone-local toolchain as the runner does.
@ -0,0 +260,4 @@
if len(a) != 3000 or multi < 1000:
print("FAIL: %d recordings, %d calls of several" % (len(a), multi))
sys.exit(1)
if gets_on >= gets_off:
Member

L7 (minor): The growth assertion pools both entry points, and both sides share the copy helper. CopyCommands can fall back to the old growth pattern while CallCommands satisfies the aggregate. Dropping the same record on both sides can also leave their hashes equal. Add a small deterministic case per API that forces two old growths and one batched growth, and assert independently constructed bytes and memory counts.

**L7** (minor): The growth assertion pools both entry points, and both sides share the copy helper. CopyCommands can fall back to the old growth pattern while CallCommands satisfies the aggregate. Dropping the same record on both sides can also leave their hashes equal. Add a small deterministic case per API that forces two old growths and one batched growth, and assert independently constructed bytes and memory counts.
siantm force-pushed perf/nvn-call-reserve from 15dbbfd830
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 21s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to eac243608f
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 4s
2026-11-21 08:21:40 +00:00
Compare
siantm changed title from WIP: perf(nvn): a call of several recordings makes room for all of them at once (HOS_NVN_CALL_RESERVE=0 to disable) to WIP: perf(nvn): a call of several recordings makes room for all of them at once (HOS_NVN_CALL_RESERVE=1 to try it) 2026-11-21 08:21:57 +00:00
Author
Contributor

Thank you for the review (#issuecomment-64237). Pushed eac24360, rebased on main @ 7c23d2be (main has since added only pipeline-state lists, #2041) (the branch's own commit unchanged by the rebase, git range-diff); test commit 2c924d1a (its new checks fail on the branch's earlier code: queue job fb-2023-5; its cases one by one on that code in fb-2023-3), fix eac24360. All checks named below ran in queue jobs on this PC (fb-2023-1, fb-2023-3, fb-2023-5); nothing has run in the game.

Default: there is no measurement in play, so HOS_NVN_CALL_RESERVE is now off by default (kind attempt): unset is main's one-at-a-time copy, =1 turns the batch on. The in-game A/B that could turn it on is in Still to test. The title says so.

  • L1 (#issuecomment-64226), the array read again after the check: fixed in eac24360: a call of 2 to 64 handles copies the guest's array once into a local snapshot with each handle's size, checks those and copies those; a larger call goes one at a time, and that loop reads each handle once and passes the same value to the watch, the check and the copy. The fixture's pin case changes the game's array to another live recording while the call makes its room: the call copies the recordings it checked (bytes compared with the records as written), for CallCommands and CopyCommands.
  • L2 (#issuecomment-64227), a checked recording freed before its copy: fixed in eac24360: the call counts itself in calls_copying from before its first check to after its last copy, and free_dead_copies frees nothing while that count is not zero (seq_cst fences on both sides, so either the call's check sees the release or the free sees the call); the next AddControlMemory or sweep frees what it left. The allocation stays outside copy_lock, which the call does not take. The pin case releases the called recordings (their control memory added again) and runs another AddControlMemory with the replay past them while the call makes its room: nothing is freed during the call, the bytes are the recordings', and the next AddControlMemory frees both. On the branch's code that case fails. The one-at-a-time path (the default) is main's, with main's shorter window; I left it as it is.
  • L3 (#issuecomment-64228), the sums and the doubling: fixed in eac24360: CMD_SEG_MAX (4 GiB, a power of two as the capacities are): the call's sum is checked before each add, and cmd_reserve_rec stops past it before its sum or its doubling can wrap, both through die (the NVN diagnostic). The fixture's huge case calls a live handle claiming 2^63 bytes alone, twice and after a small recording, on and off: each stops with that message (on the branch's code, two of them loop forever doubling through zero and the others fail allocating).
  • L4 (#issuecomment-64229), the watch's shared state: fixed in eac24360: the watched draw's command buffer is taken by one EndRecording with a compare-exchange, its slot claimed with an atomic add (past 16, none), the handle stored with a release store; the calls read each of the 16 slots once with an acquire load and skip empty ones. No test of its own: it is a race in a diagnostic, checked by reading; the fixture now runs the real watch code on every call.
  • L5 (#issuecomment-64230), the count and the array: fixed in eac24360 for both wrappers in one place (call_handles): a count of 0 reads nothing, a negative count or a count with no array stops with the diagnostic naming the call; the array must lie in the guest window as before (gw). The fixture's args case checks all three for both calls (a count with no array read address 0 before). I did not add a mapped-memory check of the array: the only one, hos_mem_mapped, takes hos_mem.c's lock, which would put a lock on every call on the default path; a guest array outside mapped memory faults as it does for every other guest pointer the NVN layer reads.
  • L6 (#issuecomment-64231), the row: fixed in eac24360: off unless set, =0 off everywhere, an empty value in longshot_env.txt on Windows removes the variable (off, as unset), and the first value described as cached, with simultaneous first callers each possibly reading the variable and all keeping the first answer stored.
  • L7 (#issuecomment-64232), the pooled growth assertion: fixed in 2c924d1a: the first case runs for CallCommands and for CopyCommands on its own: three 48 KiB recordings into a 64 KiB segment grow it twice one at a time and once with the room first, and the caller's bytes and command memory counts are compared with the records as the fixture wrote them and their sizes, not with another run.
  • L8 (#issuecomment-64233), the switch primed and only live handles: fixed in 2c924d1a: no case primes the switch; in first a call of one recording comes first and the variable is set only after it, so a single-handle call that read it would keep the other answer; the 3,000-recording run's first read is its first call of several. The real check_handle and live-handle set are cut from the source: bad calls two live handles and then one whose prefix is right but that EndRecording never made: on, the call stops with nothing copied; off, after the first two, as on main. race (a separate build of call_reserve_on with its reader replaced) has eight threads come first together, the reader answering each differently: all keep one answer. False and empty are among the cases, each with the child's environment set explicitly (empty reads as off whether the runtime keeps it or not).
  • L9 (#issuecomment-64234), the zeroed objects: fixed in 2c924d1a: the fixture runs the real nvnCommandBufferInitialize (its segment and slot given back) and Finalize, and frees every command buffer, kept recording and fixture block at the end.
  • L10 (#issuecomment-64235), the compiler: fixed in 2c924d1a: --cc and CC go through tests/compiler_command.py's parse_compiler (a path with spaces kept, a command with arguments split), then LONGSHOT_MINGW's clang, then the clone's llvm-mingw as run_all.py finds it, then clang, gcc or cc. It ran with no --cc and no CC in fb-2023-3, fb-2023-5.
  • L11 (in 64237, and #issuecomment-64236), counts and the measurement: the edit part is fixed in eac24360: the recordings' log line counts the segments' growths, the bytes those growths copied and the calls given their room at once (the first case checks the counts against its own), and the batch copies with the sizes it checked into the room made once, without a reserve per handle. The A/B is pending, for the owner: main, this PR unset and with =1, warmed and interleaved, in Kokiri Forest and Castle Town Market, with average fps, 1% lows, median and p99 frame times, peak host memory and the log line's counts, and the Tier 1 runs; Still to test lists them.
  • L12 (in 64237), the checklist and the checks: the branch is now on main @ 7c23d2be, which Tested names; main has since added only contributed pipeline-state lists (#2041), so the current-main box stays unticked. Tested lists what ran here: the PR's own checks and every check --changed 7c23d2be picks, with the failures that fail on main too. The whole suite and lint did not run on this PC (pre-commit and ruff are not installed here); they are in Still to test, and the mise run test box stays unticked.
Thank you for the review (#issuecomment-64237). Pushed eac24360, rebased on main @ 7c23d2be (main has since added only pipeline-state lists, #2041) (the branch's own commit unchanged by the rebase, `git range-diff`); test commit 2c924d1a (its new checks fail on the branch's earlier code: queue job fb-2023-5; its cases one by one on that code in fb-2023-3), fix eac24360. All checks named below ran in queue jobs on this PC (fb-2023-1, fb-2023-3, fb-2023-5); nothing has run in the game. **Default:** there is no measurement in play, so `HOS_NVN_CALL_RESERVE` is now off by default (kind attempt): unset is main's one-at-a-time copy, `=1` turns the batch on. The in-game A/B that could turn it on is in Still to test. The title says so. - **L1** ([#issuecomment-64226](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64226)), the array read again after the check: fixed in eac24360: a call of 2 to 64 handles copies the guest's array once into a local snapshot with each handle's size, checks those and copies those; a larger call goes one at a time, and that loop reads each handle once and passes the same value to the watch, the check and the copy. The fixture's `pin` case changes the game's array to another live recording while the call makes its room: the call copies the recordings it checked (bytes compared with the records as written), for CallCommands and CopyCommands. - **L2** ([#issuecomment-64227](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64227)), a checked recording freed before its copy: fixed in eac24360: the call counts itself in `calls_copying` from before its first check to after its last copy, and `free_dead_copies` frees nothing while that count is not zero (seq_cst fences on both sides, so either the call's check sees the release or the free sees the call); the next AddControlMemory or sweep frees what it left. The allocation stays outside `copy_lock`, which the call does not take. The `pin` case releases the called recordings (their control memory added again) and runs another AddControlMemory with the replay past them while the call makes its room: nothing is freed during the call, the bytes are the recordings', and the next AddControlMemory frees both. On the branch's code that case fails. The one-at-a-time path (the default) is main's, with main's shorter window; I left it as it is. - **L3** ([#issuecomment-64228](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64228)), the sums and the doubling: fixed in eac24360: `CMD_SEG_MAX` (4 GiB, a power of two as the capacities are): the call's sum is checked before each add, and `cmd_reserve_rec` stops past it before its sum or its doubling can wrap, both through `die` (the NVN diagnostic). The fixture's `huge` case calls a live handle claiming 2^63 bytes alone, twice and after a small recording, on and off: each stops with that message (on the branch's code, two of them loop forever doubling through zero and the others fail allocating). - **L4** ([#issuecomment-64229](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64229)), the watch's shared state: fixed in eac24360: the watched draw's command buffer is taken by one EndRecording with a compare-exchange, its slot claimed with an atomic add (past 16, none), the handle stored with a release store; the calls read each of the 16 slots once with an acquire load and skip empty ones. No test of its own: it is a race in a diagnostic, checked by reading; the fixture now runs the real watch code on every call. - **L5** ([#issuecomment-64230](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64230)), the count and the array: fixed in eac24360 for both wrappers in one place (`call_handles`): a count of 0 reads nothing, a negative count or a count with no array stops with the diagnostic naming the call; the array must lie in the guest window as before (`gw`). The fixture's `args` case checks all three for both calls (a count with no array read address 0 before). I did not add a mapped-memory check of the array: the only one, `hos_mem_mapped`, takes `hos_mem.c`'s lock, which would put a lock on every call on the default path; a guest array outside mapped memory faults as it does for every other guest pointer the NVN layer reads. - **L6** ([#issuecomment-64231](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64231)), the row: fixed in eac24360: off unless set, `=0` off everywhere, an empty value in `longshot_env.txt` on Windows removes the variable (off, as unset), and the first value described as cached, with simultaneous first callers each possibly reading the variable and all keeping the first answer stored. - **L7** ([#issuecomment-64232](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64232)), the pooled growth assertion: fixed in 2c924d1a: the `first` case runs for CallCommands and for CopyCommands on its own: three 48 KiB recordings into a 64 KiB segment grow it twice one at a time and once with the room first, and the caller's bytes and command memory counts are compared with the records as the fixture wrote them and their sizes, not with another run. - **L8** ([#issuecomment-64233](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64233)), the switch primed and only live handles: fixed in 2c924d1a: no case primes the switch; in `first` a call of one recording comes first and the variable is set only after it, so a single-handle call that read it would keep the other answer; the 3,000-recording run's first read is its first call of several. The real `check_handle` and live-handle set are cut from the source: `bad` calls two live handles and then one whose prefix is right but that EndRecording never made: on, the call stops with nothing copied; off, after the first two, as on main. `race` (a separate build of `call_reserve_on` with its reader replaced) has eight threads come first together, the reader answering each differently: all keep one answer. False and empty are among the cases, each with the child's environment set explicitly (empty reads as off whether the runtime keeps it or not). - **L9** ([#issuecomment-64234](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64234)), the zeroed objects: fixed in 2c924d1a: the fixture runs the real `nvnCommandBufferInitialize` (its segment and slot given back) and `Finalize`, and frees every command buffer, kept recording and fixture block at the end. - **L10** ([#issuecomment-64235](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64235)), the compiler: fixed in 2c924d1a: `--cc` and `CC` go through `tests/compiler_command.py`'s `parse_compiler` (a path with spaces kept, a command with arguments split), then `LONGSHOT_MINGW`'s clang, then the clone's `llvm-mingw` as `run_all.py` finds it, then clang, gcc or cc. It ran with no `--cc` and no `CC` in fb-2023-3, fb-2023-5. - **L11** (in 64237, and [#issuecomment-64236](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2023#issuecomment-64236)), counts and the measurement: the edit part is fixed in eac24360: the recordings' log line counts the segments' growths, the bytes those growths copied and the calls given their room at once (the `first` case checks the counts against its own), and the batch copies with the sizes it checked into the room made once, without a reserve per handle. The A/B is pending, for the owner: main, this PR unset and with `=1`, warmed and interleaved, in Kokiri Forest and Castle Town Market, with average fps, 1% lows, median and p99 frame times, peak host memory and the log line's counts, and the Tier 1 runs; Still to test lists them. - **L12** (in 64237), the checklist and the checks: the branch is now on main @ 7c23d2be, which Tested names; main has since added only contributed pipeline-state lists (#2041), so the current-main box stays unticked. Tested lists what ran here: the PR's own checks and every check `--changed 7c23d2be` picks, with the failures that fail on main too. The whole suite and lint did not run on this PC (pre-commit and ruff are not installed here); they are in Still to test, and the `mise run test` box stays unticked.
Author
Contributor

@Beanstalk This adds HOS_NVN_CALL_RESERVE, off by default: with HOS_NVN_CALL_RESERVE=1, NVN CallCommands and CopyCommands of several recordings reserve their room once (host/nvn_native.c); unset, main's way. It has not run in the game on these PCs yet. Please play Hyrule Field for three minutes, the PR against main on the same PC, on the PR's side with HOS_NVN_CALL_RESERVE=1 and with it unset, with a picture compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture, no crash, no FATAL line and no "not a live recording" line either way; set, fewer segment growths and fewer MB copied in the recordings' log line than unset, and frame time p99 equal to unset or better. Please report that line's counts.

@Beanstalk This adds HOS_NVN_CALL_RESERVE, off by default: with HOS_NVN_CALL_RESERVE=1, NVN CallCommands and CopyCommands of several recordings reserve their room once (host/nvn_native.c); unset, main's way. It has not run in the game on these PCs yet. Please play Hyrule Field for three minutes, the PR against main on the same PC, on the PR's side with HOS_NVN_CALL_RESERVE=1 and with it unset, with a picture compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture, no crash, no FATAL line and no "not a live recording" line either way; set, fewer segment growths and fewer MB copied in the recordings' log line than unset, and frame time p99 equal to unset or better. Please report that line's counts.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

Risks now says plainly that, with HOS_NVN_CALL_RESERVE=1, released recording copies held while calls of several recordings copy are not bounded at eac24360 (no cap, no wait, no point each frame where the count is zero), with the lines and the bound to add before it could be turned on; Still to test has the Market run that measures the held memory and the wait. No code change, no push.

Risks now says plainly that, with HOS_NVN_CALL_RESERVE=1, released recording copies held while calls of several recordings copy are not bounded at eac24360 (no cap, no wait, no point each frame where the count is zero), with the lines and the bound to add before it could be turned on; Still to test has the Market run that measures the held memory and the wait. No code change, no push.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 4s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/nvn-call-reserve:siantm-perf/nvn-call-reserve'
git switch 'siantm-perf/nvn-call-reserve'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2023
No description provided.