WIP: perf(nvn): a handle bind's record is written field by field, and HOS_NVN_WATCH_TEX's test is inline #1954

Draft
siantm wants to merge 5 commits from siantm/Longshot:perf/nvn-bind-fields into main
Contributor

Summary

Every texture, image and separate texture or sampler bind the game makes (BindTexture(s), BindImage(s),
BindSeparate* and their Deferred forms) is recorded by bind_handles (host/nvn_native.c). It zeroed the record
(rec()'s memset), then filled it, the handles through a memcpy of a size known only at run time. Every byte of
the record is one of its fields, so they are now stored directly with no zeroing first, and a single handle (as
BindTexture binds) as one 8-byte copy; the record's size comes from the header's and the handles' types, with its
8-byte invariant asserted. Before every texture bind, watch_tex (the HOS_NVN_WATCH_TEX debugging switch) was a
call that read its switch; its test is now an inline atomic load and the chain walk a cold function of its own. Moved
there, the watch's own faults from main are fixed in the same function: the switch is read once under a lock (threads
recording at once raced on it, and a value wider than 32 bits was read again at every bind), the chains seen are looked
up and added under that lock (after 64, one line says new ones are no longer logged), the frame walk reads only whole
frames inside the recording thread's own guest stack (hos_thread_stack, as svcQueryMemory bounds it) that are mapped,
going up it, and a count out of range is not read.

The bytes recorded are the same: tests/test_nvn_bind_handles.py (new) checks the record against a model of its bytes
and the command buffer's counters through the real allocator, and against the bind_handles before this PR, for every
count 0 to 32; it runs the watch with the switch unset, 0, not matching, too wide, matching (with a second mapped
allocation above the stack, which a frame link into must not reach) and from 8 threads.

From a7d746ac on exp/x86-asm-sync (its other half, ReportCounter's flag, is its own PR).

Read first: bind_handles, watch_tex and watch_tex_chains in host/nvn_native.c; tests/test_nvn_bind_handles.py.

Testing

Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks and a
micro-benchmark only. This change has not run in the game on this PC yet (the run is queued: Still to test). Each check
ran in a hidden console of its own, in one job of the PC's job queue (fb-1954-3), on this branch on main @ 6e1521a7
(with #1949), head fe1dd999:

  • python -I -B tests/test_nvn_bind_handles.py (fe1dd999), with no --cc (LONGSHOT_MINGW's clang) and again
    with CC set: "PASS: every case (records, unset, zero, other, wide, match, race)". At the test commit 0c4ecdf9 (the
    frame walk bounded by any mapped region, as at 519f5c7e) it fails as it should, in match: a frame link up into the
    second mapped allocation was followed and its words logged as callers ("106 < c1 < c2 < c3", reads there); the
    other cases pass there. (The round before, at the test commit 7c5dc91c, it failed in zero, other, wide, match and race
    against this PR's first code, fb-1954-2.)
  • python -I -B tests/run_all.py --strict --changed origin/main at fe1dd999 (the checks that read a changed file, 8 at
    once): 19 passed, 2 failed, 1 skipped (test_tracy_funcs.py: Linux only). The 2 failing checks
    (test_dev_packs.py, test_host_guest_writes.py) fail on main @ 6e1521a7 too, with the same messages (an
    unclassified longshot_fsr4_pipelines.bin; allow-list entries of host/hos_main.c and host/oot_menu.c), run by name
    in fb-1954-2; none fails on this branch alone.

Earlier, the same change on main @ 8859673e (3427c186): the micro-benchmark in Measurements.

Still to test:

  • Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): Castle Town's market, 60 s, this PR and main interleaved,
    no switches set, expecting the same picture and fps and frame times within the runs' spread or better (by hand,
    queued)
  • Windows 11, this PC: the same scene with HOS_NVN_WATCH_TEX set to a handle it binds, expecting its
    nvn: texture ... bound from lines, each chain once (by hand)
  • Linux: build the host and run python3 -I -B tests/test_nvn_bind_handles.py, expecting it to pass (by hand)
  • Linux, NVIDIA and AMD (Mesa RADV), and the Steam Deck: Castle Town's market and Hyrule Field, two minutes each, this PR
    against main, expecting the same frame times as main or better and the same picture (@Beanstalk play Castle Town's
    market and Hyrule Field on Linux and the Steam Deck, this PR against main, comparing frame times and screenshots)
  • Windows with an AMD GPU (Tier 1): Castle Town's market, two minutes, this PR against main, expecting the same frame
    times as main or better and the same picture (by hand or on the farm, not run yet)

Measurements

Verdict: micro-benchmark only, not measured in play yet: no gain shown. This micro-benchmark measured no difference
(1.83 ns a bind of one handle and 1.88 ns of four, on main and with this change); whether the full path gains
anything is open until the in-game runs in Still to test. The change has no switch (HOS_NVN_WATCH_TEX stays off by
default).

Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5, -O2 -march=native as the host is built; bind_handles of 1 and
4 handles cut out of each checkout's host/nvn_native.c, recording into a bump allocator (which may hide the zeroing's
cost: a harness closer to the command buffer's could show more); 21 interleaved rounds, the median and the rounds'
range, in one job of the PC's job queue (microbench.py bind). Measured on main @ 8859673e with this change as it was
there (3427c186); the rebase onto fc4a6769 changed none of its lines.

Before: main @ 8859673e After: this PR @ 3427c186 (the same change on 8859673e)
bind_handles, 1 handle, ns (median of 21 rounds, range) 1.83 (1.80 to 2.05) 1.83 (1.77 to 2.02)
bind_handles, 4 handles, ns (median of 21 rounds, range) 1.88 (1.81 to 2.06) 1.88 (1.79 to 4.75)
bind_handles, 1 handle(s)                main           median     1.83 ns  (rounds 1.80 to 2.05)
bind_handles, 1 handle(s)                PR             median     1.83 ns  (rounds 1.77 to 2.02)
bind_handles, 4 handle(s)                main           median     1.88 ns  (rounds 1.81 to 2.06)
bind_handles, 4 handle(s)                PR             median     1.88 ns  (rounds 1.79 to 4.75)

Risks

A byte of a record left unwritten would reach the replay as whatever the command buffer's memory held: a wrong texture
or sampler bound. The test checks every byte of the record against the old code for every count. HOS_NVN_WATCH_TEX
still reads its switch at the first texture bind and logs the same lines, with two differences: a value wider than
the 32 bits compared turns it off (it could never match), and after 64 chains one line says new ones are no longer
logged (each was logged at every bind before). The frame walk now asks hos_thread_stack and hos_mem_region (the
memory lock) per frame, only when a watched handle is bound. The array entry points (Bind*s) still check only the array's first byte, as on main: #2018 makes them check
the whole array (8 bytes a handle) and merges with this PR without a conflict in either order; proposed order #2018
first, after which this PR's 0 to 32 check in the watch only repeats it. No switch: a revert is enough. Players notice
nothing (no saves, longshot_options.ini or caches change).

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary Every texture, image and separate texture or sampler bind the game makes (`BindTexture(s)`, `BindImage(s)`, `BindSeparate*` and their `Deferred` forms) is recorded by `bind_handles` (`host/nvn_native.c`). It zeroed the record (`rec()`'s `memset`), then filled it, the handles through a `memcpy` of a size known only at run time. Every byte of the record is one of its fields, so they are now stored directly with no zeroing first, and a single handle (as `BindTexture` binds) as one 8-byte copy; the record's size comes from the header's and the handles' types, with its 8-byte invariant asserted. Before every texture bind, `watch_tex` (the `HOS_NVN_WATCH_TEX` debugging switch) was a call that read its switch; its test is now an inline atomic load and the chain walk a cold function of its own. Moved there, the watch's own faults from `main` are fixed in the same function: the switch is read once under a lock (threads recording at once raced on it, and a value wider than 32 bits was read again at every bind), the chains seen are looked up and added under that lock (after 64, one line says new ones are no longer logged), the frame walk reads only whole frames inside the recording thread's own guest stack (`hos_thread_stack`, as `svcQueryMemory` bounds it) that are mapped, going up it, and a count out of range is not read. The bytes recorded are the same: `tests/test_nvn_bind_handles.py` (new) checks the record against a model of its bytes and the command buffer's counters through the real allocator, and against the `bind_handles` before this PR, for every count 0 to 32; it runs the watch with the switch unset, `0`, not matching, too wide, matching (with a second mapped allocation above the stack, which a frame link into must not reach) and from 8 threads. From a7d746ac on `exp/x86-asm-sync` (its other half, `ReportCounter`'s flag, is its own PR). Read first: `bind_handles`, `watch_tex` and `watch_tex_chains` in `host/nvn_native.c`; `tests/test_nvn_bind_handles.py`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks and a micro-benchmark only. This change has not run in the game on this PC yet (the run is queued: Still to test). Each check ran in a hidden console of its own, in one job of the PC's job queue (fb-1954-3), on this branch on `main` @ 6e1521a7 (with #1949), head fe1dd999: - `python -I -B tests/test_nvn_bind_handles.py` (fe1dd999), with no `--cc` (`LONGSHOT_MINGW`'s clang) and again with `CC` set: "PASS: every case (records, unset, zero, other, wide, match, race)". At the test commit 0c4ecdf9 (the frame walk bounded by any mapped region, as at 519f5c7e) it fails as it should, in match: a frame link up into the second mapped allocation was followed and its words logged as callers ("106 < c1 < c2 < c3", reads there); the other cases pass there. (The round before, at the test commit 7c5dc91c, it failed in zero, other, wide, match and race against this PR's first code, fb-1954-2.) - `python -I -B tests/run_all.py --strict --changed origin/main` at fe1dd999 (the checks that read a changed file, 8 at once): 19 passed, 2 failed, 1 skipped (`test_tracy_funcs.py`: Linux only). The 2 failing checks (`test_dev_packs.py`, `test_host_guest_writes.py`) fail on `main` @ 6e1521a7 too, with the same messages (an unclassified `longshot_fsr4_pipelines.bin`; allow-list entries of `host/hos_main.c` and `host/oot_menu.c`), run by name in fb-1954-2; none fails on this branch alone. Earlier, the same change on `main` @ 8859673e (3427c186): the micro-benchmark in Measurements. **Still to test:** - Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): Castle Town's market, 60 s, this PR and `main` interleaved, no switches set, expecting the same picture and fps and frame times within the runs' spread or better (by hand, queued) - Windows 11, this PC: the same scene with `HOS_NVN_WATCH_TEX` set to a handle it binds, expecting its `nvn: texture ... bound from` lines, each chain once (by hand) - Linux: build the host and run `python3 -I -B tests/test_nvn_bind_handles.py`, expecting it to pass (by hand) - Linux, NVIDIA and AMD (Mesa RADV), and the Steam Deck: Castle Town's market and Hyrule Field, two minutes each, this PR against `main`, expecting the same frame times as `main` or better and the same picture (@Beanstalk play Castle Town's market and Hyrule Field on Linux and the Steam Deck, this PR against main, comparing frame times and screenshots) - Windows with an AMD GPU (Tier 1): Castle Town's market, two minutes, this PR against `main`, expecting the same frame times as `main` or better and the same picture (by hand or on the farm, not run yet) ## Measurements **Verdict:** micro-benchmark only, not measured in play yet: no gain shown. This micro-benchmark measured no difference (1.83 ns a bind of one handle and 1.88 ns of four, on `main` and with this change); whether the full path gains anything is open until the in-game runs in Still to test. The change has no switch (`HOS_NVN_WATCH_TEX` stays off by default). Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5, `-O2 -march=native` as the host is built; `bind_handles` of 1 and 4 handles cut out of each checkout's `host/nvn_native.c`, recording into a bump allocator (which may hide the zeroing's cost: a harness closer to the command buffer's could show more); 21 interleaved rounds, the median and the rounds' range, in one job of the PC's job queue (`microbench.py bind`). Measured on `main` @ 8859673e with this change as it was there (3427c186); the rebase onto fc4a6769 changed none of its lines. | | Before: main @ 8859673e | After: this PR @ 3427c186 (the same change on 8859673e) | |---|---|---| | `bind_handles`, 1 handle, ns (median of 21 rounds, range) | 1.83 (1.80 to 2.05) | 1.83 (1.77 to 2.02) | | `bind_handles`, 4 handles, ns (median of 21 rounds, range) | 1.88 (1.81 to 2.06) | 1.88 (1.79 to 4.75) | ``` bind_handles, 1 handle(s) main median 1.83 ns (rounds 1.80 to 2.05) bind_handles, 1 handle(s) PR median 1.83 ns (rounds 1.77 to 2.02) bind_handles, 4 handle(s) main median 1.88 ns (rounds 1.81 to 2.06) bind_handles, 4 handle(s) PR median 1.88 ns (rounds 1.79 to 4.75) ``` ## Risks A byte of a record left unwritten would reach the replay as whatever the command buffer's memory held: a wrong texture or sampler bound. The test checks every byte of the record against the old code for every count. `HOS_NVN_WATCH_TEX` still reads its switch at the first texture bind and logs the same lines, with two differences: a value wider than the 32 bits compared turns it off (it could never match), and after 64 chains one line says new ones are no longer logged (each was logged at every bind before). The frame walk now asks `hos_thread_stack` and `hos_mem_region` (the memory lock) per frame, only when a watched handle is bound. The array entry points (`Bind*s`) still check only the array's first byte, as on `main`: #2018 makes them check the whole array (8 bytes a handle) and merges with this PR without a conflict in either order; proposed order #2018 first, after which this PR's 0 to 32 check in the watch only repeats it. No switch: a revert is enough. Players notice nothing (no saves, `longshot_options.ini` or caches change). ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [ ] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
perf(nvn): a handle bind's record is written field by field, and HOS_NVN_WATCH_TEX's test is inline
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 7s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
5aa17e9634
bind_handles (BindTexture(s), BindImage(s), BindSeparate*, their Deferred forms) zeroed its record and then filled it,
through rec()'s memset and a memcpy of a size known only at run time; every byte of the record is one of its fields, so
they are now stored directly, a single handle as one store. watch_tex, called before every texture bind, read its
switch through a call; its test is inline now and the chain walk cold.

tests/test_nvn_bind_handles.py: the records, their sizes and the hardware-size accounting are the bytes the old
bind_handles recorded, for every count 0 to 32 with random fields.

From a7d746ac on exp/x86-asm-sync.

Assisted-by: Claude Code
Member

Possibly related, for context:

  • issue #402 Deck: the frame is CPU-bound in the submission path, not GPU-bound (16 fps at 50% internal, GPU-bound wait 0.4 ms) — renderer submission performance context; not specific to this bind change
Possibly related, for context: - issue #402 Deck: the frame is CPU-bound in the submission path, not GPU-bound (16 fps at 50% internal, GPU-bound wait 0.4 ms) — renderer submission performance context; not specific to this bind change
BeanBot left a comment

Recommendation: Request changes. The rewritten texture watch still races across recording threads, and the new test can pass with broken command-memory accounting. The reported benchmark shows no speedup, and the full path still needs an in-game comparison.

The host writes texture, image and sampler bind records directly and moves texture-watch logging off the usual bind path. This removes redundant initialization but adds a specialized record writer whose layout must stay in sync with the reader. No player benefit has been shown.

Does it help: Unproven. The PR reports 21 interleaved Windows microbenchmark rounds on an i7-9700, comparing 8859673e with 3427c186. The medians were identical at 1.83 ns for one handle and 1.88 ns for four. The harness omitted the real allocator and texture watch. No in-game FPS or frame-time data was supplied.
Other systems: The common Vulkan replay path consumes these records on NVIDIA, AMD and Steam Deck, including synchronous and threaded replay. The current payload fields and submit-time deferred source are preserved by inspection. A recording failure can still affect textures, images and samplers on every backend. The diff leaves queue ordering, descriptor registration, renderer APIs, saves, caches, audio and input untouched.
Switches: HOS_NVN_WATCH_TEX remains a hex-handle developer diagnostic, off by default, with its existing gpu/vk/switches.def row. Off is the right default. The change moves the check into the bind path and keeps lazy initialization. No switches were added or removed.

Reviewed: 5aa17e9634 against 2e1f6c32d3: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, the full two-file diff and commit, and all of tests/test_nvn_bind_handles.py. Read the changed parts of host/nvn_native.c and the relevant bind, allocator, thread, guest-memory, submit and replay code. Skimmed the rest of the host file, test-runner and CI setup, and profiling metadata. Read the run results quoted in the PR. There were no comments, earlier reviews, changed generated files or translations. No separate review checklist or sibling Tooling checkout was present.; nothing run (a code review).
Checked and fine:

  • For counts 0 through 32, the record is 32 plus 8 times the count in bytes, already aligned to eight. The writer sets the explicit pad to zero and preserves the header, source address and handle bytes for valid inputs.
  • The existing allocator and hardware-size formulas are preserved in production. The change adds no allocation, resource ownership or per-thread storage.
  • The patterned buffer will catch a missing pad write or wrong single-handle value. The fixture's compile and execution steps match the described test and require no game data.
  • The focused commit has its Assisted-by trailer. The diff contains no game-derived data, build output, vendor SDKs, generated files or third-party edits. No API version bump is needed.

Blocking (to change before merge)

  1. host/nvn_native.c:2166: The inline read races with the first-bind write. Two workers recording separate buffers can reach their first texture bind together, even with the switch unset. The sentinel check does not serialize initialization, and a parsed UINT64_MAX leaves it stuck at the sentinel. This race also existed on main. Initialize the handle before recorder threads start, or publish it through an atomic fast-path read and a once-only initializer.
  2. host/nvn_native.c:2173: The watch reads the array before bind_handles validates the count, and the array entry points check only its first byte. With the watch enabled, a count of 33 and 32 accessible handles can fault before the intended error. Validate the count and the entire readable span before either the watcher or recorder reads it, and skip the copy for zero handles. A fixed-size memcpy for one handle preserves the old byte-copy behavior for array arguments.
  3. host/nvn_native.c:2176: The cold frame walk still checks only alignment and the reserved guest window. A matching bind with x29 pointing into an unmapped guest page can crash while gathering diagnostics. Check that the whole 16-byte frame is readable in the current guest stack before reading either word, and stop the walk on a bad link.
  4. host/nvn_native.c:2182: The cold path retains an unsynchronized deduplication table. Two recording threads can both log the same new chain or race for the final slot. Once all 64 slots are full, an unseen chain logs on every subsequent match. Protect lookup and insertion together with a cold-path lock, log outside the lock, and define a bounded policy for a full table.
  5. tests/test_nvn_bind_handles.py:65: The allocator stubs never advance cmd_used or hw_used or set last_rec to the current record size. Both sides also call the same hw_fix, so a broken subtraction there still passes, and the fixture never asserts that writing stops at the record boundary. Cut the real allocator with its lower-level dependencies stubbed, assert independent counter deltas and expected record bytes, and check the trailing canary or a following record. Use malloc-backed storage so the fixture models the real segment's alignment and storage type.
  6. tests/test_nvn_bind_handles.py:17: The runner forces gcc, and standalone runs also default to gcc regardless of CC. On a Windows setup with only the documented llvm-mingw Clang, or a Clang-only Linux setup, the test fails to launch the compiler. Keep the explicit --cc override, honor CC, and default to the configured llvm-mingw or an available native C compiler.
  7. tests/test_nvn_bind_handles.py:108: The test never exercises the other changed behavior. Reversing the inline watch condition still passes all 20,000 cases. Add a game-free test that controls HOS_NVN_WATCH_TEX in the child environment and checks unset, matching, nonmatching and repeated binds, including concurrent first use once the initialization is fixed.
  8. tests/test_nvn_bind_handles.py:22: The new brace counter duplicates tests/srccut.py and counts braces in comments and strings. A brace added to a diagnostic comment can truncate the extracted function. Use the shared source cutter as required by the test guide, with an import that works under python -I.

Waiting on the author (a run, a measurement or a decision; no edit does it)

  • The isolated bind benchmark shows no gain and excludes the watch change. On the revised head, compare against its merge base in warmed, interleaved runs in the Market and a heavy area at the same settings. Report bind-path CPU time, average FPS, 1% lows and median and p99 frame times. Drop the performance change if the full path still shows no gain.
  • The PR opened before the required local game run. The supplied evidence covers an earlier Windows fixture and a subset of checks. Record the local game run, native Windows and Linux builds, full head checks and CI, and Tier 1 rendering results on affected setups, including AMD on Windows, which is missing from the test plan. Check that the same scene renders correctly with the watch off and that logging works with it enabled.

Suggestions (the author may take or leave)

  • Keep the record-size calculation tied to the header and handle element types, and assert the eight-byte layout invariant where the new writer skips rounding.
  • host/nvn_native.c:2176: Check that the whole 16-byte frame is readable in the guest stack. An aligned frame pointer in the reserved window can still point at an unmapped page.
  • host/nvn_native.c:2182: Protect the lookup and insertion together. Concurrent matches can duplicate a log or race for a slot, and an unseen chain logs on every match after the table fills.

Testing: Read-only review. The author reports 20,000 passing fixture cases at 7a8eb80c and a changed-file suite with 19 passes, two failures also reported on its base, and one skip. The skip is unnamed. No run was reported at 5aa17e96, and no Linux build, game run or head CI result was supplied.
Overlaps and merge order: The PR names a separate ReportCounter change from the same experiment. No sibling number or ordering dependency was visible in the checkout. The supplied base already includes #1946.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The rewritten texture watch still races across recording threads, and the new test can pass with broken command-memory accounting. The reported benchmark shows no speedup, and the full path still needs an in-game comparison. The host writes texture, image and sampler bind records directly and moves texture-watch logging off the usual bind path. This removes redundant initialization but adds a specialized record writer whose layout must stay in sync with the reader. No player benefit has been shown. **Does it help:** Unproven. The PR reports 21 interleaved Windows microbenchmark rounds on an i7-9700, comparing 8859673e with 3427c186. The medians were identical at 1.83 ns for one handle and 1.88 ns for four. The harness omitted the real allocator and texture watch. No in-game FPS or frame-time data was supplied. **Other systems:** The common Vulkan replay path consumes these records on NVIDIA, AMD and Steam Deck, including synchronous and threaded replay. The current payload fields and submit-time deferred source are preserved by inspection. A recording failure can still affect textures, images and samplers on every backend. The diff leaves queue ordering, descriptor registration, renderer APIs, saves, caches, audio and input untouched. **Switches:** HOS_NVN_WATCH_TEX remains a hex-handle developer diagnostic, off by default, with its existing gpu/vk/switches.def row. Off is the right default. The change moves the check into the bind path and keeps lazy initialization. No switches were added or removed. **Reviewed:** 5aa17e9634d5 against 2e1f6c32d30e: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, the full two-file diff and commit, and all of tests/test_nvn_bind_handles.py. Read the changed parts of host/nvn_native.c and the relevant bind, allocator, thread, guest-memory, submit and replay code. Skimmed the rest of the host file, test-runner and CI setup, and profiling metadata. Read the run results quoted in the PR. There were no comments, earlier reviews, changed generated files or translations. No separate review checklist or sibling Tooling checkout was present.; nothing run (a code review). **Checked and fine:** - For counts 0 through 32, the record is 32 plus 8 times the count in bytes, already aligned to eight. The writer sets the explicit pad to zero and preserves the header, source address and handle bytes for valid inputs. - The existing allocator and hardware-size formulas are preserved in production. The change adds no allocation, resource ownership or per-thread storage. - The patterned buffer will catch a missing pad write or wrong single-handle value. The fixture's compile and execution steps match the described test and require no game data. - The focused commit has its Assisted-by trailer. The diff contains no game-derived data, build output, vendor SDKs, generated files or third-party edits. No API version bump is needed. **Blocking** (to change before merge) 1. host/nvn_native.c:2166: The inline read races with the first-bind write. Two workers recording separate buffers can reach their first texture bind together, even with the switch unset. The sentinel check does not serialize initialization, and a parsed UINT64_MAX leaves it stuck at the sentinel. This race also existed on main. Initialize the handle before recorder threads start, or publish it through an atomic fast-path read and a once-only initializer. 2. host/nvn_native.c:2173: The watch reads the array before bind_handles validates the count, and the array entry points check only its first byte. With the watch enabled, a count of 33 and 32 accessible handles can fault before the intended error. Validate the count and the entire readable span before either the watcher or recorder reads it, and skip the copy for zero handles. A fixed-size memcpy for one handle preserves the old byte-copy behavior for array arguments. 3. host/nvn_native.c:2176: The cold frame walk still checks only alignment and the reserved guest window. A matching bind with x29 pointing into an unmapped guest page can crash while gathering diagnostics. Check that the whole 16-byte frame is readable in the current guest stack before reading either word, and stop the walk on a bad link. 4. host/nvn_native.c:2182: The cold path retains an unsynchronized deduplication table. Two recording threads can both log the same new chain or race for the final slot. Once all 64 slots are full, an unseen chain logs on every subsequent match. Protect lookup and insertion together with a cold-path lock, log outside the lock, and define a bounded policy for a full table. 5. tests/test_nvn_bind_handles.py:65: The allocator stubs never advance cmd_used or hw_used or set last_rec to the current record size. Both sides also call the same hw_fix, so a broken subtraction there still passes, and the fixture never asserts that writing stops at the record boundary. Cut the real allocator with its lower-level dependencies stubbed, assert independent counter deltas and expected record bytes, and check the trailing canary or a following record. Use malloc-backed storage so the fixture models the real segment's alignment and storage type. 6. tests/test_nvn_bind_handles.py:17: The runner forces gcc, and standalone runs also default to gcc regardless of CC. On a Windows setup with only the documented llvm-mingw Clang, or a Clang-only Linux setup, the test fails to launch the compiler. Keep the explicit --cc override, honor CC, and default to the configured llvm-mingw or an available native C compiler. 7. tests/test_nvn_bind_handles.py:108: The test never exercises the other changed behavior. Reversing the inline watch condition still passes all 20,000 cases. Add a game-free test that controls HOS_NVN_WATCH_TEX in the child environment and checks unset, matching, nonmatching and repeated binds, including concurrent first use once the initialization is fixed. 8. tests/test_nvn_bind_handles.py:22: The new brace counter duplicates tests/srccut.py and counts braces in comments and strings. A brace added to a diagnostic comment can truncate the extracted function. Use the shared source cutter as required by the test guide, with an import that works under python -I. **Waiting on the author** (a run, a measurement or a decision; no edit does it) - The isolated bind benchmark shows no gain and excludes the watch change. On the revised head, compare against its merge base in warmed, interleaved runs in the Market and a heavy area at the same settings. Report bind-path CPU time, average FPS, 1% lows and median and p99 frame times. Drop the performance change if the full path still shows no gain. - The PR opened before the required local game run. The supplied evidence covers an earlier Windows fixture and a subset of checks. Record the local game run, native Windows and Linux builds, full head checks and CI, and Tier 1 rendering results on affected setups, including AMD on Windows, which is missing from the test plan. Check that the same scene renders correctly with the watch off and that logging works with it enabled. **Suggestions** (the author may take or leave) - Keep the record-size calculation tied to the header and handle element types, and assert the eight-byte layout invariant where the new writer skips rounding. - `host/nvn_native.c:2176`: Check that the whole 16-byte frame is readable in the guest stack. An aligned frame pointer in the reserved window can still point at an unmapped page. - `host/nvn_native.c:2182`: Protect the lookup and insertion together. Concurrent matches can duplicate a log or race for a slot, and an unseen chain logs on every match after the table fills. **Testing:** Read-only review. The author reports 20,000 passing fixture cases at 7a8eb80c and a changed-file suite with 19 passes, two failures also reported on its base, and one skip. The skip is unnamed. No run was reported at 5aa17e96, and no Linux build, game run or head CI result was supplied. **Overlaps and merge order:** The PR names a separate ReportCounter change from the same experiment. No sibling number or ordering dependency was visible in the checkout. The supplied base already includes #1946. --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -2156,2 +2170,4 @@
if (want == ~0ull)
want = watch_tex_want = getenv("HOS_NVN_WATCH_TEX") ? strtoull(getenv("HOS_NVN_WATCH_TEX"), NULL, 16) : 0;
if (!want) return;
for (int i = 0; i < n; i++) {
Member

Validate the count and full readable handle span before the watch or the recorder reads it. An oversized count can currently make the watch run past the array before the recorder rejects it.

Validate the count and full readable handle span before the watch or the recorder reads it. An oversized count can currently make the watch run past the array before the recorder rejects it.
@ -2141,2 +2141,4 @@
NV(nvnCommandBufferBindUniformBuffersDeferred) { bind_bufs_deferred(c, 0); }
NV(nvnCommandBufferBindStorageBuffersDeferred) { bind_bufs_deferred(c, 1); }
/* (longshot) The record written field by field, pad_ too, with no zeroing first (every byte is one of them) and one
handle stored as one: the memset and memcpy calls of a record of variable size were a third of a BindTexture. The
Member

Please source or remove the one-third figure. The supplied benchmark does not measure the whole BindTexture call.

Please source or remove the one-third figure. The supplied benchmark does not measure the whole BindTexture call.
@ -2156,0 +2163,4 @@
static uint64_t watch_tex_want = ~0ull;
static __attribute__((noinline, cold)) void watch_tex_chains(Ctx *c, const uint64_t *h, int n);
static inline void watch_tex(Ctx *c, const uint64_t *h, int n) {
if (__builtin_expect(watch_tex_want != 0, 0)) watch_tex_chains(c, h, n);
Member

Two recording threads can race with first-use initialization even when the watch is off. Initialize before recording or use an atomic fast-path read and once-only initialization.

Two recording threads can race with first-use initialization even when the watch is off. Initialize before recording or use an atomic fast-path read and once-only initialization.
@ -0,0 +14,4 @@
import tempfile
from pathlib import Path
RUN_ALL_ARGS = ["--cc", "gcc"]
Member

Honor CC and choose an available native compiler by default. Forcing gcc breaks an llvm-mingw-only Windows setup.

Honor CC and choose an available native compiler by default. Forcing gcc breaks an llvm-mingw-only Windows setup.
@ -0,0 +19,4 @@
ROOT = Path(__file__).resolve().parents[1]
def block(text, marker):
Member

Use tests/srccut.py. This counter treats braces in diagnostic strings and comments as C syntax.

Use tests/srccut.py. This counter treats braces in diagnostic strings and comments as C syntax.
@ -0,0 +62,4 @@
static uint8_t blk[2][4096];
static uint32_t asked[2];
/* (the recording's next bytes: cmd_alloc's, rounded to 8 as it does; the new code's block holds a pattern) */
static uint8_t *cmd_alloc(nvn_cmd *o, uint32_t size) { (void)o; asked[0] = (size + 7) & ~7u; memset(blk[0], 0xA5, sizeof blk[0]); return blk[0]; }
Member

The stub skips the real allocator's counter writes, and both sides share hw_fix. Test independent expected counter deltas and check that the writer leaves the bytes beyond the record untouched.

The stub skips the real allocator's counter writes, and both sides share hw_fix. Test independent expected counter deltas and check that the writer leaves the bytes beyond the record untouched.
@ -0,0 +105,4 @@
p.add_argument("--cc", default="gcc")
a = p.parse_args()
nvn = (ROOT / "host" / "nvn_native.c").read_text(encoding="utf-8")
cut = "\n".join(block(nvn, m) for m in ("static void put_hdr(", "static void hw_fix(", "static void bind_handles("))
Member

All cases pass if the inline watch condition is reversed. Exercise unset, matching, nonmatching and repeated binds with the child environment set explicitly.

All cases pass if the inline watch condition is reversed. Exercise unset, matching, nonmatching and repeated binds with the child environment set explicitly.
siantm force-pushed perf/nvn-bind-fields from 5aa17e9634
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 7s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 519f5c7e7d
Some checks failed
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
beanstalk/approval waiting for a BigBean's 👍 on the request
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
2026-11-21 02:53:51 +00:00
Compare
Author
Contributor

Thank you for the review. Pushed, rebased onto current main @ 6e1521a7 (with #1949): head 519f5c7e (was 5aa17e96). git range-diff shows the PR's own commit unchanged by the rebase; on top of it, 7c5dc91c (the test, first) and 519f5c7e (the fix).

Blocking points of the review:

  1. First-use race of the watch handle (:2166): fixed in 519f5c7e. The inline test reads watch_tex_want as an atomic (relaxed) load; the first bind reads the switch once under watch_tex_lock, re-checking it there, and stores the handle atomically. A value wider than the 32 bits compared (so ffffffffffffffff too, which stayed at the sentinel) now turns the watch off; before, such a handle could never match. Test: 8 threads binding at once for the first time read the switch once and log once; ffffffffffffffff reads it once over 1000 binds.
  2. Count and span before the read (:2173, :1745): the count part is fixed in 519f5c7e: the watch reads nothing when the count is out of 0 to 32 (bind_handles then stops the game, as before); bind_handles copies no handle for a count of 0 and copies one handle with an 8-byte memcpy, so a game array of any alignment is read as before. Test: a count of 33 whose 33rd handle matches logs nothing. The span part is not changed here: the array entry points check only the first byte of P(4) on main too, and this PR does not touch those entry lines (it changes bind_handles and the watch). Checking 8 * count bytes needs every Bind*s entry line changed, which is its own change; I list it for the owner.
  3. Cold frame walk (:2176): the walk is the same as on main, but this PR moved it into watch_tex_chains, so the fix stays in that function: 519f5c7e reads a frame only when all 16 bytes are mapped guest memory (hos_mem_region), and stops at a link that does not go up the stack. Test: a frame linking to an unmapped page in the window, a frame running past the mapped stack (neither is read) and a link to itself (the walk ends).
  4. Dedup table (:2182): likewise from main, inside the moved function, so fixed in 519f5c7e: the lookup and the insertion happen together under watch_tex_lock, and the line is logged outside it. Policy for a full table: after 64 chains, one line says new ones are no longer logged, then nothing more. Test: 64 chains, then new and repeated ones: one more line in total.
  5. Allocator stubs, counters, canary (tests:65): fixed in 7c5dc91c. The check cuts the real cmd_alloc and cmd_reserve_rec and stubs the pool below them with malloc'd blocks filled with a pattern. All 20000 binds go into one recording, which grows. Each record is checked against bytes built from the inputs alone (header, fields, pad_ 0, handles). cmd_used must grow by the record's size, and hw_used and last_rec by the hardware size given (or 16 + 4 per handle), not through hw_fix. The 64 bytes after the record must keep the pattern. The comparison with the old bind_handles stays.
  6. Honor CC (tests:17): fixed in 7c5dc91c, in the order tests/test_fpx_sticky.py uses: --cc, then $CC, then on Windows LONGSHOT_MINGW's clang (which run_all.py sets to the clone's llvm-mingw), then clang, gcc or cc on PATH. It fails, not passes, with no compiler. RUN_ALL_ARGS = ["--cc", "gcc"] is gone, so the runner no longer forces gcc.
  7. The inline watch condition (tests:108): fixed in 7c5dc91c. The check runs the fixture once per case, with HOS_NVN_WATCH_TEX set in the child's environment: unset, 0, beef (a handle no bind names), ffffffffffffffff, 1234 (new chain, repeated bind, the same chain from a four-handle bind, and the variable changed after the first bind, which is not read again) and 8 threads at once. With the condition reversed the switch would never be read, so the unset case (one read expected) and the matching case (lines expected) would fail. That is read from the code; the reversed condition was not run.
  8. Use tests/srccut.py (tests:22): fixed in 7c5dc91c: block and between from tests/srccut.py, imported as the other checks do (the check's folder put on sys.path, so it works under python -I). The brace counter is gone.

Inline :2144 (the one-third figure): removed in 519f5c7e. The micro-benchmark did not time the whole BindTexture. The comment now says only what changed (no memset, no run-time-sized memcpy for one handle).

Suggestions: taken in 519f5c7e. The record's size comes from sizeof(nvn_op_hdr), offsetof(nvn_p_bind_handles, h) and the handle's size, and a _Static_assert holds the 8-byte invariant the writer relies on (no rounding). The frame walk and the dedup suggestions are points 3 and 4.

Checked through the PC's job queue (job fb-1954-2, on this head): tests/test_nvn_bind_handles.py passes at 519f5c7e (every case: records, unset, zero, other, wide, match, race), run with no --cc and again with CC set. At the test commit 7c5dc91c (this PR's code before the fix) it fails in zero, other, wide, match and race, and records and unset pass there. tests/run_all.py --strict --changed origin/main: 19 passed, 2 failed, 1 skipped. The 2 failures (test_dev_packs.py, test_host_guest_writes.py) fail on main @ 6e1521a7 with the same messages. The description's Summary, Tested, Measurements and Risks are updated; nothing has run in the game.

"Does it help": BeanBot reads the PR's numbers right. The micro-benchmark shows no gain (1.83 ns a bind of one handle and 1.88 ns of four, on main and with this change), and nothing has been measured in play. Measurements now says so plainly. The change has no switch; HOS_NVN_WATCH_TEX stays off by default.

Waiting on the author, all pending and in Still to test (none has run): interleaved in-game runs against the merge base in the Market and a heavy area (bind-path CPU time, average fps, 1 % lows, median and p99 frame times), dropping the change if the full path shows no gain; the local game run with the watch off (same picture) and on (its lines); the Windows and Linux builds, the full checks and CI on this head; the Tier 1 runs, Windows AMD included.

Thank you for the review. Pushed, rebased onto current `main` @ 6e1521a7 (with #1949): head 519f5c7e (was 5aa17e96). `git range-diff` shows the PR's own commit unchanged by the rebase; on top of it, 7c5dc91c (the test, first) and 519f5c7e (the fix). Blocking points of the [review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61948): 1. [First-use race of the watch handle](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61942) (`:2166`): fixed in 519f5c7e. The inline test reads `watch_tex_want` as an atomic (relaxed) load; the first bind reads the switch once under `watch_tex_lock`, re-checking it there, and stores the handle atomically. A value wider than the 32 bits compared (so `ffffffffffffffff` too, which stayed at the sentinel) now turns the watch off; before, such a handle could never match. Test: 8 threads binding at once for the first time read the switch once and log once; `ffffffffffffffff` reads it once over 1000 binds. 2. [Count and span before the read](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61943) (`:2173`, `:1745`): the count part is fixed in 519f5c7e: the watch reads nothing when the count is out of 0 to 32 (`bind_handles` then stops the game, as before); `bind_handles` copies no handle for a count of 0 and copies one handle with an 8-byte `memcpy`, so a game array of any alignment is read as before. Test: a count of 33 whose 33rd handle matches logs nothing. The span part is not changed here: the array entry points check only the first byte of `P(4)` on `main` too, and this PR does not touch those entry lines (it changes `bind_handles` and the watch). Checking `8 * count` bytes needs every `Bind*s` entry line changed, which is its own change; I list it for the owner. 3. [Cold frame walk](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61948) (`:2176`): the walk is the same as on `main`, but this PR moved it into `watch_tex_chains`, so the fix stays in that function: 519f5c7e reads a frame only when all 16 bytes are mapped guest memory (`hos_mem_region`), and stops at a link that does not go up the stack. Test: a frame linking to an unmapped page in the window, a frame running past the mapped stack (neither is read) and a link to itself (the walk ends). 4. [Dedup table](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61948) (`:2182`): likewise from `main`, inside the moved function, so fixed in 519f5c7e: the lookup and the insertion happen together under `watch_tex_lock`, and the line is logged outside it. Policy for a full table: after 64 chains, one line says new ones are no longer logged, then nothing more. Test: 64 chains, then new and repeated ones: one more line in total. 5. [Allocator stubs, counters, canary](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61946) (`tests:65`): fixed in 7c5dc91c. The check cuts the real `cmd_alloc` and `cmd_reserve_rec` and stubs the pool below them with malloc'd blocks filled with a pattern. All 20000 binds go into one recording, which grows. Each record is checked against bytes built from the inputs alone (header, fields, `pad_` 0, handles). `cmd_used` must grow by the record's size, and `hw_used` and `last_rec` by the hardware size given (or 16 + 4 per handle), not through `hw_fix`. The 64 bytes after the record must keep the pattern. The comparison with the old `bind_handles` stays. 6. [Honor CC](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61944) (`tests:17`): fixed in 7c5dc91c, in the order `tests/test_fpx_sticky.py` uses: `--cc`, then `$CC`, then on Windows `LONGSHOT_MINGW`'s clang (which `run_all.py` sets to the clone's llvm-mingw), then `clang`, `gcc` or `cc` on PATH. It fails, not passes, with no compiler. `RUN_ALL_ARGS = ["--cc", "gcc"]` is gone, so the runner no longer forces gcc. 7. [The inline watch condition](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61947) (`tests:108`): fixed in 7c5dc91c. The check runs the fixture once per case, with `HOS_NVN_WATCH_TEX` set in the child's environment: unset, `0`, `beef` (a handle no bind names), `ffffffffffffffff`, `1234` (new chain, repeated bind, the same chain from a four-handle bind, and the variable changed after the first bind, which is not read again) and 8 threads at once. With the condition reversed the switch would never be read, so the unset case (one read expected) and the matching case (lines expected) would fail. That is read from the code; the reversed condition was not run. 8. [Use tests/srccut.py](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61945) (`tests:22`): fixed in 7c5dc91c: `block` and `between` from `tests/srccut.py`, imported as the other checks do (the check's folder put on `sys.path`, so it works under `python -I`). The brace counter is gone. [Inline `:2144`](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61941) (the one-third figure): removed in 519f5c7e. The micro-benchmark did not time the whole `BindTexture`. The comment now says only what changed (no `memset`, no run-time-sized `memcpy` for one handle). Suggestions: taken in 519f5c7e. The record's size comes from `sizeof(nvn_op_hdr)`, `offsetof(nvn_p_bind_handles, h)` and the handle's size, and a `_Static_assert` holds the 8-byte invariant the writer relies on (no rounding). The frame walk and the dedup suggestions are points 3 and 4. Checked through the PC's job queue (job fb-1954-2, on this head): `tests/test_nvn_bind_handles.py` passes at 519f5c7e (every case: records, unset, zero, other, wide, match, race), run with no `--cc` and again with `CC` set. At the test commit 7c5dc91c (this PR's code before the fix) it fails in zero, other, wide, match and race, and records and unset pass there. `tests/run_all.py --strict --changed origin/main`: 19 passed, 2 failed, 1 skipped. The 2 failures (`test_dev_packs.py`, `test_host_guest_writes.py`) fail on `main` @ 6e1521a7 with the same messages. The description's Summary, Tested, Measurements and Risks are updated; nothing has run in the game. "Does it help": BeanBot reads the PR's numbers right. The micro-benchmark shows no gain (1.83 ns a bind of one handle and 1.88 ns of four, on `main` and with this change), and nothing has been measured in play. Measurements now says so plainly. The change has no switch; `HOS_NVN_WATCH_TEX` stays off by default. Waiting on the author, all pending and in Still to test (none has run): interleaved in-game runs against the merge base in the Market and a heavy area (bind-path CPU time, average fps, 1 % lows, median and p99 frame times), dropping the change if the full path shows no gain; the local game run with the watch off (same picture) and on (its lines); the Windows and Linux builds, the full checks and CI on this head; the Tier 1 runs, Windows AMD included.
Author
Contributor

@Beanstalk This writes the NVN texture, image and sampler bind records field by field (host/nvn_native.c) instead of building each record and copying it, and the test of the off-by-default HOS_NVN_WATCH_TEX debug watch is inline; the bytes written must be the same. It has not run in the game here with this head yet. Please play Castle Town Market and Hyrule Field, a minute each, with a picture in each compared with main's, on Linux with NVIDIA, on Linux with AMD (RADV), on Windows with an Intel GPU and on the Steam Deck, the PR against main. The pictures must match main's (no wrong or missing textures), no crash; frame time p99 equal to main's or better, within the spread.

@Beanstalk This writes the NVN texture, image and sampler bind records field by field (host/nvn_native.c) instead of building each record and copying it, and the test of the off-by-default HOS_NVN_WATCH_TEX debug watch is inline; the bytes written must be the same. It has not run in the game here with this head yet. Please play Castle Town Market and Hyrule Field, a minute each, with a picture in each compared with main's, on Linux with NVIDIA, on Linux with AMD (RADV), on Windows with an Intel GPU and on the Steam Deck, the PR against main. The pictures must match main's (no wrong or missing textures), no crash; frame time p99 equal to main's or better, within the spread.
Member

@siantm only members of BigBeans can start the farm.

One of them can approve this request by reacting 👍 to it or to this comment. This request was for 519f5c7, and the PR has changed since (fe1dd99), so a 👍 no longer approves it. Comment @Beanstalk again to ask for the new commit.

@siantm only members of `BigBeans` can start the farm. ~~One of them can approve this request by reacting 👍 to it or to this comment.~~ This request was for `519f5c7`, and the PR has changed since (`fe1dd99`), so a 👍 no longer approves it. Comment `@Beanstalk` again to ask for the new commit.
futondestroyer left a comment

Recommendation: Request changes. Bound the diagnostic frame walk to the recording thread’s guest stack and make the concurrency fixture safe when it observes duplicate logs. The performance gain is still unproven.

The rebase includes the renderer build fix. The revised writer preserves the record layout and submit-time deferred handles. The fixture now runs the real allocator, checks independent counter deltas and record bytes, and checks the trailing bytes. The watch now initializes once under a lock, uses an atomic fast-path read and bounds the deduplication table. The missing full-span check in the bind-array wrappers is inherited. Prior review

Blocking

  1. host/nvn_native.c:2191 allows a rising frame link into any mapped guest allocation. hos_mem_region also accepts the image, heap, system objects and other threads’ stacks. The watch can read unrelated words and report them as callers. Bound the full 16-byte frame to the current guest stack using hos_thread_stack, as svcQueryMemory does, and keep the mapped-range check. Model a second mapped allocation in the fixture so it catches an out-of-stack link.
  2. tests/test_nvn_bind_handles.py:137 formats every log into the same last_log array. If deduplication regresses, multiple workers can write it concurrently, causing undefined behavior in the case the fixture should report as a clean failure. Count only in the threaded case or protect the fixture’s buffer with its own lock.

Does it help? Unproven. The earlier 21-round microbenchmark had the same medians on both sides, 1.83 ns for one handle and 1.88 ns for four. After the source fixes, compare warmed and interleaved runs in the Market and a heavy scene against the updated merge base. Include bind-path CPU time, average FPS, 1% lows, median and p99 frame times. If the comparison stays within the spread, the measured result does not justify the record-writer optimization. The diagnostic fixes can be reviewed on their own merits.

Testing. The author reports all seven focused cases passed on Windows at this head. The selected suite reports 19 passes, two failures reproduced on base and one Linux-only skip. Head CI, native Linux build and gameplay remain open. The author’s farm request is awaiting approval and names Linux NVIDIA, Linux RADV, Windows Intel and Steam Deck. Add AMD Windows to the Tier 1 plan. The separate ReportCounter PR has no ordering dependency. Farm request ReportCounter PR

Reviewed 519f5c7e against 6e1521a7. Checked the interdiff, all changes to both files and revision-matched Graphify maps. Read the relevant allocator, bind, submit, replay, memory-map and thread-stack source. Nothing from the PR was run.

**Recommendation: Request changes.** Bound the diagnostic frame walk to the recording thread’s guest stack and make the concurrency fixture safe when it observes duplicate logs. The performance gain is still unproven. The rebase includes the renderer build fix. The revised writer preserves the record layout and submit-time deferred handles. The fixture now runs the real allocator, checks independent counter deltas and record bytes, and checks the trailing bytes. The watch now initializes once under a lock, uses an atomic fast-path read and bounds the deduplication table. The missing full-span check in the bind-array wrappers is inherited. [Prior review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61948) **Blocking** 1. `host/nvn_native.c:2191` allows a rising frame link into any mapped guest allocation. `hos_mem_region` also accepts the image, heap, system objects and other threads’ stacks. The watch can read unrelated words and report them as callers. Bound the full 16-byte frame to the current guest stack using `hos_thread_stack`, as `svcQueryMemory` does, and keep the mapped-range check. Model a second mapped allocation in the fixture so it catches an out-of-stack link. 2. `tests/test_nvn_bind_handles.py:137` formats every log into the same `last_log` array. If deduplication regresses, multiple workers can write it concurrently, causing undefined behavior in the case the fixture should report as a clean failure. Count only in the threaded case or protect the fixture’s buffer with its own lock. **Does it help?** Unproven. The earlier 21-round microbenchmark had the same medians on both sides, 1.83 ns for one handle and 1.88 ns for four. After the source fixes, compare warmed and interleaved runs in the Market and a heavy scene against the updated merge base. Include bind-path CPU time, average FPS, 1% lows, median and p99 frame times. If the comparison stays within the spread, the measured result does not justify the record-writer optimization. The diagnostic fixes can be reviewed on their own merits. **Testing.** The author reports all seven focused cases passed on Windows at this head. The selected suite reports 19 passes, two failures reproduced on base and one Linux-only skip. Head CI, native Linux build and gameplay remain open. The author’s farm request is awaiting approval and names Linux NVIDIA, Linux RADV, Windows Intel and Steam Deck. Add AMD Windows to the Tier 1 plan. The separate ReportCounter PR has no ordering dependency. [Farm request](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-62721) [ReportCounter PR](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1955) **Reviewed** `519f5c7e` against `6e1521a7`. Checked the interdiff, all changes to both files and revision-matched Graphify maps. Read the relevant allocator, bind, submit, replay, memory-map and thread-stack source. Nothing from the PR was run.
@ -2158,3 +2189,3 @@
if ((h[i] & 0xFFFFFFFFull) != want) continue;
uint64_t chain[8] = {c->x[30]}, fp = c->x[29];
for (int k = 1; k < 8 && fp && fp < 0x8000000000ull && !(fp & 7); k++) chain[k] = G64(fp + 8), fp = G64(fp);
for (int k = 1; k < 8 && fp && !(fp & 7) && hos_mem_region(fp, 16); k++) { /* (the whole frame mapped) */
Member

A higher frame link can land in another mapped guest allocation. hos_mem_region accepts the heap, image and other threads' stacks too. Keep the mapping check and bound the whole frame to the current guest stack via hos_thread_stack. Add a fixture case with a link into a second mapped allocation.

A higher frame link can land in another mapped guest allocation. hos_mem_region accepts the heap, image and other threads' stacks too. Keep the mapping check and bound the whole frame to the current guest stack via hos_thread_stack. Add a fixture case with a link into a second mapped allocation.
@ -0,0 +134,4 @@
static void hos_log(const char *fmt, ...) {
va_list ap;
va_start(ap, fmt);
vsnprintf(last_log, sizeof last_log, fmt, ap);
Member

If deduplication regresses, several workers can write last_log concurrently here. Make the fixture race-free even on failure by counting only in the threaded case or guarding the buffer with a fixture lock.

If deduplication regresses, several workers can write last_log concurrently here. Make the fixture race-free even on failure by counting only in the threaded case or guarding the buffer with a fixture lock.
The watch's fixture now maps a second allocation above the recording thread's stack (the heap, another thread's
stack) and gives the thread's stack through hos_thread_stack: a frame link up into that allocation must end the walk
with nothing read there. Its hos_log formats under a lock of its own, so lines logged from several threads at once
(a deduplication regression) leave last_log whole and the race case fails cleanly.

Assisted-by: Claude Code
fix(nvn): HOS_NVN_WATCH_TEX walks frames only inside the recording thread's guest stack
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 18s
fe1dd99925
A rising frame link could reach any mapped guest allocation (the image, the heap, system objects, another thread's
stack), and the watch logged the words there as callers. Each 16-byte frame must now lie inside the current guest
thread's stack (hos_thread_stack, as svcQueryMemory bounds it) and be mapped; on a thread with no guest thread nothing
past the link register is walked.

Assisted-by: Claude Code
Author
Contributor

@Beanstalk This writes the NVN texture, image and sampler bind records field by field (host/nvn_native.c) instead of building each record and copying it, and the test of the off-by-default HOS_NVN_WATCH_TEX debug watch is inline; the bytes written must be the same. It has not run in the game here with this head yet. Please play Castle Town Market and Hyrule Field, a minute each, with a picture in each compared with main's, on Linux with NVIDIA, on Linux with AMD (RADV), on Windows with an Intel GPU and on the Steam Deck, the PR against main. The pictures must match main's (no wrong or missing textures), no crash; frame time p99 equal to main's or better, within the spread.

@Beanstalk This writes the NVN texture, image and sampler bind records field by field (host/nvn_native.c) instead of building each record and copying it, and the test of the off-by-default HOS_NVN_WATCH_TEX debug watch is inline; the bytes written must be the same. It has not run in the game here with this head yet. Please play Castle Town Market and Hyrule Field, a minute each, with a picture in each compared with main's, on Linux with NVIDIA, on Linux with AMD (RADV), on Windows with an Intel GPU and on the Steam Deck, the PR against main. The pictures must match main's (no wrong or missing textures), no crash; frame time p99 equal to main's or better, within the spread.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

@futondestroyer thanks for the second look (62883). Both points are fixed at fe1dd999 (test 0c4ecdf9 first, then the fix fe1dd999; still on main @ 6e1521a7, so no rebase was needed this round):

  • 62881 (frame walk): fe1dd999 now walks a frame only when the whole 16-byte frame lies inside the current guest thread's stack, taken from hos_thread_stack(hos_self(), ...) as svcQueryMemory does, and is also mapped (hos_mem_region is still checked). A link that leaves the stack ends the walk. With no guest thread, nothing past the link register is walked. The fixture (0c4ecdf9) now maps a second allocation above the thread's stack and has a frame whose link points into it. At 0c4ecdf9, with the walk still bounded only by a mapped region, the match case fails: it logged 106 < c1 < c2 < c3 and read the other allocation. At fe1dd999 it logs 106 < c1 and reads nothing there.
  • 62882 (last_log): the fixture's hos_log now formats into last_log under its own SRWLOCK, so lines logged from several workers when dedup regresses can no longer race on the buffer. The race case then fails on its counts.

Run through this PC's job queue (fb-1954-3, Windows 11, llvm-mingw clang 23.1.5): at fe1dd999, tests/test_nvn_bind_handles.py passes all seven cases, both with no --cc and with CC set. run_all.py --strict --changed origin/main gives 19 passed and 2 failed. The two failures are test_dev_packs, test_host_guest_writes, and they also fail on main @ 6e1521a7. I added Windows with an AMD GPU to Still to test. The description's Testing section is updated.

Still pending on my side: the interleaved in-game runs against the merge base (Market and a heavy scene: bind-path CPU time, average fps, 1% lows, median and p99 frame times). If those stay within the spread, I'll say plainly that the record-writer part has no measured gain. Also pending: Linux, Tier 1 and CI at this head.

@futondestroyer thanks for the second look ([62883](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-62883)). Both points are fixed at fe1dd999 (test 0c4ecdf9 first, then the fix fe1dd999; still on `main` @ 6e1521a7, so no rebase was needed this round): - [62881](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-62881) (frame walk): fe1dd999 now walks a frame only when the whole 16-byte frame lies inside the current guest thread's stack, taken from `hos_thread_stack(hos_self(), ...)` as `svcQueryMemory` does, and is also mapped (`hos_mem_region` is still checked). A link that leaves the stack ends the walk. With no guest thread, nothing past the link register is walked. The fixture (0c4ecdf9) now maps a second allocation above the thread's stack and has a frame whose link points into it. At 0c4ecdf9, with the walk still bounded only by a mapped region, the `match` case fails: it logged `106 < c1 < c2 < c3` and read the other allocation. At fe1dd999 it logs `106 < c1` and reads nothing there. - [62882](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-62882) (`last_log`): the fixture's `hos_log` now formats into `last_log` under its own `SRWLOCK`, so lines logged from several workers when dedup regresses can no longer race on the buffer. The race case then fails on its counts. Run through this PC's job queue (fb-1954-3, Windows 11, llvm-mingw clang 23.1.5): at fe1dd999, `tests/test_nvn_bind_handles.py` passes all seven cases, both with no `--cc` and with `CC` set. `run_all.py --strict --changed origin/main` gives 19 passed and 2 failed. The two failures are `test_dev_packs`, `test_host_guest_writes`, and they also fail on `main` @ 6e1521a7. I added Windows with an AMD GPU to Still to test. The description's Testing section is updated. Still pending on my side: the interleaved in-game runs against the merge base (Market and a heavy scene: bind-path CPU time, average fps, 1% lows, median and p99 frame times). If those stay within the spread, I'll say plainly that the record-writer part has no measured gain. Also pending: Linux, Tier 1 and CI at this head.
Author
Contributor

On BeanBot's point 2, count and span before the read and the inherited span check named in your review: the span part is now its own fix, #2018 (fix/nvn-bind-array-span, test 405eb181, fix cfd24dda, on main @ 40bfceb0). Every Bind*s entry point checks the count (0 to 32) and then all 8 x count bytes of the game's array in the guest window before the watch or the record reads it, and BindVertexBuffers checks its 16-byte ranges whole. tests/test_nvn_bind_span.py runs the real entry points over a guest window followed by no-access pages: on main 13 of its cases read past the window's end and fault before any stop, with the fix all 55 pass (this PC's job queue, fb3-sep-2). It and this PR merge without a conflict in either order; not run in the game yet.

On [BeanBot's point 2, count and span before the read](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-61943) and the inherited span check named in [your review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1954#issuecomment-62883): the span part is now its own fix, #2018 (`fix/nvn-bind-array-span`, test 405eb181, fix cfd24dda, on `main` @ 40bfceb0). Every `Bind*s` entry point checks the count (0 to 32) and then all 8 x count bytes of the game's array in the guest window before the watch or the record reads it, and `BindVertexBuffers` checks its 16-byte ranges whole. `tests/test_nvn_bind_span.py` runs the real entry points over a guest window followed by no-access pages: on `main` 13 of its cases read past the window's end and fault before any stop, with the fix all 55 pass (this PC's job queue, fb3-sep-2). It and this PR merge without a conflict in either order; not run in the game yet.
Author
Contributor

Description: Risks names #2018 (the Bind*s whole-array check) and the proposed landing order; no code change.

Description: Risks names #2018 (the Bind*s whole-array check) and the proposed landing order; no code change.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 18s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/nvn-bind-fields:siantm-perf/nvn-bind-fields'
git switch 'siantm-perf/nvn-bind-fields'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1954
No description provided.