WIP: perf(nvn): recorded copies in flat per-bucket arrays instead of malloc'd list nodes #2022
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2022
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/nvn-rec-flat"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Longshot copies every command recording the game makes (
nvnCommandBufferEndRecording) and keeps the copies by the64 KiB block of their control memory, in 65,536 buckets, so that adding control memory again (
AddControlMemory) orthe sweep of overwritten recordings can let the old copies go. Each bucket was a linked list of nodes, one
malloc'dper recording and freed per release, spread over the heap: a release or a sweep walks its buckets whole, the copies
that stay included, one dependent cache miss a copy, all of it with
copy_lockheld exclusively, which everyrecording thread waits for.
This makes a bucket a flat array of
{ctl, copy, marked}entries, oldest first, walked newest first as thehead-inserted lists were; the copies that stay move back to the array's start in their order (
rec_keep). Copies arereleased, wait dead and are freed in exactly the order they were, and a recording no longer allocates a node
(
copy_track): what EndRecording allocates is its copy (from the pool), the buckets' table at the first recording, and abucket's room at its first entry and when it grows, as the handle set's when that grows. An array grows by doubling under
copy_lock, as the dead list and the handle set already do, and never shrinks. The three places thatturned a copy from a handle into a dead copy (
release_copiesand the sweep) are one function,copy_retire. Theresult expected: shorter exclusive
copy_locksections at every EndRecording and AddControlMemory; what the game seesis the same.
With the review (BeanBot 64225) it also fixes what the moved code carried or added:
rec_bucketsis read undercopy_lockonly, the table made byrec_buckets_make(the host's checked allocation, before the lock is taken,published under it,
HOS_NVN_REC_SHIFTread there once); the release's and the sweep's ticket (the submits a releasedcopy waits for) read the held count before the pushed one, under the lock, and a flush empties the held list only after
it pushed it, so with
HOS_NVN_HOLDa flush between the two reads is counted twice, never not at all; a bucket stopsthe game past 2^31 entries rather than wrap;
AddControlMemorystops the game on a range that is not guest memory (pastthe guest window, or wrapping around); the 50,000-recording log line adds the buckets' room (entries and MB), their
growths and the kept copies moved back.
No switch: the same releases and frees in the same order, so a revert is enough.
From d4eb931f on
exp/x86-asm-sync. Prior work: #1812 (GummiNaomi) changed the same functions (frees outside thelock, the sweep's ticket read at retirement, segment size hints) but kept the list nodes; its author closed it
unmerged, having measured no change at 4 or 16 cores. This one changes what #1812 left as it was: the buckets' nodes
themselves, not when they are freed.
Read first:
copy_track,copy_retire,rec_keep,rec_buckets_make,release_copiesandsweep_overwritten_copiesinhost/nvn_native.c, thentests/test_nvn_rec_release.py.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
db31aa81on mainfd7dc40d, with the Vulkan headers (RX_CPM). Build jobfb-2022-2of this PC's job queue, each check in a hidden console:tests/test_nvn_rec_release.pyat the test commit9f83a9a2fails as expected (rec_buckets read without the lock, the release's ticket missing a flushed held submit, the full bucket reallocating to 0 bytes, wrapping and past-the-window AddControlMemory ranges accepted,rec_buckets_makeabsent) and passes at the head;tests/run_all.py --strict --exactwithtests/test_nvn_rec_release.py,tests/test_recording_log_handoff.pyandtests/test_switch_table.py: 3 passed;tests/test_recording_log_handoff.pywith no--ccand noCC(llvm-mingw fromLONGSHOT_MINGW) passes;host/nvn_native.ccompiles (-fsyntax-only) with the game build's own command.tests/run_all.py --strict --changed origin/mainpicks every check (tests/cccmd.pyis new): 780 passed, 82 failed (81 failing checks andtests/test_vbytes.pytimed out at 600 s), 48 skipped, none for want of the Vulkan headers; 79 of the 81 fail on mainfd7dc40dtoo, run from a main worktree in the same job; the other two,tests/test_recording_admission.pyandtests/test_frame_wait_bounds.py(neither reads a file this changes), pass at the head run again alone in jobfb-2022-3(wheretests/test_recording_admission.pyfailed on main instead).tests/test_vbytes.py(nx2, picked only becausetests/cccmd.pyis new): not run (hangs on this PC past 20 min, at this head and on main alike: not judged). The failures on main: see the job's log2022b-main-same.log.It has NOT run in the game on this PC yet.
Still to test:
@
fd7dc40dagainst this PR, each warmed a minute, then at least three interleaved three-minute runs of each (AB ABAB), expecting no
not a live recordingline, no FATAL, the picture as on main, and fps and frame times of this PR atleast main's; a difference beyond the runs' spread is a gain or a loss, one within it is no difference measured (by
hand)
warmed and interleaved, expecting no
not a live recordingline, no FATAL and the picture as on main (@Beanstalklinux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest three minutes on main and on this PR,
warmed and interleaved, the log and a screenshot)
python3 -I -B tests/test_nvn_rec_release.py --cc gcc, expecting PASS (by hand)Measurements
Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game
comparison is in Still to test. The branch (d4eb931f) had a micro-benchmark of one EndRecording
copy_locksectionwhose time grew with the copies a bucket held with list nodes and much less with flat arrays; it ran the branch's
code (entries that also held a recording number), not this port, so no number of it is quoted here.
Risks
What could break: a copy released or freed in another order or too early would show as a wrong or missing draw, or a
not a live recordingstop, in any scene; a copy kept too long only as memory. The test runs the flat code and thelists as they were through the same 60,000 random operations and finds the same releases and frees in the same order.
The flat fixture also checks:
rec_bucketsnever read withoutcopy_lock; the tickets with a flush between theirreads; marked copies whose mark is out of the guest window's reach kept by the sweeps, unread; AddControlMemory's range
check; the full bucket; the room counters; and
rec_buckets_makewithHOS_NVN_REC_SHIFTunset,0,12,24,25, an overflowing number,xand12again. An array never shrinks: a bucket that once held many copies keeps thatroom (24 bytes an entry), and the buckets' table is 1 MiB on 64-bit hosts (512 KiB of list heads before); the log line's
room shows what they hold. AddControlMemory now stops the game, with a line saying so, on control memory that is not
guest memory, where before it let go of no copy. Left as they are on main, for a separate change:
check_handlereads acopy's magic without
copy_lockand nothing keeps the copy alive from that check to CallCommands' copy or the submit'shand-off (a game re-adding the control memory of a recording it is calling at the same time); and the sweep reads the
mark of control memory the game may have freed (on Windows the guest-window fault handler commits a zeroed 64 KiB there,
the mark reads as zero and the copy is let go). This change moves that code into
copy_retireand the walks unchanged.No switch: a revert is enough. Host code: the runtime is rebuilt; nothing changes for players' saves, options or caches.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.The recordings' copies are kept by the 64 KiB block of their control memory in 65,536 buckets. Each bucket was a linked list of nodes malloc'd one a recording (EndRecording) and freed one a release, scattered through the heap: a release or a sweep walks its buckets whole, the copies that stay included, so a walk of k copies was k dependent cache misses, all of it with copy_lock held exclusively, which every recording thread waits for. A bucket is now a flat array of {ctl, copy, marked} entries, oldest first, walked newest first as the head-inserted lists were; the copies kept move back to the array's start in their order (rec_keep). So copies are released, wait dead and are freed in exactly the order they were; EndRecording mallocs nothing (copy_track). An array grows by doubling under copy_lock, as the dead list already does, and never shrinks. The three copies of "a copy is no handle any more" (release_copies, the sweep) are one function, copy_retire. No switch: the same releases and frees in the same order. Micro-benchmark on the branch (d4eb931f, i7-9700, with 32-byte entries that also held a recording number for HOS_NVN_COPY_LOCK_ONCE): one EndRecording copy_lock section with ~50,000 live copies (the game's logs: 46,000-51,000), k copies a bucket: k=1 444 -> 349 ns, k=4 708 -> 385, k=16 1379 -> 410, k=64 2423 -> 572, k=256 6073 -> 1230; with a submit's 155 dead copies waiting, k=1 650 -> 373 ns. Not measured in play. Test: tests/test_nvn_rec_release.py runs the flat code cut from the source and the linked lists as they were through the same 60,000 random operations (tracks, releases of chunks, blocks, unaligned and over-4-GiB ranges, sweeps with marks kept, overwritten and outside the window, dead copies freed as the replay moves), in three layouts (a few busy blocks, many, none); the handles leave the set and the blocks go back to the pool in the same order. tests/test_recording_log_handoff.py's stand-ins follow EndRecording's publication (copy_track instead of the node). #1812 (GummiNaomi) changed the same functions (release_copies, the sweep, EndRecording: frees outside the lock, the ticket read at retirement) but kept the list nodes; its author closed it unmerged (no change at 4 or 16 cores). Assisted-by: Claude CodeRecommendation: Request changes. Bucket growth can stall every recording thread at the commit limit, and the retirement path still has races that can free a copy before replay is done with it. The array layout may reduce lock contention, but there are no base-versus-head game measurements yet.
Keeps command recordings in contiguous arrays so the CPU can scan them without following heap pointers. The reverse walk and stable compaction preserve retirement order in the sequential fixture. The tradeoffs are growth under the shared lock, more host memory retained after busy buckets drain, and extra writes when buckets keep their entries.
@fixwill do (17)host/nvn_native.c:1622: The first AddControlMemory reads rec_buckets without the lock while EndRecording can publish the pointer under it. (more on the line)host/nvn_native.c:1623: The retirement ticket is sampled before copy_lock and combines two counters that can change independently. (more on the line)host/nvn_native.c:1600: copy_retire preserves the plain magic store that races with check_handle's lockless read. (more on the line)host/nvn_native.c:1672: The sweep still checks only the reserved guest window before reading the mark. (more on the line)host/nvn_native.c:1610: A growing bucket calls hos_xrealloc while holding copy_lock. (more on the line)host/nvn_native.c:1771: The larger first-use calloc can return NULL, and copy_track immediately indexes the result. (more on the line)host/nvn_native.c:1609: The uint32_t doubling wraps to zero when a bucket at 2^31 entries grows. (more on the line)host/nvn_native.c:1758: The existing control-memory arithmetic still adds guest-supplied values before validating them. (more on the line)host/nvn_native.c:1674: Both walks assign every retained entry back to itself until they encounter the first removal. (more on the line)tests/test_nvn_rec_release.py:152: The -O2 fixture stores copy_prefix objects through casts into a declared uint8_t array, which provides neither the required effective type nor guaranteed alignment. (more on the line)tests/test_nvn_rec_release.py:155: The retirement test would still pass if copy_retire stopped clearing magic. (more on the line)tests/test_nvn_rec_release.py:184: The fixture claims to cover marked entries outside the guest window, but every out-of-window entry gets marked = 0. (more on the line)tests/test_nvn_rec_release.py:175: Several expressions draw from the same RNG in operands with unspecified evaluation order. (more on the line)tests/test_recording_log_handoff.py:18: The suite still forces --cc gcc for the changed handoff fixture, overriding CC. A Windows checkout with the supported llvm-mingw clang and no gcc on PATH cannot run it in strict mode. Let the fixture honor CC and fall back to the configured llvm-mingw toolchain, as the new release test does.host/nvn_native.c:1635: Use the existing Tracy lock instrumentation and add bucket growth and compaction counters so the run can attribute long lock holds.host/nvn_native.c:1487: Log the sum of allocated bucket capacities in bytes. A smaller initial capacity or reclaiming empty buckets would reduce retention, with the allocation frequency checked in the same comparison.For the author (3) (a run, a measurement or a decision)
host/nvn_native.c:1635: The planned game run gives general FPS and frame times but leaves the claimed lock saving and the tail cost of bucket growth unmeasured. (more on the line)host/nvn_native.c:1487: The fixed directory grows from 512 KiB to 1 MiB on 64-bit builds, and even a one-entry bucket retains space for four entries after it empties. (more on the line)Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The commit cites 444 → 349 ns for one entry and 6,073 → 1,230 ns for 256 entries on an i7-9700, from a microbenchmark of d4eb931f with a different entry layout. There are no interleaved game runs of
969dcfa6against75317e1fmeasuring lock time, average FPS, 1% lows, median and p99 frame time, or host memory.Other systems: The host path is shared across Windows, Linux, GPU vendors, and Steam Deck. CallCommands, CopyCommands, inline replay, threaded replay, and held submits depend on these copies staying alive. The diff leaves Vulkan queue ordering, registration, capture, and resource allocation code untouched, but earlier retirement or longer lock holds can still affect rendering and pacing in those modes. Bucket storage costs host RAM. The diff changes no save, option, cache, build output, audio, input, or renderer API format.
Switches: No switch added or removed. HOS_NVN_REC_SHIFT still defaults to 16, HOS_NVN_NO_SWEEP and HOS_NVN_HOLD still default off, and the change runs in both threaded and inline replay. HOS_TEST_ALLOC_REFUSE still defaults off and can now target the recording-bucket growth allocation by its label. The switch table already describes these behaviors.
Reviewed:
969dcfa694against75317e1f31: Read AGENTS.md, CONTRIBUTING.md, the Forgejo PR template, developer standards, the commit, all three diffs, both changed tests in full, and the full recording section of host/nvn_native.c. Traced its release, submit, replay, allocation, and guest-memory dependencies and checked source-cut consumers and test discovery. Skimmed the rest of host/nvn_native.c and the CI wiring. There was no local review checklist or Tooling checkout. The named batch logs were unavailable, so their results come from the PR description. No translations or generated files changed.; nothing run (a code review).Checked and fine:
Testing: Read-only review. git diff --check passed. The PR reports 20 passed and two failures also seen on the base in the selected Windows batch. Forgejo checks were still running, and there were no tester comments or review runs.
Overlaps and merge order: The PR names closed, unmerged #1812 as prior work on the same functions. No active overlap was identified in the supplied material or local history.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -1589,2 +1619,4 @@l->n -= from;}static void release_copies(uint64_t lo, uint64_t hi) {if (hi <= lo || !rec_buckets) return;L1: The first AddControlMemory reads rec_buckets without the lock while EndRecording can publish the pointer under it. That inherited data race also lets a release return before a concurrent first insertion becomes visible. Take copy_lock before testing the pointer, and test the two first-use paths with a controlled interleaving.
@ -1589,3 +1620,4 @@}static void release_copies(uint64_t lo, uint64_t hi) {if (hi <= lo || !rec_buckets) return;const uint64_t after = rx_nvn_pushed() + held_count(); /* (held submits name them too) */L2: The retirement ticket is sampled before copy_lock and combines two counters that can change independently. With HOS_NVN_HOLD enabled, a flush can hand off a held submit after the pushed read and clear nheld before the held read. The ticket then omits an already accepted submit, so a later release can return its recording to the pool while replay still needs it. Use a coherent submission cutoff for release and sweep, including held submissions as they move to replay. An atomic nheld read alone still permits the undercount.
@ -1742,25 +1757,20 @@ NV(nvnCommandBufferEndRecording) {/* the copy is tied to the control memory position the console driver's descriptor would take */const uint64_t at = o->ctl + (o->ctl_used < o->ctl_size ? o->ctl_used : 0);L8 (minor): The existing control-memory arithmetic still adds guest-supplied values before validating them. A wrapping base plus offset can make the mark write land at an unrelated in-window address, and a wrapping AddControlMemory end makes release_copies skip the range. Validate the full range at AddControlMemory and check the remaining descriptor space before incrementing ctl_used or forming the mark address.
@ -1481,0 +1484,4 @@EndRecording mallocs nothing. An array grows by doubling (copy_lock held, as dead does) and never shrinks. No switch:the same releases and frees, in the same order (tests/test_nvn_rec_release.py) */typedef struct { uint64_t ctl; uint8_t *copy; int marked; } rec_ent;typedef struct { rec_ent *e; uint32_t n, cap; } rec_list;L17 (minor): The fixed directory grows from 512 KiB to 1 MiB on 64-bit builds, and even a one-entry bucket retains space for four entries after it empties. Ten thousand buckets touched once retain about 0.92 MiB of entry storage before allocator overhead. The current log counts recording payloads and misses this metadata. Measure peak and post-transition host RAM and bucket capacity over a longer session, including Steam Deck.
@ -1589,0 +1597,4 @@until the replay passed `after` */static void copy_retire(uint8_t *copy, uint64_t after) {copy_prefix *p = (copy_prefix *)copy - 1;p->magic = 0; /* (no submit names it any more) */L3: copy_retire preserves the plain magic store that races with check_handle's lockless read. A reader can pass the set probe, pause, and then dereference a prefix that retirement has returned to the pool. Validation must protect the copy through CallCommands' memcpy or through submit's handoff to replay. Making magic atomic by itself would leave that lifetime gap.
@ -1589,0 +1606,4 @@static void copy_track(uint64_t ctl, uint8_t *copy, int marked) {rec_list *l = &rec_buckets[rec_bucket(ctl)];if (l->n == l->cap) {l->cap = l->cap ? l->cap * 2 : 4;L7 (minor): The uint32_t doubling wraps to zero when a bucket at 2^31 entries grows. Recordings without control memory all share one bucket and cannot be retired by a range release, so the count has no smaller enforced bound. Check capacity growth and byte multiplication before reallocating, and update cap only after success.
@ -1589,0 +1607,4 @@rec_list *l = &rec_buckets[rec_bucket(ctl)];if (l->n == l->cap) {l->cap = l->cap ? l->cap * 2 : 4;l->e = (rec_ent *)hos_xrealloc(l->e, l->cap * sizeof *l->e, "native NVN: recording bucket");L5: A growing bucket calls hos_xrealloc while holding copy_lock. If the allocation fails and then recovers, threads can stall even though the recording succeeds. Allocate replacement storage outside the lock, then recheck the bucket before installing it.
@ -1612,0 +1632,4 @@uint32_t keep = l->n;for (uint32_t i = l->n; i-- > 0;) {const rec_ent *e = &l->e[i];if (e->ctl < lo || e->ctl >= hi) l->e[--keep] = *e;L16 (minor): The planned game run gives general FPS and frame times but leaves the claimed lock saving and the tail cost of bucket growth unmeasured. Compare the base and revised head in interleaved, warmed runs at matching settings in Kokiri Forest and a recording-heavy scene such as the Market. Report average FPS, 1% lows, median and p99 frame time, and lock wait and hold times for EndRecording, release, and sweep.
@ -1643,2 +1671,2 @@if (!n->marked) {pn = &n->next;void *at = e->marked ? hos_guest_range(e->ctl, sizeof mark) : NULL;if (at) memcpy(mark, at, sizeof mark);L4: The sweep still checks only the reserved guest window before reading the mark. On Windows, FreeMemoryPages can decommit a previously marked control page inside that window, so the next sweep can fault under copy_lock. Tie retirement to unmapping or use a mapped-memory read synchronized with decommit. A separate range check leaves a check-to-read race.
@ -1645,0 +1671,4 @@void *at = e->marked ? hos_guest_range(e->ctl, sizeof mark) : NULL;if (at) memcpy(mark, at, sizeof mark);if (!at || (mark[0] == COPY_MARK && mark[1] == (uint64_t)(uintptr_t)e->copy)) {l->e[--keep] = *e;L9 (minor): Both walks assign every retained entry back to itself until they encounter the first removal. For example, a sweep of a bucket with all marks intact runs an assignment for every entry while holding the exclusive lock. Decrement keep separately and copy only when keep differs from i. Apply the same guard in release_copies.
@ -1756,3 +1769,3 @@const char *e = getenv("HOS_NVN_REC_SHIFT");if (e && atoi(e) >= 12 && atoi(e) <= 24) rec_shift = (unsigned)atoi(e);rec_buckets = (rec_node **)calloc(REC_BUCKETS, sizeof *rec_buckets);rec_buckets = (rec_list *)calloc(REC_BUCKETS, sizeof *rec_buckets);L6: The larger first-use calloc can return NULL, and copy_track immediately indexes the result. An allocation failure becomes an access violation. Use the host's checked allocation path and keep its retry outside copy_lock, then publish the initialized table under the lock.
@ -0,0 +149,4 @@static uint64_t held_count(void) { return held; }#define NCOPY 40000@SHARED@static uint8_t arena[NCOPY][64]; /* (the copies: a prefix, then the handle) */L10: The -O2 fixture stores copy_prefix objects through casts into a declared uint8_t array, which provides neither the required effective type nor guaranteed alignment. copy_id also subtracts byte pointers across different row arrays. Use an actual copy_prefix array and derive IDs by subtracting typed pointers within it, so the comparison has defined behavior under GCC and Clang.
@ -0,0 +152,4 @@static uint8_t arena[NCOPY][64]; /* (the copies: a prefix, then the handle) */static int copy_id(const void *p) { return (int)(((const uint8_t *)p - &arena[0][0]) / 64); }static FILE *out;static void live_handle_remove(uint64_t h) { fprintf(out, "R %d\n", copy_id((const uint8_t *)(uintptr_t)h - sizeof(copy_prefix))); }L11 (minor): The retirement test would still pass if copy_retire stopped clearing magic. The stand-in logs the ID and never inspects prefix state. Assert that the prefix is invalidated before handle removal, and track membership so the test rejects duplicate removals or a pool return for a handle still in the set.
@ -0,0 +172,4 @@static uint64_t blocks_of(int shape) { return shape == 0 ? 4 : shape == 1 ? 200 : 4000; }/* (a copy's control memory position: one of a few blocks (hundreds a bucket), one of many, past the window, or none) */static uint64_t pick_ctl(int shape) {uint64_t at = (rnd() % blocks_of(shape)) << 16 | (rnd() % 1024) * 64;L13 (minor): Several expressions draw from the same RNG in operands with unspecified evaluation order. The reference and flat fixtures are optimized as separate programs, so a compiler may assign those draws differently and give the two variants different inputs. Sequence the draws in separate statements, including in release ranges and mark overwrites, or drive both variants from one operation trace.
@ -0,0 +181,4 @@const uint64_t ctl = pick_ctl(shape);uint8_t *copy = new_copy();int marked = 0;if (ctl < WINDOW - 16 && rnd() % 8) {L12 (minor): The fixture claims to cover marked entries outside the guest window, but every out-of-window entry gets marked = 0. Removing the guard for a failed mark lookup would escape that case. Add a directed marked entry that the range stub refuses and assert that the sweep leaves it live.
969dcfa694db31aa814eThanks for the review. The branch is rebased on main @
fd7dc40d(the PR's own commitae69cdbbunchanged,git range-diff), with a test commit9f83a9a2and a fix commitdb31aa81on top; headdb31aa81. The checks below ran in build jobfb-2022-2of this PC's job queue (Windows 11, llvm-mingw clang 23):tests/test_nvn_rec_release.pyfails at the test commit9f83a9a2on every new check (unlocked reads, the release's ticket, the full bucket, both bad ranges, norec_buckets_make) and passes at the head; the three own checks pass undertests/run_all.py --strict --exact;nvn_native.ccompiles with the game build's command; the full suite--changedpicked: 79 of its 81 failures fail on mainfd7dc40dtoo, the other two pass at the head alone (jobfb-2022-3), andtests/test_vbytes.pytimed out (Tested in the description has the detail). Nothing has run in the game yet.rec_bucketsread without the lock: fixed indb31aa81.release_copiestakescopy_lockbefore it tests the pointer; the table is made byrec_buckets_makeand every other use reads it under the lock. The flat fixture now counts every read ofrec_bucketsmade withoutcopy_lock(a release before the first recording included); it failed at the test commit and passes at the head.db31aa81. The release's and the sweep's ticket are read under the lock, the held count first, then the pushed one;held_flush(and the full-list flush in QueueSubmitCommands) setnheldto 0 with a release store only after their submits are pushed, andheld_countloads it with acquire. A flush between the two reads is then counted twice (the copy waits a few submits longer), never missed. The fixture flushes the held submits right after a read of the pushed count and checks the dead copy's ticket covers them, for a release and for a sweep. What this does not close is the gap between a submit'scheck_handleand its push (L3).check_handle: no code change here. The plain store and the lockless check are main's (the release and the sweep each had the same three lines; this moves them intocopy_retireunchanged). As you say, an atomic magic would not close the gap; that needs the copy pinned from the check through CallCommands' copy or the submit's hand-off, which is a change tocheck_handle,inline_recordingandQueueSubmitCommandsrather than to the buckets, so it is left for a separate pull request, and Risks now says it is pre-existing on main and not changed here.host/mkw_glue.c), which commits a zeroed 64 KiB there, so the mark reads as zero and the copy is let go, with the commit (and, at the commit limit, its wait) undercopy_lock; on Linux the discarded pages read as zero. A read that is safe against decommit, or a hook in FreeMemoryPages, is a separate change; Risks says so.hos_xreallocundercopy_lock: no code change. The dead list (dead_copy_add) and the handle set (live_handle_add) already grow with the host's allocation under this lock on main; its retry only waits at the commit limit, where it gives up with an out-of-memory stop after 5 s, and main's per-recording node was amallocwith no check at all. Growth is by doubling, so it is rare once a bucket has its room.calloc: fixed indb31aa81.rec_buckets_makeallocates the table withhos_xcallocbefore takingcopy_lock, publishes it under the lock (a thread that lost the race frees its own), and readsHOS_NVN_REC_SHIFTthere once withstrtol. A fixture of it checks the allocation is made without the lock, a table another thread published first is kept, and the shift for unset,0,12,24,25,99999999999999999999,xand12again, each in its own process.db31aa81. Past 2^31 entriescopy_trackstops the game with a line naming the bucket instead of doubling; the byte count is computed insize_tandcapis set after the allocation. The fixture's full bucket tried a 0-byte reallocation at the test commit and stops before any at the head.db31aa81. AddControlMemory stops the game when its range is not guest memory (hos_guest_range: past the window, or wrapping around), before it releases anything. With the range checked, EndRecording'satstays in[ctl, ctl + size]and the mark is written only whilectl_used <= ctl_size, so it lands inside the range;ctl_used += 64would need 2^58 recordings without AddControlMemory to wrap. The fixture's wrapping and past-the-window ranges went through at the test commit and stop at the head.db31aa81: both walks copy an entry only when it moves. The equivalence run checks the order is the same.9f83a9a2. The copies arecopy_prefix arena[NCOPY][3], a handle the second element of its row, and a copy's number is taken by subtracting row pointers within the array.9f83a9a2. The handle set's stand-in tracks membership and fails on a handle removed that is not in it, or removed with its prefix's magic still set; the pool's fails on a copy returned twice or while still a handle.9f83a9a2. The run now marks some copies past the window and at its end (a mark crossing it), and a directed check sweeps every bucket with two such copies and requires both kept.9f83a9a2: everyrnd()is a statement of its own, in positions, releases and mark overwrites.--cc gccin the handoff check: fixed in9f83a9a2.RUN_ALL_ARGSis gone; the check takes its compiler from the newtests/cccmd.py(--cc, thenCC, split as a shell would, thenLONGSHOT_MINGW's clang, then clang, gcc or cc), astest_nvn_rec_release.pynow does too. Run with no--ccand noCCit passes at the head (llvm-mingw fromLONGSHOT_MINGW).db31aa81) and the description: a recording no longer allocates a node; EndRecording allocates its copy (from the pool), the table at the first recording, and a bucket's room at its first entry and when it grows, as the handle set's. The commit message ofae69cdbbkeeps the old wording (the PR's own commit is not rewritten).db31aa81: the 50,000-recording log line now also gives the buckets' room, how many times they grew and how many kept copies were moved back.copy_lockwaits are already Tracy zones:nvn_native.cdefinesHT_WRAP_LOCKS, so a wait on it is a "lock wait" zone named©_lock, inside EndRecording's own zone when it is EndRecording's. The A/B (Kokiri and the Market, warmed and interleaved, average FPS, 1% lows, median and p99 frame times, lock waits) is pending: it needs the game on this PC.db31aa81), and Risks gives the table's 1 MiB. A smaller first room or reclaiming empty buckets is not done without a measurement that shows the cost; the RAM measurement (peak and after a scene change, a long session, Steam Deck) is pending.@Beanstalk This changes how NVN command recordings' copies are kept, released and swept (host/nvn_native.c), used by CallCommands and the replay in every scene; a control memory range outside the guest window now stops with an AddControlMemory line. No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Kokiri Forest and Hyrule Field, two minutes each, the PR against main on the same PC, with a picture in each compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture as main, no crash, no FATAL line and no "not a live recording" line; frame time p99 equal to main's or better, within the run-to-run spread.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.