WIP: perf(vk): a compute program's c0 layout kept by each thread between the words it reads (RX_VK_CS_LAYOUT_FAST) #2004
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2004
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-cs-layout-fast"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Every word of c0 a compute stage reads from the driver bank (the pick's answers, each texture and image handle, three or four words of each storage buffer at the descriptors) asks
cs_layout(gpu/vk/vk_bank.cpp) for the program's layout, and each ask takes a mutex and a map lookup for five numbers that never change for a serial. WithRX_VK_CS_LAYOUT_FASTon (the default) each thread keeps the last layout it was given, good while the map has not been emptied: the map's entries are written once and never changed, andcs_layout_epochmoves, under the mutex, when the map is emptied at 4,096 programs, so what a thread kept is what the map would give. The layouts are the same; the result expected is fewer lock takes in compute work. It is not measured yet; it ships on, the value this change proposes (docs/dev/standards.md, Switches), and=0gives main's code.Switch:
RX_VK_CS_LAYOUT_FAST(kind kill, on by default; its row ingpu/vk/switches.def). Unset,1or any value the shared rule does not read as off: the layout kept by each thread.0, a blank value (spaces only),false,offornoin any case (env_on_or,gpu/vk/compat/longshot_env.h): the mutex and the map at every word, main's code. The switch is read once, at its first use (soHOS_ABcannot alternate it); its first read logs what it chose.From bd3ea307 on
exp/x86-asm-sync.Read first:
cs_layout_fast,cs_layout_epochandcs_layoutingpu/vk/vk_bank.cpp.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
614cc3eeon main @9dd41326(perf8691a615, test2dc8233c, fix650f4973as before the rebase,git range-diff; test82701b5e, perf614cc3ee), in jobsdyl-2004(light, two checks at once) anddyh-2004(build) of the PC's job queue, each check in a hidden console of its own,RX_CPMset:tests/test_cs_layout_fast.py(CsLayout,cs_layout_of,cs_layout_fast,cs_layout_epochandcs_layoutcut fromvk_bank.cpp, its mutex swapped for one that counts its locks, over a fake guest memory with the switch =1, =0 and unset: the lock takes for one program again and again, two in turn, another thread, and a miss after another thread emptied the map; a layout read once; and the switch's reader in fresh processes for unset and1,onand another word (on),0,off,false,no,OFF, a blank value and" 0 "(off), twice each, and flipped after its first read, the flip checked throughgetenv),tests/test_switch_table.py,tests/test_program_stamp_knobs.py, throughtests/run_all.py --strict --exact: pass. At the test commit82701b5e(unset expected on, the code still off)test_cs_layout_fast.pyfails (RX_VK_CS_LAYOUT_FAST unset: ... cs_layout_fast() == fast).ce3a172a, jobs fb-2002-1..3, the same check code but the default): a mutant withfalse &&before the kept-layout test fails the check; with no compiler on PATH andLONGSHOT_MINGWset the check passes throughtests/run_all.py --strict --exact.gpu/vk/vk_bank.cppcompiled (syntax) with the game build's own command: no error.tests/run_all.py --changed origin/main,RX_CPMset; jobdyh-2004): 30 passed, 8 failed, 2 skipped (Linux only). Each failure fails on main @9dd41326too, the same way, run in the same job:tests/test_crash_dialog.py,tests/test_dev_ab.py,tests/test_dev_packs.py,tests/test_frame_overlap.py,tests/test_linux_pressure.py,tests/test_present_queue_default.py,tests/test_target_reuse_memory.py,tests/test_vk_occlusion_queries.py. None stopped for want of the Vulkan headers.It has NOT run in the game on this PC yet.
Test Farm: two runs HumanBean started on
614cc3eeagainst main @9dd41326(#issuecomment-67418: RTX 4070 on Windows and RX 580 on Linux; #issuecomment-67622: RX 580 and RX 9070 on Linux): Kokiri Forest, 60 s after a 10 s warm-up, one game start a side, this PR withRX_VK_CS_LAYOUT_FAST=0and with=1.=0included: 0.09 to 0.81 in the first run and 0.84 to 0.96 in the second, against the 0.02 limit. The pictures were taken after the stick had moved and no main-against-itself picture was taken, so the runs do not show that the difference comes from this change (with=0it runs main's lookup), nor that it is the scene's own.=047.0 ms (one RX 580 start at 75.5 against its main's 36.6),=124.8 ms; second run main 39.8 and 36.4 ms,=034.9 ms,=129.8 ms. With one start a side and main 9 to 12 % from itself between cells, neither a gain nor a loss is shown.Still to test:
python -I scripts/dev.py ab --scene hyrule-field --runs 3and the same with--scene market(main @9dd41326against this PR withRX_VK_CS_LAYOUT_FAST=0: expecting no difference, both main's code), thenpython -I scripts/dev.py ab --env RX_VK_CS_LAYOUT_FAST=0 --scene hyrule-field --runs 3and with--scene market(this PR, off against on, one runtime), plus the scene with the most compute dispatches a frame (which one is to be found in a profiled run,HOS_PROFILE=1, not known yet); the same pinned options file, save and settings, each warmed up, runs interleaved; report per run and as medians the spread, average FPS, 1% lows, median and p99 frame times and the replay's CPU time a frame (vk: a frame's replay time (ms), its dispatch part); expecting the same picture every way, and a gain only where the PR unset beats=0beyond the spread. If it does not, the change is dropped (docs/dev/standards.md: no measured gain, no merge). The renderer has no count of the lookups that skip the lock (the check counts them in its fixture only) (by hand)9dd41326install to this PR, then a second (warm) start, the cache starting the same way for each run; report the time to play of both starts, thevk: pipeline cache: N files made for another translator droppedline and the preparation's lines, and first-use stutter in the first minute of Kokiri Forest; expecting the same cost as any update that changes the translator's files (by hand)RX_VK_CS_LAYOUT_FAST=0, warmed, in ABBA order, expecting the same picture every way and no crash or validation error (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+nvidia, windows+intel, steamdeck: the scene with the most compute dispatches once known, else Kokiri Forest, a minute, main against this PR unset and with RX_VK_CS_LAYOUT_FAST=0, ABBA, the same picture)tests/test_cs_layout_fast.pywith--cxx g++and--cxx clang++, expecting PASS (by hand)Measurements
Verdict: not measured in play yet: the Test Farm's two runs (Testing) show neither a gain nor a loss, and the in-game A/B on this PC (Still to test) is pending: it waits for its two builds, and its result will be added here.
RX_VK_CS_LAYOUT_FASTships on (cs_layout_fastreads it withenv_on_or("RX_VK_CS_LAYOUT_FAST", 1)ingpu/vk/vk_bank.cpp; kind kill, its row ingpu/vk/switches.def;0gives main's code), the value this change proposes (docs/dev/standards.md, Switches); if the A/B shows no gain beyond the spread, the change is dropped.Risks
A layout kept past the map's emptying would read a compute program's bank words at the wrong places (wrong compute results); the epoch is moved under the mutex and the check empties the map from another thread and counts the lock takes after it. On by default;
RX_VK_CS_LAYOUT_FAST=0runs main's code.gpu/vk/vk_bank.cppis in the translator's stamp (nx2/translator_files.py), and any change to its code moves the stamp, whatever the switch says. At the first start after updating,pcache_drop_staledeletes the Vulkan pipeline cache files made for the old stamp in the game folder'scache/(the main file and the play files; the log saysvk: pipeline cache: N files made for another translator dropped), the program cache (longshot_programs.bin) and the prepare record start over, every program is translated again (the translations themselves are unchanged) and the preparation makes the driver's stage libraries again. This change keeps none of those caches; it costs a player what any update that changes the translator's files costs (Still to test measures it). The renderer is compiled again after updating.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). Not yet: only the checks in Testing ran.main, or Measurements says why there is nothing to measure. Not yet (Measurements).third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def. Its row is in; it ships on, as the change proposes (docs/dev/standards.md, Switches), not measured yet.tests/that reads changed source is updated, not weakened.vk_bank.cppcompiled (syntax) on Windows only (Testing).Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Recommendation: Request changes. This can remove a mutex and map lookup from repeated compute-bank reads, but the fixture passes even if every read still takes the mutex. The update also discards compatible pipeline caches and can force a full shader preparation before players see any proven frame-time gain.
The renderer keeps the last compute c0 layout on each thread. The epoch and by-value copy are sound on inspection. The change adds a small amount of thread-local state and extra work on a miss, and changes the shader cache stamp for everyone who updates.
@fixwill do (8)tests/test_cs_layout_fast.py:85: The read counter stays flat on both a thread-local hit and a locked map hit. (more on the line)gpu/vk/vk_bank.cpp:94: Keep the existing stamp for this cache-policy-only change through a narrowly scoped compatibility change, or land it with a translator change that already requires the same rebuild.gpu/vk/vk_bank.cpp:103: Count lookups and lock entries in a profiling build. If frequent misses erase the saving, pass one resolved layout through the repeated bank reads for a dispatch or program query.tests/test_cs_layout_fast.py:23: The runner resolves{cxx}from CXX or PATH before the fixture can use its LONGSHOT_MINGW fallback. (more on the line)gpu/vk/switches.def:313: The row and PR say any other set value enables the switch. (more on the line)tests/test_cs_layout_fast.py:123: The fixture ignores whether_putenv_sorsetenvactually changed the value. (more on the line)75317e1f. It also omits the applicable source-reading-check box and the template's Ready to merge section. Restore those boxes and update the current-main claim when the head is retested.For the author (3) (a run, a measurement or a decision)
gpu/vk/vk_bank.cpp:94: A player updating an already prepared install can lose the existing Vulkan pipeline files as well as the translated programs. (more on the line)gpu/vk/vk_bank.cpp:103: The planned forest walk only calls for replay time and can miss frame-pacing costs or workloads that alternate compute programs. (more on the line)Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),PR description / Description follows the template (pull_request_target),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The author reports the Windows fixture passed and the changed checks returned 30 passes and 8 failures also seen on base
75317e1f. There are no in-game runs, lock-count or replay-time measurements, or frame-time comparisons for head46b04b8b.Other systems: The shared CPU path serves compute translation and program picks, descriptor reads, bulk bank fills, and upscale slot selection on Windows and Linux across GPU vendors, including Steam Deck. Seed translation still bypasses this cache, and graphics banks and c2 take their existing paths. The stamp also reaches graphics and compute program records, preparation records, and Vulkan pipeline caches. The diff adds no GPU allocations or direct save, audio, or input changes.
Switches: RX_VK_CS_LAYOUT_FAST is a developer kill switch, on when unset and read and logged once at first non-seed use. Empty, 0, false, off, and no disable it under the shared parser. HOS_AB cannot alternate it. The row is present, but its set-value wording is inaccurate and the on default has no hardware measurements yet.
Reviewed:
46b04b8b74against75317e1f31: Read AGENTS.md, CONTRIBUTING.md, the Forgejo template, developer standards, vk_bank.cpp and test_cs_layout_fast.py in full. Read the full diff and commit, the changed switch row and neighboring rows, and the relevant renderer, host, cache, and test-runner code. Skimmed the switch table outside the changed section and the reported runs. docs/review-checklist.md, the sibling Tooling checkout, and the named run logs were absent. The diff contains no generated files or translations.; nothing run (a code review).Checked and fine:
Testing: Read-only review. No builds, tests, or project scripts were run. The Windows results are author-reported. The supplied CI snapshot has checks pending and lists no head-only failures.
Overlaps and merge order: No overlapping open PR was visible in the supplied evidence. Two commits are already on local main since the merge base. The shared switch table changed at an unrelated row.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -310,6 +310,7 @@ SW(RX_VK_COMPUTE_SERIAL, hw, bool, "on with Intel's Windows driver, off elsewherSW(RX_VK_COMPUTE_WARM_APART, hw, bool, "on with Mesa's Intel driver, off elsewhere", "1 or 0 says on any GPU whether the preparation compiles a compute program's variants as items of their own, after the translation.")SW(RX_VK_COND_ALWAYS, diag, bool, "off: render conditions are evaluated", "=1 (test) makes every conditional-render condition hold, so everything conditional is drawn.")SW(RX_VK_CPU_OVER_GPU, attempt, bool, "off: CPU writes are taken over GPU contents only for textures the GPU merely cleared", "Takes whole pages the CPU wrote after a GPU write over any GPU-written texture, not only cleared ones.")SW(RX_VK_CS_LAYOUT_FAST, kill, bool, "on: a compute program's c0 layout is kept by each thread between the words it reads", "=0 (or false, off, no) looks the layout up under its lock at every word, as before; any other value is on. Read once, at its first use (HOS_AB cannot alternate it).")L7 (minor): The row and PR say any other set value enables the switch. Inherited
RX_VK_CS_LAYOUT_FAST=and a whitespace-only value disable it throughenv_on_or. Describe the shared boolean rule in both places and add a whitespace-only switch case, which also avoids any platform-specific treatment of empty environment values at process startup.@ -78,0 +91,4 @@}();return on;}std::atomic<uint64_t> cs_layout_epoch{1}; // (moved, under cs_layout's mutex, when its map is emptied)L2: A player updating an already prepared install can lose the existing Vulkan pipeline files as well as the translated programs.
pcache_drop_stale()deletes the old files and preparation can rebuild the driver's stage libraries. Measure an in-place update from a prepared base and the following warm start, with the same cache starting state for each run. Record time to play, the preparation and cache-drop log lines, and first-use stutter on a slow compiler such as RADV.@ -80,0 +100,4 @@};static thread_local Last last{0, 0, {}};const bool fast = cs_layout_fast();if (fast && last.epoch == cs_layout_epoch.load(std::memory_order_acquire) && last.serial == s.prog.serial) return last.l;L4 (minor): The planned forest walk only calls for replay time and can miss frame-pacing costs or workloads that alternate compute programs. Compare the merge base, head off, and head on with repeatable inputs and interleaved runs at identical settings. Include a compute-heavy scene and a representative heavy gameplay scene. Report the spread, average FPS, 1% lows, median and p99 frame times, replay CPU time, and how many lookups bypass the lock. Use that evidence to choose the default.
@ -0,0 +20,4 @@sys.path.insert(0, str(Path(__file__).resolve().parent)) # (tests/srccut.py: python -I leaves this folder off the path)from srccut import block, between # noqa: E402RUN_ALL_ARGS = ["--cxx", "{cxx}"]L6 (minor): The runner resolves
{cxx}from CXX or PATH before the fixture can use its LONGSHOT_MINGW fallback. A Windows checkout with bundled llvm-mingw but no C++ driver on PATH gets a missing-tool failure under--stricteven though the fixture can use the installed compiler. Use one discovery path for direct and suite runs that honors--cxx, CXX, and bundled llvm-mingw.@ -0,0 +82,4 @@CHECK(la.ubo == 1 && la.ssbo == 2 && la.tex == 3 && la.img == 4 && la.push == 32);CHECK(lb.ubo == 6 && lb.ssbo == 7 && lb.tex == 8 && lb.img == 9 && lb.push == 48);}const int read_before = reads.load();L1: The read counter stays flat on both a thread-local hit and a locked map hit. For example, adding
false &&to the fast-hit condition leaves the runtime checks and the source-text checks at line 179 passing. Assert lock or map entries through a fixture hook and check repeated same-serial calls with the switch on and off. Use the same hook to check that a miss after eviction repopulates the fast path, and replace the source-text assertions with those behavior checks.@ -0,0 +120,4 @@if (argc < 2) return 2;const bool first = t::@FN@();#ifdef _WIN32_putenv_s("@NAME@", argv[1]);L8 (minor): The fixture ignores whether
_putenv_sorsetenvactually changed the value. If the update fails, a reader that rereads the environment but logs only once could pass the read-once check. Fail on a nonzero return and verify the replacement value is visible throughgetenvbefore calling the reader again.@fix
I didn't fix this pull request: it failed (the Codex runner answered 409: busy). Nothing was pushed.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
@fix
46b04b8b74ce3a172a21Thanks for the review. The branch is rebased on main @
40bfceb0(the PR's own commit unchanged,git range-diff), with a test commit and a fix commit on top; headce3a172a. The checks below ran in build jobsfb-2002-1(onf1f2dc5b, with the mutants and the old heads) andfb-2002-2(on85a59e6f) andfb-2002-3(on40bfceb0, website files only since) of this PC's job queue (Windows 11, llvm-mingw clang 23). Nothing has run in the game yet. These commits answer the review that HumanBean's "@fix" asked for (BeanBot's review of46b04b8b, every point below); the branch's head is nowce3a172a, so a next "@fix" starts from it.5a928283. In the cut,cs_layout'sstd::mutexis swapped for one that counts its locks (the only edit to the cut code), and the source-text checks are gone. The fixture runs with the switch=1,=0and unset and checks: the same program 100 times takes no lock when on and 100 when off; two programs in turn take one a call either way; another thread's first call takes one and its next ones none when on; after another thread emptied the map and the program's control section changed, the next call takes the lock once and reads the new layout, and the 100 after it take none when on. A copy of the head withfalse &&before the kept-layout test now fails (100 lock takes where none were expected, =1); at the test commit the check fails only on the new default (L4).nx2/translator_files.pystampsvk_bank.cppby its tokens, so any change to its code moves the stamp, and every place this lock could be skipped is in that file; a narrower exception would need the stamp to leave part of a stamped file out, which can hide a real translator change. So the change keeps the stamp move, and Risks now says what it costs; whether it lands beside another change that moves the stamp anyway is the maintainer's choice of merge order. The update measurement you describe (an in-place update from a prepared install, then a warm start, time to play, the cache-drop and preparation lines, first-use stutter, RADV included) is in Still to test, pending for the author.pcache_drop_staledeletes the pipeline cache files made for the old stamp (the main file and the play files, with its log line), that the program cache and the prepare record start over, that every program is translated again and the stage libraries are made again, whatever the switch says, and that this change keeps none of those caches.RX_VK_CS_LAYOUT_FASTnow ships off (ce3a172a, kind attempt): unset is main's code. Still to test asks for main against this PR unset (expected the same) andscripts/dev.py ab --env RX_VK_CS_LAYOUT_FAST=1(on against off, one runtime) in Hyrule Field, the Market and the scene a profiled run shows with the most compute work, with the spread, average FPS, 1% lows, median and p99 frame times and the replay's CPU time; pending for the author, and the default is chosen from those runs. Passing one resolved layout through a dispatch's bank reads is the other route if the kept layout misses often; it is a larger change and waits for those numbers.{cxx}: fixed in5a928283, as in #2002: noRUN_ALL_ARGS; the check finds its compiler itself, one way for a direct run and the suite's:--cxx,CXX,LONGSHOT_MINGW's clang++ (toolchain.py, linked static), then PATH. Run throughtests/run_all.py --strict --exactwith no compiler on PATH,CXXunset andLONGSHOT_MINGWset, the check failed at the PR's old head46b04b8b(not found: cxx (--strict)) and passes at the new head.ce3a172a) and the description now give the shared rule: unset, a blank value (spaces only) and0,false,offornoin any case are off, any other value on. The switch's cases (5a928283) addno,OFF," "and" 0 "(off), each in a fresh process, twice. All pass at the head._putenv_s/setenvunchecked: fixed in5a928283. The child fails (exit 3, saying why) when the call returns nonzero orgetenvdoes not give the new value, before the reader is called again.f1f2dc5bnow, the description's Tested names that base, and the source-reading checks box (ticked) and the Ready to merge section are back.@Beanstalk This adds RX_VK_CS_LAYOUT_FAST, off by default: with RX_VK_CS_LAYOUT_FAST=1 each thread keeps the last compute program layout it was given instead of taking a lock for every driver bank read (gpu/vk/vk_bank.cpp). Whatever the switch says, the first start after updating translates the compute programs once again and drops the old pipeline cache (the log's "files made for another translator dropped"). It has not run in the game on these PCs yet. Please play Kokiri Forest for three minutes, the PR against main on the same PC, on the PR's side with RX_VK_CS_LAYOUT_FAST=1 and with it unset, with a picture compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture, no crash and no validation error; set, frame time p99 equal to unset or better; unset, frame times as main's apart from the one retranslation.
@siantm only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was force3a172, and the PR has changed since (614cc3e), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Description brought up to the head (no push; the head is still
ce3a172aon main @40bfceb0). The description shown since 06:21 still named the old head46b04b8bon75317e1f, its Tested section, and the switch as on by default. It now gives what ran atce3a172ain fb-2002-1/2/3 (as in the reply above), the switch off by default asce3a172aships it, and the Risks and Still to test from that round. Three of the 8--changedfailures,tests/test_present_queue_default.py,tests/test_target_reuse_memory.pyandtests/test_vk_occlusion_queries.py, had not compiled in those runs, at the head or on main: this PC's runs had no Vulkan headers, and under--strictthey stopped atnot found: vulkan_headers. Run again with the headers (RX_CPMset,--strict, 20 min per check) in build jobfb-vkfix-1atce3a172aand on main @40bfceb0: each fails on main too, with the headers found. None fails only at this head. Main has moved past40bfceb0(b62932a9, renderer changes among them), so "on current main" stays unticked until the branch is updated and retested.Verdict: Request changes. The code is sound: the per-thread layout is invalidated correctly and the new check catches the mutants that matter. But
ce3a172aturnsRX_VK_CS_LAYOUT_FASToff because it is unmeasured, which docs/dev/standards.md rules out ("Never off just because it is new or unmeasured"). Merged like that, it moves the translator stamp for every player and gains nothing for anyone.For a player: with the switch on, the compute work's driver-bank reads (every c0 word a compute stage reads) stop taking a mutex and a map lookup for a layout that never changes for a program. That means less CPU time and lock contention in compute-heavy frames, with the same picture. That is the expected gain; nobody has measured it yet. The cost, whatever the switch says, is one retranslation and a pipeline-cache drop on the first start after updating, because
vk_bank.cppis in the translator's stamp. Risks now says so.Reviewed:
ce3a172aagainst40bfceb0(the merge base with main; it also merges cleanly with main at9dd41326). Read in full:gpu/vk/vk_bank.cpp(theCsLayoutblock throughcs_layout, and the file's top),gpu/vk/switches.def's row, andtests/test_cs_layout_fast.py. Also readenv_on_or(gpu/vk/compat/longshot_env.h),nvn_prog.serial(gpu/render/rx_nvn.h),translator_stamp's environment prefixes (vk_shader.cpp),nx2/translator_files.pyandtests/test_program_stamp_knobs.py. Compared the fix commits against BeanBot's review of46b04b8b. Nothing run (a code review).Fixed in this review: nothing.
Checked and fine:
cs_layout_epochmoves there, under the mutex, after the clear. Every store intolasthappens under that same mutex, with the epoch that is current at that moment, so the relaxed loads there are right. A fast-path hit that races an eviction is ordered before the bump, and it returns what the map held for that serial. The{0, 0}start can never match, because the epoch starts at 1. 64 bits will not wrap.rx_nvn.h), so a serial whose control section changes would be just as wrong in main's map. The kept layout adds no staleness the map did not already have.seed_cs_controlstill comes first and never toucheslast.lastis a function-localthread_localof a trivially constant-initialized type, so it has no dynamic TLS initializer (the ThinLTO TLS rule). With the switch off, the fast path is a short-circuited branch andlastis never written: main's code.RX_VK_CS_LAYOUT_FASTmatches none oftranslator_stamp's environment prefixes. Setting it keeps the program cache, so one runtime can A/B it.false &&on the fast path; no epoch compare; no epoch bump; no store intolaston a map hit (the other thread's case). The eviction case rewrites the control section, so a stale layout shows up as wrong numbers, not only as a lock count.toolchain.mingw/find/native_*exist), L7 (the row and the description give the shared rule; blank and" 0 "are tested) and L8 (_putenv_s/setenvchecked throughgetenv) are resolved in the code. L2's answer, to keep the stamp move rather than carve a stamped file into parts, is the right call, and Risks states the cost.Assisted-bytrailers. No game data, no generated files,third_party/untouched.Blocking (to change before merge)
gpu/vk/vk_bank.cpp:85:env_on_or("RX_VK_CS_LAYOUT_FAST", 0)(and theattemptrow,gpu/vk/switches.def:313). ce3a172a's message gives the reason: "not measured in the game yet, so it ships off". docs/dev/standards.md (Switches) says a switch "ships at that value from the branch's first push, WIP or not ... Never off just because it is new or unmeasured". BeanBot's L4 asked for the evidence to choose the default. It did not ask for the switch to be turned off.attemptswitch is by definition "an experiment nobody ships".env_on_or(..., 1), kindkill, and the row's unset text saying the layout is kept by each thread. Then measure =1 against =0 as Still to test plans. If there is no gain beyond the spread, drop the change ("No measured gain, no merge"), rather than ship it off.Suggestions (the author may take or leave)
gpu/vk/vk_bank.cpp:132: two plain statements in braces read more easily than the comma operator. As it is, the epoch bump is easy to miss on that line.memory_order_acquireload is stronger than needed, becauselastis thread-local and is only written under the mutex. It is free on x86, and a comment saying why it is safe (the hit returns the map's own value even if it races the bump) would help the next reader more than the ordering does.Questions
Testing: The author's Windows 11 / i7-9700 / RTX 4060 runs are of the check (it passes at the head, and fails with the mutant) and of the
--changedchecks (8 failures, each also failing on main). Nothing has run in the game anywhere, so the gain is unmeasured. CI is pending, waiting on the farm approval. Still missing: the author's in-game A/B (=1 against =0 in a compute-heavy scene, with spread, 1% lows, median and p99 frame times and replay CPU time), the update-in-place cost on a slow pipeline compiler (RADV), and the Linux renderer build and check with GCC and Clang. The @Beanstalk request should run on the head after the default is put back.Overlaps and merge order: none in code.
gpu/vk/switches.defis touched by most open PRs; a textual conflict there is expected and trivial. Sincevk_bank.cppis stamped, landing this in the same release as another translator change makes its retranslation cost free.@ -310,6 +310,7 @@ SW(RX_VK_COMPUTE_SERIAL, hw, bool, "on with Intel's Windows driver, off elsewherSW(RX_VK_COMPUTE_WARM_APART, hw, bool, "on with Mesa's Intel driver, off elsewhere", "1 or 0 says on any GPU whether the preparation compiles a compute program's variants as items of their own, after the translation.")SW(RX_VK_COND_ALWAYS, diag, bool, "off: render conditions are evaluated", "=1 (test) makes every conditional-render condition hold, so everything conditional is drawn.")SW(RX_VK_CPU_OVER_GPU, attempt, bool, "off: CPU writes are taken over GPU contents only for textures the GPU merely cleared", "Takes whole pages the CPU wrote after a GPU write over any GPU-written texture, not only cleared ones.")SW(RX_VK_CS_LAYOUT_FAST, attempt, bool, "off: a compute program's c0 layout is looked up under its lock at every word it reads", "=1 keeps it by each thread between the words it reads. Off when unset, blank (spaces only) or 0, false, off or no in any case; any other value is on. Read once, at its first use (HOS_AB cannot alternate it). Not measured in play yet.")With the default back on, this row becomes kill, and its unset text says the layout is kept by each thread (=0: the lock at every word).
@ -78,0 +82,4 @@// map's entries are written once and never changed, so what a thread kept is what the map would give)bool cs_layout_fast() {static const bool on = [] {const bool v = env_on_or("RX_VK_CS_LAYOUT_FAST", 0);Blocking: this ships off because it is unmeasured, which docs/dev/standards.md (Switches) rules out: 'Never off just because it is new or unmeasured'. Merged off, every player pays the stamp move and nobody runs the kept layout. Put back env_on_or(..., 1), kind kill, then A/B =1 against =0. If there is no gain beyond the spread, drop the change.
@ -104,3 +130,3 @@break;}if (by_serial.size() > 4096) by_serial.clear();if (by_serial.size() > 4096) by_serial.clear(), cs_layout_epoch.fetch_add(1, std::memory_order_acq_rel);Nit: two statements in braces would read more easily than the comma operator. The epoch bump is easy to miss here.
ce3a172a21614cc3ee06@Beanstalk This makes compute driver-bank reads keep the c0 layout per thread instead of taking a lock (gpu/vk/vk_bank.cpp), on by default; RX_VK_CS_LAYOUT_FAST=0 turns it off. Either way, the first start after updating translates the compute programs once again and drops the old pipeline cache (expected). This head is on main
9dd41326: please compare with main at that merge base. Please play Kokiri Forest for three minutes, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_CS_LAYOUT_FAST=0, with a picture compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture, no crash or validation error; unset, frame time p99 equal to main's or better and to the switch off, apart from the one retranslation.@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Thank you for the review (#issuecomment-65664). Pushed
614cc3ee, rebased on main @9dd41326(the branch's earlier commits unchanged by the rebase,git range-diff): test commit82701b5e, then614cc3ee. The checks named below ran in jobdyl-2004of this PC's queue (Windows 11, llvm-mingw clang 23):test_cs_layout_fast.py,test_switch_table.pyandtest_program_stamp_knobs.pypass at the head, andtest_cs_layout_fast.pyfails at the test commit (unset expected on);vk_bank.cppcompiles with the game build's command. The--changedchecks and their failures on main are in jobdyh-2004, queued, and the description will give them. Nothing has run in the game.vk_bank.cpp:85, and #issuecomment-65662,switches.def:313): you are right.RX_VK_CS_LAYOUT_FASTis on by default again, restored per docs/dev/standards.md (Switches), in614cc3ee:env_on_or(..., 1), kindkill, the row's unset text says the layout is kept by each thread and=0(or blank,false,off,no) takes the lock at every word, as before; the log line matches.test_cs_layout_fast.pyruns unset as on, in the lock counts and in the reader's cases (82701b5e). Still to test plans=0against unset; if there is no gain beyond the spread, the change is dropped rather than shipped off, and the description says so.vk_bank.cpp:133): taken in614cc3ee: the map's emptying and the epoch move are two statements in braces, the move with its own comment.614cc3ee):lastis the thread's own and written only under the mutex, the epoch moves only under it, and a hit that races another thread's emptying returns what the map held for that serial, which never changes, so the acquire is not what makes it safe.Please test in game and make the default flag value is good for everyone instead of off!
@Beanstalk
Beanstalk test farm · an earlier run: commit
614cc3e(the newer results are in a comment below)Commit
614cc3e· approved by HumanBean (reviewed) · dashboard⚠️ Note: The explicit 1 cell uses the same fast path as the unset default; the 0 cell provides the meaningful off comparison. Main is run for both cells without the PR-only switch.
⚠️ Note: One run per cell cannot establish a gain beyond run-to-run spread; the most compute-heavy scene and update-in-place cache costs are not measured.
Done (started 15:59 UTC, last report 16:21 UTC). Comment
@Beanstalkto run it again.anykokiri-walk· frame time p99 (ms)RX_VK_CS_LAYOUT_FAST=0(reference)RX_VK_CS_LAYOUT_FAST=1anyWhy it failed
any:kokiri-walk · RX_VK_CS_LAYOUT_FAST=0 (PR only)screenshot.max_diff:limit 0.0any:kokiri-walk · RX_VK_CS_LAYOUT_FAST=1 (PR only)screenshot.max_diff:limit 0.0What was tested
What will be tested · AI plan (
gpt-6.1-terra)Compare the default-on compute layout cache with the off setting and main during a short walk in Kokiri Forest. The two switch values require separate launches; the run takes about 23 minutes per tester. Frame-time regressions, changed pictures, crashes or Vulkan validation errors fail the plan.
0or1(PR only).kokiri-start, then teleport to Kokiri Forest, move the stick, take the picture forest to compare with main, measure forest for 60 s (after 10 s warm-up); 1 run on each side for each of 2 cells (2 game starts a run).Measurements on `any`, for the PR description
Verdict: fail on any (2 reports):
kokiri-walk screenshot.max_difffailed.Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 4070, 616.56.0AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM; 2: linux+amd+radv+dgpuAMD Radeon RX 580 Series (RADV POLARIS10), 26.2.7AMD Ryzen 5 5500F 6-Core Processor, 32 GB RAMScene: kokiri-walk, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); kokiri-walk: 1 game start a side, forest 60 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)9dd4132614cc3eRX_VK_CS_LAYOUT_FAST=0 (PR only)· forest · frame time (p99) (ms)RX_VK_CS_LAYOUT_FAST=0 (PR only)· forbidden log linesRX_VK_CS_LAYOUT_FAST=1 (PR only)· forest · frame time (p99) (ms)RX_VK_CS_LAYOUT_FAST=1 (PR only)· forbidden log linesFlags and settings
RX_VK_CS_LAYOUT_FAST0,1As Markdown for the PR description:
The testers' hardware
anyNVIDIA GeForce RTX 4070, 616.56.0AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAManyAMD Radeon RX 580 Series (RADV POLARIS10), 26.2.7AMD Ryzen 5 5500F 6-Core Processor, 32 GB RAM@Beanstalk
Beanstalk test farm · ❌ failed on any
Commit
614cc3e· approved by HumanBean (reviewed) · dashboard⚠️ Note: The explicit 1 cell uses the same fast path as the unset default; the 0 cell provides the meaningful off comparison. Main is run for both cells without the PR-only switch.
⚠️ Note: One run per cell cannot establish a gain beyond run-to-run spread; the most compute-heavy scene and update-in-place cache costs are not measured.
Done (started 2026-11-21 16:58 UTC, last report 2026-11-21 17:36 UTC). Run 2: started again by HumanBean. Comment
@Beanstalkto run it again.anykokiri-walk· frame time p99 (ms)RX_VK_CS_LAYOUT_FAST=0(reference)RX_VK_CS_LAYOUT_FAST=1anyWhy it failed
any:kokiri-walk · RX_VK_CS_LAYOUT_FAST=0 (PR only)screenshot.max_diff:limit 0.0any:kokiri-walk · RX_VK_CS_LAYOUT_FAST=1 (PR only)screenshot.max_diff:limit 0.0What was tested
What will be tested · AI plan (
gpt-6.1-terra)Compare the default-on compute layout cache with the off setting and main during a short walk in Kokiri Forest. The two switch values require separate launches; the run takes about 23 minutes per tester. Frame-time regressions, changed pictures, crashes or Vulkan validation errors fail the plan.
0or1(PR only).kokiri-start, then teleport to Kokiri Forest, move the stick, take the picture forest to compare with main, measure forest for 60 s (after 10 s warm-up); 1 run on each side for each of 2 cells (2 game starts a run).Measurements on `any`, for the PR description
Verdict: fail on any (2 reports):
kokiri-walk screenshot.max_difffailed.Hardware: 1: linux+amd+radv+dgpu
AMD Radeon RX 580 Series (RADV POLARIS10), 26.2.7AMD Ryzen 5 5500F 6-Core Processor, 32 GB RAM; 2: linux+amd+radv+dgpuAMD Radeon RX 9070 (RADV GFX1201), 26.0.8AMD Ryzen 7 5800X 8-Core Processor, 32 GB RAMScene: kokiri-walk, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); kokiri-walk: 1 game start a side, forest 60 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)9dd4132614cc3eRX_VK_CS_LAYOUT_FAST=0 (PR only)· forest · frame time (p99) (ms)RX_VK_CS_LAYOUT_FAST=0 (PR only)· forbidden log linesRX_VK_CS_LAYOUT_FAST=1 (PR only)· forest · frame time (p99) (ms)RX_VK_CS_LAYOUT_FAST=1 (PR only)· forbidden log linesFlags and settings
RX_VK_CS_LAYOUT_FAST0,1As Markdown for the PR description:
The testers' hardware
anyAMD Radeon RX 580 Series (RADV POLARIS10), 26.2.7AMD Ryzen 5 5500F 6-Core Processor, 32 GB RAManyAMD Radeon RX 9070 (RADV GFX1201), 26.0.8AMD Ryzen 7 5800X 8-Core Processor, 32 GB RAM@HumanBean thank you. The default is on, not off:
cs_layout_fastreads the switch withenv_on_or("RX_VK_CS_LAYOUT_FAST", 1)(gpu/vk/vk_bank.cpp, line 86 at614cc3ee), and its row ingpu/vk/switches.defis kind kill, on when unset; only=0(or a blank value,false,off,no) gives main's code. It has been on since614cc3ee; the head before it,ce3a172a, had it off, and the farm request of 06:12 still said so.Your two farm runs (#issuecomment-67418, #issuecomment-67622) failed on the picture alone, in both cells,
=0as well (0.09 to 0.96 against the 0.02 limit). The pictures were taken after stick input and with no main-against-itself picture beside them, so they show neither that this change makes the difference nor that the scene does. The frame times had one start a side, with main 9 to 12 % from itself between cells, so they show neither a gain nor a loss (=1against main: 24.8 against 24.6 ms, then 29.8 against 36.4 ms); no crash and no forbidden log line. A rerun with the pictures at an input-free point, a main-against-itself picture and more starts a side will be asked for.The in-game A/B on this PC (unset against
=0, four interleaved runs a side in Kokiri Forest, then main against this PR in Hyrule Field) is pending: it waits for its two builds. Its result goes into Measurements; it decides whether on is the right default for everyone, and the change is dropped if it shows no gain. The description's Testing and Measurements now say this. No push.@Beanstalk This replaces my request 65984 on this head, as HumanBean asked (67417): runs 67418 and 67622 took the picture after the stick moved, so the camera differed between sides, and had no main-against-itself picture. This makes compute driver-bank reads keep the c0 layout per thread instead of taking a lock (gpu/vk/vk_bank.cpp), on by default; RX_VK_CS_LAYOUT_FAST=0 turns it off. Please start from the kokiri-start save and take the picture right after it loads and settles, BEFORE any stick input, with a main-against-itself picture on the same PC; then, for frame times only, walk Kokiri Forest for 60 s after a 10 s warm-up. The PR against main on the same PC, on the PR's side unset and with RX_VK_CS_LAYOUT_FAST=0, four interleaved starts a side, and please report main's own p99 spread between its starts. On Linux with AMD (RADV), Linux with NVIDIA, Windows with Intel and the Steam Deck. The same picture as main within main's own difference from itself, no crash or validation error; frame time p99 unset equal to =0 and to main, or better, beyond main's own spread before it counts as a gain.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.@Beanstalk One correction to my request 69639 above, for its picture point: please take each picture 5 s (300 frames at 60 fps) after the first frame whose picture is not a single uniform colour, with no input in between, and take the main-against-itself picture from the same point. A picture taken during the load's fade-in from black is not comparable: run 70269 on #2156 caught the fade in 3 of 4 starts.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.