fix(vk): eleven changes from the renderer rewrite research, one commit per issue #1542
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1542
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:research/rewrite-fixes-1"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: high · Tested on: Linux, RTX 4090 (NVIDIA 615.78.08), windowless, including a paired timing on a quiet PC; the first six commits also independently on Windows 11, RTX 4070 (a tester's run, quoted in Testing); cross-built for Windows (llvm-mingw); game-free checks · Needs: a Windows run of the five newest commits; AMD and Intel; a Steam Deck's 8-processor case
TL;DR
Eleven changes from the research behind #1493, one commit per issue, in order of risk. Three make the port answer the console's graphics interface the way the console does, one stops every player re-preparing all shaders when a struct changes, three add counters, and four cut waiting: a cheaper memory fence in the translated game, a Linux lock, the virtual cores' gate and wake calls nobody sleeps on. Each has a game-free test that fails on main and passes here, the head played 11 and 17 minutes on Linux with no crash, and two paired timings on a quiet PC (against the PR's base and against today's main) show no difference beyond the spread.
Breakdown
HOS_OS_SIGNAL_ALWAYS,HOS_VCORE_LOCKED,RX_VK_LOG_VIEW_FORMATS) and one default that follows the machine (HOS_VCORE_STARVE_US: 20 ms with 8 or fewer logical processors, 1 s otherwise); no setting, no interface version change.Why
The research behind #1493 compared the port with the console and with its own history and found three places where the port answers the game differently from the console (#1513, #1514, #1515), a cache key that makes every player re-prepare all shaders when a struct gains a member (#1510), things nobody could see (what the shader stock holds, #1509; what the game's short sleeps do, #1506; which formats colour images are viewed as, #1512), and four costs in the host and the lifted code: a full fence called out of line for barriers x86-64 already orders (#1507), a Linux lock that sleeps at once and wake calls made with nobody to wake (#1504), a global lock at every take and give-up of a virtual core (#1503), and wake calls for semaphores, queues, locks and timers nobody sleeps on (#1505). Each issue has its evidence. This pull request is one pull request for all of them, as one commit per issue, so it splits along the commits if a reviewer prefers.
Fixes #1515, #1514, #1513 and #1510; parts of #1509, #1506, #1512, #1507, #1504, #1503 and #1505. Part of #1493.
How to reproduce
Seen on: found by reading the port against the console's behaviour and against its own history; shown on main by the new tests (below), on any OS and GPU (the host commits' tests are Linux).
Settings and switches: defaults (
HOS_SLEEP_LOG=1for the sleep census,RX_VK_LOG_PROGRAM_LIFE=1for the shader stock's draw notes,RX_VK_LOG_VIEW_FORMATS=1for the view-format census).Steps: run each commit's test against main's sources (Testing, section 1). For the cache key: change a member of
Programon main and watch every player's cache be dropped.Expected: the console's answers (FAILED for a never-fenced sync wait; compute runs whatever render enable says; the report word is written); a cache that survives a struct change; no wake call, futex call or lock for a signal, wake or core take nobody waits on.
Actual: on main the wait answers TIMEOUT_EXPIRED, compute is skipped while render enable is off,
nvnCommandBufferReportValueis the "not implemented" stub, and the key moves withsizeof(Program); the new tests count, on main, 2,000,000 futex calls for 1,000,000 wakes and 1,000,000 wake-alls with no waiter, 2,000,000 locks of the cores' gate for 1,000,000 take and give-up pairs, and 600,000 wake calls for 100,000 rounds of signals with no waiter.What changed
Eleven commits on main @
430eef4f2: the three interface answers, the cache key, three counters, and four commits that remove work from the lifted code and the host. Three switches are added and one default changes (Flags and settings).7d7b8a77anvnSyncWait(host/nvn_native.c) on a sync that was initialised but never fenced answers FAILED, as the console does; the first 8 such waits are logged with the caller73a666298gpu/vk/vk_replay.cpp) gates draws and clears only; a compute dispatch runs and is marked whatever it says; a counter reports dispatches recorded while it was offdd1907db7nvnCommandBufferReportValueis recorded as a new op (appended last, so no op number moves) and its value and a scaled timestamp are written in stream order, as a counter's report is4e1154a38translator_stamp()(gpu/vk/vk_shader.cpp) no longer hashessizeof(Program); it takes the C++ library's major version instead; a test fails if a struct size enters the key again68046eb7bgpu/vk/program_life.h,vk_programs.cpp) logs what the shader stock holds, what the game freed and when, every 600 presents and at close; counting only, nothing is released321e88068HOS_SLEEP_LOG=1, short sleeps are counted by thread class and run length (host/hos_thread.c), and the log says which calls ended each run; counting onlyf58b64f0eRX_VK_LOG_VIEW_FORMATS=1, a census (gpu/vk/view_format_census.h,vk_texture.cpp) counts the formats each colour image is viewed as and which images a format list could cover for AMD's colour compression; log only, no image or view is made differently; removal date 2026-12-31ca5021509nx2/insn.py,nx2/rtgen.py) writes a compiler-only barrier on x86-64 for the eight load-only and store-only DMB options instead of an out-of-line full fence; the full DMB, every DSB, SB and ISB keep the fence; a test pins every ordering class; the next build translates the game againeaa14833dhost/posix/windows.h,win32_posix.c) is one futex word that spins up to 200 pauses (not with one processor) and then sleeps on the word, and a condition variable wake with no waiter makes no system call; includes the header fix the whole suite found in the first version, with its own test (tests/test_posix_windows_h.py)7a94a791ehost/hos_thread.c);HOS_VCORE_LOCKED=1gives the lock back; the 5 svcores:line gains the longest queue182aa141dhost/nn_os_sync.c) and a signal with nobody asleep makes no wake call; a queue's receive pokes no WaitAny that waits for items;HOS_OS_SIGNAL_ALWAYS=1gives the old behaviour backAgainst main @
430eef4f2the stack changes 51 files (7,285 lines added, 298 removed); most of the added lines are the new tests.Measurements
Verdict: no cost found, and no gain beyond the spread, in two paired timings. At the Castle Town market square on a quiet PC (others' CPU 0.8 to 1.1 cores): against the PR's base
430eef4f2, main 13.91 ms a frame (14.01, 13.82) and this head 13.76 ms (13.82, 13.71), 1.1% lower; and against today's mainc507b98c8with this head merged into it, main 13.94 ms (13.89, 14.00) and main with the PR 13.90 ms (13.75, 14.06), 0.3% lower; the bar was 3% and the same-arm spreads were 0.8% to 2.3%, so both are no difference beyond the spread. The 99th-percentile frame, the 1% low, the GPU time (5.5 ms), the process's memory (resident set, anonymous memory and the video memory the log reports) and the virtual cores' waits are the same in both arms, with no escapes. A gain under about 1.5% would not show here, and it is one view, one PC and one GPU: nothing on walking, other areas, a CPU with fewer cores (where the lock and wake changes should matter most) or Windows. The rows from the sixth on in the table are the game-free tests' counts of calls and locks, not time.Hardware: Linux (Arch, kernel 7.2.15), RTX 4090, NVIDIA 615.78.08, i9-13900K, 94 GB; and a tester's Windows 11, RTX 4070 12 GB, NVIDIA 621.01 (the first six commits).
Scene: the long runs go through Kokiri Forest, Hyrule Field, the Market and Kakariko Village, headless 1280x720; the Windows route is Kokiri Forest, Hyrule Field and the Deku Tree interior at 1920x1080, frame generation off; the timing: the Castle Town market square, child, by day, the camera aimed at one fixed view, headless 1280x720.
Method: the game-free tests and the whole suite as in Testing; the game runs read the game's log (
longshot_log.txt) for each commit's lines. The long runs had other agents' tests and builds beside them, so no frame rate in them is a speed result. The timings are separate: two exclusive holds of four 110 s runs each (the first main against the head, the second today's main against the head merged into it), in the order main, PR, main, PR, under the workbench's paired-timing worker (the quiet-window and measuring locks held, others' CPU under 3 cores mean and 6 at the peak, same-arm spread at most 5%, a gap of 3% or more beyond the spread counts as a result), frame times from a 50 s window of each run, memory from the process's own counters at the end of each run, both builds as shipped (the head's lifted code differs from main's by this PR's barrier change).How to reproduce: Testing, sections 1 to 6b.
430eef4f2182aa141dtests/test_prepare_stall.py, a load-flaky 12 s watchdog test, Testing section 2)c507b98c8(this head merged into it, local scratch merge never pushed): frame time, mean ms (each run)4b68c42be)63c69f5d3; 53 failing names shared, none unique; the five newest commits were not run there)nvnCommandBufferReportValuesizeof(Program)Flags and settings
Three switches are added (
HOS_OS_SIGNAL_ALWAYS,HOS_VCORE_LOCKED,RX_VK_LOG_VIEW_FORMATS), the default of one changes (HOS_VCORE_STARVE_US) and two rows say more than before (HOS_SLEEP_LOG,RX_VK_LOG_PROGRAM_LIFE); no menu row, ini key or build flag. Each row is ingpu/vk/switches.defanddocs/development.md.HOS_OS_SIGNAL_ALWAYS(commit 11)HOS_VCORE_LOCKED(commit 10)=1: every take, give-up and yield goes through the lockHOS_VCORE_STARVE_US(commit 10; its category changes from tune to hw)taskset 0-7on a build of the stack before the final commits: no wait of 50 ms or more at 20 ms, 36 at 1 s, the machine's load differed, so not a controlled pair; not run on a DeckRX_VK_LOG_VIEW_FORMATS(commit 7)=1: log lines when the census changes (at most every 900 presents) and at closeHOS_SLEEP_LOG(commit 6)RX_VK_LOG_PROGRAM_LIFE(commit 5)program censusline every 600 presentsshader stocklines, with or without this switch; only this switch notes first drawsTesting
All commands run from a checkout of the head with the project's pinned Python (
python -I -B ...), through the workbench's idle-class runner for anything heavy (https://github.com/mooki000/ls-workbench/tree/main/harness). Each test was run first against main's sources with the new test copied in (red), then on the branch (green), then once per deliberate break of the changed code (every break must fail the test, and none may fail only by a compile error).Tested:
1. The new tests, per commit.
tests/test_nvn_sync_wait.py --cc clangPASS: a sync never fenced answers FAILED and logs its first 8 waits; the other answers are unchangedtests/test_render_enable_compute.py --cxx g++(and clang++)PASS: render enable gates draws and clears only; a dispatch runs and is marked whatever it says, and is countedtests/test_nvn_report_value.py --cc clang --cxx g++PASS: nvnCommandBufferReportValue is in the proc table, recorded with its value and address, and written in stream ordertests/test_stamp_abi.py --cxx g++(and clang++)sizeof(Program)is in the key, no libstdc++ version branch, compiled words 3, 0 against 3, 16; exit 1PASS: translator_stamp holds only sizes the cache writes as they lie (Shader::Info) and the library's version (number 3, 16)tests/test_program_life.py,tests/test_program_load_wait.pyvk_programs.cppas on main the wiring check failsprogram_life: ok,program_load_wait: oktests/test_sleep_census.pyFAIL: host/hos_thread.c has no census of short sleeps; exit 1ok: 23 census variants and 10 wiring variants fail(gcc and clang)tests/test_view_format_census.py --headers <vulkan include>FAIL: gpu/vk/view_format_census.h: no census of the formats colour images are viewed as, exit 1; with only the new header added to main's source the wiring check stops (srccut: 'bool view_census_on() {' is not in the source), exit 1PASS: the census counts each colour image made and the formats it is viewed as, judges which a format list could cover for AMD's colour compression (...), is told by every view and new image, and changes nothing in how an image is made; 55 checks of the header's fixture and a 9-image scene compared line by linetests/test_lift_ordering.pyrecomp_barrier();for the eight load-only and store-only options; 3 header checks, 5 emitter checks, 160 IR checks, 160 assembly checks); everything else passes on main, i.e. the other classes are pinned as they are (391 words checked as text)PASS: the ordering the lifter gives guest accesses and barriers is as pinned; 391 words as text, 588 compiled functions at -O1 and -O2 with and withoutA64_WRITE_TRACK, 11 litmus linestests/test_posix_srwlock.py(20 cases),tests/test_posix_windows_h.pywake_no_waiter(1000000 wakes and 1000000 wake-alls with no waiter made 2000000 futex calls) andtimedfail, the other cases pass (they guard the rewrite; the newblocked_sleepscase is an upper bound that passes on main's pthread lock by design);test_posix_windows_h.pyis red on the first version of the header1000000 wakes and 1000000 wake-alls with no waiter made 0 futex calls;test_posix_windows_h.pypassestest_posix_windows_h.py: 3 of 3tests/test_vcore_gate.pyfree_core_no_lock(1000000 take and give-up pairs took vc_lock 2000000 times) andwait_deadlines(a 200 ms wait slept 191 times) fail; the cases that need the test seams skip on mainmask_changeplain only) on the commit's own base:1000000 take and give-up pairs took vc_lock 0 times,a 200 ms wait slept 2 times; the stack's file addskick_cleared_in_yield(the yield race of Risks), which passes at the commit's position and has no mutation run of its owntests/test_os_sync_waiters.py(39 cases)no_waiter(600000 wake calls, 0 locks of the multi-wait) andqueue_take(200000 locks of the multi-wait and 200000 wakes of it); 29 pass (they guard), 8 skipWhat each test does: #1 compiles the real
nvnSyncInitializeandnvnSyncWaitover a fake GPU, clock and log and checks every answer; #2 cuts the real clear, draw and dispatch cases out ofvk_replay.cppand runs single ops with render enable on and off in the replay thread's pass, a worker's pass and the dry pass; #3 compiles the real handler and the real replay case over a fake guest memory and checks the written bytes, the ordering with a conditional-render compare and the reset behaviour; #4 compiles the source's ownabi[]initialiser and compares its words with the compiler's library, and fails if anysizeofin the key is of a type the cache does not write as one block (it replaces the five-name guard onProgramintests/test_linux_pressure.py, whose premise this change removes); #5 pins the census counts and the call that notes a program's first draw; #6 compiles the real census code over stubs and runs 12 scenarios (idle pollers, class names, which calls end a run, two threads interleaved, the widest line); #7 compiles the census header with-Wall -Wextra -Werrorand checks RADV's family, atomics and size rules, the bookkeeping and the report's lines on a made-up scene, and the wiring (a view tells the census after it succeeds, a new image once, none for a pooled one, no format list chained, flags and usage unchanged); #8 builds the translator and the generated header, pins the C written for every ordering class (the exclusives, LDAR, STLR, LDAPR, the LSE forms, SWP, CAS, all sixteen DMB and DSB options, SB, ISB), compiles the real emitter's output at -O1 and -O2 and reads the optimised IR and assembly, and runs litmus tests of the lifted code (message passing under the compiler barriers, store buffering under the full ones, with a positive control); #9 links the realwin32_posix.cwith a fixture that counts the futex calls it makes and runs 2 to 8 threads (exclusion, sharing, wake-all, timed waits, every wake seen with the futex wait delayed at random, a thread that finds the lock held sleeping on it a few times and not once per pass of a loop), andtest_posix_windows_h.pyincludes the header the way a fixture does, with-Werror; #10 compiles the real gate cut out ofhos_thread.cwith real threads and counters in front of the lock, the sleeps and the wakes (never two holders on a core, priority order, a starved waiter first, the starvation bound, a mask change, the starve default for 4, 8 and 9 processors) and, through three seams that are empty in the game, makes the lost hand-off happen on purpose in three interleavings and the lost kick request in a fourth; #11 compiles the real waiter counts with real threads and delays forced before every lock: counts that fail on main, and invariants that pass on main and guard the change (no lost wake-up for a semaphore, a queue's receiver and sender, a lock or a timer start with the signal held at the lock while the waiter counts in and sleeps, WaitAny ping-pongs with its sleep a minute long so only a poke can end it).The checks that read the changed files, run at each of the eleven positions of the stack, fail only
tests/test_linux_pressure.pyandtests/test_slow_frames.py, and both fail on main. Existing tests changed to follow the code they cut:test_hle_stats,test_short_sleep_timer,test_image_view_retryand its fixture,test_program_lifeandtest_program_load_waitwith their fixtures,test_linux_pressure(its five-name guard onProgramgoes, see #4), and for commits 10 and 11test_vcore_escape,test_thread_wake_cleanup,test_slow_frames,test_multiwait_wakeandtest_cv_signal_race.2. The whole suite (Linux).
python -I -B tests/run_all.py --jobs 6 --strict, clean worktrees of each tree, 6 at once, on the quiet machine:430eef4f2182aa141dCompared test by test, the only failing name that is not main's is
tests/test_prepare_stall.py; no test fails on main alone. The 41 failing names on main, which fail the same on the head: test_bug_report_defaults, test_capture_readback_visibility, test_cpu_report, test_crt_options, test_damage_debug_menu, test_dev_packs, test_display_startup_failure, test_dlss_feature_size, test_draw_stream_visibility, test_failed_compile_keeps_objects, test_fsr_image_retirement, test_fsr_uniform_ownership, test_guest_imported, test_hud_layout_row, test_invalidate_quick, test_invalidate_vcore, test_libc_hooks, test_linux_preflight, test_linux_pressure, test_linux_update, test_log_open_failure, test_mirror_budget, test_missing_nup, test_mods_size_table, test_nonuniform_image_arrays, test_parallel_init_order, test_pending_guest_import, test_pipeline_play_cache, test_present_source_crop, test_present_thread, test_profiling_mode, test_psychov35_fsr3_fg, test_readback_timeout_source, test_replay_write_order, test_run_all_changed, test_slow_frames, test_staging_burst, test_switch_table, test_upscaler_rows, test_vk_occlusion_queries, test_walk_key.tests/test_prepare_stall.pyis a 12 s stall-watchdog test and is load-flaky: no file of the stack is among its inputs; it also failed once on main in another worker's checks (the--changedruns of #1512's author), in a third worker's whole-suite run it failed under load on its head and passed alone on both trees, in 3 reruns here it failed once on the head and 0 of 3 on main, and in 5 further reruns it passed on both trees.The first push's one extra failure,
nx2/test_depth_sample_types.py, was a clang++ compile of 63 sources killed at a load average of about 180; it passes in 4 s alone on both trees and passes in this quiet suite.The suite also found one real regression in the first version of #1504:
host/posix/windows.hdeclared its SRW helpersstatic inline __attribute__((noinline))(an error under-Werror=attributes) and read the CPU count withsched_getaffinity, which needs_GNU_SOURCEtoo early for a file that included a libc header first; six tests failed only on the stack. The helper declarations and the CPU-count call (syscall(SYS_sched_getaffinity)with a popcount, spinning only with more than one usable processor, as before) are fixed in commit 9 with the newtests/test_posix_windows_h.py.3. The Windows-target syntax check on the final head. The workbench's fixture checker (https://github.com/mooki000/ls-workbench/tree/main/skills/ls-pr-evidence/scripts) compiled 16 translation units of the new and changed fixtures with llvm-mingw: 15 pass and only
tests/test_hle_stats.pyfails, as it does on main (inherited). Seven files are unjudged:tests/test_linux_pressure.py(main no longer has the text its fixture looks for,pool_pressed = true;) andtests/test_slow_frames.py(the checker stopped before its driver), both unjudged on main too;tests/test_vcore_gate.py,tests/test_os_sync_waiters.py,tests/test_posix_srwlock.pyandtests/test_posix_windows_h.py(their drivers skip Windows: they link the Linux layer of the host); andtests/test_lift_ordering.py(the checker's own driver could not load its generated library; its compile passes). The twomemcpyfailures the first push's check showed (commits 2 and 3) are fixed in those commits with the define the repository uses. The checker stops before running the drivers: it is a syntax check, not a Windows execution.4. In the game, Linux. RTX 4090, headless 1280x720, four areas (Kokiri Forest, Hyrule Field, the Market, Kakariko Village), with
RX_VK_LOG_PROGRAM_LIFE=1:182aa141d(buildg4): 690.1 s, 680.7 s of play, 40,859 frames, 22 pictures; fatal 0, crash 0, device lost 0, kernel and NVRM lines 0,nocore0 (noran without a core after 2000 msline), 136 five-secondvcores:lines with 0 escapes in all, longest queue 25; 47,867 program keys kept at start; the render-enable counter 0, the shader-stock lines, and the interface commits' lines as before. The machine's load was 19 to 43 from other work at the time (mean 60.0 fps is not a speed result).c0f37df02(buildg3,RX_VK_LOG_PROGRAM_LIFE=1andRX_VK_LOG_VIEW_FORMATS=1): 1,012.5 s, 1,006.0 s of play, 64,259 frames, 35 pictures; fatal 0, crash 0, device lost 0, kernel 0, NVRM 0,nocore0, 201vcores:lines with 0 escapes, longest queue 23; 47,867 program keys kept; the census's render-target answer in Measurements. This is the head before the last change to commit 9 and a test line of commit 11: the two builds differ only inhost/posix/windows.h(the SRW helpers' declarations and the CPU-count read) and in two tests, so the tour stands for the final head too.c0f37df02withHOS_SLEEP_LOG=1: 227.4 s, 18,179 frames, the same zeros, 133sleep: censuslines.0 program keys kept, 38,809 programs prepared in 33.2 s (the driver's cache was warm, so this is not a player's time); the second start kept 47,867 program keys and prepared nothing.63c69f5d3, the old gate), the same route: 1,051 s, 55,559 frames, 30 pictures; fatal 0, crash 0, device lost 0, kernel and NVRM lines 0; its save thread logged 28ran without a core after 2000 mslines (the same line appears in runs of main; the load average was about 200 at the start); a 227 s census run on it ran clean. Its first start on the old cache dropped the same 10 files and kept 0 keys, in 63.9 s of preparation.tasksetto 8 processors) on a ten-commit build of the stack: at the 20 ms default 16,079 frames,nocore0, longest wait for a core 35.6 ms and none of 50 ms or more; withHOS_VCORE_STARVE_US=100000036 waits of 50 ms or more, the longest 976.9 ms; the machine's load differed between the two, so this is not a controlled pair.5. In the game, Windows (an independent run of the first six commits). futondestroyer ran the first push's head on Windows 11, RTX 4070 12 GB, NVIDIA 621.01, headless Vulkan at 1920x1080, frame generation off, and posted the results (#1542 (comment)). He has not run the five newest commits.
4b68c42band the head63c69f5d) built cleanly, reached Hyrule Field and the Deku Tree interior from the Kokiri checkpoint, and all three launches exited cleanly with no device-lost marker (a smoke route with sampled frames).ReportValueor never-fencedSyncWaitlog hits: the route did not exercise those three paths. All three passed their Windows fixtures.6. The paired timing (Linux). One exclusive hold of the workbench's paired-timing worker on the quiet PC, main
430eef4f2(its warm copy) against a copy of this head's build (the binaries equal the build receipt's, the lift not rebuilt), the Castle Town market square by day with the camera aimed at one fixed view, four 110 s runs in the order main, head, main, head, a 50 s frame-time window in each. Each run was valid, undisturbed and quiet (others used 0.8 to 0.9 cores on average and at most 1.2 at the peak; 0 kernel lines; every log carries its own build stamp and "scene ok"):vcores:lines: 8,550 and 8,643 waits a second against main's 8,541 and 8,776 (the mean wait 80 and 81 us against 79 and 79), 0 escapes and 0 starved grants in all four runs; the longest wait 50.3 ms in the head's first run (one wait of 50 ms or more) and 25.4 ms in its second, 23.4 and 24.3 ms on main.6b. The second paired timing, against today's main. Forge main moved ten commits (several of them perf(vk)) while this was open, so the head was also timed merged into today's main
c507b98c8: a local scratch merge04239b7e2(clean, never pushed, the same 51 files as this PR) built as a full game with its own lift, against mainc507b98c8built the same way, the same scene, plan and rules (an exclusive hold of four 110 s runs: main, merged, main, merged). Both copies started with an empty pipeline cache and were warmed the same way (a cold start, a 200 s stand, a dry run of the scene), because the head's translator stamp drops a cache made by main: pipeline cache files 1,114,405 and 1,117,231 KB, the host's allocation at boot 2,381 to 2,386 MB in all four runs. All four runs were valid, undisturbed and quiet (others 0.8 to 1.1 cores, 0 kernel lines):vcores:lines a run, 8,405 to 8,747 waits a second, a mean wait of 79 to 81 us, 0 escapes and 0 starved grants in every run;c507b98c8averaged 13.94 ms against 13.91 ms for430eef4f2in the first pair, so its ten commits did not move this scene.7. The Windows cross-build. The workbench's cross-build route (llvm-mingw) on the final head
182aa141d: exit 0, coverage complete. The renderer (210 units, 40 compiled new, 170 cached, 0 failed) linked intolongshot_vk.dll(14,856,192 bytes, SHA-25621d215d4...), and the host with the Opus, zstd and WebGPU parts (214 units, 67 new, 147 cached, 0 failed) compiled with 0 errors and 0 warnings, includinghos_thread.candnn_os_sync.c(commits 10 and 11) and the renderer changes of commits 5, 7 and 1 to 3; the one Windows-compiled file of commit 9 is not part of that build (win32_posix.cis the Linux layer's). The game exe was not linked: commit 8 changes the lift and no Windows lift of the head was made, which is a 15 to 30 minute job.Not tested:
tasksetrun; macOS and ARM hosts (the wake-count skip rests on x86's atomics).LONGSHOT_TRACY=1) of the stack.Must be tested:
vcores:lines and theran without a corelines of each, expecting no crash, no device loss, no freeze at loading,escapes0 and no moreran without a corelines than main (the Linux-only lock is skipped there; the cores' gate and the waiter counts run there for the first time) (Test Farm: @Beanstalk windows, the long session on main and on this head, thevcores:lines side by side; or by hand: read the log)never fenced(absent or explained), the render-enable line'sM in all(above 0 means that change is live in that scene: look at fades, cutscenes and effect-heavy areas), theshader stocklines and the cache re-preparation on a cold driver cache, expecting no crash, no device loss, no glitch after the shader preparation, and a second start that prepares nothing (Test Farm: @Beanstalk amd and intel, a start on a cache from an earlier build, then a second start and the long session; or by hand: read the log lines above)HOS_VCORE_STARVE_US=1000000, counting thevcores:lines that saywaited50 ms or more for a core, expecting fewer at the default and noran without a coreline (Test Farm: @Beanstalk steamdeck, the two sessions and the counts; or by hand: count the lines in each log)escapes0 and noran without a coreline (Test Farm: @Beanstalk, the frame-time plan with main as the base; or by hand: the workbench's paired-timing worker)Risks
What could break: a program that read the report word ReportValue writes now sees it written (that is the fix; before, it kept stale data), and a caller that treated a never-fenced sync wait as a timeout now sees failed; both match the console, and neither path was seen in any run. The counters cost a walk of their tables every 600 presents and every 5 s with their logs on, and near nothing with them off; the paired timing in Measurements (logs off) found no cost.
The lifter's barrier change (#1507) alters the C of every function that holds one of the eight load-only or store-only barriers (about 920 of the game's 3,823 barrier words); it rests on x86-64 ordering loads with loads, loads with stores and stores with stores in write-back memory, the litmus runs of the test agree, and no run showed a fault, but a missing order would show as a hang, a stuck load or garbage state in a scene that uses worker threads. The Linux lock (#1504) replaces every SRW lock on Linux, so a lost wake there is a freeze; the 20-case test, TSan runs and 16 mutants say it is not, and the long runs showed none. The waiter counts (#1505) can lose a wake-up of a thread asleep on a semaphore, queue, lock or timer: a freeze (they wait with no deadline), a stutter as long as a timed wait, or a WaitAny that answers 5 ms late; the test holds each signal at its lock while the waiter counts in and sleeps, and
HOS_OS_SIGNAL_ALWAYS=1is the control. The cores' gate (#1503) is lock-free now. A yield with nobody queued for its core clears the holder's kick request without the lock; a request a waiter makes in the instant between that look and that clear is lost, and the waiter asks again when its slice ends (2 ms), so a core can be handed over about 3 ms late in that race; a game-free case reproduces the loss and checks that bound. HOS_VCORE_LOCKED=1 now locks every take, give-up and yield, so it also locks the yield that main already lets go without the lock when nothing is queued: it is the gate from before #1553, not main's exact gate, as a bisect baseline; the docs row now says so. With 8 or fewer logical processors the starve limit is 20 ms: a waiter that has waited that long goes before better waiters, which is what the Steam Deck's 184 long waits asked for and is unmeasured as a frame-time effect. The game runs' numbers come from a machine that other agents' tests were using, so the clean runs show no fault, not that a rare race is absent.Merge status: rebased onto main
430eef4f2(the first push was on4b68c42be) and merges with today's main: the forge's main is nowc507b98c8, ten commits further on, andgit merge-treeof this head with it reports no conflict. Three conflicts were resolved by hand in the rebase: commit 5 with the program-pool changes of #1564 and #1567 (gpu/vk/vk_programs.cpp,tests/program_load_wait_test.cpp; their behaviour and the census counts kept, the tests and mutations rerun), commit 9 with #1553 inhost/posix/windows.handwin32_posix.c(this commit's version kept: main's change had the waiter count), and commit 10 with main's Tracy zones inhost/hos_thread.c(the gate kept, the zones kept).Overlaps: trial-merged against the open pull requests that touch the same files (for commits 7 to 11 on 17 and 18 Nov against main
4b68c42beand430eef4f2, not re-run against today's main): none conflicts only because of this change. #1185 (lifter) needs two trivial hunks kept from both sides against commit 8 (nx2/insn.py, two pure additions; the lift tests pass in the merged tree); #288 and #772 each need one trivial hunk in the frame statistics or the dry-pass marking; our own #966, after commit 2, must drop its render-enable test in its dry-pass marking lines, and against commits 9 to 11 has one hunk in the SRW typedef ofhost/posix/windows.hand 7 hunks inwin32_posix.cagainst main's 4, for whoever lands second. #1469 (locals by default) merges cleanly with commit 8, but both shorten what a lifted function does around a barrier, so their gains may not add (an interleaved timing would say); its own test already fails on main plus #1469 alone. #157 (broadcast condition variable, timed semaphore acquires) has one hunk inwait_untilofhost/nn_os_sync.cagainst commit 11, and the same hunk conflicts on plain main (main's #1553 carries an equivalent clamp, and #157's own check fails against main): #157 needs a rebase; keep commit 11's line. #1284 (log writer on Linux) has one docs-table hunk beside commit 11's row and is compatible with commit 9's wake rule. Of the open pull requests, 152 touch a file of commit 7 and none conflicts with it unless it already conflicts with main. #1339 and #799 are independent of every commit here.Turning it off: revert the commit; the first six are each independent, and the five newest touch separate source files (the docs and switch tables aside), so each should revert on its own (not tried). Without a rebuild:
HOS_VCORE_LOCKED=1puts the cores' lock back andHOS_OS_SIGNAL_ALWAYS=1makes every signal wake as before; the lock (#1504) and the lifter's barriers (#1507) have no switch.Compatibility: every player re-prepares the shader cache once when this lands (about 4 minutes on a Steam Deck in a review run, 33.2 s here on a warm driver cache); #1507 changes the lift's key, so the next build translates the game again and compiles every lifted object again; the starve limit default changes on machines with 8 or fewer logical processors (Flags and settings); saves,
longshot_options.iniand the pipeline cache's format are untouched; the renderer interface version does not change (the new op is appended last, so no op number moves).Issues
Fixes #1515, #1514, #1513 and #1510. Part of #1509, #1506, #1512, #1507, #1504, #1503, #1505 and #1493.
Checklist
Everyone
(One topic: the first batch of fixes from the renderer-rewrite research (#1493), as one pull request with one commit per issue; it splits along its commits if a reviewer prefers.)
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
(The verdict is in Measurements: no cost found and no gain beyond the spread in the paired timing at one scene.)
in Risks. (Speed, CPU and memory: two paired timings found no difference beyond the spread and equal memory, Measurements; the one setup cost, the shader cache's one-time re-preparation and the dropped pipeline cache, is measured and in Compatibility; one scene on one PC, and memory at the end of a run only: both are in Not tested.)
main.(Rebased onto
430eef4f2; main is nowc507b98c8, ten commits further, and this head merges with it with no conflict.)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).(732 passed, 42 failed against main's 722 and 41; the 41 are main's and named in Testing section 2; the 42nd,
tests/test_prepare_stall.py, is a load-flaky 12 s watchdog test that also failed once on main and passes alone on both trees.)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.(Linux built and ran the final head. The first six commits built cleanly and ran on Windows 11 in futondestroyer's run, Testing section 5. The host and the renderer of the final head were cross-built for Windows with llvm-mingw with 0 errors and 0 warnings (Testing section 7); the game exe was not linked there (the lift changed) and the five newest commits were not run on Windows.)
(Three new switches and one changed default, each with its row in
docs/development.mdandgpu/vk/switches.def; the two control switches and the census switch ship off.)New on this forge; not opened on GitHub.
NVN's render enable (nvnCommandBufferSetRenderEnable, a conditional render that comes out false) is read by 3D work only, draws and clears. The replay also skipped NOP_DISPATCH and NOP_DISPATCH_INDIRECT while it was off, and the dry pass left a skipped dispatch's outputs unmarked, so a later draw that read what the dispatch writes could read stale data. Both dispatch ops now run whatever render enable says, and the dry pass marks compute work as written; draws, clears and their marking keep the rule they had. A counter says how many dispatches were recorded with render enable off, which the old rule would have skipped: the frame statistics print it ("vk: render enable off: N dispatches in the last 120 frames were recorded ..., M in all"), so a run shows the size of the effect. New tests/test_render_enable_compute.py cuts the real clear, draw and dispatch cases and the work marking out of vk_replay.cpp and runs them on single ops with render enable on and off in the replay thread's, a worker's and the dry pass; the dispatch cases and the dry marking fail on main. Part of #1493. Fixes #1514. Assisted-by: Claude Sonnet 5.5 Co-Authored-By: Claude Sonnet 5.5 <[email protected]>Windows correctness and cache upgrade passed on
63c69f5dagainst merge base4b68c42b. Tested on Windows 11, RTX 4070 12 GB, NVIDIA 621.01, headless Vulkan at 1920×1080, 100% render scale, with frame generation off.HOS_SLEEP_LOG=1. Render-enable-off dispatches stayed at zero, and there were noReportValueor never-fencedSyncWaitlog hits. All three paths passed their Windows fixtures. This gameplay route did not exercise them.4b68c42b63c69f5dAll six change-specific fixtures passed. The two suites shared 53 failing test names, with zero unique to the PR. The base alone failed the unchanged
tests/test_longshot_ctl.pywith a locked-log cleanup error and an exit-state assertion. The additional Linux failure reported in the PR description remains unidentified.63c69f5d3a182aa141d8New push: the branch now has eleven commits, rebased onto main
430eef4f2. Today's main (9dff58e6e) is nine commits further on and this head merges with it without a conflict. The description is updated to match.Against the first push (
git range-diff): commits 1, 4 and 6 are unchanged; 2 and 3 changed only in their fixtures (the define the repository uses for the Windowsmemcpywarning); 5 changed because the rebase conflicted with #1564 and #1567 and I resolved it by hand, keeping their behaviour and the census counts, then reran its tests and mutations; 7 to 11 are new: the view-format census (#1512), the lifter's load-only and store-only barriers (#1507), the Linux futex SRW lock (#1504), the virtual cores' gate without the global lock (#1503) and the waiter counts (#1505).What the whole suite found: the first version of #1504 broke
host/posix/windows.h(the SRW helpers' declarations failed under-Werror=attributes, and the CPU count needed_GNU_SOURCEtoo early); six tests failed only on the stack. It is fixed in that commit, with a new test,tests/test_posix_windows_h.py. The earlier extra Linux failure,nx2/test_depth_sample_types.py, was a clang++ compile killed at a load average of about 180; it is gone in the quiet rerun. The suite is now 732 passed, 42 failed, 17 skipped against main's 722, 41, 17: the 41 are main's, and the extra name,tests/test_prepare_stall.py, is a load-flaky 12 s watchdog test that also failed once on main and passes alone on both trees.Game runs on Linux: the final head played 680.7 s (40,859 frames), and a 16-minute tour (64,259 frames) ran on the head before two small last changes (the header fix in commit 9, one test flag in commit 11); both had no crash, device loss or kernel line, no
ran without a coreline and 0 escapes of the cores' gate. The cache key fix re-prepared 38,809 programs once from an old cache and kept 47,867 keys on the next start.Thanks to futondestroyer for the Windows run of 18 Nov in the comment above: it covers the first six commits only (no unique failure, the six fixtures pass). The five new commits have not run on Windows; they were only syntax-checked with llvm-mingw.
Not done yet: the night timing, so no speed or memory claim is made for any commit; Windows execution of the new five; AMD and Intel. The description's Must be tested list has the steps.
WIP: fix(vk): six changes from the renderer rewrite research, one commit per issueto WIP: fix(vk): eleven changes from the renderer rewrite research, one commit per issue@Beanstalk
@mooki approval of
182aa14recorded. This diff touches sensitive paths, so a second member ofBigBeansmust approve it too.Beanstalk test farm
Commit
182aa14· tier untrusted · approved by mooki · dashboardThis diff touches sensitive paths, so it needs two members of
BigBeans:nx2/insn.py,nx2/rtgen.pyA member of
BigBeanscan review this commit and comment@Beanstalkto test it.The paired timing of this head against main is in, and the pull request is out of work in progress.
Timing (Linux, RTX 4090, i9-13900K, a quiet PC: others used 0.8 to 0.9 cores). One exclusive hold of four 110 s runs in the order main, head, main, head at the Castle Town market square (child, by day, one fixed view), a 50 s window of each. Main is
430eef4f2and the head is182aa141d, both builds as shipped.All four runs were valid and quiet. Main's two runs differ by 1.3%, the head's by 0.8%, and the head is 1.1% lower on average: under the 3% bar, so the result is no difference beyond the spread. The virtual cores' waits are the same (8,541 and 8,776 a second on main, 8,550 and 8,643 on the head), with no escapes and no starved grants in any run. A gain under about 1.5% would not show here, and this is one view on one PC: nothing on walking, other areas, fewer cores, Windows, AMD or Intel, or memory.
Windows cross-build of the final head. The renderer and the host compiled and linked as units with llvm-mingw, 0 errors and 0 warnings, including the files of the cores' gate and the waiter counts. The game exe was not linked because the lifter commit changes the lift; the five newest commits have not run on Windows.
The description now has the timing in Measurements, the run in Testing section 6, the cross-build in section 7, and three boxes ticked: one topic (the first batch of the renderer-rewrite research, splitting along its commits if a reviewer prefers), nothing gets worse (speed measured as above; the memory the new counters use is not measured and is listed under Not tested), and host code builds on Windows and Linux (with the exe link stated as not done). What still has to be tested is in Must be tested.
WIP: fix(vk): eleven changes from the renderer rewrite research, one commit per issueto fix(vk): eleven changes from the renderer rewrite research, one commit per issueVerdict: Request changes. The new virtual-core fast path can grant an excluded core and lose a preemption request. The weaker default DMB lowering still needs an isolated Windows boot test.
Reviewed
182aa141against430eef4f, including the five additions since the earlier Windows smoke. No source changes pushed.host/hos_thread.c:2013. A queued waiter can still be granted a core excluded by a completed affinity change. The picker uses a cached mask and the waiter updates that mask only after it runs. If the old owner releases the excluded core before the woken waiter resumes, the grant can win first. Update the queued mask and waiter counts under the gate lock at the affinity change. Add a fixture that releases the excluded core while the waiter is held at the wake.host/hos_thread.c:2285. The idle yield can clear a kick published after the waiting-count read. The existing seam forces this schedule and accepts the lost request. Keep the idle fast path read-only and serialize clearing an existing kick. Have the seam require prompt handover.nx2/rtgen.py:49. Keep the full fences by default, or split the weaker lowering pending a Windows test. A separate Windows stack reportedly failed at boot with weaker DMB ISHLD/ISHST and booted with full fences. That report is from a different stack. For this PR, change only those two lowerings on a pinned affected configuration and repeat boot-to-title and first gameplay. DMB tracking #1507gpu/vk/vk_texture.cpp:2683. The opt-in close report reads the replay thread's plain frame counter from the display or control close path. The census mutex doesn't synchronize the replay write. Use a synchronized frame snapshot.Independent Windows checks: lift ordering passed for 391 instruction words and 588 compiled functions. The vcore escape, multiwait wake, thread wake cleanup, view-format census, and HLE stats fixtures also passed. The threaded ordering litmus requires Linux and was skipped. These fixtures do not establish current-head Windows boot and gameplay or exercise the affinity interleaving above. The earlier Windows result applies to the six-commit head
63c69f5d.@ -2608,0 +2680,4 @@{std::lock_guard lk(view_census_mu);lines = view_census_data.report("close", frame_no);}The close thread reads a replay-owned plain counter here. Use a synchronized snapshot.
@ -1803,3 +2013,1 @@vc_take(best, c);HOS_CHAIN(HOS_CHAIN_SIGNAL, vc_chain_key(best));WakeConditionVariable(&best->vc_cv);if ((vc_queue[i]->vc_wait_mask >> c & 1) && (!best || vc_better(vc_queue[i], best))) best = vc_queue[i];An affinity change can complete before the queued waiter refreshes this mask. Block granting a newly excluded core and cover releasing it first.
@ -1987,2 +2284,2 @@A stale kick goes through the lock so it is cleared without losing a newer request. */if (!__atomic_load_n(&vc_nq, __ATOMIC_ACQUIRE) && !__atomic_load_n(&t->vc_kick, __ATOMIC_RELAXED)) return;if (!vc_gate_locked() && !__atomic_load_n(&vc_core[t->vc_held - 1].waiting, __ATOMIC_SEQ_CST)) {VC_SET(t->vc_kick, 0);The unlocked idle path can clear a kick published after the count read. Keep this path read-only and require prompt handover in the seam.
@ -38,0 +46,4 @@BARRIERS = r"""/* (nx2/rtgen.py, nx2/insn.py dmb) DMB LD / DMB ST in lifted code: x86-64 never reorders what they order, so only thecompiler is held back. Anything else: the full fence, as recomp_barrier(). */#if defined(__x86_64__) || defined(_M_X64)#define A64_BARRIER_LOAD() __atomic_signal_fence(__ATOMIC_SEQ_CST)Please keep the full fence by default or isolate the lowering in an affected Windows boot comparison.
The same timing against today's main, with CPU and memory, is in the description (Measurements and Testing section 6b). The short version:
Against main
c507b98c8(ten commits past this PR's base, several of them perf(vk)), the head was merged into it as a local scratch merge (clean, never pushed, the same 51 files) and both were built as full games with their own lifts. One exclusive hold of four 110 s runs at the Castle Town market square, in the order main, PR, main, PR, on a quiet PC (others 0.8 to 1.1 cores, 0 kernel lines):Means: main 13.94 ms, main with the PR 13.90 ms, 0.3% lower; same-arm spreads 0.8% and 2.3%, bar 3%. No difference beyond the spread. The third and fourth runs have a slower tail in both arms (the machine drifted equally). The game process used 5.40 to 5.51 cores in all eight runs of both pairs, the GPU time was 5.5 ms and the log's video memory 7.3 to 7.5 GB in every run, there were 0 escapes and 0 starved grants, and main
c507b98c8itself averaged 13.94 ms against 13.91 ms for430eef4f2, so its ten commits did not move this scene.One correction to what I would have read from the first pair. There the head's anonymous memory was 9% lower (about 3.5 GB against 3.86 GB). That is not this PR: main's copy had loaded a 315 MB larger pipeline cache, and the host's allocation already differed by that much at boot. With both copies at the same cache size, the gap is gone.
A cost players will see once: commit 4 changes the translator stamp, so the first start after this lands drops the old pipeline cache (1.4 GB here) and prepares the shaders again (38,809 programs in 33 s on this PC with a warm driver cache). It is in Compatibility. After that a second start prepares nothing.
Still one scene on one PC and one GPU: nothing on walking, other areas, a CPU with fewer cores, Windows, AMD or Intel. The merge also merges clean with main today.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.