WIP: feat(vk): a Recorder thread row on the Debug tab turns the recorder thread on (off by default) #564
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!564
Loading…
Reference in a new issue
No description provided.
Delete branch "klimovich008/Longshot:feat/recorder-thread-setting"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Windows 11, RX 9070 XT · Needs: Windows with an NVIDIA or Intel GPU; Linux, any GPU
Why
The recorder thread is the largest speed-up we have measured in Hyrule Field on a CPU-limited PC, but a player can only
turn it on with environment variables (
RX_VK_RECORDER=threadwithRX_VK_RECORDER_OWN=1, #162). This adds a menu rowfor it, off by default. The tracker has no issue asking for it (searched "recorder", "RX_VK_RECORDER", "replay thread").
What changed
A "Recorder thread" row on the Insert menu's Debug tab, under Developer: On makes the renderer hand its Vulkan commands
to a second thread, after a restart. The environment variables keep working and go over the row for a run. Nothing changes
while the row is Off.
mode_from_envandmode_initingpu/vk/vk_recorder.hread the row whenRX_VK_RECORDERis unset. The rowmeans
thread, with the slot's own resets (RX_VK_RECORDER_OWN) on unless that variable is set;RX_VK_PARALLEL2 ormore refuses it, as it refuses the variable.
RX_VK_RECORDERset to any value decides alone, exactly as before. Thestart line and the refusal line say which of the two chose (
vk: the Recorder thread row (recorder_thread=1 ...)orvk: RX_VK_RECORDER=thread:).g_row_oninvk_recorder.h, set bygpu/vk/vk_draw.cppto itsmenu_env_on(thereader the Descriptor reuse and Sampler cache rows already use: the row's line in
longshot_options.iniin the gamefolder), so there is no second parser. The mode is still read once, at its first use.
gpu/vk/vk_menu.cpp(Debug tab, Developer; Off/On; a restart row; preset-exempt) andopts.recorder_threadinvk_menu.h.hn_recorder_threadshows a line under the row when it was On at the start andthe recorder is not running this time (the variable said otherwise,
RX_VK_PARALLELrefused it, or its thread or ringcould not be made).
gpu/vk/menu_i18n.h(machine-made, as the file's others).docs/development.mdgets a row forRX_VK_RECORDERitself (ithad none) and
recorder_threadin the list of rows read at the start.tests/test_recorder_thread_row.py(new);tests/test_damage_debug_menu.pyandtests/test_recorder_own_idle.pyfollow the new row and the new
g_ownline.Not changed:
gpu/vk/vk_api.hand every file the shader-cache stamp hashes.Measurements
Verdict: it helps on this PC: 61.5 -> 71.4 fps (+16.1 %) in Hyrule Field with the row On, 3 alternating pairs, all
admitted; CPU per frame rises 5.6 ms (the work moves to the recorder thread).
Hardware: Windows 11, RX 9070 XT, AMD 26.10.2 (Vulkan driver 0x8001a0), Ryzen 7 5700X3D, 64 GB
Scene: Hyrule Field, the standard test: the save loaded, the camera turning at full speed; 1280x720 window, 2048x1152 output
(16:9), no upscaler, no frame generation, frame rate unlocked (
HOS_FPS_MAX=240)Method: fps and CPU ms a frame from the present log and the process times over a 20 s window after the load; 3 alternating
pairs, no warm-up runs, the median; game clock 0.999-1.000 in every run
How to reproduce: two game folders from one base, differing only in
longshot_vk.dll; in the PR's folderrecorder_thread=1inlongshot_options.ini(or the row On and a restart), noRX_VK_RECORDER*variable in either; ourstandard-test script, the two alternated
Baseline: Longshot
839e85ff, the base our A/Bs share on this PC, without the row. Main was not the comparison: the PR'scolumn is its renderer commits on that base (test/recrow-839 @ 91bf45f3: the same
gpu/lines as this branch's), with#162's commits under them, as the row needs them (
RX_VK_RECORDER_OWN); #162 is not in839e85ff.Measured on main
839e85ff; main is at546dc212since. Of its commits in between, #162 is in the PR's column, and #60 and#250 touch
vk_recorder.horvk_work.hwithout changing how the recorder runs its packets; the fps on today's main willdiffer (#569 alone is 3-5 % less frame time), the difference between the columns is what the row turns on. A measurement
on the next base will replace these.
839e85ff, no recorder row839e85ff+ #162 @ 91bf45f3, Recorder thread OnTwo of the three control runs had a stall of about half a second in the window, which lowers their fps: against the control
run without one (64.04 fps) the row's median is +11.6 %. Those stalls come from this PC's drive, not from either build:
small file writes to it block for hundreds of ms for some minutes after a multi-gigabyte write (a runtime folder had just
been copied), and the test's own present trace is flushed on the presenting thread. An earlier A/B of the same switches by
the variables (#162's description) gave 66.52 -> 76.57 fps on this PC.
Flags and settings
One new menu row with its options-file key; three existing variables unchanged, now also related to the row.
recorder_threadinlongshot_options.ini)vkCmd*calls are packed and run in order by a second thread, a recording slot's own resets made without the wait: whatRX_VK_RECORDER=threadwithRX_VK_RECORDER_OWN=1doRX_VK_RECORDERRX_VK_RECORDER_OWNRX_VK_PARALLELRX_VK_RECORDER=thread, with a log line that names the rowTesting
Tested: game-free, on Windows (MSVC-target clang 23 and MinGW g++ on PATH):
tests/test_recorder_thread_row.py: the mode code cut fromvk_recorder.hbuilt withvk_draw.cpp's realmenu_env_onandthe line that hands it to the recorder, over a temporary game folder. All 135 cases of
RX_VK_RECORDER{unset, thread, sync,off, 0} x the row {no file, 0, 1} x
RX_VK_PARALLEL{unset, 1, 4} xRX_VK_RECORDER_OWN{unset, 0, 1}: the mode, the slot'sown resets, the one start or refusal line and which source it names;
mode()first called from four threads at once. Theoptions line as it is read (no file, no line, 0, 1, a longer key with the same start). The row in the table once, a restart
row with a slot, preset-exempt, default 0, its key
recorder_thread; the real hint over start value x mode x a recorderthat could not start. 12 broken variants of the cut code must fail, and do.
0 lines set rec::g_row_on, want 1), before its behavioural parts run.python -I -B tests/run_all.pyon this head against main5d3ecc78: 532 passed, 11 failed, 34 skipped here; 531 / 11 / 34 on main, the same eleven (test_appimage_player_build,test_cpm_discovery,test_lift_cleanup_error,test_linux_preflight,test_make_game_cache,test_oot_state_tap,test_psychov35_dlss,test_task_mesh_fallback_devices,test_tested_configurations,test_vk_occlusion_queries,test_walk_key).546dc212(one commit: the mods page and its test): the nine tests that read this PR's files (test_recorder_thread_row,test_recorder_own_idle,test_damage_debug_menu,test_menu_i18n,test_menu_restart_slots,test_preset_exempt,test_readme_menu_rows,test_env_on,test_check_pr_description) rerun there: 9 passed.vk_draw.cpp,vk_menu.cpp,vk_replay.cppandvk_menu_i18n.cppcompile as whole files with the renderer's flags forWindows, and to objects on Linux (Ubuntu under WSL, clang 21). The renderer DLL the game runs below used was built with
llvm-mingw from the row's code on
839e85ff(without the translations, which that older tree's tables do not take).In the game (Windows 11, RX 9070 XT, AMD 26.10.2): four runs of Hyrule Field with
recorder_thread=1in the options file andno recorder variable (the A/B above and an earlier attempt's one run), about 70 s each: the row's start line and the
recorder's stats in each log, 0 calls made directly, no device lost, no error line; the control's logs have no recorder
line. The menu on screen (two windowless runs with the renderer's menu script and shots): the row on the Debug tab under
Developer with its help text; with
RX_VK_PARALLEL=4set, the refusal's log line and the row's "Not running this time" line.Where the row is read, by reading the code (not by runs): the menu saves and the recorder reads the options file through the
same
game_module_path, so on every start (the launcher, the AppImage, the menu's Restart now) the line saved is the lineread;
make_game.py(--cli,--update) rewrites only its owncompile_shaders=line and keeps the others. A shaderpreparation makes no
vkCmd*call through the recorder, so it should neither read the row nor start the thread.Not tested: the row on this branch's head on main (the game runs are its commits on the base
839e85ff); the row changedby hand in the menu and the restart question after it (the runs set the options file's line); the translations on screen;
a shader preparation with the row on; a full Linux build; long sessions through the row; no content check (picture
comparison) was run on the row-on path.
Earlier runs of the same recorder thread through the variables, for context: about 25 minutes of play on this PC without a
crash, and #369 / #389 (a Ryzen 5 5600X on Linux, the Market: +6.5 % standing, flat while moving, nothing drawn wrong).
Must be tested:
vk: the Recorder thread row (...), nothing drawn wrong, no device lost, and fps at a fixed spot not lower than with the row Off (by hand: test request T-TTLW)Risks
What could break: nothing while the row is Off. With it On, the recorder thread's own risks: a second thread making the
driver's command calls (a driver that misbehaves with it), and less gain or a loss on CPUs without spare cores.
Turning it off: the row Off and a restart, or
RX_VK_RECORDER=0for a run without touching the saved options.Compatibility: Nothing changes for a player who leaves the row Off.
longshot_options.inigains arecorder_thread=line when the menu next saves; saves, the shader and pipeline caches and the vk ABI are untouched.
Screenshots
The row on the Debug tab with its help text, and the same with the line it shows when it is On but not running (refused by
RX_VK_PARALLEL=4): two pictures inthis comment, from this PR's renderer code
on
839e85ff@ 91bf45f3 (the After column's build). Main has no such row, so there is no Before picture.Reviewer notes
mode_from_env/mode_initingpu/vk/vk_recorder.h, thenhn_recorder_threadingpu/vk/vk_menu.cpp.its first packet); a row for
RX_VK_RECORDER_IDLE=wait, which cost fps when it was measured for #162.tests/test_menu_i18n.pychecks that every table key is a menu text, not that every menutext has a key, so a new row can stay English unnoticed; the Intel shader compiles row has no translations.
Issues
See #162
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).mainand passing here.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Measured: the standard test in Hyrule Field, 3 alternating pairs, all admitted: 61.54 -> 71.44 fps (+16.1 %) with the row On through the options file and no recorder variable; CPU 63.82 -> 69.45 ms a frame. Two of the three control runs had a half-second stall in the window (against the control run without one: +11.6 %); the description says where those come from as far as it is known. New commit
392c179fadds the two docs/tested files of this A/B. Still WIP for one thing: the screenshots of the row in the menu.Description only: the flags table's empty cells filled and RX_VK_PARALLEL listed, why the numbers from
839e85ffstill hold on today's main, and the earlier recorder runs moved out of Not tested. No code change.Test requests opened: T-TTLW (NVIDIA or Intel, Windows) and T-XA4S (Linux). They need the testing wanted label to be claimable.
Screenshots of the row, from this PR's renderer code on
839e85ff(test/recrow-839 @ 91bf45f3: the samegpu/lines, without the translations), two windowless runs with the renderer's own menu script and shots (HOS_HEADLESS,RX_VK_MENU_SCRIPT,RX_VK_SHOTS,RX_VK_SHOT_MENU),recorder_thread=1in the options file and no recorder variable.The row On with the recorder running (the log:
vk: the Recorder thread row (recorder_thread=1 in longshot_options.ini; RX_VK_RECORDER unset): ..., then the recorder's stats lines):The same with
RX_VK_PARALLEL=4set for the run, which refuses the row (the log:vk: the Recorder thread row (recorder_thread=1 in longshot_options.ini) refused: RX_VK_PARALLEL=4 ...): the row's line says it is not running this time.Rebased on main
546dc212(no conflict; the five commits are otherwise unchanged) and the description moved to the new template. The full suite on the rebased branch against main: the same eleven failures on both, this PR's own tests pass. Screenshots of the row and of its hint are in the comment above.392c179f4e51c4b43e08Closing as superseded by #588, which merged while this was being finished: its Threaded rendering row (Auto / On / Off) turns the recorder thread on together with the new render thread, through its own hook in
gpu/vk/vk_recorder.h. A second row for the recorder thread alone would duplicate half of that row, and this branch now conflicts with it invk_recorder.h,vk_menu.cpp, the README anddocs/development.md.What stays useful from here, for whoever picks it up on top of #588:
RX_VK_RECORDER_OWN), turned on through the options file with no variable set, 61.5 -> 71.4 fps in Hyrule Field on an RX 9070 XT / Ryzen 7 5700X3D under Windows (3 alternating pairs; the description above has the table and its caveats). #588's row turns the recorder thread on withoutRX_VK_RECORDER_OWN;tests/test_recorder_thread_row.py's matrix of the variable, the row,RX_VK_PARALLELandRX_VK_RECORDER_OWN, which would port tog_setting_thread;gpu/vk/menu_i18n.h, andtests/test_menu_i18n.pydoes not catch it.The two test requests of this PR (T-TTLW, T-XA4S) are cancelled.
Pull request closed