fix(vk): a fixed frame generation multiplier's sets follow the game's presents, not the render thread's hand-over #2184
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2184
Loading…
Reference in a new issue
No description provided.
Delete branch "Jiro113/Longshot:fix/fg-fixed-sets-game-clock"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
With a fixed frame generation multiplier (2x to 8x) the generated frames are paced on the game's timeline (#1362, and under FIFO #1442), and that
timeline was the render thread's hand-over:
framegen_recordput each set at its own call, which comes after the mainthread's signal and the frame's replay. On an RTX 4090 (DLSS Frame Gen 2x at a 60 fps cap, FIFO, a 240 Hz VRR display)
the hand-over came about 4.4 ms after the game's present, its intervals' standard deviation 0.94 ms where the
presents' was 0.28 ms, and the game frames' intervals on screen followed the hand-overs' (r 0.81; against the presents',
r 0.23): the game's frames reached the screen as unevenly as the replay ran.
To see it on
main: frame generation 2x at a 60 fps cap, V-Sync on, a VRR display; pan the camera in Hyrule Field andrecord the presents with PresentMon: the game frames' intervals on screen have a standard deviation of 2.36 ms (six runs
here: 2.27 to 2.44), 28 % of them within 0.5 ms of 16.67 ms.
At the game's cap, the sets now follow the game's presents:
presenttakes the game's thread's time just after the FPS counter's stamp; the present's job carries it to bothframegen_recordcalls (present_game_at);fixed_clockputs the set at that present, the render thread's lag after the presents later, smoothed over 16sets: the presents give the intervals and the lag the phase, so the GPU's delay (
pace_delay) takes up only how muchthe lag varies;
hand-over, the lag kept, and the next set is on the presents again; a hitch takes the lag again;
pace_feedskeeps a set that came after a long game frame from feeding the GPU's delay back, on either timeline.Below its cap the game's thread waits in its present (for the main thread's signal, then for room in the render
thread's queue), so its presents carry the game's own work too, and sets on them would take in how both vary.
enqueuenow takes how long the thread waited (
game_wait);fixed_clocksmooths it, and over 1 ms the sets are at theirhand-over as on
main, under 0.5 ms on the presents again. At the cap here, 4 to 47 presents of 600 waited 0.1 to0.2 ms.
RX_VK_FG_GAME_CLOCK=0puts every set at its hand-over again (a kill switch, its row ingpu/vk/switches.def). Auto isunchanged. The log's frame interpolation line counts the sets on the presents, their lag (mean, most), those at their
hand-over because the game ran below its cap, and the times the delay was fed back at a game frame.
docs/dev/reference/architecture.md(frame generation's rule 2) says when the presents hold.This and #2183 (the replay's queue at global priority low) change some of the same files (
vk_framegen.cpp/.h,vk_replay.cpp,switches.def, the guide images' README) but merge with no conflict in either order; the "both" columnbelow is the two merged.
Possibly related: #1568 (frame generation 2x paced unevenly in
Jabu-Jabu's Belly, Linux, RTX 5080) and #2105 (stutter in busy
areas with frame generation, the game's frames down to 51 to 55 fps, RTX 4070 SUPER, Windows): below the cap this keeps
main's pacing when the replay has its own thread (Risks), so it should not change those drops (inferred). Neitherwas tried there.
Read first:
gpu/vk/vk_framegen.cpp(fixed_clock,pace_feeds,framegen_record),gpu/vk/vk_replay.cpp(
present,enqueue:present_game_at,game_wait).Testing
Tested: Windows 11, RTX 4090, NVIDIA 621.24, a 240 Hz VRR display, full screen. Four runs of this PR and four with #2183 too, against six of
main(Measurements). In every runthe log's line put each 5 s window's sets on the game's presents (a run's first set aside), but in the window of the
teleport's load, where 17 to 57 were at their hand-over with the game below its cap. Over the pan the hand-over came 4.0
to 5.3 ms after the game's present, as on
main(Summary); a window of the pan:No device was lost. This PR's head, on
maina2dc53859, ran again 30 s in a window (in the Market: the run'steleport to Hyrule Field didn't take): past its first log window, each 5 s window put 299 to 301 sets on the game's
presents, 0 with the game below its cap; no device lost.
On earlier builds of this work, on the same PC (with the replay's queue at global priority low too, before the work was
split in three), below the cap under a GPU load, three runs of two series: past the first log windows every window put
its sets at their hand-over, and the game frames' calls followed the hand-overs (their intervals' slope against them
1.06, 1.07 and 1.04, r 0.99) as with
RX_VK_FG_GAME_CLOCK=0(1.05, 1.06, 1.04, and 1.11 in another scene, r 0.98 to0.99), where without the last commit they followed the presents (1.03 and 1.06); at the cap, unloaded,
300 or 301 sets a log window on the presents.
mise run test(tests/run_all.py --strict) on this PC, Windows with llvm-mingw's clang++: 37 checks fail onmaina2dc53859here (listed below), and the same 37 fail here, on the same lines, and no other (test_sampler_admission'sconcurrent cases fail a varying number of times from run to run on
main: 2 of 147 in its full run, 3 in this PR's). The renderer builds on Linux (RX_VK_ONLY=1).tests/test_framegen_fixed_pace.pyfails at each test commit for the bug's reason and passes at its fix (clang++): onmain's code, the screen moves with the hand-over (sd 1.04 ms); below the cap on the first fix's, the GPU's delay ranto 11.4 ms on the presents against 7.2 at the hand-over. It passes here with g++ (11.4, Linux) and clang++; it catches
the 13 mutations of the last fix its commit lists, and the 3 a cold review tried on the first.
tests/present_source_crop_test.cppfollowsframegen_record's new arguments; it passes onmainand here, withclang++ and with g++.
The 37 checks that fail on
maina2dc53859on this PCnx2/test_import_windows,test_auto_handheld,test_bug_report,test_close_waits_for_save,test_compile_zero_byte,test_crash_dialog,test_damaged_rom_files,test_dev_ab,test_dev_build,test_dev_migrate,test_dev_packs,test_draw_stream_visibility,test_fs_save_commit,test_fsr3fi_hud_layer,test_fsr3fi_hudless_colour,test_fsr_uniform_ownership,test_game_menu_mouse_buttons,test_guest_copies,test_hold_toggle,test_input_movie,test_linux_preflight,test_lower_atomg_cas,test_menu_i18n,test_menu_keys,test_menu_slider_ends,test_pads_after_menu,test_perf_report,test_portable_builder_home,test_portable_builder_startup,test_recording_admission,test_sampler_admission,test_sdiv_overflow,test_sim_turf_pools,test_switch_table,test_tls_objects,test_vibration_menu,test_work_lockStill to test:
camera pan,
mainagainst this PR, expecting the game frames at least as even on screen and the log's line countingthe sets on the presents (Test Farm)
Measurements
Verdict: a fix for the game frames' pacing: at the cap their intervals on screen have a standard deviation of
1.49 ms (1.48 to 1.52, four runs) against 2.36 on
main(2.27 to 2.44, six runs), and no longer follow the renderthread's hand-overs (r 0.35 against 0.81). The intervals between a game frame and a generated frame do not change
beyond the runs' spread (9.5 % within 0.5 ms of 8.33 ms against 10.0 %): where the generated frame lands is
#2183's part. With both, 30.4 % within 0.5 ms (20.1 to 35.4) against 29.8 % (26.5 to 33.6) for that PR alone,
the game frames' standard deviation 1.29 ms against 2.24, and a game frame's two halves (game to generated frame,
generated to game frame) within 0.2 ms of each other on average in three runs of four (+0.04 to +0.36 ms), against
none of four without this PR.
Setup: Windows 11, RTX 4090, NVIDIA 621.24, 240 Hz VRR display; DLSS Frame Gen 2x, 60 fps cap, FIFO, full screen,
Hyrule Field with the camera panning for 60 s after a teleport, PresentMon over the pan; every run with the same logging
switches on (
HOS_PRESENT_TRACE,RX_VK_FG_LATENCY_LOG,RX_VK_REFLEX_LOG),RX_VK_PRESENT_QUEUE=0. Runs interleavedwith the other heads of this work, after one warm-up run each; the mean of each head's runs, (lowest-highest). One more
run of
mainwas left out: PresentMon saw 5309 of its 13240 frames composed by the desktop (Composed: Flip), notflipped straight to the display.
8aa749834Baseline:
main8aa749834for both After columns. 38e673852 is this PR's commits (760888e08) and one diagnosis commiton top,
RX_VK_FG_MADE_SHIFT_US, left unset (it shifts nothing unset, and it is not in this PR); 451bc026c is the sameon top of #2183's commits.
Measured on
main8aa749834. This PR is now onmaina2dc53859, its commits' changes the same (checked withgit range-diff) and, since, a line in the guide images' README.main's commits since change neither the presents'timing, the queues nor the fixed sets' pacing; in the renderer, one keeps the HUD's colours with FSR 3.1 frame
generation in Native HDR (DLSS here), one starts the replay's threads through a guard that reports their C++
exceptions, one logs fewer memory imports, #2171 moves two includes, spells
rx_nvn.h's static assertions so g++ builds them, changes menu texts and mendspresent_source_crop_test.cpp'shud_sourcestand-in, and #2174 keeps the simulated-turf lists at the game's own size in areas without turf, so that HyruleField, the scene measured, no longer stalls 250 ms at a time on the way in: that changes frame times there, and both
sides above ran without it. #1722 adds a save.
Risks
RTX 4090 with DLSS only (Still to test).
RX_VK_FG_GAME_CLOCK=0puts it back as onmain.more waits of 2 ms or more at the cap puts them at their hand-over for about 22 presents.
its sets stay on its presents.
fixed_clock's state is not reset when the device or the swapchain is made again, or frame generation is turned offand on; a hitch takes the lag again.
longshot_options.ini, the saves or the caches. A revert is enough.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here.gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.No screenshots: the change is in the frames' timing, which a still picture does not show; nothing in the menus changes.
The guide's Graphics capture reviews
gpu/vk/vk_framegen.h, which this changes (framegen_record's arguments): theimage is retained and
docs/user/images/guide/README.mdrecords the review. That check(
scripts/check_guide_screenshots.py --review-sources) fails onmainalready (ten files changed since thecapture's reviewed revision); this adds that one to its list.
Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Tested in game: Windows 11, RTX 4090, NVIDIA 621.24, a 240 Hz VRR display, full screen; DLSS Frame Gen 2x at a 60 fps cap, V-Sync on, Hyrule Field with the camera panning, PresentMon recording, four runs of this PR and four with the queue priority PR too, against six of main: the log put 300 or 301 sets a 5 s window on the game's presents, and the game frames' intervals on screen had a standard deviation of 1.49 ms against 2.36 on main. No device was lost. The final head, on main
a2dc53859, ran again 30 s in a window: past its first log window, 299 to 301 sets a 5 s window on the game's presents, none below the cap, no device lost.Possibly related, for context:
WIP: fix(vk): a fixed frame generation multiplier's sets follow the game's presents, not the render thread's hand-overto fix(vk): a fixed frame generation multiplier's sets follow the game's presents, not the render thread's hand-overRead of
dd4be442(code =0546b296test +4a273d61fix + docs) on maina2dc5385. The hand-off change is only the set's place —fixed_dues(place.before, place.at, …)from the presents + the smoothed lag instead of(before, now)— and the rest of the set's life is untouched. The off path is main exactly and the test is real. One concrete gap at 3x and up, under FIFO.1. At 3x+, a lag step or a held present can make
fifo_catch_updrop the set's made frames. The game frame's due is nowplace.at + pace_lead(frame, made) + extra(vk_framegen.cpp:1844-1856), withplace.at = game_at + c.lag(:263) — the smoothed lag. When the actual hand-over is later than the smoothed lag by more than the lead,due[made]falls before the set is taken.fifo_catch_upthen still measures fromdue[made]: itsnextis clamped tonowand its loop wantsshownframes a refresh apart ending within a refresh ofdue[made](:346-354); with the game frame already late it cannot fit even three, and returnsmade - 1(:354), leaving out all but one made frame — and the game frame goes late anyway, so this drops generated frames for nothing. Its own doc says the premise is "so that the set's game frame … is shown no more than a refresh after its own time" — true on main's hand-over timeline, not once the place is behind the hand-over. Reachable atmade >= 2(3x to 8x, which DLSS FG runs at 6x):hold_ms()= 50 ms (cpu_wait_unmet,vk_replay.cpp:7592-7593), whose owngame_waitdoes not see the hold (it is taken atenqueueafter the signal/room,:7752, not when the held present runs,:7557) — so it is accepted as an at-cap present and placed with a 4.5 ms lag while it was handed over 50 ms late;The tests cannot see it: every case with a non-zero lag uses
made = 1(tests/test_framegen_fixed_pace.py:510,513,568,570,584,587,604,608,617,620), andfifo_catch_upreturns 0 atmade < 2(vk_framegen.cpp:346); the 3x/4x/6x cases (:425,438,454,472,486) pass no lag, sofixed_clock's place there is the hand-over and the new path is not exercised. Please add one 3x case with the measured lag and a held present; the fix is to measure the catch-up against the hand-over when the game frame is already late (e.g.max(due[made], now)), or to skip it then.2. The other questions.
due[]is strictly increasing whenevertimed(fixed_clockrequiresat > before,:267), and a place not after the one before, a hitch, or a present not known/stale (>100 ms,:253) leaves the set at its hand-over or with no times at all — main's path.set->made, the frames' order and the slots are untouched, so nothing is duplicated or reordered; the only drop path I found is the catch-up above.=0reduces to main exactly:c.attracksnowandp.at = now, soplace.before/atare main'sbefore/now;c.frameequalsfg.game_ns(same sequence, same guards), andpace_feeds(span, frame, hand_span, hand_frame)then hasspan == hand_spanandframe == hand_frame, i.e. the old single check. Frame generation off never callsframegen_record; Auto is structurally untouched because theauto_ratebranch returns before theby_timeblock (:1747-1800) — that claim holds.fixed_clock, andcheck_game_atasserts the time's path frompresent_taggedto bothframegen_recordcalls, neither of which exists on main — and it covers the measured lag distribution, a lag step, a hitch, a present not known and one >100 ms stale, the below-cap loop with both hysteresis ends, and=0. Its held-present coverage is plumbing only (:849-853), which is the gap in 1.present_image_with_trial, not the present job'sgame_at; nothing here makes it worse. A held image makes the display thread's presents late, and the sets go out at their times when it ends, as on main (FIFO's catch-up may still leave made frames out there, which is the existing rule).RX_VK_QUEUE_PRIORITYand #2182'sRX_VK_PRESENT_PROBE/RX_VK_QUEUE_PRIORITYmust be on the same side in both arms (and #2182 is diagnostic anyway);RX_VK_FG_GAME_CLOCKon the same side in both arms is the whole comparison; keepRX_VK_PRESENT_QUEUE=0as the measured runs did, plus the FG logs. For our #1988 present/texture-stall A/B, pinRX_VK_FG_GAME_CLOCKandRX_VK_ASYNC_PARTS/RX_VK_ASYNC_FENCESon both arms — this changes the game frames' pacing, so it moves present stalls for a reason that is not #1988. #2184 and our #1988 fix both editdo_presentwithin a few lines (theframegen_recordcall at:5508vs theSlowWatch/texture_framenote) — rebase, no semantic overlap.:254-256) — measured only on the RTX 4090. On a PC whose present thread waits more at the cap,heldstays false? no: it goes true, the sets return to the hand-over, and the fix does nothing (not worse, but not the claim). Say "where the thread does not wait in its present"; the Still-to-test's log line is the acceptance. The screen table's Before/After aremain 8aa749834/38e673852(pre-split + a diagnosis commit), both without #2174, which the body itself says changes Hyrule Field's frame times — the measured scene; the head has a 30 s log window only. One row on the head (or a re-measure on current main) closes the status warning.No approval, and no claim on it.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.