WIP: feat(vk): live switches and ab (in session, or across restarts) for A/B runs; RX_VK_OCC_QUERIES the first #36
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
needs measurements
question
testing wanted
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!36
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:feat/live-switches"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
Comparing a switch's two values meant two launches, a minute apart, with another walk to the scene and other warm caches, so differences of a few percent were lost. This makes a switch changeable while the game runs, so both values are measured in one session, alternating, with the game's own numbers for each half.
What changed
The diff against
mainholds #23 (the occlusion queries) under this PR's own commits; #23's part is reviewed there.From #23 (stacked under this PR):
occ_queries_on,occ_room,occ_take,occ_collectandocc_resetingpu/vk/vk_replay.cppgive the draws between a samples-passed reset and its report Vulkan occlusion queries from a 65,536-entry ring (kOccPool), numbered inNvnState(occ_next,occ_first,occ_limit,occ_on,occ_overflowingpu/vk/vk_state.h);replay'sNOP_RESET_COUNTERandNOP_REPORT_COUNTERopen and close a run, andbegin_recordingwrites the finished sums into the game's slots (occ_collect).RX_VK_OCC_LOGlogs their counts.draw()ingpu/vk/vk_draw.cppbegins the query in its pass and ends it as it returns (OccGuard);DrawArgscarriesocc_poolandocc_query(gpu/vk/vk_draw.h).Device::occlusion_preciseandhost_query_reset(gpu/vk/vk_internal.h), set increate_device(gpu/vk/vk_main.cpp).tests/test_vk_occlusion_queries.py(new in #23);tests/fsr_uniform_ownership_test.cppstubsocc_collectandocc_value.This PR:
gpu/vk/vk_live.h(new):LiveSwitch,live_switches(),live_on,live_findandlive_marks. Each switch starts as its environment variable says (longshot_env.h's rule). The first isRX_VK_OCC_QUERIES:occ_queries_onaskslive_on(LIVE_OCC_QUERIES)once a command buffer, before its dry pass (replay_handle'socc_limit, throughocc_room), so the dry pass and its workers agree. Turned off, the run already open goes past its limit and its report stays "visible"; turned on, the next samples-passed reset opens a run; its query pool is made the first time it is on.live_tickingpu/vk/vk_replay.cpp, called fromstatus_tick, takes an ask at the next frame and first logs the time since the last ask, e.g.vk: live: 590 frames in 9.5 s with RX_VK_OCC_QUERIES=0: 62.1 fps (1% low 51.0), GPU 8.12 ms a frame, 1830 draws a frame. The first half second after a change is left out. An ask for the value a switch already has changes nothing and still ends the period.menu_settingingpu/vk/vk_menu.cppends its list with the live switches (tablive, keyenv.<NAME>,live_fill) and the mark (live.mark,live_mark_fill);menu_setting_settakes those keys withoutmenu_mu. They are not in the menu and not saved tolongshot_options.ini. No interface change:setting/setting_setcarry them, as inlongshot_ctl.py option set RX_VK_OCC_QUERIES 1.scripts/longshot_ctl.py ab NAME [--every 10] [--rounds 4](ab,live_halves,ab_summary,ab_text,scene_flag,ab_counted): starts from the switch's value with a mark, flips it every N seconds R times each way, ends on the start value, and prints each half, each value's medians (fps, 1% low, GPU ms, draws), the difference, and the fps spread between halves of the same value (saying when the difference is within it). A half in which the game left play, changed map, loaded or opened a menu is marked, listed and left out of the medians.longshot_ctl.py ab NAME --restart GAME_DIR [--values A,B] [--teleport 3] [--settle 15](ab_restart,to_play,teleport_to,start_game), for switches that need a restart: A B then B A, each launch taken into play and teleported (asked again where refused), left to settle, measured between two marks, then quit; the same table as live mode.tests/test_live_switches.py(new);tests/test_vk_occlusion_queries.py: the switch is invk_live.hnow, andocc_queries_onasks it before the pool is made.Flags and settings
RX_VK_OCC_QUERIESenv.RX_VK_OCC_QUERIES)RX_VK_OCC_LOGRX_VK_X,RX_VK_Ytests/test_live_switches.py's fake bridge and fake games; the game reads neitherlongshot_ctl.py ab--restart)Measurements
Measured on the GitHub version (NotRexed/Hookshot#2932), not yet on Longshot: this PR stays WIP until it is.
These runs show the tool at work: both columns are this PR's build, with
RX_VK_OCC_QUERIESoff and on. main was not the comparison, since the change is the A/B itself.Baseline: this PR (GitHub), RX_VK_OCC_QUERIES=0
Hardware: RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, 64 GB, Windows 11
Scene: Castle Town Market from a save, standing, then again with the camera turning
Method: live: one headless launch,
ab RX_VK_OCC_QUERIES --every 10 --rounds 4; the medians of each value's halves, from the game'svk: live:linesHow to reproduce:
python -I scripts/longshot_ctl.py ab RX_VK_OCC_QUERIES --every 10 --rounds 4against a running game in Castle Town Marketabsays so.Restart mode,
ab RX_VK_OCC_QUERIES --restart GAME_DIR --rounds 2 --teleport 3 --every 20: four launches, unattended, about 10 minutes.Testing
Tested: the runs under Measurements (on the GitHub version) and the game-free checks this description names.
Not tested: Linux. The occlusion queries underneath (#23) have not run on AMD, the Steam Deck or integrated GPUs.
Needs other testers:
longshot_ctl.py ab RX_VK_OCC_QUERIESandab ... --restart, expecting their tables, with no lost device (request to open)Screenshots
Nothing on screen.
Risk and rollback
RX_VK_OCC_QUERIESis read once a command buffer, from an atomic onlylive_tickwrites (live_on); a change while the game runs leaves the run already open "visible", which costs culling, never a missing object.live_tickruns at every present; it reads the asks and logs only when there is one.env.andlive.markkeys take no menu lock and are not saved, so the options file is untouched.RX_VK_OCC_QUERIESis set or the bridge asks for it; reverting the commits after #23 is enough to take the live switches andabout.Reviewer notes
Carried over from NotRexed/Hookshot#2932 with the move to Longshot: the same change on Longshot's
main, with its rename applied. Its review and discussion are there.mainholds both, so the flags and files from #23 are listed above as #23's.run_all.py): every check that reads the changed files (179), throughrun_all.py, Windows: 176 pass.test_linux_driver_fault_guardandtest_readback_wait_boundedfail the same way onmainthere, andtest_input_backgroundfails the same way at NotRexed/Hookshot#2871's head (its older base; it passes on that day'smain).tests/test_live_switches.py:vk_live.h,live_tick,live_fillandmenu_setting_set'senv.branch are cut out and run against stub counters (the start value from the variable; an ask taken at the next frame, not before; the settle time left out; each half's frame counts, fps, 1% low, GPU ms and draws, read back withlongshot_ctl's own pattern; a mark; a half too short).abruns against a fake bridge (the order of the asks, the halves kept, the medians and spread, a half with a map change marked, a menu row refused) andab --restartagainst fake games (A B then B A, each launch's value and environment, its second mark counted, each one quit and waited for; a refused press and a refused teleport asked again; a load inside a launch flagged and left out; every launch flagged is an error). In the source: the tick is in the present, the listing comes after the rows, the ask is not saved and takes no menu lock, and onlylive_tickwrites what the readers take. Two deliberate bugs fail it: no settle time, and an ask applied before its half is logged.vk_live.honly once its reader takes it where a change is safe (where a piece of work begins, with nothing made or sized at the start depending on it); restart mode covers the rest (thread modes, cache keys, resources sized at the start).Issues
None.
Checklist
For everyone
type(scope): description).main.python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.docs/tested-configurations.mdfor each measured run.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).mainand passing here.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
feat(vk): live switches and ab (in session, or across restarts) for A/B runs; RX_VK_OCC_QUERIES the firstto WIP: feat(vk): live switches and ab (in session, or across restarts) for A/B runs; RX_VK_OCC_QUERIES the firstBack to WIP under #48's process: this was carried over before the new template. Its description will be brought to the template and measured on Longshot (main against this PR, at named commits) when it's wanted for review; ask here if you'd like it sooner.
Description brought to Longshot's PR template (#48). Still WIP: its numbers are from the GitHub version, and it isn't rebased on current main; it gets measured on Longshot (main against this PR, at named commits) before it is marked ready.
Description moved to the template's Testing section and whole checklist (#72): what was tested, what wasn't, and what needs other testers (their test requests to open while this is WIP).
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.