WIP: feat(host): switches that change during a run: set and apply them through the control bridge, and HOS_AB any of them #2185
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2185
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:feat/switch-store"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Every switch was read once, as the game starts, so trying another value meant quitting, setting it and starting again. That costs a boot, a save load and the scene's warm-up for each value, and runs on either side of a restart differ by more than many changes are worth. This makes 16 of the host's switches change during a run. Each is decided again at every use, so a change between two uses leaves nothing the old value built:
HOS_SPIN_US,HOS_SPIN_OTHER_US,HOS_SHORT_SLEEP_TIMER,HOS_SHORT_SLEEP_POLLERS,HOS_POLL_SLEEP_NS;HOS_VCORE_SLICE_US,_STARVE_US,_ESCAPE_US,_ESCAPE_EQ_US;HOS_PRESENT_NOWAIT,HOS_PRESENT_POLL,HOS_NVN_SYNC_BLOCK,HOS_NO_PACING,HOS_VI_DOUBLE,HOS_GAME_INTERVAL,HOS_VSYNC_YIELD.How it works:
host/switch_store.cholds their text, taken from the environment (andlongshot_env.txt) before the game's threads start, until an apply gives one another.hos_sw_due,hos_sw_seen): a use costs one more load, and no read takes a lock.switch_setgives a switch a value for the next apply;switch_applyapplies;switch_setwith"apply": truedoes both.switch_listandswitch_getsay what each switch reads now and has pending.at_start: it takes a restart), a name that is no switch, and a value its type does not take.HOS_ABalternates a store switch by setting and applying its value at each change of side, with no reader askinghos_ab_value. So any of the 16 can be A/B'd in one run.This is the first step toward testing many flags in one run: a sweep sets and applies values through the bridge, with no restart between them. The list is
gpu/vk/switches_live.def. Seven more host switches the audit found safe to change during a run wait for their checks to be rewritten: those checks hold them to one read on purpose (HOS_SPIN_ALWAYS,HOS_CV_SIGNAL_ALWAYS,HOS_MW_WAKE,HOS_NO_HANDOFF,HOS_NO_OBJ_CACHE,HOS_COLLECT_VECTOR,HOS_TEXELS_KEEP_CORE).HOS_SYNC_TABLE_LOCKstays read-once: it is read undermw_lockinWaitAny's scan (tests/test_multiwait_wake.py).On main:
switch_setis an unknown command, andHOS_AB=HOS_SPIN_US=0alternates nothing; its end line says "HOS_SPIN_US was never read during the A/B".Read first:
host/switch_store.h(the pattern a reader follows),host/switch_store.c, the four commands inhost/control.c,hos_ab_tickinhost/hos_ab.c; then one converted reader,hos_spin_nsinhost/hos_thread.c.Testing
Tested: Windows 11, RTX 3080 (NVIDIA 621.14), from the
kokiri-startsave with Skip title screen, through the PC's job queue. The runtime was this branch's pack over main971f77f52's base (main since then changed five lines ofnx2/build.py, none of this).switch_listwithlivegave the 16;switch_get HOS_SPIN_USgavevalue: null(unset);switch_set HOS_AUDIO 0was refused,at_start: "HOS_AUDIO is read once, as the game starts: set it in the environment or longshot_env.txt and start again";switch_set HOS_SPIN_US abcwas refused,bad_request: "… is no number";switch_set HOS_SPIN_US 0left it pending (switch_get:value: null, pending: "0"), andswitch_applyapplied it (changed: 1, generation 2; the log:switches: applied (2): HOS_SPIN_US=0);HOS_VI_DOUBLEwithapply: true, both unset again withunset: true, and an apply with nothing pending (changed: 0).HOS_AB=HOS_SPIN_US=0,3,20in the game: 12 applies, one at each change of side after the warm-up. The end line:side 0 (normal) 1648 frames, 10.299 ms a frame (97.1 fps); side 1 (the A/B's value) 1481 frames, 10.984 ms a frame (91.0 fps); side 1 +6.66 % frame time. In Link's house, uncapped: no spin before a wait sleeps costs frame time here.tests/test_switch_store.py: every read of a listed switch goes through the store. It compiles the store with a fixture: set, apply, the same text again, unset, a number in another base, the three refusals, a path not said in the log, every row listed, and four threads reading through 2,000 applies. Three broken copies of the store fail it.test_control: the four commands over the real transport.test_hos_ab: the A/B sets and applies a store switch at each change of side, and nothing for another.tests/switch_store_stub.h):test_short_sleep_timer,test_spin_by_role,test_pacing_policy,test_frame_rate_cap_30,test_frame_rate_cap_step,test_threaded_rendering,test_vcore_escape,test_vcore_mask0.tests/run_all.py --changed, 131): every one that failed also failed on main1101b6127. Main's #2171 has since repaired many of its checks,test_vcore_escapeandtest_vcore_mask0among them; both cut the converted virtual-core block, and now take the store's stand-in too (ed6ce62df). Rebased onto maina2dc53859; nothing has run on this head, and the whole suite not at all.Still to test:
HOS_CONTROLset, sendswitch_setwithnameHOS_SPIN_US,value0 andapplytrue over the bridge, expecting the build to pass, the reply"changed":1,"applied":"HOS_SPIN_US=0"and the log lineswitches: applied (2): HOS_SPIN_US=0(@Beanstalk)Measurements
Verdict: no cost with nothing set. The PR's undisturbed runs are at or above main's in both series: 71.8 and 71.6 fps against 71.1, and 67.9 against 65.3.
Setup: Windows 11, RTX 3080 (NVIDIA 621.14).
dev.py ab --runs 2, this branch's pack against its base, main1101b6127, in two series of four interleaved timing jobs (A1, B1, A2, B2). Hyrule Field, child, the far view; 60 s measured after the warm-up, uncapped. Three of the eight runs are left out, each spoiled by another session's runtime build running beside it and preparing shaders on the GPU:rt-pr1937, 16:41-16:49, during series 1's B1 (27.9 fps) and A2 (29.1 fps);rt-1937b, 17:00-17:02, during series 2's A1 (41.1 fps). Such a build is a light job, which the queue lets start beside a timing one.1101b6127d9e16a38cThe 16 switches ship unset, as on main: their defaults are unchanged.
Risks
tests/test_switch_store.pyholds its reads to the store.<a path>.switch_set; reverting is enough.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). The 131 checks that read the changed files ran (Testing); the whole suite has not.main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Tested in game: Windows 11, RTX 3080, NVIDIA 621.14, from the kokiri-start save. Through the control bridge in the game: switch_list (the 16), switch_set refused for HOS_AUDIO (at_start) and for a value that is no number, HOS_SPIN_US=0 pending then applied (switches: applied (2)), HOS_VI_DOUBLE applied, both unset again; the game ran on. HOS_AB=HOS_SPIN_US=0,3,20: 12 applies, side 1 +6.66 % frame time in Link's house uncapped. dev.py ab against main, Hyrule Field: the undisturbed runs 71.8 and 71.6 fps against 71.1.
d9e16a38c8ed6ce62dfeRead of
ed6ce62don maina2dc5385(three commits: the store and the commands,"unset": true, the two vcore checks' stand-in). The store's write/read protocol and the live boundary hold up. One concrete mismatch between what the bridge accepts and what the readers actually parse is the finding.1. The store's int rule is not the readers' rule.
hos_sw_intandvalue_fitsparse withstrtol(v, &end, 0)(host/switch_store.h:57-64,host/switch_store.c:106-118), soswitch_set HOS_SPIN_US 0x10is accepted, applied and reported back asHOS_SPIN_US=0x10. Every converted numeric reader parses in decimal, though:hos_spin_nsusesatoll(host/hos_thread.c:727),vc_sw_nsatoll(:2044-2051),hos_sleep_logged's poll capatoll(:1453-1456), andhos_short_sleep_pollersstrtol(asked, &end, 10)with the whole string (:1231). So0x10reaches the game as 0 for spin/vcore/poll (and the pollers fall back to the processor rule, a third meaning), whileswitch_getsays0x10andtests/test_switch_store.py:114asserts the other meaning — its fixture reads throughhos_sw_int, which no live reader calls. That is the silent-misread class this feature exists to remove, now on the bridge's side, and a tester's hex value becomes a measurement of 0. The same applies to"010"(octal 8 vs decimal 10). Fix: validate invalue_fitswith the readers' rule (base 10 and the whole string, ashos_short_sleep_pollersdoes), or convert the readers tohos_sw_intand say in the row that the env text now takes any base (a behaviour change), or refuse a base prefix; then the test's0x10case matches. (hos_sw_intalso returns the default for an empty text whereatoll("")gives 0 — only the helper, not a live reader.)2. The store writes and the live boundary. One SRW lock serialises
hos_sw_set/hos_sw_apply/hos_sw_pending/hos_sw_each;now_vis an atomically loaded pointer whose text is never freed, and the generation is bumped with release after the texts, so a reader that sees the new generation sees the new text (switch_store.c:148-173,switch_store.h:26-38).hos_sw_initruns afterlongshot_env.txt(hos_main.c:2139before:2446) and beforehos_ab_start, and the lazytake_envseeds a switch under the lock for a read before init. A reader'shos_sw_due/seencan at worst work a value out twice around an apply, never keep a value for a generation it was not worked out at. I checked each of the 16: all recompute inside the due block and none keeps a derived value outside it (hos_thread.c:727, 1292, 1453, 1501, 2044;nn_vi.c:51, 225, 248-250;nvn_native.c:1187, 1241), and I agree with the listed ones that a change between two uses leaves nothing the old value built (theHOS_VI_DOUBLE/HOS_NO_PACINGpair only decides the wait and the pacing of the present in flight; the short-sleep mode is decided per sleep).HOS_AB's live path sets and applies under the store's lock with side 0 from the store's text at setup (hos_ab.c:141-144, 161), and the source scan forbidshos_ab_valueon a live name.3. The ini keys. None: these are environment/
longshot_env.txtkeys, notlongshot_options.inirows, sotests/test_menu_ini_keys.pyand the saved options are untouched. All 16 areswitches.defrows and the list is sorted (switches_live.def; the store test checks both). The bridge's JSON keys (name,value,apply,unset,live) matchdocs/dev/reference/control-bridge.mdandtests/control_test.c;HOS_AB'sNAME=value[,value...]syntax is unchanged. Theswitch_set/switch_getreplies redact a value that names a folder as<a path>inappliedas well as the log, which is the host's rule.4. The test. It fails on main (the store files do not exist), and it covers the store's set→pending→apply→unset, the same text changing nothing, the three refusals, the path redaction,
hos_sw_eachover every row, four threads reading through the pattern while 2000 applies run, and three broken copies. The source scan holds every literal read of a live name to the store and forbidshos_ab_valueon one. Two gaps: it does not enforce thehos_sw_due/hos_sw_seenpattern at each reader (a reader that read the store once, without a due check, would pass the scan and never change), and its numeric rule is the helper's, not the readers' (finding 1). The converted readers' own checks includeswitch_store_stub.h, whosehos_sw_envisgetenvand whose generation never moves, so they exercise main's behaviour, not the live apply — which is fine while the store has its own test, but it is why 1 slipped through.5. The default. The 16 ship unset and the readers' rules/defaults are unchanged as far as I read them:
HOS_SPIN_US20,HOS_SPIN_OTHER_USunset → the 5 us rule,HOS_VI_DOUBLE/HOS_NO_PACINGoff,HOS_GAME_INTERVALoff,HOS_PRESENT_NOWAITunset → the renderer's option,HOS_SHORT_SLEEP_TIMERunset → the processor rule (Linux: the timer),HOS_SHORT_SLEEP_POLLERSunset → the thread count.switch_getreportingnullwhile the reader uses its own default is correct but worth one line in the docs.6. Process. WIP, 0 reviews. The Measurements are against main
1101b6127and the headd9e16a38c; the PR is now ona2dc5385(the status warning stands), and Testing says nothing has run on this head. The Linux bridge run in Still to test is the one that matters for the new files.No approval, and no claim on it.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.