fix(host): the OptimizeChangeBank hold is keyed to the bank's identity and checks the guest layout (follow-up to #1814) #1870
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1870
Loading…
Reference in a new issue
No description provided.
Delete branch "GummiNaomi/Longshot:fix/ocb-hold-review"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
A follow-up to #1814 (
HOS_OCB_HOLD_MS), which was merged while futondestroyer's review asking for changes was still open(its review of
48b45e7b, #1814). It fixes the review's threepoints, and a fourth case ballotstomper found on Linux (4. below); the hold itself, its default (2000 ms) and the patched
line are unchanged.
when the map changed and the same slot was filled with another neighbour, that neighbour could be taken on the old
one's time (on main: chosen at 1.6 s after an earlier occupant had been chosen from 0, it is taken at 2.0 s instead of
3.6 s). A choice is now the manager, the unit and the map in it (an FNV-1a of the unit's own path), seen by every
update with no gap over 500 ms; a change of any of these, or a pause in the updates (a loading screen, a freeze),
starts its time again.
+0x1e8, each unit's +0x10) wherever they pointed. The constructor (
f_80d32c00) points each of them at a 0x100-bytebuffer inside the manager, and nothing repoints them, so the host now checks exactly that before each read: the
vtable (0x84bc1a28: the build and the class), each path pointer equal to its own buffer, the buffer sizes, the current
length, a state the game has (0-7), in-use flags of 0 or 1, and the chosen unit in use. The strings are then read from
those buffers only: every read stays inside the manager the update is running on, and no pointer the game could have
freed is followed. Any mismatch gives the game's own choice at that update (counted; logged once). The patch is also
made only in the supported build's lift (
patch_file's supported flag, like the other build-specific patches), and thehost compares main's build id with
e0f3ccfa...once; another build gets the game's own choice.HOS_OCB_HOLD_MS=-5gave the 2000 ms default, although #1814 documented a negativevalue as off. A negative whole number is now 0 (the game's own), like
0andoff; malformed values (-,-x,12x)keep the default; the switch is read once whatever its value (no "not read yet" value it could also take).
ballotstomper, in their comment of 21 Nov on this PR; Linux, RX 6750 XT). Hyrule Field lists Gerudo Valley in two
units: unit 1 (the Lake Hylia side, LoadRadius 30) and unit 4 (the valley's gate, LoadRadius 240). Inside unit 4's
radius the update chooses 4 and asks for the valley; at the next update
oot_ocb_heldresolved the held map to thefirst in-use unit naming it, unit 1, so
oot_ocb_pickheld choice 4 back for unit 1, and the update's release ofthe units it did not choose (
f_832ba400, by path) cancelled the load it had just asked for. The valley's ~250 fileswere opened and decompressed again every ~76 ms while Link stayed in the radius; main has the same lookup.
oot_ocb_heldnow takes the update's choice: when that unit is in use and names the held map, it is the held unit,the same preload;
oot_ocb_steppasses it in (ballotstomper's fix). A choice is held back only when it names anothermap, which is what the hold is for.
held map whose
+1is set (asked for:f_832b9880sets it whenf_832ba540succeeds and clears it when it lets theunit go), else the first one as before. Without it, a choice flickering between unit 4 and another
neighbour's unit went on with unit 1 while it was held back, and the release of unit 4 cancelled the valley once.
Rebased onto main @
6e1521a7(head1727599a; before it,a106df16atf066ea85). No conflicts this time. Point 4 is onenew commit on top, crediting ballotstomper.
Read first:
host/oot_ocb_hold.h(oot_ocb_pick,oot_ocb_layout,oot_ocb_held,oot_ocb_key,oot_ocb_step,oot_ocb_hold_ms), thenhost/oot_ocb_hold.cand the gate innx2/oot_patches.pypatch_file. See #1814.Testing
Tested: Windows 11 Pro 26200, RTX 4070 SUPER (NVIDIA 617.42), Ryzen 9 9950X3D with SMT off, the game held to 4
cores (affinity 0xF00), points 1-3 at
7afc46f8(their code unchanged since): seven fountain-basin runs against main (below), one run withHOS_OCB_HOLD_MS=-1, and six area changes plus a walk to the market's exit on each side. In every run the hold's logline is there and the "not laid out" line is not: the layout check accepts the game's own manager (a run long enough to
log the counters says
39 choices of another neighbour held back, 4 taken, 0 updates refused, in 18000 updates). Areachanges: 6 of 6 arrived on both, 4.9 / 4.1 / 2.8 / 4.5 / 3.1 / 2.8 s here against 4.9 / 3.0 / 2.8 / 4.5 / 3.1 / 2.8 s on
main (the second, the first arrival in Hyrule Field, is the only one apart); the walk south ends at the same spot and the
same "Hyrule Castle Town" card on both.
Point 4 in the game here, at this head (
1727599a) against main @6e1521a7, interleaved (main, PR, PR, main) plus one mainrun with the hold off: the debug teleport into Gerudo Valley, Link walked back towards the field (four stick presses of
225 frames at a 61 fps cap, the same spot every run), then standing 60 s. Gerudo Valley's map must list Hyrule Field in
two units too (the change only does anything where two units name the held map), and there main cancels and reloads the field again and again: its layout opened 561 and 541 times, against 2
with this PR and 2 on main with
HOS_OCB_HOLD_MS=0(so the loop is the hold's lookup, not the game's). Numbers underMeasurements. ballotstomper measured their fix (the first bullet of 4.) on Linux on #1814's code: also there.
Checks:
tests/test_ocb_hold.pypasses on Windows (clang) and Linux (WSL Ubuntu 24.04, gcc 13.3) at this head. Its casesfor points 1-3 fail on main for the review's reasons: run against main's header, the same-slot refill is taken at 2000 ms
instead of 3600 ms, the held unit is read through a pointer outside the manager (and held), and
-1,-5,-2000give2000 ms (at
7afc46f8). Point 4's cases are ballotstomper's (two in-use units naming one map, that map loading:oot_ocb_heldgives 4 with choice 4 and 1 with none,oot_ocb_stepkeeps 4), the+1fallback, and a model of the restof the update (it asks for the unit it goes on with and lets go of the other units asked for, by path): 10 s standing in
unit 4's radius, then 10 s on the edge of another neighbour's radius, 0 loads of the valley cancelled. Run against the
previous head's lookup (main's: the first unit naming the map), they fail: 312 and 313 cancels,
oot_ocb_held1 for choice4; with ballotstomper's rule alone, the edge case still cancels once.
test_switch_table(--cxx clang++) andtest_doc_anchorspass.run_all.py --changedat this head ran 72 checks (those that read the changed files): 32 pass, 26 areskipped (no gcc or Vulkan headers on this PC) and 14 fail, each the same way on main @
6e1521a7(test_bc_small_scratch,test_crash_dialog, test_dev_ab, test_dev_packs, test_frame_overlap, test_frame_whole, test_high_fps_fixes, test_lift_cache,
test_linux_pressure, test_oot_cloth_60, test_oot_cloth_time, test_quick_pipelines, test_texture_ahead,
test_water_substeps: MSVC headers under
-Werror,HOS_HEAP_BASE, an unclassified FSR 4 file, main's own). The host filewas built by the builder's Windows (llvm-mingw) build of this head.
Still to test:
HOS_OCB_HOLD_MS=0,expecting fewer long frames with the hold and no
ocb: the preload manager ... is not laid outline in the log(@Beanstalk steamdeck, the market fountain circles with HOS_OCB_HOLD_MS=0 against unset)
fountain circles)
Test Farm)
the valley's side, expecting Gerudo Valley's files opened once (by hand,
HOS_FS_TRACE=1)Measurements
Verdict: the hold works as on main: in the 4-core fountain basin 70.4 fps against main's 71.1 (-1.0 %, within the
runs' spread: 69.6-71.7 here, 70.5-71.5 on main), the loader threads 6.4-8.2 % of a core against 5.0-7.4 %, and 43 % with
the hold off (
HOS_OCB_HOLD_MS=-1, now off as documented: 56.5 fps). Area changes as on main. Nothing to gain wasexpected from points 1-3: they are correctness fixes. Point 4 is a real gain where a map has two units: ballotstomper's
ranch walk (Linux, RX 6750 XT, #1814's code with their fix) went from 39.3 / 36.9 / 37.0 fps to 54.8 / 47.6 / 55.8, p99
from 67-160 ms to 27-82 ms, game time a frame from ~21 ms to 15-17 ms; a watch on the manager showed the valley's load
reaching state 4 and staying. Here, the same case from Gerudo Valley's side (Testing), 60 s standing at a 61 fps cap
(so fps cannot rise; the long frames, the game's time a frame and the loaders' CPU show it):
6e1521a7(two runs)1727599a(two runs)HOS_OCB_HOLD_MS=0: main @6e1521a7(one run)Setup: as above (this PC, all 16 cores this time), 1920x1080 headless (the control tool's test profile),
RX_VK_FRAME_TRACE=1 HOS_FPS_MAX=61 HOS_FS_TRACE=1 HOS_THREAD_TIMES=5, thekokiri-startsave. Loaders: ResourceMemory, ARes Loading 0/1 and the ParallelDecompThreads. Standing at the valley's arrival spot without the walk, main
opens the field's layout twice too: the loop needs Link inside both units' radii.
Setup: Windows 11 Pro 26200, RTX 4070 SUPER 12 GB (NVIDIA 617.42), Ryzen 9 9950X3D SMT off, 32 GB, the game held to 4
cores (process affinity 0xF00), as #1814 measured it. Castle Town's market (child) from the
kokiri-startsave: fourcycles of teleport, a walk into the fountain's basin and six small circles, the circles' frames only; 2560x1440, uncapped
(
HOS_FPS_MAX=1000), headless,RX_VK_FRAME_TRACE=1 HOS_THREAD_TIMES=5and #1814'sRX_VK_ASTC_BC_ALL=1 RX_VK_IMAGE_BLOCK_MB=32 RX_VK_LOAD=create RX_VK_BC_LAYERS=8 RX_VK_LATE_FIX=0. Two interleaved rounds(main, PR, PR, main), the executables swapped in one game folder; one main run of the first round never got past its
first teleport ("an event is playing") and was run again. The difference of the means: a bootstrap over 3 s blocks.
Loaders: ARes Loading, OptimizeChangeBanc, ParallelDecompThread and Resource Memory from the 5 s thread-time lines.
8f462db57afc46f8HOS_OCB_HOLD_MS=-1: this PR @7afc46f8The switch's default stays 2000 ms, as #1814 measured it; only what a negative value means changes (off, as documented).
The block bootstrap puts
7afc46f8up to 0.27 ms a frame behind main, but the runs overlap (the fastest run of all isthis PR's); the per-update work added is a few hundred byte and word loads inside the manager (its layout and one path's hash),
microseconds a frame. Not measured: 16 cores, the Steam Deck, Linux.
Risks
What could break: if the game's manager were ever laid out otherwise than the check expects, the hold would quietly do
nothing (the game's own choice at every frame, as with
HOS_OCB_HOLD_MS=0) and the log would say so once(
ocb: the preload manager at ... is not laid out as host/oot_ocb_hold.h knows it); on this build the check passes (0refused). A neighbour that comes back into the same slot after a pause now waits its full hold again: at most 2 s more
before its preload starts, as with any new choice. Point 4: a choice of another unit naming the held map is no longer held
back at all (it is the same preload; holding it was never the point), and a choice held back goes on with the unit that
asked for the held map; if the
+1byte ever meant something else, it would pick another unit naming the same map, neveranother map. Turning it off:
HOS_OCB_HOLD_MS=0,offor now any negative number.After updating: the lifted code is unchanged, so no new translation; nothing else changes for players.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). (The checks that read the changed files, on Windows;test_ocb_holdalso on Linux; the failures are main's own here.)main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here.gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
@Beanstalk steamdeck and linux: Castle Town's market (child) from kokiri-start, walk into the fountain's basin and circle for a minute, with
HOS_OCB_HOLD_MSunset againstHOS_OCB_HOLD_MS=0,RX_VK_FRAME_TRACE=1 HOS_THREAD_TIMES=5. Expecting fewer long frames and less loader-thread CPU with the hold (unset), the hold's log line, and noocb: the preload manager ... is not laid outline.(Earlier runs on #1846 and #1847 today ended in
errorbefore building: the farm's checkout of main failedgit rev-parse --verify HEAD^{commit}in packs.py's cmd_base, so nothing was measured there.)@GummiNaomi approval of
7afc46frecorded. This diff touches sensitive paths, so a second member ofBigBeansmust approve it too.Beanstalk test farm
Commit
1727599· approved by nobody yet · dashboardA member of
BigBeanscan review this commit and comment@Beanstalkto test it.Results look worst?
7afc46f877f066ea8578@HumanBean slightly, yes, by about 1 %: 70.4 against 71.1 fps, +0.14 ms a frame. That is within the runs' spread, though (main 70.5-71.5 fps, this PR 69.6-71.7), and the single fastest run of all was one of this PR's. The extra work is a few reads of the manager once a frame, so a real 0.14 ms cost is unlikely.
This PR is not meant to be faster. It fixes the three points of futondestroyer's review of #1814, which merged while that review was open:
HOS_OCB_HOLD_MS, documented as off, gave the default.Rebased onto main @
a106df16(only aswitches.defrow conflicted); its checks pass again. The farm needs a second BigBeans approval, since it touchesnx2/oot_patches.py.A case the hold still gets wrong on this head, found on Lon Lon Ranch (Linux, RX 6750 XT).
What happens. Hyrule Field's map lists Gerudo Valley in two units: unit 1 (the Lake Hylia side, LoadRadius 30) and unit 4 (the valley's gate, LoadRadius 240). Inside unit 4's radius:
GerudoValley_Child(f_832ba540); unit 4's asked-for byte (+1) is set and the load starts.oot_ocb_heldresolves the held map to a unit by path and returns the first in-use unit naming it: unit 1.oot_ocb_pickcompares by unit index, so the choice 4 is held back: the update goes on with unit 1. Asking for it does nothing, since the manager's current map is already that one.f_832ba400releases by path, so it cancels the load just asked for: state 2/3 → 5 → 0, the current path emptied.The map's ~250 files (layouts, 39 actor packs, 177 models, 32 textures) are opened and decompressed again every ~76 ms for as long as Link stays in the radius. A watch on the manager at one spot showed state 0 in 403 updates, 2–3 in 369, and loaded once in 811, with no unit left asked for. The fountain basin's maps have one unit each, so it doesn't show there.
The fix.
oot_ocb_heldtakes the update's choice as well. When that unit is in use and its own path is the held map's, it is the held unit; otherwise it is the first unit naming the map, as now.oot_ocb_steppasseschosenin.So a choice is held back only when it names another map than the one held, which is what the hold is for. A unit naming the same map is the same preload, and the release by path can no longer cancel it.
Measured on #1814's code, which has the same lookup, with the fix: the load reaches state 4 and stays. A recorded walk through the ranch goes from 39.3 / 36.9 / 37.0 fps to 54.8 / 47.6 / 55.8 (p99 67–160 ms → 27–82 ms; game time a frame ~21 ms → 15–17).
A test for it: two in-use units naming one map with that map loading;
oot_ocb_heldwith choice 4 returns 4, with no choice 1, andoot_ocb_stepkeeps 4. I can port both onto this head if you'd like them in this PR.f066ea85781727599a3a@ballotstomper thank you: a clear report, the cause traced through to the release by path, numbers, and the fix. It is in this PR now, so there is no need to port it.
What changed (
1727599a, on main @6e1521a7; the commit credits you for the report and the fix's design):oot_ocb_heldtakes the update's choice: when that unit is in use and names the held map, it is the held unit;oot_ocb_steppasseschosenin. A choice is held back only when it names another map.+1is set (the update sets it whenf_832ba540succeeds and clears it when it lets the unit go), else the first one as before. Without that, a choice flickering between unit 4 and another neighbour's unit went on with unit 1 while it was held back, and the release of unit 4 cancelled the valley once. Where your rule gives the choice, the result is the same.How it was tested: your test as you described it (two in-use units naming one map, that map loading:
oot_ocb_heldgives 4 with choice 4 and 1 with no choice;oot_ocb_stepkeeps 4), the+1fallback, and a small model of the rest of the update (it asks for the unit it goes on with and lets go of the other units asked for, by path): 10 s in unit 4's radius, then 10 s on the edge of another neighbour's radius. At this head the valley's load is never cancelled. With the previous lookup it is cancelled 312 and 313 times, and with the choice rule alone once on the edge.test_ocb_holdpasses on Windows (clang) and Linux (gcc). I haven't run it in the game on Windows yet (the test PC is busy with other runs). Your ranch numbers are in the description, credited to you.@ballotstomper the in-game check on Windows (RTX 4070 SUPER, 9950X3D) is done, at
1727599aagainst main @6e1521a7(main, PR, PR, main). I found the same loop from the other side: Gerudo Valley's map must list Hyrule Field twice too. I teleported into the valley, walked Link back towards the field to the same spot each run (61 fps cap), then stood 60 s:HOS_OCB_HOLD_MS=0also gave 2, so the loop comes from the hold's lookup, not the game.The table is in the description. I haven't checked the field side (inside unit 4's radius) here; it's listed under Still to test.