perf(host): FreeAddressRegion gives its virtual core up while the region's pages are discarded #1646
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
6 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1646
Loading…
Reference in a new issue
No description provided.
Delete branch "Jiro113/Longshot:perf/vcore-improvements"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Windows 11, RTX 4090 · Needs: Linux, any GPU
TL;DR
When the game frees a large block of memory, the runtime hands its pages back to the system, for milliseconds, and the
freeing thread kept its virtual core throughout, holding up the game's threads waiting for it. Now it gives the core
up for that work. Against current main, in 3 runs a side in Castle Town's market, frames are the same (within the
spread); waits for a core of 50 ms or more fell from 287 to 19 (medians). On an older main (
c507b98c) the change tookaway waits of 300 ms or more and raised the 1 % low from 29 to 35 fps; main no longer has those waits.
Breakdown
loader threads wait less for a core there. On main
c507b98c, anyone walking around the market (hitches).region's pages back to the system, so the game's threads on that core were held up for the whole of it.
after; the region stays reserved until its pages are handed back.
HOS_KEEP_FREED_PAGESas before), saves,options and caches.
868f6be8, interleaved after a warm-up of each(medians): frames within the spread (mean frame time 18.39 against 18.54 ms, 1 % low 24.3 against 23.8 fps, the
same-seed pairs going either way); windows with a wait for a core of 300 ms or more, none on either side; waits of 50
ms or more for a core, every thread's, 287 on main, 19 here. A game-free test fails on main for the bug's reason and
passes here. The rebased tree ran in the game: the four runs of this PR in the series (a warm-up and three measured,
300 s each in the market), no error or hang.
before the rebase (GrimVisage's comment, an RX 9070 XT); Intel GPUs and the Steam Deck are not run.
bookkeeping keeps it from being handed out again during the hand-back. The freeing thread then waits to take a core
back, the kind of wait #1632 found freezing loads on an Arc iGPU for a decode that gave its core up at every call;
on main since #1339 that is no new wait, as the renderer's import check before the hand-back already takes one back
for each region freed. In every run of both series, on both sides, that thread never waited 50 ms or more for a core,
but once (124 ms, in a load before the market, in the first series) (Risks).
Why
In Castle Town's market, walking in its south part, the
vcores:log line shows game threads waiting for a virtualcore; on main
c507b98c(the first series here) they waited 0.3 to 0.9 s, and the frame rate dipped in hitches. Most ofthose long waits were for core 1. The game's resource memory thread (
Resource Memory, pinned to core 1) frees memoryregions as it streams resources there; each free hands the region's pages back to the system (
DiscardVirtualMemoryon Windows,
MADV_REMOVEon Linux), milliseconds for a large region, and the thread kept core 1 throughout. The game'score-1 job workers (
ModuleSystemWorker1,BaseProcExecutor1,NavMeshCAStepDepWorker1), of a higher priority thanthe freeing thread, could not have the core until it was done, and the main thread waited for them to finish their
jobs. A profiled run on main
c507b98cput most of the resource memory thread's extra work in slow frames into thathand-back, with core 1 held.
See #1150 (merged): it keeps the core for an invalidate that has nothing to wait for, where giving it up cost more than
it saved; this is the other side, a host task long enough that keeping the core holds up the game. See #327 (open: mask
0 and the guest's core cache) and #249 (closed: named threads first) for the other work on the virtual cores: neither
touches this path. See #1553 (merged): it gives the core up around the recording diagnostics and large decodes, for
waits of the same kind in the market; both series here are on a main that has it. See #1543 (merged): its Tracy zones
found a market frame's main thread spending about 10 of 18.6 ms in one job-loop wait. See #1503: the virtual-core gate
lets a low-priority thread wait up to a second for a core (
HOS_VCORE_STARVE_US); this PR does not change the gate,it takes away the host work that kept core 1 busy. See #1542 (open): one of its commits makes taking and giving up a
core cheaper (part of #1503); it doesn't touch
host/hos_mem.cand keepshos_vcore_away/hos_vcore_back, so the twomerge in either order. See #1632 (merged; on main in the second series): on an Arc iGPU, loads froze while the native
zstd decode, which gave its core up around every call, waited to take one back; the decode now keeps its core. This PR
gives the core up once for each region freed, for milliseconds of work; Risks has the waits measured. See #1020: its
leads find the market's frame waiting on the main thread, which waits on its job workers.
See #1610:
hos_region_freetook the region off the list before handing its pages back, so an allocation meanwhilecould be given pages about to be wiped. Once the hand-back no longer holds the core, the window is wider, so this PR
does what #1610 proposes in its items 1 to 3: the region stays listed, marked as being freed, until its pages are
handed back, and a second free of it meanwhile does nothing (item 4 is #432's).
See #1339 (merged; on main in the second series): before the hand-back, the runtime now asks the renderer, on Windows
too (Linux already did), whether it imported the pages, and zeroes imported pages in place instead of handing them
back. Asking gives the core up and takes one back, once for each region freed. With this PR the thread gives its core
up before the question and takes one back once, after the hand-back or the zeroing: the question's own give-up does
nothing for a thread that holds no core. On main the hand-back itself still runs with the core held. Main no longer has
the market's waits of 300 ms or more (Measurements); which of the changes merged since
c507b98ctook them away is notestablished.
Players' logs with the same waits: #1436 (Windows, rc.7, an i5-12400 with 6 cores) has 31
vcores:lines with a wait of300 ms or more for a core, 23 of them after it reaches the market, many at the one-second limit; #996 (Linux, rc.4) has
waits of 0.5 to 1 s in the market in 9 of its 196
vcores:lines. Both are older than #1339 and #1576; whether currentmain still has those waits on those PCs, and whether this PR changes them there, is not measured. #597 (rough frame
times in the market on a Ryzen 7 9800X3D) has no log yet. See #1576 (merged; on main in the second series): it makes
the game's own work a frame cheaper in the same market. The two touch different files.
What changed
hos_region_free(nn::os::FreeAddressRegion) used to take the region off its list, then hand its pages back with thethread's virtual core held. Now it gives the core up for the hand-back, as a file read does, and takes the region off
the list only once the hand-back is done, so a region allocated meanwhile cannot be given pages that are being handed
back.
hos_region_freeinhost/hos_mem.c: under the lock, finds the region, marks it as being freed and marks itspages; lets the lock go,
hos_vcore_away(),discard_pages(); then removes the region under the lock andhos_vcore_back()if the thread had a core. An address that is no region, or a region already being freed, returnsat once, keeping the core. A region being freed stays in the list, so
hos_region_allocskips it and the controlbridge's range checks (
hos_mem_region,hos_regions) still read it as mapped;hos_region_allocwrites its newentry unmarked. The comments in
hos_region_freeandhos_mem_regionsay why only its ownhos_vcore_backtakes acore back, and that a region being freed still reads as mapped.
tests/test_vcore_region_free.pycompiles the actualhos_region_freeandhos_region_allocwith stand-insand checks that the hand-back runs once, without the core and outside the lock, with the region still listed; that
the region then leaves the list with the others kept in order and the core taken back once; that an unknown address
keeps the core and a thread holding no core takes none; that a second free of the region during its hand-back does
nothing and leaves it listed; that a region of no size leaves the list; and that a region allocated while another
is being freed can itself be freed. On main 7 of its checks fail: "the discard runs without the virtual core", "the
region stays listed through its discard (no allocation is handed its pages)", "the core is held again after it,
taken once, and the lock let go", "an allocation while the region is discarded is not handed its pages (#1610)",
"after the discard the region allocated meanwhile stays listed and the freed one is gone", "a second free of the
region while it is discarded leaves it listed (its address is not handed out again yet)" and "after both frees the
core is held again, taken once, and the lock let go".
tests/test_host_free_marks.py(from #1339) compileshost/hos_mem.cwith stand-ins of its own, which sincethe rebase have to include
hos_vcore_awayandhos_vcore_back; its fixture now stands in for them, counting thecores given up, and checks after each region free that every core given up was taken back. Its other checks are
unchanged.
Measurements
Verdict: no change in frames against current main. On main
868f6be8every frame row is within the spread (mean andmedian frame time, p99, 1 % low, frames of 33 and of 50 ms or more), and the same-seed pairs go either way. Neither
side had a 10-s window with a wait of 300 ms or more for a virtual core. What this PR still changes is the waits for
a core: the longest wait in a window fell from 177 to 45 ms (medians), the waits of 50 ms or more, every thread's, from
287 to 19, and the time all threads spent waiting for a core by 13 to 17 % in each same-seed pair. The long waits left
on main are low-priority loader threads (
tera::FileResourceMgr,PreActorModelCompose,aglTextureDataMgr::Compaction: 275 of the 287 in main's seed-55 run), never the main thread, which is why the framesdon't move.
Hardware: Windows 11 Home 10.0.26300, RTX 4090 24 GB (NVIDIA 621.24), Core i9-13900K (32 logical processors), 64 GB
RAM.
Scene: Castle Town's market as a child, its south part, moving; output 2560x1440 in a window (shown, not in front),
HDR10, NVIDIA DLAA (2560x1440 to 2560x1440), DLSS Frame Generation 2x, V-Sync Auto (FIFO), capped at 60 by
HOS_FPS_MAX=60. One options file for every run; it is not Auto's pick (Auto on this PC would choose DLSS Quality).The numbers count the game's own frames, not the made ones.
Method: a scripted route through the control bridge (the same seed on both sides) walking the south market for 300 s
after arriving there, 29 windows of about 10 s. Per run: the windows with a virtual-core wait of 300 ms or more and the
longest wait of each window (the
vcores:lines), the waits of 50 ms or more for a core, every thread's, in thewindows (
HOS_PROFILE=1records each one with its thread and the thread that gave the core), the time all threadsspent waiting for a core, the frames of 50 ms or more (a count) and of 33 ms or more (a minute), the 1 % low (1000 over
the mean of the slowest 1 % of frame times), p99, the mean and median frame time. The same switches on both sides:
HOS_PROFILE=1with its sampler held off,RX_VK_PASS_TIMING=0,HOS_FPS_MAX=60,HOS_WINDOW_NOACTIVATE=1. Bothbuilds are packs over one base (v3.0.0,
544229a2), each from its commit, with DLSS. Main868f6be8does not build onWindows (#475's
near, fixed by #1729), so both sides carry #1729's commit44ab7f74, which touches neitherhost/hos_mem.cnor anything this PR changes: main as 15e051cb, this PR (18bf5321) as 227ceaa5; each run's log reportsits version (v3.0.0-rc.8-46-g15e051cb, v3.0.0-rc.8-51-g227ceaa5). The runs went through the PC's job queue in timing
mode. A warm-up run of each side first (seed 57, not counted: a build prepares its shaders on its first run), then
seeds 55, 56 and 58 in the order main, this PR, this PR, main, main, this PR. Main's first seed-56 run had another
agent's runtime build beside it for about 70 s (the job queue let a job that prepares shaders on every core run beside
it: 18 to 28 cores busy outside the game in 7 of its windows); it is not counted, and main's seed 56 ran again after
the other five. Outside load: 2.0 to 3.1 cores busy outside the game (each run's mean over its windows; same-seed
pairs, this PR minus main: +0.38, +0.62, −0.11). Process Lasso runs on this PC; its configuration has no rule naming
the game.
How to reproduce: a save in Castle Town's market as a child and the settings above; walk the south part (z below ~31)
for 5 minutes, then read the
vcores:lines' longest waits and the frame times.Each run by seed (55, 56, 58), then the median:
868f6be8(+44ab7f74)18bf5321(+44ab7f74)¹ In the first window. On both sides the load just before the market (3.3 s before that window) has waits of 114 to
129 ms; this PR's line in the first window reaches back over them, main's two lines there (9 and 10 ms) were written at
a moment that doesn't. Without the first window, 81 ms.
Same-seed pairs, this PR minus main (seeds 55, 56, 58): mean frame time −0.03, +0.14, −0.13 ms; 1 % low +0.02, −0.55,
−0.17 fps; frames of 22 ms or more −0.9, +0.5, −0.2 points; time waiting for a core −13, −17, −14 %.
Who waited 50 ms or more in main's seed-55 run:
tera::FileResourceMgr124 times,PreActorModelCompose98,aglTextureDataMgr::Compaction53,Porter7,BaseProcCreateAndDeleteThread5; the cores were given by the core-1workers (
NavMeshCAStepDepWorker1,BaseProcExecutor1,ModuleSystemWorker1) and the loaders themselves. In thisPR's seed-55 run the same loaders, 19 times.
The first series, on main
c507b98c(before the rebase). The same scene, settings and method on mainc507b98c,before #1576, #1339, #1632 and #1663 merged (2026-11-18; its six files in
docs/tested/are2026-11-18-vcore-region-free-jiro113-s20*.md). There the change was a gain in the hitches:c507b98c2664bb7cMain's frames are worse in the second series than in the first (frames of 33 ms or more 13 → 55 a minute, 1 % low 29
→ 24 fps; this PR's side too, 2 → 59 a minute). The two series are different sessions, not interleaved, so that
comparison is not claimed; a series on this PC earlier the same day, on main at v3.0.0-rc.8, gave main the same
numbers as the second series. The likely cause is #1632's new default,
HOS_NATIVE_ZSTD=2, in which the game's streamdecode keeps its virtual core: c507b98c's default was 1 (the core given up while it decodes), and every run of the
second series logs
HOS_NATIVE_ZSTD=2. A separate series on this PC (rc.8, the same market, profiling switches of itsown, so not comparable with the tables above) gave 54.8 fps at the default (one valid run) against 58.3 with
HOS_NATIVE_ZSTD=1(#1477 (comment)). This PR doesn'ttouch the decode. The first series ran at c507b98c's default (mode 1); whether this PR's frame gain comes back with
HOS_NATIVE_ZSTD=1on current main is not measured.Flags and settings
None.
Testing
Tested: The rebased tree in the game: this PR's four runs of the second series (
18bf5321with #1729's commit, as227ceaa5; Windows 11 Home 10.0.26300, RTX 4090, NVIDIA 621.24), a warm-up and three measured runs, each the game's
start, the load to the market and 300 s walking in it: DLSS started; no error and no hang (no
the game read no controller forline); no wait for a core of 300 ms or more in the market. The longest wait of each run, of both sides(0.49 to 0.81 s), comes during the start and the loads before the market, from a loader thread. Before the rebase, on
main
c507b98c: a 23-minute run in the game driven by the control bridge, with no profiling switch: the market (2minutes), then Hyrule Field, Kokiri Forest, Kakariko Village and the Graveyard, back and forth with 35 area changes,
then the market again (4 minutes), with no error and no hang.
tests/test_vcore_region_free.pyfails on main (7 of itschecks, as above) and passes here (Windows llvm-mingw, Linux gcc and clang).
After the rebase onto main
868f6be8, on this PC:tests/test_host_free_marks.py(#1339's) failed on the rebased tree before its fixture stood in for the core(What changed); it passes now, on Windows and Linux, and passes on main as before.
python -I -B tests/run_all.pyon Windows, main868f6be8and this PR's head (ad3c871e) each from a cleangit archive: main 735 passed, 52 failed, 64 skipped; here 736 passed, 52 failed, 64 skipped. The same 52 fail onboth, none only here; here one more passes (
test_vcore_region_free). One istest_vbytes, which runs past its600-s limit on this PC on both; the other 51:
byte_compile_tracked,test_auto_handheld,test_bug_report,test_bug_report_defaults,test_build_long_paths,test_capture_read_once,test_capture_readback_visibility,test_crash_dialog,test_crt_options,test_damage_debug_menu,test_dev_packs,test_discover,test_display_startup_failure,test_dlss_feature_size,test_draw_stream_visibility,test_failed_compile_keeps_objects,test_fsr_image_retirement,test_fsr_uniform_ownership,test_game_menu_mouse_buttons,test_guest_imported,test_host_guest_writes,test_hud_layout_row,test_job_queue,test_libc_hooks,test_lift_lookups,test_linux_preflight,test_linux_pressure,test_missing_nup,test_pads_after_menu,test_parallel_init_order,test_pending_guest_import,test_pipeline_play_cache,test_prepare_cache_save,test_present_source_crop,test_present_thread,test_profiling_mode,test_psychov35_fsr3_fg,test_readback_timeout_source,test_recorder_sync,test_replay_write_order,test_sdiv_overflow,test_slow_frames,test_song_note_cue,test_staging_burst,test_tested_configurations,test_texel_buffer_ownership,test_upscaler_rows,test_vcore_escape,test_vk_occlusion_queries,test_walk,test_walk_key. (test_tested_configurationsfails on main the same way on this PC;this PR's six new files in
docs/tested/pass its checks.)git archiveof each, this PR at72a7a581, before the comment-only18bf5321): here 690 passed, 48 failed, 114 skipped;main 688 passed, 49 failed, 114 skipped. The 48 fail on main too; main's one more is
test_perf_report, whichpasses here. The three that read
host/files and fail (test_host_guest_writes,test_libc_hooks,test_vcore_escape) fail on main with the same message on both systems.ad3c871e,test_vcore_region_free(g++ and clang++) andtest_host_free_marks(gcc and clang) pass.72a7a581) on Linux (Ubuntu 22.04 under WSL,nx2/make_game.pywith the emsdkclang and the Linux sysroot, shaders not prepared: no GPU there):
longshotandlongshot_vk.sobuilt, 0 filesfailed, no warning from the build.
44ab7f74) on top, with DLSS: main868f6be8alone doesn't build on Windows (#475 names a counter
nearingpu/vk/descriptor_array_census.h, which the Windowsheaders define as nothing; #1729 fixes it). That build ran the runs above.
Not testable by hand: the two cases the region's mark is for, an allocation and a second free of the same address
while its pages are handed back, can't be made to happen from the game on purpose; the game-free test runs both.
Not tested: the rebased tree in the game on Linux; Intel GPUs, the Steam Deck and CPUs with fewer cores (#1632's
freezes were on a Core Ultra 7 155H laptop): the change is the same code on every platform and the hand-back itself is
unchanged; while it runs, the freeing thread now runs beside the game's six virtual cores instead of on one of them
(not measured on a small CPU).
Must be tested:
the
vcores:lines' longest waits no longer than main's in the market (Test Farm: linux, the market walked for 5minutes, the
vcores:longest waits against main)expecting no load to hang (no
the game read no controller forover 20 s in longshot_log.txt) and thevcores:lines' longest waits in the market no longer than main's (Test Farm: steamdeck, the market walked and an area change,
against main)
Risks
What could break: another game thread now runs on the core while a region is handed back. The region stays listed,
marked as being freed, until then, so an allocation meanwhile is not given its pages (#1610). A second free of the same
address meanwhile (the guest freeing it twice) finds the mark and does nothing, as on main, where it found no region;
the test checks it.
The freeing thread now waits to take a core back after the hand-back. #1632 found such waits froze loads for 20 s to 5
minutes on an Arc 155H laptop, for a decode that gave its core up around every call; this gives it up once for each
region freed. On main since #1339 that is no new take-back: the renderer's import check before the hand-back gives the
core up and takes one back, once for each region freed; with this PR the check runs without the core (its own give-up
does nothing for a thread that holds none), and the thread takes a core back once, after the hand-back (or the
zeroing) instead of before it. Measured: in the second series (on main
868f6be8, with #1339 and #1632) theResource Memorythread never waited 50 ms or more for a core in any run, warm-ups included, on either side, over the whole run(the start, the loads and the market; the profile records every such wait with its thread). In the first series (on
main
c507b98c) it waited once, 124 ms, in a load before the market in one of this PR's runs, and never on main; in the23-minute run with its 35 area changes, never. No run of either series had a load hang. A CPU with fewer cores and an
iGPU laptop are not run (Must be tested).
Turning it off: revert the commit.
Compatibility: Nothing changes.
Issues
second free meanwhile does nothing (its items 1 to 3; item 4 is #432's).
too; the check gives the core up and takes one back for each region freed.
host/hos_mem.c, and the two merge in either order.back; the decode now keeps its core. Risks has this PR's waits for the same kind of take-back.
near); both sides of the second series carry its commit.vcores:linesin the market; not measured on that CPU.
vcores:lines; the steady drop it reports while running circles is not claimed here.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main. (On868f6be8, measured there.)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md): six for eachseries,
2026-11-18-vcore-region-free-jiro113-s20*.mdand2026-11-19-vcore-region-free-jiro113-rv*.md, eachset in its own commit.
mainand passing here.tests/that reads changed source updated in this PR, not weakened. (test_host_free_marks:its fixture stands in for the core given up, and checks it is taken back.)
Windows: the whole game, with #1729's commit on top: Testing.)
🤖 Generated with Claude Code
Tested in game: Windows 11 Home 10.0.26300, RTX 4090 (NVIDIA 621.24), i9-13900K: this PR's build (
2664bb7c) in the game: a 23-minute scripted run through Castle Town's market, Hyrule Field, Kokiri Forest, Kakariko and the Graveyard and back (35 area changes), no error or hang; after the start no wait for a core reached 300 ms; and 3 measured 300-s runs in the south market against mainc507b98c(Measurements).WIP: perf(host): FreeAddressRegion gives its virtual core up while the region's pages are discardedto perf(host): FreeAddressRegion gives its virtual core up while the region's pages are discarded@Beanstalk
Beanstalk test farm · an earlier run: commit
ba18666(the newer results are in a comment below)Commit
ba18666· tier reviewed · approved by llun · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)The plan compares main and the pull request in Castle Town Market and across repeated loads through Hyrule Field and Kakariko Village. It measures frame-time tails, hitches, load times and memory, with pictures to catch visible corruption. Crashes, stalled loads, significant performance or memory regressions, and substantial picture differences fail the plan. The two paired runs take about 22 minutes per tester; the market measurement is stationary after movement, so it does not reproduce the reported walking benchmark exactly.
kokiri-start, then teleport to Market, move the stick, measure market for 60 s (after 5 s warm-up), take the picture market to compare with main, 2 times: teleport to Hyrule Field, teleport to Kakariko Village, teleport to Market, measure return-market for 20 s (after 5 s warm-up), take the picture return-market to compare with main; 2 runs on each side.Done (started 20:17 UTC, last report 21:48 UTC). Comment
@Beanstalkto run it again.NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 3090, 621.24.0Measurements · any
Verdict: could not be judged on any (2/1 report).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 3090, 621.24.0AMD Ryzen 9 5900X 12-Core Processor, 128 GB RAMScene: market-and-loads, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-and-loads: 2 game starts a side, market 60 s after 5 s warm-up, return-market 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
mooki referenced this pull request2026-11-18 23:27:04 +00:00
The code looks right to me. What holds this back is the evidence: it is on a main 49 commits old, and two of those commits change this exact path.
Blocking
c507b98c. Since then main has merged #1339, which rewritesdiscard_pages(on Windows too, it now asks the renderer whether the pages are imported, and that question gives the core up and takes it back, then zeroes imported pages), and #1663, which changes how the game'sYieldThreadgives up its virtual core. Both change the waits this PR measures. The description says itself that the PR "is rebased and measured again before it leaves WIP", and "Rebased on currentmain" is unchecked, but the PR is out of WIP without either. The merged tree has never run in the game. The fix: rebase ontomain@origin(it merges cleanly), run it in the game once, repeat the 3+3 market series against current main, update Measurements anddocs/tested/, and say whether the gain holds now that main itself gives the core up once per free.PR description / Description follows the template, run 38042). Please look at what it reports. The two unchecked boxes and the "before this leaves WIP" wording are likely causes. Fix it with the rebase.Thread safety (read, no change needed)
hos_region_freereleases thehos_mem.clock beforehos_vcore_away()and takes it again only to unlist the region. No lock is held across the discard or acrosshos_vcore_back(). Its only caller is the HLE forFreeAddressRegion, which holds no other lock.hos_region_alloc,hos_region_free,hos_mem_regionorhos_regions. The region stays listed withfreeingset, so first-fit allocation steps over it. A second free sees the mark and returns, as main does when it finds nothing. The unlisting looks the entry up by address and mark, so amemmovefrom an allocation in between cannot make it remove the wrong one. A new entry clears the mark it inherits from the slot it takes. All of this is covered by the test.rx_guest_imported's ownhos_vcore_away()seesvc_held == 0and returnsvc_escaped, which the outer away has already cleared. So only the outerhos_vcore_back()takes a core back, once per free, the same count as on main since #1339. The question'shos_guest_fillnow runs without a core, whichhost/hos.hallows ("Any thread may call them at any time").DiscardVirtualMemory/MEM_RESET, andMADV_REMOVEon the memfd) already ran outside the lock on main. Only the core is new. The large-pages return and the web build's stubhos_vcore_awayare unaffected. There is no Linux build or run of this branch yet.Measurements: present, done properly for the main they ran on (Windows 11, RTX 4090, 3 interleaved runs a side after warm-ups, same seeds, per-run values given). The gain is well outside the spread: 1% low 27.0-30.1 against 34.1-36.6 fps, p99 29.6-30.3 against 25.5-26.5 ms, windows with a core wait of 300 ms or more 24-26 against 0 of 29. The median frame time change is within the spread, as the description says. But these numbers are for
c507b98c, not for what would merge (point 1). The Test Farm's "failed" label is the farm, not the change: both reports (RTX 3050 Ti laptop, RTX 3090) show no crash and "Nothing was measured: no metric came back" on either side, so there was nothing to judge. Run it again after the rebase.Still needed for hardware: Linux (AMD RX 6000-9000 and NVIDIA) and the Steam Deck. The Deck matters here because the freeing thread now takes its core back against busier cores on a small CPU (the kind of wait #1632 found on an Arc laptop). The PR's own Must be tested asks for both, which is right. Ask
@Beanstalkfor them after the rebase, walking the market (the earlier plan stood still after moving).host/hos_mem.c:79(outside the diff): Nit, not blocking: while a region is being discarded, the control bridge still reports it as mapped, so a bridge read there can return zeros or a mix of old and new bytes. That is intended (the description says so) and harmless, but a word inhos_mem_region's comment would save the next reader the question.@ -198,0 +212,4 @@}UNLOCK();if (!found) return;const int away = hos_vcore_away();On current main,
discard_pagescallsrx_guest_imported, whose ownhos_vcore_away()now returns 0 here (no core held,vc_escapedcleared by this away), so only this function'shos_vcore_back()takes a core back. That is correct, and it would be worth one clause in the comment above after the rebase, since the description leans on it.@Beanstalk
Beanstalk test farm · an earlier run: commit
ba18666, run 2 (the newer results are in a comment below)Commit
ba18666· tier double-reviewed · approved by HumanBean, llun · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)The plan compares main and the pull request in Castle Town Market and across repeated loads through Hyrule Field and Kakariko Village. It measures frame-time tails, hitches, load times and memory, with pictures to catch visible corruption. Crashes, stalled loads, significant performance or memory regressions, and substantial picture differences fail the plan. The two paired runs take about 22 minutes per tester; the market measurement is stationary after movement, so it does not reproduce the reported walking benchmark exactly.
kokiri-start, then teleport to Market, move the stick, measure market for 60 s (after 5 s warm-up), take the picture market to compare with main, 2 times: teleport to Hyrule Field, teleport to Kakariko Village, teleport to Market, measure return-market for 20 s (after 5 s warm-up), take the picture return-market to compare with main; 2 runs on each side.Done (started 07:46 UTC, last report 08:59 UTC). Run 2: started again by HumanBean. Comment
@Beanstalkto run it again.NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: windows+amd+amd-proprietary+dgpuAMD Radeon RX 9070 XT, 2.0.416Measurements · any
Verdict: could not be judged on any (2/1 report).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: windows+amd+amd-proprietary+dgpuAMD Radeon RX 9070 XT, 2.0.416AMD Ryzen 9 5900X 12-Core Processor, 32 GB RAMScene: market-and-loads, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-and-loads: 2 game starts a side, market 60 s after 5 s warm-up, return-market 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Tested in game on Linux: Fedora 45 (kernel 7.3.4), AMD Radeon RX 9070 XT (RADV GFX1201, Mesa 26.2.7), Ryzen 5 5600X (12 logical processors), 32 GB RAM. This PR's head
ba186660against its basec507b98c, two test builds made the same way from one game dump (a host-only change, so both sides prepare the same shaders). No errors, hangs or device losses in any run.Castle Town market (the main check): the confirmation on Linux. Long waits for a virtual core fall by about 80% and none of 100 ms or more remain in the first run; frame times are within the spread.
PR 2's four long waits are another kind: all are
sead::MainThreadholding its own core (615 and 648 ms, the others the threads queued behind it), not the free path this PR changes.Hyrule Field (supporting): within the spread on every number, waits included. The field has far fewer core waits to begin with (median 4 of 100 ms or more on base, 1 here), and its worst drops are the main thread's own work.
Settings: the player's own (Custom preset), 2560x1440 fullscreen, HDR10, AMD FSR 4 (1920x1080 to 2560x1440), V-Sync Auto (mailbox), no frame cap, frame generation off, FPS counter off; one kept options file for every run. Threaded rendering Auto (on).
HOS_PROFILE=1on both sides (frames from its presents trace, the waits from its virtual-core wait records).Method: a scripted route through the control bridge from the repository save
kokiri-start. Market: the debug teleport to the Market, the clock at 12:00, then laps between the gate and the fountain, position-steered, 120 s. Field: the same teleport, out of the gate into Hyrule Field, then laps along Link's facing between 20 m and 320 m out, 120 s. A warm-up run of each side first, then interleaved (base, PR, base, PR); Link's hearts protected (HOS_OOT_PROTECT_HEARTS=1) so an enemy can't end a run.ba1866604372a7a581dfBeanstalk test farm
Commit
ad3c871· approved by nobody yet · dashboardA member of
BigBeanscan review this commit and comment@Beanstalkto test it.868f6be8(#1729's fix on both sides)@HumanBean thanks for the review. Point by point:
1. Rebased, run in the game, measured again. The branch is on main
868f6be8(72a7a581, withtest_host_free_marks's fixture standing in for the core the discard now gives up). The rebased tree ran in the game: this PR's four runs of the new series (a warm-up and three measured, 300 s each in the market), no error and no hang. The 3+3 south-market series is redone against main868f6be8(main868f6be8doesn't build on Windows, #475'snear, so both sides carry #1729's commit44ab7f74), with its six files indocs/tested/(2026-11-19-vcore-region-free-jiro113-rv*.md), and Measurements rewritten.Does the gain hold? Not in frames. On current main every frame row is within the spread (mean frame time 18.39 against 18.54 ms, 1 % low 24.3 against 23.8 fps, same-seed pairs going either way), and main no longer has the market's waits of 300 ms or more (0 windows on both sides, against 24-26 of 29 on
c507b98c). What still changes is the waits for a core: the longest wait in a window 177 → 45 ms, waits of 50 ms or more 287 → 19 (medians), the time all threads spend waiting for a core −13 to −17 % in each same-seed pair. The long waits left on main are low-priority loader threads (tera::FileResourceMgr,PreActorModelCompose,aglTextureDataMgr::Compaction), never the main thread, which is why the frames don't move. The TL;DR, Breakdown and Measurements now say this; thec507b98ctable stays as the first series. I keptperf, as the change still cuts the core waits; if you'd rather it not merge for that alone, say so.Main's frames are worse in this series than on
c507b98c, on both sides. The likely cause is #1632's new defaultHOS_NATIVE_ZSTD=2(c507b98c's default was 1; every run here logs mode 2), as in #1477 (comment). This PR doesn't touch the decode; whether its frame gain comes back withHOS_NATIVE_ZSTD=1on current main is not measured.2. The description check passes locally (
scripts/check_pr_description.py): rebased, boxes ticked, no WIP wording left.Nits: done in
18bf5321:hos_region_free's comment says why only its ownhos_vcore_backtakes a core back (the nested away inrx_guest_importedfinds no core held), andhos_mem_region's says a region being freed still reads as mapped, so a bridge read can see zeros or a mix.Tests:
python -I -B tests/run_all.pyon Windows: main 735 passed / 52 failed / 64 skipped, this PR 736 / 52 / 64; the same 52 fail on both (test_vbytesruns past its time limit on this PC on both), andtest_vcore_region_freepasses here. Linux (WSL): the whole game builds from the rebased tree, and atad3c871etest_vcore_region_freeandtest_host_free_markspass with gcc and clang; run_all as in Testing.Still needed: Linux and the Steam Deck, as Must be tested says.
Two more looks at the market's core-1 waits on Windows, on main
8b0ea786without this PR: same PC, scene, options file and route as the series in the description (Windows 11, RTX 4090, NVIDIA 621.24, i9-13900K; south market, cap 60,HOS_PROFILE=1, DLSS on). The captures and exports stay on this PC; below are only numbers read from them.1. Who holds core 1 while the loaders wait (Tracy). A
LONGSHOT_TRACY=1build (its Windows link: #1765) with local zones arounddiscard_pages's callers and branches (an experiment, in no PR), 60 s captured 30 s into the market leg, seed 55, run as a user (TRACY_NO_SAMPLING=1). Two captures, the same picture:tera::FileResourceMgr,PreActorModelCompose,aglTextureDataMgr::Compactionand a few others) waited 50 ms or more for a core 121 times in the minute: 11.9 s summed, 6.8 s with the overlaps merged.Resource Memory55 % of the time,BaseProcExecutor123 %,ModuleSystemWorker119 % andNavMeshCAStepDepWorker15 % (the other capture: 54, 22, 20, 5).Resource Memory's time on the core during those waits, 72 % is the hand-back of freed regions, all of it fromFreeAddressRegion(FreeMemoryPageswas never called): 58 %DiscardVirtualMemory(14,524 calls, 16.5 GB, median 85 µs, p99 4.7 ms), 11 % zeroing pages the renderer imported (24,331 calls, 14.4 GB), 3 % asking the renderer whether it imported them. About 30 GB are freed a minute.So this PR's give-up covers about three quarters of what
Resource Memorydoes on core 1 while the loaders wait, about 40 % of the core's time in those waits; the rest goes to the game's own core-1 workers and toResource Memory's other work.2. No hand-back at all (
HOS_KEEP_FREED_PAGES=1). One runtime of main8b0ea786, the switch unset (A) against set (B), the series' protocol: 300 s each in the south market, a warm-up of each side (seed 57, not counted), then A55, B55, B56, A56, A58, B58, through the job queue in timing mode. A program outside the queue took about 30 cores for about 70 s during A55, B56 and A56 (the queue's noise check reads 0 on Windows), so only the 10-s windows where other programs used 5 cores or fewer are counted:HOS_KEEP_FREED_PAGES=1HOS_KEEP_FREED_PAGES=1HOS_KEEP_FREED_PAGES=1Keeping the freed pages takes the long waits away much as this PR does (the longest wait a window from 170–192 to 66–78 ms; this PR on
868f6be8: 177 to 45 ms, a different main and series, so not a direct comparison), and like this PR moves no frame: fps and p99 frame time (33–35 ms) are within the spread; the 1 % low is about 1 fps higher with the switch, from few windows, not established. It costs about 0.6 GB of resident memory, flat over the 5 minutes. Nothing here argues forHOS_KEEP_FREED_PAGES=1as a default: this PR takes the waits away without keeping the memory. And both agree that in this scene at cap 60 the loaders' waits for core 1 no longer bound the frame.Tested in the game on Windows 11 with an RTX 4070 (NVIDIA 616.56) and a Ryzen 9 7900X: this head
ad3c871ewith #1729 merged locally so it compiles on Windows, against main868f6be8with #1729. It was an A/B in Hyrule Field while looking into #1720.The route, the same in every run: a Fire Temple save, teleport to Hyrule Field, Switch age to child, Set time 21:00, then about 2 minutes running around the field fighting the Stalchildren that spawn, with camera turns. Settings: cap 90, DLSS Quality, frame generation off, Reflex and V-Sync on, with
RX_VK_PASS_TIMING=1andHOS_THREAD_TIMES=5. Each run started cold in its own game folder, and a person played. The numbers come from 120-present windows from the field's arrival; a fight window is one with 200 or more dispatches a frame.tera::FileResourceMgr9,PreActorModelCompose3,aglTextureDataMgr::Compaction2, others 3GeneralFormatAsync_)No crash, device loss or fatal in any run, and each ended with a clean quit (exit 0). One run per side for main, so the fps rows are within the noise of playing by hand.