fix(vk): read the registrations kept from before the renderer started from the host's copies (#1117) #2289
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2289
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/vk-early-registration-copies"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The textures and samplers the game registers before the renderer has started can reach the Vulkan back end as all zeros: the game's sampler 258 then filters nearest instead of trilinear and a few texture slots have no image. Six of the logs attached to #1117 show exactly that, and they are the logs of the two reporters with the blocky light halos.
Refs #1117.
The cause
rx_start.host/rx_backend.ckeeps those calls (keep_early) and afterwards hands them to the renderer as pointers to its own copies on the heap (flush_early,forward): the log'srx: 123 texture and sampler calls made before the renderer started ..., given to it.descriptor_ok(gpu/vk/vk_bank.cpp) decides which memory a registration is copied from. A pointer outside the guest window was first given tohost->gpu_ptr()and only then treated as host memory.nvn_native_on()returns 1),hos_gpu_ptranswers throughnvn_gpu_guest(host/nvn_native.c): every value from 16 MB (HOS_SYSTEM_BASE) to 512 GB (HOS_SPACE_END) is a guest address, and a value from 1 TB (VBASE) up is one where a virtual pool has a page mapped. A kept copy at such a host address was therefore read from guest memory at that address, not from the copy.nvnTexturePoolRegisterTexture,nvnTexturePoolRegisterImageandnvnSamplerPoolRegisterSamplerpass host pointers inside the guest window (P(i)), whichdescriptor_ok's first check takes;forwardpasses the kept copies. So askinggpu_ptrcould only ever misread a host pointer.What changed
gpu/vk/vk_bank.cpp:descriptor_okno longer askshost->gpu_ptr(). A pointer inside the guest window is taken as before; any other pointer is the host's own memory whenVirtualQuerysays it is committed and readable, and is refused otherwise (as before for memory that is not there).tests/test_registration_host_copies.pywithtests/registration_host_copies_test.cpp: runsdescriptor_ok, cut from the source, againstnvn_gpu_guest, cut from the source, with the guest window's real bounds read from the headers. Seven cases: a pointer in the guest window up to its last bytes; a kept copy at 0x9873eec (the address from the log below); a kept copy at a mapped virtual pool's address above 1 TB; a copy above the window; no memory at the address; memory that is not readable; a region that ends inside the registration. It also checks that the four registration entries still go throughdescriptor_ok.vk_bank.cppis one of the files the shader cache's stamp is made from (nx2/build.py), so the program cache is prepared again at the first start after this change, as after any change to that file.Evidence: the reporters' logs
Counted by a script with asserts over the 13 logs attached to #1117 and #1727 (7 and 6):
RX_VK_FORCE_LINEAR=1)vk: sampler 258 is nearest: min filter 0, mag filter 0, wrap 0/0/0, lod 0..0 bias 0, aniso 0, compare 0andvk: texture slot 0x102 (handle 0x10200102): no image for it; NVN target 0 format 0 0x0x0 levels 0 gpu 0, .... In the logs without them the firstsampler anisotropyline isthe game asks for 1 (min filter 5, ...).mem: 0x9873eec committed on first touch (nothing handed it out)betweenrx: renderer: Vulkan (hookshot_vk.dll)andrx: 123 ... given to it, whererx_startonly hands the kept calls over; 0x9873eec is inside the 16 MB to 512 GB range. The other five logs with the zeros have no such line there. Why one run of the same PC was right is not shown; the address of the heap block is my guess.Added 2026-11-11 13:54 UTC, two more pieces of evidence (no game run):
early_callinhost/rx_backend.c) is 256 bytes on both Windows targets, with the texture at offset 8, the view at 136 and the sampler's parameters at 156 (static asserts over the struct cut from the source). 0x9873eec minus 156 is 0x9873e50, a 16-byte-aligned address, asreallocreturns; minus 8 or minus 136 it is not aligned. So the first byte that was read from guest memory instead of from the copy is the first byte of a kept sampler registration's parameters, which is whatregister_samplercopies.bdfe09c, RTX 4090, NVIDIA 615.78.08, a 1920x1080 window: the log hasrx: 123 texture and sampler calls made before the renderer started (33 texel writes), given to it, the firstsampler anisotropyline isthe game asks for 1 (min filter 5, ...), and no line about sampler 258 or a texture slot without an image. That is what the reading expects where the heap lies above 512 GB: the hand-over itself is not at fault, the address check is.Added 2026-11-11 16:20 UTC: the Windows hand-over emulated in the game. A local test switch (not part of this pull request) hands the first 123 calls' registrations to the renderer as host copies at 0x9870000, the address range of the Windows log. On Linux with an RTX 4090, main
49f75dc8then logs the reporters' lines (sampler 258 is nearest: min filter 0, ...; texture slots 0x102, 0x107, 0x124 all zero) and draws the glow around Navi and the lanterns in blocks; with this change and the same hand-over the lines are gone and the glow is smooth. The table and the pictures are in this comment.How it was tested
python -I -B tests/test_registration_host_copies.py --cxx g++and--cxx clang++(g++ 16.2.1, clang 23.1.3, Arch Linux): 7 cases pass on the branch. With--sourcepointed at main49f75dc8the same test fails at the casecopy-in-guest-address-range.python -I -B tests/run_all.pyon the branch (5f0de4d2, Linux): 363 passed, 2 failed, 10 skipped. The two failures are this machine's, the same on main:tests/test_updater_asset_digest.py(only the .NET 10 SDK is installed, the fixture wants net8.0) andtests/test_wayland_libdecor_hint.py(clang 23's-Wunused-but-set-globalunder-Werrorin the test's own harness).Not tested
bdfe09c: no kept calls, sampler 258 trilinear, smooth glows around the lanterns and Navi).Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 [email protected]
A Linux data point: this fault does not happen there, and with the PR the logs come out the same.
Setup:
49f75dc8plus about 30 open PRs. The two renderers differ only in this PR: one built with it, one with its merge reverted (vk_bank.cpponly).rx: N ... calls made before the renderer startedvk: sampler 258 is nearest(the all-zero set)Why it doesn't happen here: this fits your cause. On Linux the kept copies are ordinary glibc heap allocations, far above the range
nvn_gpu_guesttakes for guest addresses, sodescriptor_okreads the copies even without the fix. Windows heap addresses can fall inside that range.Results:
🤖 Generated with Claude Code
Thanks for building it both ways: that is the check for a regression on a second Linux setup, which I could not give.
Your reason is the one I see here too. On my RTX 4090 (NVIDIA 615.78.08) a run with a window on main keeps the same 123 calls and reads them correctly, because the kept copies lie above the range that is taken for guest addresses.
What stays open is the Windows side: a run of this branch on one of the PCs from #1117 that show the halos (both are RTX 5080s). If someone there can build it, the two log lines to look for are
vk: sampler 258 is nearest: min filter 0, ...andvk: texture slot 0x102 ...: no image for it. With the fix they should be gone, and the firstsampler anisotropyline should readthe game asks for 1 (min filter 5, ...).The Windows hand-over emulated in the game (Linux, RTX 4090): blocks on main, none with this change.
I cannot get a low heap address on Linux, so I built a local test switch (not part of this pull request): it hands the registrations among the first 123 texture and sampler calls to the renderer as host copies in a block mapped at 0x9870000, the address range of the Windows log. Nothing else differs from a normal run without a window. Three runs in one hold, Kokiri Forest from the same save (Link on his porch, the lanterns over the door, Navi), output 2048x1152 (the game's own), default settings:
49f75dc8+ the switchsampler 258 is nearest: min filter 0, mag filter 0, wrap 0/0/0, lod 0..0 bias 0, aniso 0, compare 0; texture slots 0x102, 0x107, 0x124 registered all zero: the same lines as the Windows log with the blocks49f75dc8+ this change + the switchthe game asks for 1 (min filter 5, ...)), no texture slot registered all zero49f75dc8In run 1 the game samples through sampler 258 in at least 30 programs (a local log line, capped at 64 lines), three of them among the seven this thread named as the bloom and light-shaft passes:
27ee9883e2a20323(textures of 128x72, 64x36 and 4x4),1e2fcee33a0c7209(128x72) andb7914ae65792d0c8(2048x1152). Small buffers read without filtering are the blocks. So the zero sampler alone gives the picture of this issue, and reading the kept copies from where they are removes it.All three runs: RTX 4090, NVIDIA 615.78.08, Arch Linux, no window, exit at the wall, no FATAL, crash or device-lost line, no driver line in the kernel log (one
NVRM: memMap_IMPLline each, the known refusal of the dma-buf import). Load average 9 to 40 during the hold.Still not shown: a Windows run, with or without the change. The emulation shows what the zeros do to the picture and that the change stops them under a low address; it does not show which addresses a Windows heap really gives.
Looks right to me. The four callers of
descriptor_okonly ever pass a host pointer:nvnTexturePoolRegisterTexture/RegisterImage/RegisterSamplerpassP(i)inside the guest window, andforwardpasses&c->t/&c->v/&c->pfrom the kept array. So thegpu_ptrprobe could only ever misread a heap copy, and dropping it doesn't lose a case. The0x9873eec - 156alignment argument and the emulated low-address hand-over make the cause convincing.Thread safety of the kept registrations: fine.
flush_earlyrunsforwardunderearly_lock, and both renderer paths copy synchronously before it returns.register_texture_locked/register_samplercopy into the bank, and the_queuedversions copy*t/*v/*pinto theJobbeforeenqueue. So nothing reads the array afterfree(early). A game thread registering during the flush blocks on the lock, seesstarted, and goes straight through after the kept calls, same as before.Windows/Linux parity: the change is the same on both. On Linux the shim's
VirtualQuery(host/posix/win32_posix.c) answers from/proc/self/maps, so a glibc copy (brk or mmap, rw-p) is still accepted. That matches Joey-Magnus's before/after run (123 calls, no zero sampler either way). It also costs nothing new at run time: game registrations still take the guest-window check first, and only the 123 kept calls reachVirtualQuery, as they already did on Linux. The test covers both sides of the 512 GB line plus the refusal cases, and it's declared forrun_all.pyviaRUN_ALL_ARGS.Still open, and not a blocker for the code: a real Windows run from one of the #1117 reporters (look for the
sampler 258 is nearestline). CI also hasn't run: the Windows builder, Linux AppImage and game-free test workflows are waiting on maintainer approval (action_required), so someone needs to approve them before merge.Thanks, also for checking the flush's locking and the Linux side. About the workflows: GitHub now starts them for my pushes without an approval (it did for #2380). Main's game-free tests fail until #2380 is in, so I'll rebase this onto main right after that merge and push; the runs should then show here on their own. The Windows run by a #1117 reporter is still open, as you say; I asked for it in the issue and will link the result here.
Rebased onto main
b29967bfso the workflows run here: they had waited for an approval on the old head, and main's game-free tests are green again since #2380. The patch is unchanged (git range-diff: the commit is equal), and its test passes on the new base.github-sync referenced this pull request2026-11-12 06:22:54 +00:00