MagicBeanDev/Longshot

Watch

Fork 99

perf(vk): the all-zero check of a big texture reads only what the game wrote of it (Linux) #567

Open

p0lux wants to merge 2 commits from p0lux/Longshot:perf/zero-upload-holes into main

p0lux commented

2026-11-14 19:46:14 +00:00

Contributor

Type: perf · Risk: low · Tested on: Linux (SteamOS), Steam Deck OLED (RADV VANGOGH) · Needs: none

Why

When an area loads, the renderer reads the whole storage of a big new texture to see whether it is all zero, so it can clear the image instead of unswizzling it. On Linux that read is wasted on storage the game never wrote: the shadow map (12288 x 12288 D16, 288 MB) took 108 ms of the replay thread at each arrival in Kokiri Forest on a Steam Deck, and the read itself made about 190 MB of pages that then stayed in the game's memory. See #388.

What changed

On Linux the all-zero check of a big texture now asks the kernel which parts of the storage the game ever wrote, and reads only those; the rest is known to be zero without a read. On Windows, and where the question has no answer, all of the storage is read as before.

The check asks what was written (upload_all_zero, gpu/vk/vk_texture.cpp). On Linux the guest window is a memfd's (host/hos_main.c), and a part of it never written, or given back since (MADV_REMOVE), is a hole of that file: it reads as zero, and reading it through the mapping makes its pages. The check now asks the file which parts hold data (lseek SEEK_DATA / SEEK_HOLE) and reads only those. The log's is all zero: cleared on the GPU line says how many MB were read.
New gpu/vk/guest_holes.h: data_parts (the written parts of a range of a file) and all_zero (the threaded read, moved out of upload_all_zero so it can be tested).
memory_guest_data_parts (gpu/vk/vk_memory.cpp, declared in vk_internal.h): finds the window's memfd the way import_dma_buf does (hos_posix_guest_memfd) and turns a host pointer into the file's offset. Renderer only: no host change, vk_api.h untouched.
A rounding fix in the read, found on the way: a thread's share was the floor of the size over the thread count, rounded up to 4096. When that floor was already a multiple of 4096 and the division had a remainder, up to threads - 1 bytes at the end were never read. The share is now the ceiling. tests/test_guest_holes.cpp has the case (4 MB and 3 bytes on 4 threads).
Windows, and a window that is anonymous memory: the question answers "not known" and all of the storage is read, as before.
docs: a row for RX_VK_ZERO_HOLES in docs/development.md; one run file in docs/tested/.

Measurements

Verdict: it helps one frame and the memory: the all-zero check of the 288 MB shadow map goes from 108 ms to 2 ms at each arrival in Kokiri Forest on a Steam Deck OLED, the load's transition frame from 1,069 to 910 ms, and about 190 MiB of pages are no longer made. The arrival as a whole is not measurably shorter, and nothing changes in Hyrule Field. No frame rate gain is claimed. Passed.

Hardware: SteamOS 3.9.5 (kernel 7.2.7-valve2), Steam Deck OLED: AMD Custom APU 0932, RADV VANGOGH, Mesa 26.2.0-devel, 16 GB shared with the GPU; the GPU's clock fixed at 1600 MHz
Scene: the arrival in an area: a save in Kokiri Forest and one in Hyrule Field loaded by script; no screen (headless), handheld profile, Minimum preset, the game's own upscaler, no frame generation, no cap
Method: the game and host built from main 9c9d69ed; the back end built alone from each commit (clang, same flags) and swapped in that game folder. Each run starts the game and loads the save, with RX_VK_FRAME_TRACE=1. The check's time is the log's own line. Frame times are the trace's, over the load's transition frame (the first frame over 600 ms that uploads over 60 MB) and the 600 presents after it. Memory is /proc/<pid>/status at the end of the run, about half a minute after the arrival. Main, this PR, and a third renderer (another pull request's), alternated, 3 runs a scene; the median of 3
How to reproduce: load a save in Kokiri Forest, then grep "is all zero" logs/longshot_log.txt; RX_VK_ZERO_HOLES=0 for the old read

this PR: perf/zero-upload-holes @ 8fa1f2c1 (this PR's code commit on main 9c9d69ed; on the fork as measured/zero-upload-holes-8fa1f2c1). The branch is now on main 9e6f231d (ae150395 and a docs commit): git range-diff shows the same patch, with only neighbouring rows of docs/development.md differing, and main has changed nothing in upload_all_zero or in how the guest window is mapped in between, so the numbers hold. Not measured again on this main.

Medians; each run in brackets, in the order run:

Kokiri Forest	Before: main @ `9c9d69ed`	After: this PR @ 8fa1f2c1	Δ
all-zero check of the 288 MB shadow map	108.3 ms (108.3 / 108.0 / 111.3)	2.2 ms (2.7 / 2.0 / 2.2), 16 MB of it read	-106 ms
upload time in the load's transition frame	224 ms (222 / 224 / 227)	118 ms (125 / 118 / 118)	-106 ms
the transition frame	1,069 ms (1,102 / 1,069 / 1,039)	910 ms (1,013 / 896 / 910)	-159 ms
`RssShmem` (the guest window's pages)	4,913 MiB (4,913 / 4,870 / 5,028)	4,725 MiB (4,722 / 4,725 / 4,745)	-188 MiB
time in frames over 60 ms, the 600 presents after	8.6 s (6.0 / 9.1 / 8.6)	8.2 s (8.4 / 8.2 / 6.7)	within the spread

Hyrule Field	Before: main @ `9c9d69ed`	After: this PR @ 8fa1f2c1	Δ
all-zero checks at this arrival	none	none
the transition frame	999 ms (1,033 / 976 / 999)	1,013 ms (1,013 / 1,013 / 1,083)	within the spread
`RssShmem`	4,634 MiB (4,551 / 4,660 / 4,634)	4,640 MiB (4,640 / 4,676 / 4,556)	none
time in frames over 60 ms, the 600 presents after	3.0 s (2.9 / 3.0 / 3.3)	2.8 s (2.8 / 3.2 / 2.7)	within the spread

main      vk: upload: 12288x12288 NVN 0x32 (288 MB) is all zero: cleared on the GPU (checked in 108.3 ms)
this PR   vk: upload: 12288x12288 NVN 0x32 (288 MB) is all zero: cleared on the GPU (checked in 2.7 ms, 16 MB of it read)
this PR   vk: upload: 4096x4096 NVN 0x32 (32 MB) is all zero: cleared on the GPU (checked in 6.0 ms, 32 MB of it read)

Read with care. This is one check in one frame: it takes 106 ms out of a frame of about a second and gives back about 190 MiB, and that is all it does. The arrival as a whole (6 to 9 s spent in slow frames in Kokiri Forest) does not get measurably shorter, and nothing changes in Hyrule Field, where no texture of 32 MB or more is checked. The 288 MB storage had 16 MB written, so about 272 MB of pages are no longer made by the check; the 188 MiB is what is left of that half a minute later, between runs that differ by up to 158 MiB on main.

Flags and settings

One new A/B switch.

Flag	What it is	Before this PR	After this PR	Effect	Helps / doesn't help / untested	Recommendation
`RX_VK_ZERO_HOLES=0`	the all-zero check reads all of the storage	that behaviour, always	unset: only what the game wrote is read (Linux, a memfd window); `=0`: the old read	none on the picture; 106 ms less in the frame that checks a 288 MB texture, about 190 MiB fewer pages	helps: Linux where the guest window is a memfd's (Steam Deck above); doesn't help: Windows, and Linux with an anonymous window (the read is as before); untested: other Linux PCs in the game	leave unset: on for everyone, the switch is for A/B only

Testing

Tested:

python -I -B tests/run_all.py on macOS with the Vulkan headers given, on the rebased head: 507 passed, 48 failed, 39 skipped. The 48 fail the same way on main 9e6f231d on that machine (no /proc, no posix_fallocate, Python 3.9, the Linux launch scripts and the like). The suite ran beside four others and 10 more checks timed out in that run (on main too, 10); each passes when run alone.
tests/test_guest_holes.py (new): on macOS the read and the "not known" answer; on Linux (Debian, g++ 14, kernel 7.2.9, and on the Deck's kernel 7.2.7) the question against a real memfd: never written, written with zeros, a range not page aligned, given back with MADV_REMOVE, the last page, past the end. It fails on main (no such check) and with the old rounding. The existing checks that read vk_texture.cpp, vk_memory.cpp and vk_internal.h needed no change and pass.
In the game, a Steam Deck OLED, without a screen: the 6 runs above of this commit and 6 of main, no error, no device lost, 0 GPU timeouts in the kernel's log. With the picture, by a person: builds holding this change with other renderer-only ones played on the same Deck on 2026-11-14: reported smooth, nothing wrong reported.
Linux: the back end builds from the rebased head (a1fdd59c; clang, x86-64-v3, on the Steam Deck): 0 failed. Windows: longshot_vk.dll cross-builds from the same head with llvm-mingw 20261104 on Linux: 0 failed. Not run on Windows.

Not tested:

Windows in the game: the code path there is the old read, now in guest_holes::all_zero (the test runs it on plain memory).
Other Linux GPUs and drivers in the game. A Linux PC without /dev/udmabuf (anonymous window): the "not known" path is the test's first case only.
A storage partly written with something other than zero in its written parts was not met in the game; the test covers it.

Must be tested: None: what a hole reads as is the kernel's, checked by the test on any Linux, and Windows keeps the read it had.

Risks

What could break: a texture cleared when it should have been uploaded, if a part of the window held bytes that are not the memfd's at that offset. The check's answer does not change otherwise: a hole reads as zero, and what was written is still read byte by byte. Today the only mapping made in the window is the whole-window one at offset 0 (hos_main.c; VirtualAlloc in it only changes protection, discards are MADV_REMOVE), and memory_guest_data_parts refuses a pointer outside the window. lseek moves the memfd's file position, which nothing reads (the mapping and udmabuf take offsets).

Turning it off: RX_VK_ZERO_HOLES=0 is the old read; reverting the commit removes the change.

Compatibility: Nothing changes: saves and longshot_options.ini are untouched, no shader or pipeline cache is rebuilt (no stamped file is touched), MKWVK_API_VERSION is as it was.

Reviewer notes

The read used to stop at the first non-zero byte and still does; all_zero is the same loop over the written parts, with at most one thread a MB of data.

Issues

See #388

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how.
Rebased on current main.
python -I -B tests/run_all.py passes (any failure that also fails on main is named).
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md).
A game-free test added for the fix, failing on main and passing here.
Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
Docs updated.

**Type:** perf · **Risk:** low · **Tested on:** Linux (SteamOS), Steam Deck OLED (RADV VANGOGH) · **Needs:** none ## Why When an area loads, the renderer reads the whole storage of a big new texture to see whether it is all zero, so it can clear the image instead of unswizzling it. On Linux that read is wasted on storage the game never wrote: the shadow map (12288 x 12288 D16, 288 MB) took 108 ms of the replay thread at each arrival in Kokiri Forest on a Steam Deck, and the read itself made about 190 MB of pages that then stayed in the game's memory. See #388. ## What changed On Linux the all-zero check of a big texture now asks the kernel which parts of the storage the game ever wrote, and reads only those; the rest is known to be zero without a read. On Windows, and where the question has no answer, all of the storage is read as before. - **The check asks what was written** (`upload_all_zero`, `gpu/vk/vk_texture.cpp`). On Linux the guest window is a memfd's (`host/hos_main.c`), and a part of it never written, or given back since (`MADV_REMOVE`), is a hole of that file: it reads as zero, and reading it through the mapping makes its pages. The check now asks the file which parts hold data (`lseek` `SEEK_DATA` / `SEEK_HOLE`) and reads only those. The log's `is all zero: cleared on the GPU` line says how many MB were read. - **New `gpu/vk/guest_holes.h`**: `data_parts` (the written parts of a range of a file) and `all_zero` (the threaded read, moved out of `upload_all_zero` so it can be tested). - **`memory_guest_data_parts`** (`gpu/vk/vk_memory.cpp`, declared in `vk_internal.h`): finds the window's memfd the way `import_dma_buf` does (`hos_posix_guest_memfd`) and turns a host pointer into the file's offset. Renderer only: no host change, `vk_api.h` untouched. - **A rounding fix in the read, found on the way**: a thread's share was the floor of the size over the thread count, rounded up to 4096. When that floor was already a multiple of 4096 and the division had a remainder, up to `threads - 1` bytes at the end were never read. The share is now the ceiling. `tests/test_guest_holes.cpp` has the case (4 MB and 3 bytes on 4 threads). - **Windows, and a window that is anonymous memory**: the question answers "not known" and all of the storage is read, as before. - docs: a row for `RX_VK_ZERO_HOLES` in `docs/development.md`; one run file in `docs/tested/`. ## Measurements **Verdict:** it helps one frame and the memory: the all-zero check of the 288 MB shadow map goes from 108 ms to 2 ms at each arrival in Kokiri Forest on a Steam Deck OLED, the load's transition frame from 1,069 to 910 ms, and about 190 MiB of pages are no longer made. The arrival as a whole is not measurably shorter, and nothing changes in Hyrule Field. No frame rate gain is claimed. Passed. Hardware: SteamOS 3.9.5 (kernel 7.2.7-valve2), Steam Deck OLED: AMD Custom APU 0932, RADV VANGOGH, Mesa 26.2.0-devel, 16 GB shared with the GPU; the GPU's clock fixed at 1600 MHz Scene: the arrival in an area: a save in Kokiri Forest and one in Hyrule Field loaded by script; no screen (headless), handheld profile, Minimum preset, the game's own upscaler, no frame generation, no cap Method: the game and host built from main 9c9d69ed; the back end built alone from each commit (clang, same flags) and swapped in that game folder. Each run starts the game and loads the save, with `RX_VK_FRAME_TRACE=1`. The check's time is the log's own line. Frame times are the trace's, over the load's transition frame (the first frame over 600 ms that uploads over 60 MB) and the 600 presents after it. Memory is `/proc/<pid>/status` at the end of the run, about half a minute after the arrival. Main, this PR, and a third renderer (another pull request's), alternated, 3 runs a scene; the median of 3 How to reproduce: load a save in Kokiri Forest, then `grep "is all zero" logs/longshot_log.txt`; `RX_VK_ZERO_HOLES=0` for the old read this PR: perf/zero-upload-holes @ 8fa1f2c1 (this PR's code commit on main 9c9d69ed; on the fork as `measured/zero-upload-holes-8fa1f2c1`). The branch is now on main 9e6f231d (ae150395 and a docs commit): `git range-diff` shows the same patch, with only neighbouring rows of `docs/development.md` differing, and main has changed nothing in `upload_all_zero` or in how the guest window is mapped in between, so the numbers hold. Not measured again on this main. Medians; each run in brackets, in the order run: | Kokiri Forest | Before: main @ 9c9d69ed | After: this PR @ 8fa1f2c1 | Δ | |---|---|---|---| | all-zero check of the 288 MB shadow map | 108.3 ms (108.3 / 108.0 / 111.3) | 2.2 ms (2.7 / 2.0 / 2.2), 16 MB of it read | -106 ms | | upload time in the load's transition frame | 224 ms (222 / 224 / 227) | 118 ms (125 / 118 / 118) | -106 ms | | the transition frame | 1,069 ms (1,102 / 1,069 / 1,039) | 910 ms (1,013 / 896 / 910) | -159 ms | | `RssShmem` (the guest window's pages) | 4,913 MiB (4,913 / 4,870 / 5,028) | 4,725 MiB (4,722 / 4,725 / 4,745) | -188 MiB | | time in frames over 60 ms, the 600 presents after | 8.6 s (6.0 / 9.1 / 8.6) | 8.2 s (8.4 / 8.2 / 6.7) | within the spread | | Hyrule Field | Before: main @ 9c9d69ed | After: this PR @ 8fa1f2c1 | Δ | |---|---|---|---| | all-zero checks at this arrival | none | none | | | the transition frame | 999 ms (1,033 / 976 / 999) | 1,013 ms (1,013 / 1,013 / 1,083) | within the spread | | `RssShmem` | 4,634 MiB (4,551 / 4,660 / 4,634) | 4,640 MiB (4,640 / 4,676 / 4,556) | none | | time in frames over 60 ms, the 600 presents after | 3.0 s (2.9 / 3.0 / 3.3) | 2.8 s (2.8 / 3.2 / 2.7) | within the spread | ``` main vk: upload: 12288x12288 NVN 0x32 (288 MB) is all zero: cleared on the GPU (checked in 108.3 ms) this PR vk: upload: 12288x12288 NVN 0x32 (288 MB) is all zero: cleared on the GPU (checked in 2.7 ms, 16 MB of it read) this PR vk: upload: 4096x4096 NVN 0x32 (32 MB) is all zero: cleared on the GPU (checked in 6.0 ms, 32 MB of it read) ``` Read with care. This is one check in one frame: it takes 106 ms out of a frame of about a second and gives back about 190 MiB, and that is all it does. The arrival as a whole (6 to 9 s spent in slow frames in Kokiri Forest) does not get measurably shorter, and nothing changes in Hyrule Field, where no texture of 32 MB or more is checked. The 288 MB storage had 16 MB written, so about 272 MB of pages are no longer made by the check; the 188 MiB is what is left of that half a minute later, between runs that differ by up to 158 MiB on main. ### Flags and settings One new A/B switch. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_ZERO_HOLES=0` | the all-zero check reads all of the storage | that behaviour, always | unset: only what the game wrote is read (Linux, a memfd window); `=0`: the old read | none on the picture; 106 ms less in the frame that checks a 288 MB texture, about 190 MiB fewer pages | helps: Linux where the guest window is a memfd's (Steam Deck above); doesn't help: Windows, and Linux with an anonymous window (the read is as before); untested: other Linux PCs in the game | leave unset: on for everyone, the switch is for A/B only | ## Testing **Tested:** - `python -I -B tests/run_all.py` on macOS with the Vulkan headers given, on the rebased head: 507 passed, 48 failed, 39 skipped. The 48 fail the same way on main 9e6f231d on that machine (no `/proc`, no `posix_fallocate`, Python 3.9, the Linux launch scripts and the like). The suite ran beside four others and 10 more checks timed out in that run (on main too, 10); each passes when run alone. - `tests/test_guest_holes.py` (new): on macOS the read and the "not known" answer; on Linux (Debian, g++ 14, kernel 7.2.9, and on the Deck's kernel 7.2.7) the question against a real memfd: never written, written with zeros, a range not page aligned, given back with `MADV_REMOVE`, the last page, past the end. It fails on main (no such check) and with the old rounding. The existing checks that read `vk_texture.cpp`, `vk_memory.cpp` and `vk_internal.h` needed no change and pass. - In the game, a Steam Deck OLED, without a screen: the 6 runs above of this commit and 6 of main, no error, no device lost, 0 GPU timeouts in the kernel's log. With the picture, by a person: builds holding this change with other renderer-only ones played on the same Deck on 2026-11-14: reported smooth, nothing wrong reported. - Linux: the back end builds from the rebased head (a1fdd59c; clang, x86-64-v3, on the Steam Deck): 0 failed. Windows: `longshot_vk.dll` cross-builds from the same head with llvm-mingw 20261104 on Linux: 0 failed. Not run on Windows. **Not tested:** - Windows in the game: the code path there is the old read, now in `guest_holes::all_zero` (the test runs it on plain memory). - Other Linux GPUs and drivers in the game. A Linux PC without `/dev/udmabuf` (anonymous window): the "not known" path is the test's first case only. - A storage partly written with something other than zero in its written parts was not met in the game; the test covers it. **Must be tested:** None: what a hole reads as is the kernel's, checked by the test on any Linux, and Windows keeps the read it had. ## Risks **What could break:** a texture cleared when it should have been uploaded, if a part of the window held bytes that are not the memfd's at that offset. The check's answer does not change otherwise: a hole reads as zero, and what was written is still read byte by byte. Today the only mapping made in the window is the whole-window one at offset 0 (`hos_main.c`; `VirtualAlloc` in it only changes protection, discards are `MADV_REMOVE`), and `memory_guest_data_parts` refuses a pointer outside the window. `lseek` moves the memfd's file position, which nothing reads (the mapping and udmabuf take offsets). **Turning it off:** `RX_VK_ZERO_HOLES=0` is the old read; reverting the commit removes the change. **Compatibility:** Nothing changes: saves and `longshot_options.ini` are untouched, no shader or pipeline cache is rebuilt (no stamped file is touched), `MKWVK_API_VERSION` is as it was. ## Reviewer notes - The read used to stop at the first non-zero byte and still does; `all_zero` is the same loop over the written parts, with at most one thread a MB of data. ## Issues See #388 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated.

p0lux added 2 commits

2026-11-14 19:46:15 +00:00

perf(vk): the all-zero check of a big texture reads only what the game wrote of it (Linux) 742cb89f79

upload_all_zero read a texture's whole storage to find it zero. On Linux the guest window is a memfd's, and a storage
the game never wrote (the shadow map, 12288 x 12288 D16, 288 MB, fresh at each arrival in an area) is holes of that
file: zero without a read, and reading them through the mapping made their pages. The check now asks the file what
holds data (lseek SEEK_DATA / SEEK_HOLE, gpu/vk/guest_holes.h) and reads that.

- gpu/vk/guest_holes.h: data_parts (the written parts of a range of the file) and all_zero (the read, moved out of
  upload_all_zero; a thread's share is now the data's ceiling over the threads, the floor left up to threads - 1
  bytes unread when it was a multiple of 4096).
- vk_memory.cpp: memory_guest_data_parts, the window's memfd looked up as import_dma_buf does.
- RX_VK_ZERO_HOLES=0: all of it read, as before. The log's "is all zero" line says how many MB were read.
- tests/test_guest_holes.py: the question against a real memfd on Linux, the read on plain memory.

Windows and an anonymous window: not known, all of it read, as before.

Assisted-by: Claude Fable 5.2
Co-Authored-By: Claude Fable 5.2 <[email protected]>

docs(tested): the all-zero check that reads only what was written, on a Steam Deck OLED

PR description / Description follows the template (pull_request_target) Successful in 3s

Details

beanstalk/approval Test farm not requested

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

Third-party parts / The same as upstream (pull_request) Has been cancelled

Details

Website / Website on Cloudflare (pull_request) Has been cancelled

Details

e6430ec2f7

Main 9c9d69ed against the commit on it (8fa1f2c1), alternated, 3 runs a scene: the arrival in Kokiri Forest and in Hyrule Field.

Assisted-by: Claude Fable 5.2

Co-Authored-By: Claude Fable 5.2 <[email protected]>

p0lux changed title from ~~WIP: perf(vk): the all-zero check of a big texture reads only what the game wrote of it (Linux)~~ to perf(vk): the all-zero check of a big texture reads only what the game wrote of it (Linux)

2026-11-14 19:46:50 +00:00

p0lux referenced this pull request

2026-11-14 20:30:08 +00:00

Game: on a Steam Deck the Kokiri Forest hang does not come back without RX_VK_MESH_NOCULL (main 9c9d69ed), and the line costs 2 to 10% of the GPU time #579

p0lux commented

2026-11-15 09:05:50 +00:00

Author

Contributor

Rebased onto main 9e6f231d (head a1fdd59c): no conflict; git range-diff shows the same code patch (neighbouring rows of docs/development.md differ). The numbers are the ones measured before; the description says why they still hold.

The description now follows the new template: the status line, a summary under What changed, the verdict, Flags and settings under Measurements, Must be tested, and Risks with Compatibility.

On the rebased head: python -I -B tests/run_all.py on macOS gives 507 passed, 48 failed, 39 skipped, the 48 failing the same way on main 9e6f231d on that machine; the back end builds for Linux (clang, on a Steam Deck) and for Windows (llvm-mingw 20261104, cross-built), 0 failed.

Rebased onto main 9e6f231d (head a1fdd59c): no conflict; `git range-diff` shows the same code patch (neighbouring rows of `docs/development.md` differ). The numbers are the ones measured before; the description says why they still hold. The description now follows the new template: the status line, a summary under What changed, the verdict, Flags and settings under Measurements, Must be tested, and Risks with Compatibility. On the rebased head: `python -I -B tests/run_all.py` on macOS gives 507 passed, 48 failed, 39 skipped, the 48 failing the same way on main 9e6f231d on that machine; the back end builds for Linux (clang, on a Steam Deck) and for Windows (llvm-mingw 20261104, cross-built), 0 failed.

p0lux force-pushed perf/zero-upload-holes from e6430ec2f7

PR description / Description follows the template (pull_request_target) Successful in 3s

Details

beanstalk/approval Test farm not requested

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

Third-party parts / The same as upstream (pull_request) Has been cancelled

Details

Website / Website on Cloudflare (pull_request) Has been cancelled

Details

to a1fdd59cd3

PR description / Description follows the template (pull_request_target) Successful in 15s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

Third-party parts / The same as upstream (pull_request) Has been cancelled

Details

Website / Website on Cloudflare (pull_request) Has been cancelled

Details

2026-11-15 09:05:52 +00:00

Compare

omgitsgela referenced this pull request

2026-11-15 09:46:08 +00:00

Game: Hyrule Town spin costs ~30% fps (Deck LCD) - GPU-written 691x432 texture re-uploaded per present + page-capture storm #750

omgitsgela referenced this pull request

2026-11-15 10:32:15 +00:00

Game: Hyrule Town spin costs ~30% fps (Deck LCD) - GPU-written 691x432 texture re-uploaded per present + page-capture storm #750

p0lux referenced this pull request

2026-11-15 11:53:07 +00:00

fix(vk): a mesh fallback program with bigger payloads reuses the record arena #551

p0lux referenced this pull request

2026-11-15 11:53:08 +00:00

perf(vk): scan pending uniform updates in place #472

omgitsgela referenced this pull request

2026-11-15 12:46:01 +00:00

Game: Hyrule Town spin costs ~30% fps (Deck LCD) - GPU-written 691x432 texture re-uploaded per present + page-capture storm #750

mooki added the

labels

2026-11-15 13:55:58 +00:00

HumanBean force-pushed perf/zero-upload-holes from a1fdd59cd3

PR description / Description follows the template (pull_request_target) Successful in 15s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

Third-party parts / The same as upstream (pull_request) Has been cancelled

Details

Website / Website on Cloudflare (pull_request) Has been cancelled

Details

to e832dd0760

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 4s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 4s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

2026-11-15 18:44:29 +00:00

Compare

HumanBean commented

2026-11-15 18:46:13 +00:00

Owner

Rebased onto main @ bbe6fdf2. No tests were run here; the test boxes stay yours. One conflict in gpu/vk/vk_memory.cpp: the include list (main's linux_pressure.h and this PR's guest_holes.h), both kept.

Rebased onto `main` @ bbe6fdf2. No tests were run here; the test boxes stay yours. One conflict in gpu/vk/vk_memory.cpp: the include list (main's `linux_pressure.h` and this PR's `guest_holes.h`), both kept.

HumanBean reviewed

2026-11-15 19:04:23 +00:00

HumanBean left a comment

Code review only (nothing built or run). This PR duplicates 790 and the two cannot both land.

Both add gpu/vk/guest_holes.h, tests/test_guest_holes.cpp and tests/test_guest_holes.py, both rewrite the zero check in upload_all_zero (gpu/vk/vk_texture.cpp) and both add the RX_VK_ZERO_HOLES switch (SEEK_DATA/SEEK_HOLE on the guest memfd). The second to merge gets an add/add conflict. This is the older PR and the better base, in my reading: it fixes the share rounding in the old read (a tail of up to threads - 1 bytes was never read when total / threads was already a multiple of 4096), it spreads the threads over the data rather than over the address range, it is rebased, and its lookup sits in vk_memory.cpp with no vk_api.h change. The authors should agree on one PR; 790's author should close theirs or rebase onto this one.

Code review of this PR, no blocking bug found:

data_parts: offsets are relative to lo; ENXIO means no more data; a failed or non-advancing seek clears the parts and returns false, so the caller reads everything. A page in swap counts as data in shmem, so a swapped-out page is read, not skipped.
all_zero: per is the ceiling rounded to 4096, so threads * per >= data; the skip and left bookkeeping across parts is right; the early return is in a void lambda, fine.
memory_guest_data_parts: bounds check cannot overflow, the memfd is looked up once, and non-Linux returns false.

Small things, none blocking, and good to take from 790 while merging the two:

The number of parts is not capped. 790 caps at 1024 extents and reads the rest whole. A heavily fragmented storage could give tens of thousands of parts, each costing two lseek calls. Probably harmless, but a cap is cheap.
RX_VK_ZERO_HOLES is read with atoi(...) == 0, so off, false or no do not turn it off; only 0 does. The docs row says =0, so it matches, but the rest of the switches use env_value_off. Also the static is named read_holes and is true when the switch is off, which reads backwards.
The log line on the "is not all zero" path does not say how much was read; the all-zero line does.

Leaving this as a comment, not an approval, because it cannot merge alongside 790 and the description should name 790 and say which PR supersedes which.

Code review only (nothing built or run). **This PR duplicates [790](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/790) and the two cannot both land.** Both add `gpu/vk/guest_holes.h`, `tests/test_guest_holes.cpp` and `tests/test_guest_holes.py`, both rewrite the zero check in `upload_all_zero` (`gpu/vk/vk_texture.cpp`) and both add the `RX_VK_ZERO_HOLES` switch (SEEK_DATA/SEEK_HOLE on the guest memfd). The second to merge gets an add/add conflict. This is the older PR and the better base, in my reading: it fixes the share rounding in the old read (a tail of up to `threads - 1` bytes was never read when `total / threads` was already a multiple of 4096), it spreads the threads over the data rather than over the address range, it is rebased, and its lookup sits in `vk_memory.cpp` with no `vk_api.h` change. The authors should agree on one PR; 790's author should close theirs or rebase onto this one. Code review of this PR, no blocking bug found: - `data_parts`: offsets are relative to `lo`; `ENXIO` means no more data; a failed or non-advancing seek clears the parts and returns false, so the caller reads everything. A page in swap counts as data in shmem, so a swapped-out page is read, not skipped. - `all_zero`: `per` is the ceiling rounded to 4096, so `threads * per >= data`; the skip and left bookkeeping across parts is right; the early `return` is in a void lambda, fine. - `memory_guest_data_parts`: bounds check cannot overflow, the memfd is looked up once, and non-Linux returns false. Small things, none blocking, and good to take from 790 while merging the two: 1. The number of parts is not capped. 790 caps at 1024 extents and reads the rest whole. A heavily fragmented storage could give tens of thousands of parts, each costing two `lseek` calls. Probably harmless, but a cap is cheap. 2. `RX_VK_ZERO_HOLES` is read with `atoi(...) == 0`, so `off`, `false` or `no` do not turn it off; only `0` does. The docs row says `=0`, so it matches, but the rest of the switches use `env_value_off`. Also the static is named `read_holes` and is true when the switch is off, which reads backwards. 3. The log line on the "is not all zero" path does not say how much was read; the all-zero line does. Leaving this as a comment, not an approval, because it cannot merge alongside 790 and the description should name 790 and say which PR supersedes which.

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 4s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 4s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

This pull request has changes conflicting with the target branch.

gpu/vk/vk_texture.cpp

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title

perf(vk): the all-zero check of a big texture reads only what the game wrote of it (Linux) (#567)

Merge commit body

Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/567

perf(vk): the all-zero check of a big texture reads only what the game wrote of it (Linux) (#567)

Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/567