WIP: perf(vk): Stage 0 of the memory-sync redesign: the ceiling report, a content oracle for the write marks, and four unmarked writes fixed #1034

Draft
TrashPanda wants to merge 4 commits from TrashPanda/Longshot:perf/stage0-ceiling-oracle into main
Contributor

Type: perf · Risk: low · Tested on: Linux, RTX 4070 Ti · Needs: none (measurement only; the host fixes are guest-memory marks)

Why

To make the renderer faster we first need to know how far it is from what the game itself allows on a PC, and a way to catch a renderer change that loses a CPU write. Neither existed: scripts/chain.py showed the frame's critical path but couldn't tell the game's own work from the renderer's, and nothing checked that every CPU write to guest memory the GPU reads is marked. This is Stage 0 of a staged renderer memory-sync redesign. Each later stage is judged by these numbers.

What changed

Three measurements, all off unless asked for, plus four guest writes the new check found unmarked.

  • The ceiling report (scripts/chain.py --log): the renderer brackets its work in spans (MkwVkHost.chain_span, MKWVK_SPAN_* in gpu/vk/vk_api.h: the replay, a submit's memory work, the present, and its waits for the GPU) recorded by HOS_CHAIN. The report splits each thread's running time on the frame's path into the game's own and the renderer's, takes the GPU's time from the same run's vk: GPU time lines, and prints the frame time over the ceiling (the larger of the GPU's time and the game's own path; 1.0 means nothing of the port's is above it). One load per span when nothing records.
  • A content oracle (RX_VK_MARK_AUDIT=N, gpu/vk/vk_memory.cpp): every Nth submit it copies a window of the synced pages (mirrored pools, texture storage, the capture ranges) and, a submit later, compares them with guest memory. A page that changed while its mark is still clear is a missed write, logged with its address.
  • A memory line every 600 submits: capture buffers and spares, staging spares, mirrors against their budget, live imports, retired pool records, shadows, stamps and VRAM. scripts/memtour.py sessions can be compared with it.
  • Four writes found unmarked by the oracle and fixed (host/nvn_native.c, host/hos_mem.c, host/hos_svc.c): nvnCommandBufferEndRecording's copy mark in control memory, nvnQueueSubmitCommands' submit counter, and the zeroing in FreeMemoryPages/discard_pages and UnmapPhysicalMemory. Each lay in a mirrored pool, so a capture could keep the page's old bytes and a write-back could put them back.
  • Tests: tests/test_collect_concurrent.py (collect_in finds every mark set before it starts and loses none set while it runs, with threads marking at the same time), tests/test_chain.py (the spans and the report).
  • MKWVK_API_VERSION 25 to 26 (two members appended after #1102's mic_test); tests/test_audio_output_devices.py now checks that the playback and microphone callbacks follow options_path, rather than that they're the struct's last seven members, and it and tests/test_microphone_test_gain.py accept any version from 25 on.

Measurements

Verdict: On this PC both scenes run at 1.26 times the ceiling, which is the GPU here. The renderer adds 1.5-1.7 ms to the frame's path, about 1.1-1.3 ms of it waiting for the GPU on the main thread. Recording costs nothing measurable: 120.1 fps recording against 120.1 not (Kokiri Forest), 114.5 against 112.9 (Hyrule Field). The oracle finds no missed write after the fixes, and the renderer's memory is bounded over a 25-minute tour except for retired pool records (Stage 1).

Hardware: Ubuntu 26.04.1 LTS (kernel 7.0.0-43), RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D, 32 GB
Scene: Kokiri Forest (teleport 0) and Hyrule Field (teleport 1) from the repo save kokiri-start, the camera turning; 1920x1080, no upscaler, HOS_FPS_MAX=240, headless (HOS_NO_SDL=1)
Method: a 20 s window with HOS_CHAIN and a 20 s window without in the same run, fps from HOS_PRESENT_TRACE; python -I scripts/chain.py <chain> --log <log> for the report
How to reproduce: HOS_CHAIN=<file> and touch <file>.go for the window, then the command above
Baseline: main has neither the spans nor the oracle, so there is no main column. The first table is this PR's build with HOS_CHAIN off (Before) and on (After): what recording costs. The build is 37085a07, this branch's first commit before its rebases, with uncommitted Stage 0 changes (-dirty in the docs/tested/ rows); the ceiling table below is from the same runs.

Before, HOS_CHAIN off: this PR @ 37085a07 After, HOS_CHAIN on: this PR @ 37085a07
Kokiri Forest (kok1) fps 120.1 120.1
Hyrule Field (hyr1) fps 112.9 114.5

The ceiling report, the same runs:

Kokiri Forest (kok1) Hyrule Field (hyr1)
frame (ms) 8.28 8.68
GPU (ms) 6.58 6.90
the game's own path (ms) 5.11 5.81
the renderer on the path (ms) 1.70 (1.27 GPU waits) 1.51 (1.06 GPU waits)
frame over ceiling 1.26 1.26
fps recording / not 120.1 / 120.1 114.5 / 112.9
-- the ceiling report
  frame 8.28 ms; the game's own path 5.11 ms; the renderer on the path 1.70 ms (1.27 of it waiting for the GPU)
  GPU 6.58 ms a frame (5 log lines); the ceiling 6.58 ms (the GPU): frame time over ceiling 1.26
-- the ceiling report
  frame 8.68 ms; the game's own path 5.81 ms; the renderer on the path 1.51 ms (1.06 of it waiting for the GPU)
  GPU 6.90 ms a frame (6 log lines); the ceiling 6.90 ms (the GPU): frame time over ceiling 1.26

The content oracle (RX_VK_MARK_AUDIT), the same scenes:

before the host fixes (3 runs) after
Kokiri Forest 1,547, 1,739 and 3,172 missed, of 34.8 M, 35.7 M and 10.9 M pages compared (control memory at +0x1c8, the queue's submit counter, freed texture storage zeroed) 0 missed of 16,482,304
Hyrule Field not run 0 missed of 30,162,944
vk: mark audit 2124: 16384 pages, ..., 0 missed; in all 1547 missed of 34783232 compared      (before)
vk: mark audit 1007: 16384 pages, 25 changed since copied (25 taken by a capture since, 0 marked now, 0 written back by the GPU), 0 missed; in all 0 missed of 16482304 compared      (after)

The renderer's memory over a session: scripts/memtour.py through Zora's Domain, Hyrule Field, Kakariko, the Market, Death Mountain Trail and Goron City, 1 lap and 2 areas (an event at Goron City then refused the teleports), 1,321 memory lines on d66b4ac6 (this branch before its rebase). Captures 70-276 MB, spares 2-32 MB, mirrors up to 1,223 MB of a 2,048 MB budget, live imports flat at 100-137 (about 1.8-2.1 GB) after the first areas, shadows 124-384 MB, stamps 0-7 MB, VRAM 5.4-5.6 GB of 5.65. Only the retired pool records grow: 0 to 159, about 10 per area change, though all of their imports are given back. The process's private memory rose about 1 GB during the first lap, as each new area loaded, and was flat on the second.

vk: renderer memory (600 submits): captures 25, 70 MB (spares 7, 18 MB); staging spares 0 MB; mirrors 15 of 2048 MB; imports 6, 36 MB; retired pools 0, 0 MB (0 mirrors), 0 MB waiting to go back; shadows 0 MB, stamps 0 MB; VRAM 459 of 5825 MB
vk: renderer memory (600 submits): captures 25, 108 MB (spares 5, 10 MB); staging spares 0 MB; mirrors 1047 of 2048 MB; imports 103, 1812 MB; retired pools 159, 5288 MB (0 mirrors), 0 MB waiting to go back; shadows 164 MB, stamps 2 MB; VRAM 5460 of 5652 MB
vk: imports: 103 live (1812 MB), 0 retired still imported (0 MB), 159 retired given back (5288 MB)

Flags and settings

One new switch and its window size, off by default (testing only); HOS_CHAIN gains the spans.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_MARK_AUDIT=N the content oracle, every Nth submit - off a 16,384-page copy and compare a submit; logs missed writes finds unmarked guest writes (four found here) off: for testing renderer changes
RX_VK_MARK_AUDIT_MB=N the oracle's window: how many MB of the synced ranges each audit compares, the next window along each time - 64 a larger window covers the ranges in fewer audits and costs more a submit untested beyond the default 64: only with RX_VK_MARK_AUDIT

Testing

Tested: python -I -B tests/run_all.py --jobs 6 on Linux at the head of the stacked #1037 (this PR plus four commits): 649 passed, 24 failed, 21 skipped. All 24 also fail on main ac778751 on this PC (25 there: test_fsr_uniform_ownership fails on main and passes here, with this PR's change to it): the import checks test_guest_imported, test_pending_guest_import and test_import_reach; test_replay_write_order, test_walk_key, test_run_all_changed, test_make_game_cache and test_bug_report_defaults; and tests this PC can't run (the lz4 and numpy Python modules missing, Windows-only fixtures, a Vulkan 1.3 offscreen device). The renderer (RX_VK_ONLY=1 nx2/build.py --standalone, llvm-mingw 20261104) cross-builds and links for Windows, and the four changed host files (host/rx_backend.c, host/hos_mem.c, host/hos_svc.c, host/nvn_native.c) compile with llvm-mingw for Windows and with clang for Linux (-fsyntax-only with the build's flags and a local lift's headers). No path handling changed. In the game: the scenes above on Linux with an RTX 4070 Ti, with and without HOS_CHAIN, with RX_VK_MARK_AUDIT=1, and the 25-minute memory tour, all with no error in the logs.

Not tested: Windows in the game, and a full Windows link of the game (the host files were compiled for Windows, not linked into the game). That code is portable C, and the host fixes are one HOS_WRITTEN each.

Must be tested: None. The new switch is off by default, and the host fixes only add marks the renderer already handles.

Risks

What could break: the four new marks make the renderer copy those pages again at the next submit: a few bytes a submit, and a freed or unmapped range once.

Turning it off: RX_VK_MARK_AUDIT unset (the default); revert for the marks.

Compatibility: MKWVK_API_VERSION 26, so the game and its renderer must come from the same build (as always). Nothing else changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md).
  • MKWVK_API_VERSION bumped (gpu/vk/vk_api.h's MkwVkHost or MkwVkRenderer changed).
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
**Type:** perf · **Risk:** low · **Tested on:** Linux, RTX 4070 Ti · **Needs:** none (measurement only; the host fixes are guest-memory marks) ## Why To make the renderer faster we first need to know how far it is from what the game itself allows on a PC, and a way to catch a renderer change that loses a CPU write. Neither existed: `scripts/chain.py` showed the frame's critical path but couldn't tell the game's own work from the renderer's, and nothing checked that every CPU write to guest memory the GPU reads is marked. This is Stage 0 of a staged renderer memory-sync redesign. Each later stage is judged by these numbers. ## What changed Three measurements, all off unless asked for, plus four guest writes the new check found unmarked. - **The ceiling report** (`scripts/chain.py --log`): the renderer brackets its work in spans (`MkwVkHost.chain_span`, `MKWVK_SPAN_*` in `gpu/vk/vk_api.h`: the replay, a submit's memory work, the present, and its waits for the GPU) recorded by `HOS_CHAIN`. The report splits each thread's running time on the frame's path into the game's own and the renderer's, takes the GPU's time from the same run's `vk: GPU time` lines, and prints the frame time over the ceiling (the larger of the GPU's time and the game's own path; 1.0 means nothing of the port's is above it). One load per span when nothing records. - **A content oracle** (`RX_VK_MARK_AUDIT=N`, `gpu/vk/vk_memory.cpp`): every Nth submit it copies a window of the synced pages (mirrored pools, texture storage, the capture ranges) and, a submit later, compares them with guest memory. A page that changed while its mark is still clear is a missed write, logged with its address. - **A memory line every 600 submits:** capture buffers and spares, staging spares, mirrors against their budget, live imports, retired pool records, shadows, stamps and VRAM. `scripts/memtour.py` sessions can be compared with it. - **Four writes found unmarked by the oracle and fixed** (`host/nvn_native.c`, `host/hos_mem.c`, `host/hos_svc.c`): `nvnCommandBufferEndRecording`'s copy mark in control memory, `nvnQueueSubmitCommands`' submit counter, and the zeroing in `FreeMemoryPages`/`discard_pages` and `UnmapPhysicalMemory`. Each lay in a mirrored pool, so a capture could keep the page's old bytes and a write-back could put them back. - **Tests:** `tests/test_collect_concurrent.py` (`collect_in` finds every mark set before it starts and loses none set while it runs, with threads marking at the same time), `tests/test_chain.py` (the spans and the report). - `MKWVK_API_VERSION` 25 to 26 (two members appended after #1102's `mic_test`); `tests/test_audio_output_devices.py` now checks that the playback and microphone callbacks follow `options_path`, rather than that they're the struct's last seven members, and it and `tests/test_microphone_test_gain.py` accept any version from 25 on. ## Measurements **Verdict:** On this PC both scenes run at 1.26 times the ceiling, which is the GPU here. The renderer adds 1.5-1.7 ms to the frame's path, about 1.1-1.3 ms of it waiting for the GPU on the main thread. Recording costs nothing measurable: 120.1 fps recording against 120.1 not (Kokiri Forest), 114.5 against 112.9 (Hyrule Field). The oracle finds no missed write after the fixes, and the renderer's memory is bounded over a 25-minute tour except for retired pool records (Stage 1). Hardware: Ubuntu 26.04.1 LTS (kernel 7.0.0-43), RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D, 32 GB Scene: Kokiri Forest (teleport 0) and Hyrule Field (teleport 1) from the repo save `kokiri-start`, the camera turning; 1920x1080, no upscaler, `HOS_FPS_MAX=240`, headless (`HOS_NO_SDL=1`) Method: a 20 s window with `HOS_CHAIN` and a 20 s window without in the same run, fps from `HOS_PRESENT_TRACE`; `python -I scripts/chain.py <chain> --log <log>` for the report How to reproduce: `HOS_CHAIN=<file>` and touch `<file>.go` for the window, then the command above Baseline: main has neither the spans nor the oracle, so there is no main column. The first table is this PR's build with `HOS_CHAIN` off (Before) and on (After): what recording costs. The build is `37085a07`, this branch's first commit before its rebases, with uncommitted Stage 0 changes (`-dirty` in the `docs/tested/` rows); the ceiling table below is from the same runs. | | Before, HOS_CHAIN off: this PR @ 37085a07 | After, HOS_CHAIN on: this PR @ 37085a07 | |---|---|---| | Kokiri Forest (kok1) fps | 120.1 | 120.1 | | Hyrule Field (hyr1) fps | 112.9 | 114.5 | The ceiling report, the same runs: | | Kokiri Forest (kok1) | Hyrule Field (hyr1) | |---|---|---| | frame (ms) | 8.28 | 8.68 | | GPU (ms) | 6.58 | 6.90 | | the game's own path (ms) | 5.11 | 5.81 | | the renderer on the path (ms) | 1.70 (1.27 GPU waits) | 1.51 (1.06 GPU waits) | | frame over ceiling | 1.26 | 1.26 | | fps recording / not | 120.1 / 120.1 | 114.5 / 112.9 | ``` -- the ceiling report frame 8.28 ms; the game's own path 5.11 ms; the renderer on the path 1.70 ms (1.27 of it waiting for the GPU) GPU 6.58 ms a frame (5 log lines); the ceiling 6.58 ms (the GPU): frame time over ceiling 1.26 -- the ceiling report frame 8.68 ms; the game's own path 5.81 ms; the renderer on the path 1.51 ms (1.06 of it waiting for the GPU) GPU 6.90 ms a frame (6 log lines); the ceiling 6.90 ms (the GPU): frame time over ceiling 1.26 ``` **The content oracle** (`RX_VK_MARK_AUDIT`), the same scenes: | | before the host fixes (3 runs) | after | |---|---|---| | Kokiri Forest | 1,547, 1,739 and 3,172 missed, of 34.8 M, 35.7 M and 10.9 M pages compared (control memory at `+0x1c8`, the queue's submit counter, freed texture storage zeroed) | 0 missed of 16,482,304 | | Hyrule Field | not run | 0 missed of 30,162,944 | ``` vk: mark audit 2124: 16384 pages, ..., 0 missed; in all 1547 missed of 34783232 compared (before) vk: mark audit 1007: 16384 pages, 25 changed since copied (25 taken by a capture since, 0 marked now, 0 written back by the GPU), 0 missed; in all 0 missed of 16482304 compared (after) ``` **The renderer's memory over a session:** `scripts/memtour.py` through Zora's Domain, Hyrule Field, Kakariko, the Market, Death Mountain Trail and Goron City, 1 lap and 2 areas (an event at Goron City then refused the teleports), 1,321 memory lines on `d66b4ac6` (this branch before its rebase). Captures 70-276 MB, spares 2-32 MB, mirrors up to 1,223 MB of a 2,048 MB budget, live imports flat at 100-137 (about 1.8-2.1 GB) after the first areas, shadows 124-384 MB, stamps 0-7 MB, VRAM 5.4-5.6 GB of 5.65. Only the retired pool records grow: 0 to 159, about 10 per area change, though all of their imports are given back. The process's private memory rose about 1 GB during the first lap, as each new area loaded, and was flat on the second. ``` vk: renderer memory (600 submits): captures 25, 70 MB (spares 7, 18 MB); staging spares 0 MB; mirrors 15 of 2048 MB; imports 6, 36 MB; retired pools 0, 0 MB (0 mirrors), 0 MB waiting to go back; shadows 0 MB, stamps 0 MB; VRAM 459 of 5825 MB vk: renderer memory (600 submits): captures 25, 108 MB (spares 5, 10 MB); staging spares 0 MB; mirrors 1047 of 2048 MB; imports 103, 1812 MB; retired pools 159, 5288 MB (0 mirrors), 0 MB waiting to go back; shadows 164 MB, stamps 2 MB; VRAM 5460 of 5652 MB vk: imports: 103 live (1812 MB), 0 retired still imported (0 MB), 159 retired given back (5288 MB) ``` ### Flags and settings One new switch and its window size, off by default (testing only); `HOS_CHAIN` gains the spans. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_MARK_AUDIT=N` | the content oracle, every Nth submit | - | off | a 16,384-page copy and compare a submit; logs missed writes | finds unmarked guest writes (four found here) | off: for testing renderer changes | | `RX_VK_MARK_AUDIT_MB=N` | the oracle's window: how many MB of the synced ranges each audit compares, the next window along each time | - | 64 | a larger window covers the ranges in fewer audits and costs more a submit | untested beyond the default | 64: only with `RX_VK_MARK_AUDIT` | ## Testing **Tested:** `python -I -B tests/run_all.py --jobs 6` on Linux at the head of the stacked https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1037 (this PR plus four commits): 649 passed, 24 failed, 21 skipped. All 24 also fail on `main` ac778751 on this PC (25 there: `test_fsr_uniform_ownership` fails on main and passes here, with this PR's change to it): the import checks `test_guest_imported`, `test_pending_guest_import` and `test_import_reach`; `test_replay_write_order`, `test_walk_key`, `test_run_all_changed`, `test_make_game_cache` and `test_bug_report_defaults`; and tests this PC can't run (the `lz4` and `numpy` Python modules missing, Windows-only fixtures, a Vulkan 1.3 offscreen device). The renderer (`RX_VK_ONLY=1 nx2/build.py --standalone`, llvm-mingw 20261104) cross-builds and links for Windows, and the four changed host files (`host/rx_backend.c`, `host/hos_mem.c`, `host/hos_svc.c`, `host/nvn_native.c`) compile with llvm-mingw for Windows and with clang for Linux (`-fsyntax-only` with the build's flags and a local lift's headers). No path handling changed. In the game: the scenes above on Linux with an RTX 4070 Ti, with and without `HOS_CHAIN`, with `RX_VK_MARK_AUDIT=1`, and the 25-minute memory tour, all with no error in the logs. **Not tested:** Windows in the game, and a full Windows link of the game (the host files were compiled for Windows, not linked into the game). That code is portable C, and the host fixes are one `HOS_WRITTEN` each. **Must be tested:** None. The new switch is off by default, and the host fixes only add marks the renderer already handles. ## Risks **What could break:** the four new marks make the renderer copy those pages again at the next submit: a few bytes a submit, and a freed or unmapped range once. **Turning it off:** `RX_VK_MARK_AUDIT` unset (the default); revert for the marks. **Compatibility:** `MKWVK_API_VERSION` 26, so the game and its renderer must come from the same build (as always). Nothing else changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). - [x] `MKWVK_API_VERSION` bumped (`gpu/vk/vk_api.h`'s `MkwVkHost` or `MkwVkRenderer` changed). - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
HOS_CHAIN records ENTER/LEAVE spans around the renderer's calls (submit, present, GPU waits,
texel reads, memory work, slot and late waits); scripts/chain.py splits the critical path into
the game's own work, the renderer's, and GPU waits, and joins it with the GPU timestamps
(--log) into a ceiling report. MKWVK_API_VERSION 24 (chain_span, chain_on).

RX_VK_MARK_AUDIT=N compares a copy of the synced pages with guest memory every Nth submit and
logs differences no write mark covered.

tests/test_collect_concurrent.py: collect_in loses no mark set while it runs.
RX_VK_MARK_AUDIT found three host writes of guest memory that set no write mark, so the
captures kept the pages' old bytes:

- nvnCommandBufferEndRecording's copy mark in the command buffer's control memory
  (9 pages a window in Kokiri Forest, at offset 0x1c8), found with a hardware watchpoint;
- nvnQueueSubmitCommands' submit count in the game's queue object (offset 0x28);
- FreeMemoryPages, discard_pages and UnmapPhysicalMemory zeroing freed pages (12 MB of
  texture storage at one load).

Kokiri Forest and Hyrule Field now audit with 0 missed pages.
Co-authored-by: Cursor <[email protected]>
docs(tested): the Stage 0 baseline runs with the ceiling report (Kokiri Forest, Hyrule Field, RTX 4070 Ti, Linux)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
5b156bfdda
Co-authored-by: Cursor <[email protected]>
Author
Contributor

Tested in game: Linux (Ubuntu 26.04.1, kernel 7.0.0-43), RTX 4070 Ti, NVIDIA 595.99.02, Ryzen 7 9800X3D: Kokiri Forest and Hyrule Field with and without HOS_CHAIN (120.1/120.1 and 114.5/112.9 fps, frame over ceiling 1.26 in both), RX_VK_MARK_AUDIT=1 in both (0 missed writes after the host fixes, of 16.5 M and 30.2 M pages compared), and a 25-minute memtour.py session through six areas; no errors in the logs.

Tested in game: Linux (Ubuntu 26.04.1, kernel 7.0.0-43), RTX 4070 Ti, NVIDIA 595.99.02, Ryzen 7 9800X3D: Kokiri Forest and Hyrule Field with and without HOS_CHAIN (120.1/120.1 and 114.5/112.9 fps, frame over ceiling 1.26 in both), RX_VK_MARK_AUDIT=1 in both (0 missed writes after the host fixes, of 16.5 M and 30.2 M pages compared), and a 25-minute memtour.py session through six areas; no errors in the logs.
TrashPanda force-pushed perf/stage0-ceiling-oracle from 5b156bfdda
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 34c15dcc6f
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-16 04:21:44 +00:00
Compare
TrashPanda force-pushed perf/stage0-ceiling-oracle from 34c15dcc6f
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 52a3547745
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-16 13:55:20 +00:00
Compare
Author
Contributor

Rebased onto main @ 71b1edd8 (head 52a35477). Main's #1102 took MKWVK_API_VERSION 25 for mic_test, so this PR's two chain_span/chain_on members now follow mic_test as version 26. tests/test_audio_output_devices.py checks that the playback and microphone callbacks follow options_path wherever later versions append, and it and tests/test_microphone_test_gain.py accept any version from 25 on, so the next bump doesn't break them. The description says 26 now.

python -I -B tests/run_all.py on 52a35477 (Linux): 635 passed, 22 failed, 21 skipped. All 22 fail the same way on main @ 71b1edd8 on this PC (lift prerequisites and Windows-only fixtures), none from this branch.

Rebased onto main @ 71b1edd8 (head 52a35477). Main's #1102 took `MKWVK_API_VERSION` 25 for `mic_test`, so this PR's two `chain_span`/`chain_on` members now follow `mic_test` as version 26. `tests/test_audio_output_devices.py` checks that the playback and microphone callbacks follow `options_path` wherever later versions append, and it and `tests/test_microphone_test_gain.py` accept any version from 25 on, so the next bump doesn't break them. The description says 26 now. `python -I -B tests/run_all.py` on 52a35477 (Linux): 635 passed, 22 failed, 21 skipped. All 22 fail the same way on main @ 71b1edd8 on this PC (lift prerequisites and Windows-only fixtures), none from this branch.
TrashPanda force-pushed perf/stage0-ceiling-oracle from 52a3547745
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 7046f6da36
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
2026-11-16 19:44:57 +00:00
Compare
Author
Contributor

Rebased onto main @ ac778751 (head 7046f6da; was 52a35477).

Conflict: one hunk in gpu/vk/vk_memory.cpp's take_capture. Main's #977 (1b2e7555, early reads) replaced the late pages' wait for c->late_value with if (wait) { vk_timeline_wait(wait); ... }, where late_defer_split leaves some late pages to their reads. This PR had wrapped the old wait in ChainSpan span(MKWVK_SPAN_LATE_WAIT). Resolved by keeping main's block whole and wrapping its vk_timeline_wait(wait) in the same span. The new wait #977 adds in late_defer_cpu_read (a read reached from memory_read_as_submitted) is left unspanned, like the late_fix waits on that path were already. MKWVK_API_VERSION stays 26: main is still at 25. The other three commits applied as they were.

Tests: the touched checks (test_collect_concurrent, test_chain, test_audio_output_devices, test_microphone_test_gain, test_retired_give_back and every check that reads vk_memory.cpp) pass, except the import and write-order checks that also fail on main. python -I -B tests/run_all.py --jobs 6 at the stacked #1037's head c328a0c4: 649 passed, 24 failed, 21 skipped. All 24 also fail on main ac778751 on this PC (25 there; test_fsr_uniform_ownership passes only with this PR). The Windows renderer cross-builds and links with llvm-mingw 20261104. The four changed host files compile for Windows (llvm-mingw) and Linux (clang).

Description: the Measurements table now names its commit: recording off against on, both on build 37085a07, with a Baseline line explaining why there's no main column. Also added RX_VK_MARK_AUDIT_MB to Flags and the host-code box, and refreshed the test results to this run.

Rebased onto main @ ac778751 (head 7046f6da; was 52a35477). **Conflict:** one hunk in `gpu/vk/vk_memory.cpp`'s `take_capture`. Main's #977 (1b2e7555, early reads) replaced the late pages' wait for `c->late_value` with `if (wait) { vk_timeline_wait(wait); ... }`, where `late_defer_split` leaves some late pages to their reads. This PR had wrapped the old wait in `ChainSpan span(MKWVK_SPAN_LATE_WAIT)`. Resolved by keeping main's block whole and wrapping its `vk_timeline_wait(wait)` in the same span. The new wait #977 adds in `late_defer_cpu_read` (a read reached from `memory_read_as_submitted`) is left unspanned, like the `late_fix` waits on that path were already. `MKWVK_API_VERSION` stays 26: main is still at 25. The other three commits applied as they were. **Tests:** the touched checks (`test_collect_concurrent`, `test_chain`, `test_audio_output_devices`, `test_microphone_test_gain`, `test_retired_give_back` and every check that reads `vk_memory.cpp`) pass, except the import and write-order checks that also fail on main. `python -I -B tests/run_all.py --jobs 6` at the stacked #1037's head c328a0c4: 649 passed, 24 failed, 21 skipped. All 24 also fail on main ac778751 on this PC (25 there; `test_fsr_uniform_ownership` passes only with this PR). The Windows renderer cross-builds and links with llvm-mingw 20261104. The four changed host files compile for Windows (llvm-mingw) and Linux (clang). **Description:** the Measurements table now names its commit: recording off against on, both on build 37085a07, with a Baseline line explaining why there's no main column. Also added `RX_VK_MARK_AUDIT_MB` to Flags and the host-code box, and refreshed the test results to this run.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/stage0-ceiling-oracle:TrashPanda-perf/stage0-ceiling-oracle'
git switch 'TrashPanda-perf/stage0-ceiling-oracle'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1034
No description provided.