perf(lift): lifted stores mark only GPU pages, by a page bitmap (RX_MARK_BITMAP) #1617

Merged
HumanBean merged 3 commits from dyst/Longshot:perf/mark-bitmap into main 2026-11-18 18:43:45 +00:00
Member

Type: perf · Risk: medium · Tested on: Windows 11, RTX 3080, Ryzen 9 3900X · Needs: a check that the picture is unchanged; Linux, AMD, Intel and Steam Deck runs; this branch measured on its own

This is ballotstomper's work (their commit 72db7c7d on their fork's perf/rdna2), brought onto main so the measurements below have a place to live. It is opened as WIP to be discussed, not merged as it is; ballotstomper, take it over, fold it into your own PR, or say how you'd like it done.

TL;DR

Each store the translated game makes also wrote a mark for its memory page, so that the renderer could find what the game changed. The renderer only reads the marks of pages the GPU uses, and most of the game's stores are to other memory. This skips the mark unless the store is on a page the GPU has mapped, tested with a small bitmap. On a CPU-limited PC it raised the Market's frame rate by about 11% on its own, and by 17-22% together with building all the translated code at -O2. It is on by default (RX_MARK_BITMAP=0 when building turns it off), and the limitations below are real: read them before trusting the numbers.

Breakdown

  • Who it helps: players whose CPU limits the frame (the Market, Castle Town, Hyrule Field on mid-range CPUs).
  • What was wrong: every store of the translated code wrote 2 bytes into the page-mark map. That was about 2.2-2.7 ms of the game threads' time a frame while moving (ballotstomper's no-marks timing build).
  • What changes:
    • host/mark_bitmap.c (new): a bit for each page the GPU can read. A page's bit is set when a GPU mapping covering it is made, and for the page before it too. Bits are never cleared.
    • host/nvdrv.c: map_add sets the bits before the mapping counts.
    • nx2/rtgen.py: in an A64_MARK_BITMAP build, A64_MARK marks only when the store's first byte is on such a page.
    • nx2/build.py: the translated code is compiled with that define unless RX_MARK_BITMAP=0. The host's own writes keep marking every page.
  • What does not change: a build with RX_MARK_BITMAP=0: the translated C and the runtime header are as before, every lifted store marks its page.
  • How it was checked: timed A/B runs in the game (below); the checks that read the marks and rtgen.py pass with it off.
  • What is still open: everything in Risks.
  • Risk: medium. A missed mark means the renderer misses a write and draws stale data.

Why

The wiki's CPU rework plan (track D, the write marks rule; question 1) asks what the marks cost the frame. ballotstomper measured it on Linux: a build without the lifted code's marks ran the game's threads 2.2-2.7 ms a frame faster while moving. With this bitmap the Market walk went from 40.4 to 46.1 fps on a Ryzen 5 5600X and RX 6750 XT. With every page's bit set again (HOS_MARK_BITMAP_ALL=1), it walked at 41.3, so the bitmap is nearly all of that gain. The numbers below check it on another OS, GPU vendor and CPU.

#1558 (HumanBean) does the same thing another way, filtering the marks by the GPU's backing ranges, on Linux. dukenheimer measured no frame-rate gain from it on a Steam Deck. The two should become one.

What changed

  • host/mark_bitmap.c (new): a64_gpu_pages, 16 MB of zeroed static data for the 512 GB guest window, of which only the heap's part is ever touched (~210 KB). hos_gpu_pages_set(guest, size) sets the bits. HOS_MARK_BITMAP_ALL=1 sets every bit at the start.
  • host/nvdrv.c: map_add calls hos_gpu_pages_set when the mapping's GPU address is the guest address.
  • nx2/rtgen.py: A64_GPU_PAGE(a) and the bitmap form of A64_MARK, under #if defined(A64_MARK_BITMAP).
  • nx2/build.py: MARK_BITMAP, added to the translated code's flags unless RX_MARK_BITMAP=0 (or off, false, no). Adapted from ballotstomper's hunk, which joined a RX_NO_GUEST_MARKS switch main doesn't have.
  • gpu/vk/switches.def: HOS_MARK_BITMAP_ALL's row.

Measurements

Verdict: faster where the CPU limits the frame: +11% in the Market from the bitmap, +17-22% with all translated code at -O2; +5-7% in Hyrule Field. But the measured builds were not this branch alone (see "What was measured").

What was measured

  • The PC: Windows 11, Ryzen 9 3900X (12 cores), RTX 3080 10 GB (NVIDIA driver), 64 GB. CPU-limited: in the Market the game's main thread waits about 12 ms of its 15 ms a frame for the game's workers.
  • The route: PR #801's A/B kit from a Hyrule Field save: a walk and a look round in Hyrule Field, then a teleport to the Market and a walk there; the frame trace (RX_VK_FRAME_TRACE=1) gives the median frame time by area.
  • The method: each run a timing job of the PC's queue (nothing else on the GPU or building). Each side warmed up once, then 3 rounds with the order reversed every other round.
  • The builds: all on 310e7c9d, through scripts/dev_build.py pack. main is the base alone. The other sides also carry:
    • #1560 (the lift's emit hooks; its translated C is byte-identical to main's);
    • #1469's local registers (6 registers, flags left in the context) with #1587's fix. Measured alone they are within noise: +0.2% Hyrule Field, +1.6% Market over 3 rounds;
    • the whole-buffer zstd decode in 32 KB pieces (an unopened branch of mine, aimed at hitches while streaming, not the median frame);
    • the bitmap, with A64_MARK_BITMAP always defined. The same code as here, with the switch on.
  • "-O2": LONGSHOT_CHUNK_OPT=all LONGSHOT_CHUNK_LEVEL=O2, an existing build option, not part of this PR. On this PC the pack took 51 minutes against 18 at -O1.

Results

Median frame rate by area, from the frame trace's median frame time:

Series Side Hyrule Field Market Game threads, Market
2 (3 rounds) main 45.9 fps 30.9 fps 31.1 ms
2 bitmap 48.6 (+6.0%) 34.3 (+10.9%) 28.5 ms
2 bitmap, -O2 48.0 (+4.7%) 36.2 (+17.0%) 26.0 ms
3 (1 round) main 49.2 33.0 29.2 ms
3 bitmap, -O2 52.6 (+6.9%) 40.1 (+21.6%) 23.9 ms
3 bitmap, -O3 51.5 (+4.7%) 39.1 (+18.4%) 24.6 ms
  • main differs between series: 30.9 fps in the Market in series 2, 33.0 in series 3, and 33.3 in an earlier series. Against the faster main, series 2's bitmap at -O2 is about +9%. Series 3 stopped after its first round when the disk filled, so it is one pair.
  • The tails (medians of series 2's runs, Market): p99 72.8 ms on main, 57.6 with the bitmap, 59.6 with -O2; frames over 50 ms 139, 54 and 59. Hyrule Field's p99 is about the same on every side (48.6-51.1 ms).
  • No gain: -O3 against -O2 (one round, -2.5%); 8 virtual cores against 6 (HOS_CORES=8, the warm-up only, slightly slower).

Flags and settings

On by default at build time; RX_MARK_BITMAP=0 turns it off. HOS_MARK_BITMAP_ALL=1 at run time: every page's bit set, the A/B's control.

Testing

Tested:

  • In the game, Windows 11, RTX 3080, Ryzen 9 3900X: 14 runs of the route in builds carrying this change (with the additions under "What was measured"; at -O1, -O2 and -O3). Each played to its end: no crash or hang, and every game patch applied.
  • With it off (RX_MARK_BITMAP=0), tests/test_mark_groups.py, test_exclusive_marks.py, test_fpx_sticky.py, test_fastfp_override.py, test_split_giants.py and test_switches_removed.py pass. test_switch_table.py's failures are main's own; this row isn't among them. test_libc_hooks.py fails here and on main alike.

Not tested:

  • Whether the picture is right. No screenshots were compared, and nobody watched these runs. A missed mark draws stale data, so this is the first thing to check.
  • This branch alone, built and run. The numbers are from builds with the additions above.
  • The HOS_MARK_BITMAP_ALL=1 control, here. ballotstomper's Linux run showed it gives the old speed back.
  • No game-free check covers A64_MARK_BITMAP yet.
  • Linux, AMD and Intel GPUs, the Steam Deck, and any other CPU. ballotstomper's Linux runs (RX 6750 XT, Ryzen 5 5600X) were on their branch, with other changes.
  • tests/run_all.py in full.

Must be tested:

  • Any PC: the same scenes with and without it, picture compared (screenshots, or RX_VK_DUMP_FILE): the Market, Kokiri Forest, a cutscene, a menu, a load.
  • Linux, the Steam Deck, AMD, Intel: the Market walk with and without it (Test Farm: linux, steamdeck, amd, intel; the Market walk, this branch against main).

Risks

What could break:

  • Pages a mapping doesn't cover. map_add sets bits only for mappings whose GPU address is the guest address (va == guest). The game's stores to a page the GPU reads through any other mapping, or reads without a mapping map_add sees, would no longer be marked, and the renderer would draw that memory stale.
  • A store that starts on a page without a bit and crosses into one with a bit. The page before each mapping is set for exactly this case.
  • Bits are never cleared. Memory the GPU once used stays marked, which is as before, so no correctness risk, only a little speed.
  • 16 MB of static data in the executable's zero-filled data, untouched except the heap's part.

Turning it off: build with RX_MARK_BITMAP=0; HOS_MARK_BITMAP_ALL=1 in a build with it.

Compatibility: nothing changes for a player's files.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why. In builds with this change and others; the picture not compared.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks. The picture is unchecked.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main. On 310e7c9d, where it was measured.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). The checks above only.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A new RX_, HOS_ or LONGSHOT_ switch has its row in gpu/vk/switches.def.

🤖 Generated with Claude Code

**Type:** perf · **Risk:** medium · **Tested on:** Windows 11, RTX 3080, Ryzen 9 3900X · **Needs:** a check that the picture is unchanged; Linux, AMD, Intel and Steam Deck runs; this branch measured on its own **This is ballotstomper's work** (their commit 72db7c7d on their fork's `perf/rdna2`), brought onto main so the measurements below have a place to live. It is opened as WIP to be discussed, not merged as it is; ballotstomper, take it over, fold it into your own PR, or say how you'd like it done. ## TL;DR Each store the translated game makes also wrote a mark for its memory page, so that the renderer could find what the game changed. The renderer only reads the marks of pages the GPU uses, and most of the game's stores are to other memory. This skips the mark unless the store is on a page the GPU has mapped, tested with a small bitmap. On a CPU-limited PC it raised the Market's frame rate by about 11% on its own, and by 17-22% together with building all the translated code at `-O2`. It is on by default (`RX_MARK_BITMAP=0` when building turns it off), and the limitations below are real: read them before trusting the numbers. ## Breakdown - **Who it helps:** players whose CPU limits the frame (the Market, Castle Town, Hyrule Field on mid-range CPUs). - **What was wrong:** every store of the translated code wrote 2 bytes into the page-mark map. That was about 2.2-2.7 ms of the game threads' time a frame while moving (ballotstomper's no-marks timing build). - **What changes:** - `host/mark_bitmap.c` (new): a bit for each page the GPU can read. A page's bit is set when a GPU mapping covering it is made, and for the page before it too. Bits are never cleared. - `host/nvdrv.c`: `map_add` sets the bits before the mapping counts. - `nx2/rtgen.py`: in an `A64_MARK_BITMAP` build, `A64_MARK` marks only when the store's first byte is on such a page. - `nx2/build.py`: the translated code is compiled with that define unless `RX_MARK_BITMAP=0`. The host's own writes keep marking every page. - **What does not change:** a build with `RX_MARK_BITMAP=0`: the translated C and the runtime header are as before, every lifted store marks its page. - **How it was checked:** timed A/B runs in the game (below); the checks that read the marks and `rtgen.py` pass with it off. - **What is still open:** everything in Risks. - **Risk:** medium. A missed mark means the renderer misses a write and draws stale data. ## Why The wiki's CPU rework plan (track D, the write marks rule; question 1) asks what the marks cost the frame. ballotstomper measured it on Linux: a build without the lifted code's marks ran the game's threads 2.2-2.7 ms a frame faster while moving. With this bitmap the Market walk went from 40.4 to 46.1 fps on a Ryzen 5 5600X and RX 6750 XT. With every page's bit set again (`HOS_MARK_BITMAP_ALL=1`), it walked at 41.3, so the bitmap is nearly all of that gain. The numbers below check it on another OS, GPU vendor and CPU. #1558 (HumanBean) does the same thing another way, filtering the marks by the GPU's backing ranges, on Linux. dukenheimer measured no frame-rate gain from it on a Steam Deck. The two should become one. ## What changed - `host/mark_bitmap.c` (new): `a64_gpu_pages`, 16 MB of zeroed static data for the 512 GB guest window, of which only the heap's part is ever touched (~210 KB). `hos_gpu_pages_set(guest, size)` sets the bits. `HOS_MARK_BITMAP_ALL=1` sets every bit at the start. - `host/nvdrv.c`: `map_add` calls `hos_gpu_pages_set` when the mapping's GPU address is the guest address. - `nx2/rtgen.py`: `A64_GPU_PAGE(a)` and the bitmap form of `A64_MARK`, under `#if defined(A64_MARK_BITMAP)`. - `nx2/build.py`: `MARK_BITMAP`, added to the translated code's flags unless `RX_MARK_BITMAP=0` (or off, false, no). Adapted from ballotstomper's hunk, which joined a `RX_NO_GUEST_MARKS` switch main doesn't have. - `gpu/vk/switches.def`: `HOS_MARK_BITMAP_ALL`'s row. ## Measurements **Verdict:** faster where the CPU limits the frame: +11% in the Market from the bitmap, +17-22% with all translated code at `-O2`; +5-7% in Hyrule Field. **But the measured builds were not this branch alone** (see "What was measured"). ### What was measured - **The PC:** Windows 11, Ryzen 9 3900X (12 cores), RTX 3080 10 GB (NVIDIA driver), 64 GB. CPU-limited: in the Market the game's main thread waits about 12 ms of its 15 ms a frame for the game's workers. - **The route:** PR #801's A/B kit from a Hyrule Field save: a walk and a look round in Hyrule Field, then a teleport to the Market and a walk there; the frame trace (`RX_VK_FRAME_TRACE=1`) gives the median frame time by area. - **The method:** each run a timing job of the PC's queue (nothing else on the GPU or building). Each side warmed up once, then 3 rounds with the order reversed every other round. - **The builds:** all on `310e7c9d`, through `scripts/dev_build.py pack`. **main** is the base alone. The other sides also carry: - #1560 (the lift's emit hooks; its translated C is byte-identical to main's); - #1469's local registers (6 registers, flags left in the context) with #1587's fix. Measured alone they are within noise: +0.2% Hyrule Field, +1.6% Market over 3 rounds; - the whole-buffer zstd decode in 32 KB pieces (an unopened branch of mine, aimed at hitches while streaming, not the median frame); - the bitmap, with `A64_MARK_BITMAP` always defined. The same code as here, with the switch on. - **"-O2":** `LONGSHOT_CHUNK_OPT=all LONGSHOT_CHUNK_LEVEL=O2`, an existing build option, not part of this PR. On this PC the pack took 51 minutes against 18 at `-O1`. ### Results Median frame rate by area, from the frame trace's median frame time: | Series | Side | Hyrule Field | Market | Game threads, Market | |---|---|---|---|---| | 2 (3 rounds) | main | 45.9 fps | 30.9 fps | 31.1 ms | | 2 | bitmap | 48.6 (+6.0%) | 34.3 (+10.9%) | 28.5 ms | | 2 | bitmap, `-O2` | 48.0 (+4.7%) | 36.2 (+17.0%) | 26.0 ms | | 3 (1 round) | main | 49.2 | 33.0 | 29.2 ms | | 3 | bitmap, `-O2` | 52.6 (+6.9%) | 40.1 (+21.6%) | 23.9 ms | | 3 | bitmap, `-O3` | 51.5 (+4.7%) | 39.1 (+18.4%) | 24.6 ms | - **main differs between series:** 30.9 fps in the Market in series 2, 33.0 in series 3, and 33.3 in an earlier series. Against the faster main, series 2's bitmap at `-O2` is about +9%. Series 3 stopped after its first round when the disk filled, so it is one pair. - **The tails** (medians of series 2's runs, Market): p99 72.8 ms on main, 57.6 with the bitmap, 59.6 with `-O2`; frames over 50 ms 139, 54 and 59. Hyrule Field's p99 is about the same on every side (48.6-51.1 ms). - **No gain:** `-O3` against `-O2` (one round, -2.5%); 8 virtual cores against 6 (`HOS_CORES=8`, the warm-up only, slightly slower). ### Flags and settings On by default at build time; `RX_MARK_BITMAP=0` turns it off. `HOS_MARK_BITMAP_ALL=1` at run time: every page's bit set, the A/B's control. ## Testing **Tested:** - In the game, Windows 11, RTX 3080, Ryzen 9 3900X: 14 runs of the route in builds carrying this change (with the additions under "What was measured"; at `-O1`, `-O2` and `-O3`). Each played to its end: no crash or hang, and every game patch applied. - With it off (`RX_MARK_BITMAP=0`), `tests/test_mark_groups.py`, `test_exclusive_marks.py`, `test_fpx_sticky.py`, `test_fastfp_override.py`, `test_split_giants.py` and `test_switches_removed.py` pass. `test_switch_table.py`'s failures are main's own; this row isn't among them. `test_libc_hooks.py` fails here and on main alike. **Not tested:** - **Whether the picture is right.** No screenshots were compared, and nobody watched these runs. A missed mark draws stale data, so this is the first thing to check. - **This branch alone, built and run.** The numbers are from builds with the additions above. - **The HOS_MARK_BITMAP_ALL=1 control,** here. ballotstomper's Linux run showed it gives the old speed back. - **No game-free check** covers `A64_MARK_BITMAP` yet. - Linux, AMD and Intel GPUs, the Steam Deck, and any other CPU. ballotstomper's Linux runs (RX 6750 XT, Ryzen 5 5600X) were on their branch, with other changes. - `tests/run_all.py` in full. **Must be tested:** - Any PC: the same scenes with and without it, picture compared (screenshots, or `RX_VK_DUMP_FILE`): the Market, Kokiri Forest, a cutscene, a menu, a load. - Linux, the Steam Deck, AMD, Intel: the Market walk with and without it (Test Farm: linux, steamdeck, amd, intel; the Market walk, this branch against main). ## Risks **What could break:** - **Pages a mapping doesn't cover.** `map_add` sets bits only for mappings whose GPU address is the guest address (`va == guest`). The game's stores to a page the GPU reads through any other mapping, or reads without a mapping `map_add` sees, would no longer be marked, and the renderer would draw that memory stale. - **A store that starts on a page without a bit and crosses into one with a bit.** The page before each mapping is set for exactly this case. - **Bits are never cleared.** Memory the GPU once used stays marked, which is as before, so no correctness risk, only a little speed. - **16 MB of static data** in the executable's zero-filled data, untouched except the heap's part. **Turning it off:** build with `RX_MARK_BITMAP=0`; `HOS_MARK_BITMAP_ALL=1` in a build with it. **Compatibility:** nothing changes for a player's files. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [ ] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. *In builds with this change and others; the picture not compared.* - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. *The picture is unchecked.* - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [ ] Rebased on current `main`. *On `310e7c9d`, where it was measured.* - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). *The checks above only.* - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A new `RX_`, `HOS_` or `LONGSHOT_` switch has its row in `gpu/vk/switches.def`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Every store of the lifted game wrote its page's mark into the dirty map, which the renderer reads for GPU pages only; OoT's
GPU pools lie in the heap among its CPU data. host/mark_bitmap.c keeps a bit per page the GPU can read, set when a GPU
mapping is made (nvdrv.c map_add) and for the page before it, and A64_MARK (nx2/rtgen.py) marks only when a store's first
byte is on such a page. HOS_MARK_BITMAP_ALL=1 sets every bit at the start: every store marks again, plus the test.

ballotstomper's work: their commit 72db7c7d on their fork's perf/rdna2, brought onto main as it is, except the build.py
hunk (main has no RX_NO_GUEST_MARKS for it to join).

Co-Authored-By: ballotstomper <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
chore(vk): HOS_MARK_BITMAP_ALL's row in the switch table
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 9s
PR description / Description follows the template (pull_request_target) Successful in 11s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 12s
CI / Workflow lint (actionlint) (pull_request) Successful in 13s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 15s
beanstalk/approval needs a second approval (sensitive paths)
410b07458c
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

Tested in game: Windows 11, RTX 3080 (NVIDIA driver), Ryzen 9 3900X: 14 runs of PR #801's route (a Hyrule Field walk and look round, then the Market) in packs over 310e7c9d that carry this change with A64_MARK_BITMAP on, together with #1560, #1469's local registers + #1587, and a zstd change of mine; all played to the end with no crash or hang. Not yet: this branch built and run alone, and the picture compared with main's (no screenshots were taken).

Tested in game: Windows 11, RTX 3080 (NVIDIA driver), Ryzen 9 3900X: 14 runs of PR #801's route (a Hyrule Field walk and look round, then the Market) in packs over 310e7c9d that carry this change with A64_MARK_BITMAP on, together with #1560, #1469's local registers + #1587, and a zstd change of mine; all played to the end with no crash or hang. Not yet: this branch built and run alone, and the picture compared with main's (no screenshots were taken).
Owner

@Beanstalk

@Beanstalk
Member

@HumanBean approval of 410b074 recorded. This diff touches sensitive paths, so a second member of BigBeans must approve it too.

@HumanBean approval of `410b074` recorded. This diff touches sensitive paths, so a second member of `BigBeans` must approve it too.
Member

Beanstalk test farm

Commit ce2dd9e · tier untrusted · approved by nobody yet · dashboard

This diff touches sensitive paths, so it needs two members of BigBeans: nx2/build.py, nx2/rtgen.py

A member of BigBeans can review this commit and comment @Beanstalk to test it.

<!-- beanstalk:sticky --> ### Beanstalk test farm Commit `ce2dd9e` · tier **untrusted** · approved by nobody yet · [dashboard](https://beanstalk.magicbean.dev/prs/1617) This diff touches sensitive paths, so it needs two members of `BigBeans`: `nx2/build.py`, `nx2/rtgen.py` A member of `BigBeans` can review this commit and comment `@Beanstalk` to test it.
First-time contributor

A data point from a second PC, since the description says the two were not yet measured together: this change stacks with #1576 (registers as locals, vector rewrites).

Windows 11, Ryzen 9 9900X, RTX 4070 SUPER, 4K DLSS Quality, 90 cap, the Castle Town market on a recorded route, ballotstomper's commit 72db7c7d picked onto #1576's code (not this branch's head), RX_MARK_BITMAP=1. Median game work a frame, one run each, the same five runtime switches on every side:

build game work
main 430eef4f 16.4 ms
#1576 14.2 ms
#1576 + the mark bitmap, -O1 12.7 ms
#1576 + the mark bitmap, all chunks -O2 12.0 ms

So about 1.5 ms from the bitmap on top of #1576 here. At 4K this PC is GPU-limited near 70 fps with it, so the fps hardly moves between the last two rows. No picture difference seen by eye in several hours of play on the -O2 build; not compared by screenshots.

Measured and written with the help of an AI coding agent.

A data point from a second PC, since the description says the two were not yet measured together: this change stacks with #1576 (registers as locals, vector rewrites). Windows 11, Ryzen 9 9900X, RTX 4070 SUPER, 4K DLSS Quality, 90 cap, the Castle Town market on a recorded route, ballotstomper's commit 72db7c7d picked onto #1576's code (not this branch's head), `RX_MARK_BITMAP=1`. Median game work a frame, one run each, the same five runtime switches on every side: | build | game work | |---|---| | main 430eef4f | 16.4 ms | | #1576 | 14.2 ms | | #1576 + the mark bitmap, `-O1` | 12.7 ms | | #1576 + the mark bitmap, all chunks `-O2` | 12.0 ms | So about 1.5 ms from the bitmap on top of #1576 here. At 4K this PC is GPU-limited near 70 fps with it, so the fps hardly moves between the last two rows. No picture difference seen by eye in several hours of play on the `-O2` build; not compared by screenshots. Measured and written with the help of an AI coding agent.
perf(lift): the page bitmap on by default; RX_MARK_BITMAP=0 when building turns it off
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 12s
beanstalk/approval not approved yet
PR description / Description follows the template (pull_request_target) Successful in 4s
afb21bbfed
The lifted game's stores mark only GPU pages unless the build sets RX_MARK_BITMAP=0 (or off, false, no). The
measured gain was with it on (+11% in the Market); the picture check and the Linux, AMD, Intel and Steam Deck runs
the PR names are still owed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

afb21bbf: the page bitmap is now on by default (nx2/build.py: the translated code gets -DA64_MARK_BITMAP unless RX_MARK_BITMAP=0, off, false or no); HOS_MARK_BITMAP_ALL's row says so. The description's default lines are updated. Still owed before this can leave WIP: the picture check (same frames with and without it), and the Linux, Steam Deck, AMD and Intel runs. ballotstomper, the default flip is the only change to your code.

afb21bbf: the page bitmap is now on by default (nx2/build.py: the translated code gets -DA64_MARK_BITMAP unless RX_MARK_BITMAP=0, off, false or no); HOS_MARK_BITMAP_ALL's row says so. The description's default lines are updated. Still owed before this can leave WIP: the picture check (same frames with and without it), and the Linux, Steam Deck, AMD and Intel runs. ballotstomper, the default flip is the only change to your code.
Author
Member

Rebased onto main c507b98c (30 commits on), cleanly: 96e50eb5, 566c43a5, ce2dd9e8. test_mark_groups, test_exclusive_marks, test_fastfp_override, test_split_giants and test_switches_removed pass; test_fpx_sticky and test_switch_table fail on this PC as they do on main c507b98c (no C++ headers for that check's compiler here; the switch table's rows main lacks), neither about this change.

Rebased onto main c507b98c (30 commits on), cleanly: 96e50eb5, 566c43a5, ce2dd9e8. test_mark_groups, test_exclusive_marks, test_fastfp_override, test_split_giants and test_switches_removed pass; test_fpx_sticky and test_switch_table fail on this PC as they do on main c507b98c (no C++ headers for that check's compiler here; the switch table's rows main lacks), neither about this change.
dyst force-pushed perf/mark-bitmap from afb21bbfed
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 12s
beanstalk/approval not approved yet
PR description / Description follows the template (pull_request_target) Successful in 4s
to ce2dd9e8a3
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 11s
PR description / Description follows the template (pull_request_target) Failing after 5s
beanstalk/approval not approved yet
2026-11-18 18:35:47 +00:00
Compare
HumanBean changed title from WIP: perf(lift): lifted stores mark only GPU pages, by a page bitmap (RX_MARK_BITMAP) to perf(lift): lifted stores mark only GPU pages, by a page bitmap (RX_MARK_BITMAP) 2026-11-18 18:43:39 +00:00
HumanBean deleted branch perf/mark-bitmap 2026-11-18 18:43:45 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1617
No description provided.