WIP: perf(lift): RX_MARK_DIRECT=1, page marks reach their table directly, not through a .refptr slot (compile only, off) #1979

Draft
siantm wants to merge 5 commits from siantm/Longshot:perf/lift-mark-mask into main
Contributor

Summary

A page mark is a shift, a mask into the soft write watch's table and the byte store, plus the table's address, which
MinGW code loads from its .refptr slot, again after guest stores that might alias the slot. With RX_MARK_DIRECT=1
the generated a64.h declares the table hidden (clang; it is the executable's own), so the lifted code and the host's
files reach it directly. The index keeps its mask and the marks stay release stores: only the table's address changes.
Unset or off, a64.h is main's byte for byte, so no lifted object's key changes. Compile only; the build's log names
the mode, and dev.py pack builds the game when it is on. From 1bf5203e on a local performance branch, reworked in
review: the switch was RX_MARK_MASK=0 and also dropped the mask; the reviewers asked to keep the bounds, so that half
is gone and the switch is renamed for what it still does.

The bitmap lookup's plain load (A64_GPU_PAGE, which hos_gpu_pages_set updates atomically) is main's and this PR no
longer touches it: making it atomic changes the default a64.h for every player, so it is left for a change of its own.

Read first: rt_header in nx2/build.py, MARK_DIRECT in nx2/rtgen.py, docs/dev/reference/page-marks.md,
tests/test_mark_direct.py.

Testing

Tested: Windows 11, i7-9700, RTX 4060, llvm-mingw clang 23.1.5, Python 3.14.7 (the Builder's); game-free checks
only. This change has not run in the game on this PC yet, with the switch on or off (Still to test). Each check ran in
a hidden console of its own, through the PC's job queue (job fb2-1979-probe), on this change's tree before its rebase
onto main @ f1f2dc5b (whose one commit touches none of its files; head 327c4f8a after it), the test run with no
arguments (its compiler found through LONGSHOT_MINGW, as tests/run_all.py gives it):

  • python -I -B tests/test_mark_direct.py: exit 0, "all passed": the header main's for every off value and different
    in the table's declaration alone for every on value; the table hidden in the IR with the switch on alone, release
    marks each after its guest store; .refptr.a64_dirty in main's MinGW assembly and none with the switch; seeds 1 to 3,
    O1 and O2, bitmap off and on, the same digest, polls and marks either way (e.g. "seed 3, O2, bitmap on: off
    3b699ce7951861dc 7657 3701, on 3b699ce7951861dc 7657 3701"), the index inside the table at the window's edge.

The same test with tests/test_vk_pgo.py, tests/test_split_giants.py and tests/test_light_passes.py is queued on
327c4f8a itself (job fb2-1979-final); its result goes in a comment.

Earlier, at dd39b843 (the first form, RX_MARK_MASK=0), python -I -B tests/run_all.py --strict --changed origin/main on main @ 2e1f6c32: 98 passed, 17 failed (named in that run's comment as failing on main too; two of them, test_pipeline_states.py and test_recorder_sync.py, had found no Vulkan headers in that run (RX_CPM unset) and compiled nothing on either side),
7 skipped. That run was not repeated on this head. Those two, run again with the headers at 327c4f8a and on main @ f1f2dc5b: test_pipeline_states.py passes on both; test_recorder_sync.py fails on main too with the same lines.

Round 2 (dev.py base and the shared base): the base is keyed by its commit alone, and dev.py base passed the
caller's environment to its build, so a base first built with RX_MARK_DIRECT=1 became the off side of every later A/B. Now
the shared base's build leaves the switch out (scripts/devtools/build.py cmd_base, packs.GAME_COMPILE_SWITCHES).
The branch was rebased unchanged onto main @ 9dd41326 (git range-diff: "=" for the PR's commit). Through the PC's job
queue (job fb2-lift-r2, RX_CPM set, each check in a hidden console of its own, 20 minutes at most):

  • python -I -B tests/test_mark_direct.py at the test commit dfa8408e: exit 1, "1 failed": "dev.py base, RX_MARK_DIRECT=1 then unset: its
    build leaves RX_MARK_DIRECT out, as main builds it"; at the fix, head de30a7f9: exit 0, "all passed" (both orders, one base build,
    without the switch).
  • tests/test_vk_pgo.py and tests/test_light_passes.py at de30a7f9: exit 0 each.

Round 3 (dev.py packs base and a base you name): round 2 left the switch out only in dev.py base. The
documented dev.py packs base runs packs.cmd_base directly, whose build still took RX_MARK_DIRECT from the caller's
environment, so a named base built with it on was what a later dev.py packs rt with it unset copied. Now
packs.cmd_base, where both commands converge, builds with the game's compile-only experiments
(GAME_COMPILE_SWITCHES) taken out of the environment after packs.run merges the caller's; the wrapper's own strip is
gone. A pack with the switch on still builds its own game in that mode. On main @ 9dd41326 (main has not moved; the
earlier commits are unchanged), through the PC's job queue (job fb2-lift-r3b, light, two checks at once at most,
RX_CPM set, each check in a hidden console of its own, 20 minutes at most):

  • python -I -B tests/test_mark_direct.py at the test commit fa3212a5: exit 1, "2 failed": "dev.py packs base, RX_MARK_DIRECT=1 first,
    unset second: the base's build (make_game.py's environment) has no RX_MARK_DIRECT, as main builds it" and "...: dev.py packs
    rt with the switch unset and no pack runs a game built off (got on)"; at the fix, head 4388b51c: exit 0, "all passed"
    (dev.py packs base and dev.py base, each with the switch on first and unset second and the other way round, then
    dev.py packs rt with no pack, and dev.py packs pack and rt with the switch on).
  • tests/test_dev_packs.py and tests/test_dev_build.py at 4388b51c: one failure each, the same on main @ 9dd41326
    in the same job ("every file the renderer keeps beside the game is classified (unknown:
    ['longshot_fsr4_pipelines.bin'])"; Linux.test_folders: C:\Users\User\Games against C:\Users\User/Games).

In the game, on this PC (Windows 11, i7-9700, RTX 4060), 2026-11-22: Kokiri Forest 800x600, RX_MARK_DIRECT unset against
set at build, both at 4388b51c, a warm-up a side then 8 interleaved runs a side, every run in the same scene state: median
calm game time a frame 10.05 against 10.23 ms with it on, 1.8 % higher, no gain shown (leaning to a loss) (Measurements).

Still to test:

  • Windows 11, this PC (i7-9700): a clean build of the game with RX_MARK_DIRECT unset and with it set, its build time and the
    executable's size, expecting figures for Measurements (by hand)
  • Linux: python3 -I -B tests/test_mark_direct.py, expecting it to pass; and the game built with the switch unset and set, played in
    Kokiri Forest, expecting no crash and frame times alike (by hand)
  • Steam Deck: the game with RX_MARK_DIRECT unset and set, Kokiri Forest or the Market, a few interleaved runs a side, expecting
    no crash and frame times alike (by hand)
  • Windows and Linux (Test Farm): the existing request, default lifted C and lift timing with the switch unset against
    main @ 9dd41326, expecting the same lifted code; held until the farm pairs this head with 9dd41326 (the Test Farm request already on this PR)
  • Windows 11, this PC, by hand: if the reviewer wants the lean settled, the same Kokiri Forest comparison again, 8 a side,
    expecting the medians within about 2 % of each other or the lean confirmed (by hand)

Measurements

Verdict: RX_MARK_DIRECT ships off (the maintainer's request, while the enabled and disabled results are gathered). In play,
no gain: the median calm game time a frame is 1.8 % higher with the switch on (10.23 against 10.05 ms), with more long-tail runs on that side (five of eight with p99 above 24 ms, one at 24.3, against one of eight off); inside the spread, but leaning to a loss. Eight runs a side resolve roughly 1.5 to 2 % on the median (an estimate: identical runs of one build spread
about 5 % end to end in calm game time). The switch stays off.

In play: Kokiri Forest, off against on at 4388b51c, 8 runs a side (the current reading)

Setup: Windows 11, i7-9700, RTX 4060, run alone on the PC on 2026-11-22, 06:48 to 07:33 UTC; Kokiri Forest (child), 800x600
(CPU-paced), frame cap off, 60 s a run; a warm-up a side, then 8 interleaved runs a side (off on on off, four times). The
teleport into the scene checked as taken in every run, so every run is in the same (light) scene state; the GPU cooled
to 70 °C before each run, its median clock over each measured stretch its full 2,745 to 2,760 MHz (the few dips are in
the clk_min column). Calm game time: the measured 60 s in six
10 s windows, each window whose median game time is more than 20 % above the run's lowest dropped (a burst), then the
median game time of the frames left; blocked: the game thread's wait over the same frames.

Baseline: this PR with RX_MARK_DIRECT unset (off), which builds with main's flags and header, so it stands for main @ 9dd41326
(the merge base); on = set at build.

Kokiri Forest, 8 runs a side, median (range) Before, switch unset: this PR @ 4388b51c After, RX_MARK_DIRECT=1: this PR @ 4388b51c
calm game ms a frame 10.05 (9.96 to 10.79) 10.23 (10.05 to 10.46)
fps 90.7 (78.0 to 91.6) 85.2 (77.0 to 88.5)
p99 frame ms 16.6 (15.6 to 57.8) 25.4 (17.1 to 64.9)
blocked ms a frame 8.13 (8.02 to 8.78) 8.39 (8.27 to 8.55)
burst windows dropped none 2 runs, 4 windows

Each run (the run's table; game and calm in ms a frame; clock in MHz over the measured stretch):

tag           runtime   fps   med_ms  p99_ms  low1  class  game   calm   bursts  blocked  gpu_start  gpu_max  slowed  cool_wait  clock  clk_min  idle_s
md4-warm-off  mdir-off  86.3  10.88   38.62   25.9  light  9.89   9.76   1       8.33     71         80       0.00    5 s        2760   1875     3
md4-warm-on   mdir-on   90.6  10.66   16.69   59.9  light  10.00  10.00  0       8.16     73         81       0.00    10 s       2760   2760     0
md4-off1      mdir-off  91.4  10.54   16.71   59.8  light  10.01  10.01  0       8.02     74         79       0.00    10 s       2760   2760     0
md4-on2       mdir-on   88.1  10.84   24.33   41.1  light  10.05  10.05  0       8.29     74         81       0.00    10 s       2760   2760     0
md4-on3       mdir-on   78.6  11.26   43.57   22.9  light  10.61  10.23  2       8.55     74         84       0.00    10 s       2745   2685     1
md4-off4      mdir-off  90.2  10.72   15.99   62.5  light  10.14  10.14  0       8.15     73         81       0.00    5 s        2760   2760     0
md4-off5      mdir-off  78.0  10.99   57.75   17.3  light  10.14  10.14  0       8.45     74         84       0.00    10 s       2760   1350     2
md4-on6       mdir-on   77.0  11.02   64.86   15.4  light  10.27  10.27  0       8.42     71         75       0.00    5 s        2760   1860     2
md4-on7       mdir-on   83.4  11.12   26.37   37.9  light  10.46  10.46  0       8.39     70         83       0.00    0 s        2760   2760     0
md4-off8      mdir-off  91.6  10.54   15.72   63.6  light  10.02  10.02  0       8.09     73         81       0.00    5 s        2760   2760     0
md4-off9      mdir-off  91.1  10.59   15.63   64.0  light  10.07  10.07  0       8.04     74         84       0.00    10 s       2760   2760     0
md4-on10      mdir-on   88.0  10.85   20.36   49.1  light  10.24  10.24  0       8.27     73         82       0.00    5 s        2760   2760     0
md4-on11      mdir-on   87.0  10.96   17.72   56.5  light  10.23  10.23  0       8.43     72         84       0.00    5 s        2760   2760     0
md4-off12     mdir-off  91.1  10.58   16.56   60.4  light  9.96   9.96   0       8.11     73         77       0.00    5 s        2760   2760     0
md4-off13     mdir-off  88.1  10.86   16.55   60.4  light  10.03  10.03  0       8.36     74         85       0.00    10 s       2760   2760     0
md4-on14      mdir-on   82.3  11.01   34.41   29.1  light  10.35  10.09  2       8.38     74         81       0.00    5 s        2760   2760     0
md4-on15      mdir-on   88.5  10.83   17.09   58.5  light  10.13  10.13  0       8.32     73         84       0.01    5 s        2745   2745     0
md4-off16     mdir-off  84.2  11.45   18.66   53.6  light  10.79  10.79  0       8.78     73         82       0.00    5 s        2760   2760     0

Clean build time and size, Linux and the Steam Deck are not measured.

Earlier: 4 runs a side, before the wrapper checked the teleport (2026-11-22, 00:15 to 01:03 UTC)

Reading then: not decided, no loss seen. Every run sits in one of two scene states, and both sides land in both.

Setup: Windows 11, i7-9700, RTX 4060, run alone on the PC on 2026-11-22; Kokiri Forest (child), 800x600 (CPU-paced),
frame cap off, 60 s a run; one warm-up a side, then 4 interleaved runs a side (off on on off off on on off). off = RX_MARK_DIRECT
unset, whose lifted code, object keys and executable are those of main @ 9dd41326 (the merge base); on = set at build.

run side fps median frame ms p99 frame ms state
off1 off 69.05 13.57 46.69 heavy
on2 on 81.69 11.98 19.85 light
on3 on 83.19 11.95 17.27 light
off4 off 70.45 13.35 34.72 heavy
off5 off 80.33 12.21 17.56 light
on6 on 69.43 13.47 33.52 heavy
on7 on 80.32 11.97 18.44 light
off8 off 78.29 12.47 25.07 light

Range: off 69.1 to 80.3 fps, on 69.4 to 83.2. The two states: light about 1,100 textures a frame and 76 to 83 fps, heavy
about 1,380 and 67 to 73. The run wrapper asks the Debug teleport into Kokiri Forest, where the save already puts Link; asked
within about 22 frames of entering play, the game refuses it and nothing reloads (heavy), asked later it is taken (light).
Within one state the two sides overlap (heavy: off 69.1 and 70.5 fps, on 69.4; light: off 78.3 and 80.3, on 80.3, 81.7 and 83.2): inside the spread.

Game-free checks

Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5; tests/test_mark_direct.py through the PC's job queue

Before: main @ f1f2dc5b After: this PR @ 327c4f8a
tests/test_mark_direct.py not run (no rt_header on main) all passed: seeds 1 to 3, O1 and O2, bitmap off and on, off and on alike; .refptr.a64_dirty in main's MinGW code, none with the switch

Risks

Only the table's declaration changes: the index keeps its mask, the marks their release order. Off by default: unset,
the build and its header are main's. Turning it off is unsetting it and building again: an executable keeps what it
was built with. Changing it compiles the lifted code and the host's files again, and translates the game again where
the translated C is gone (a player's build folder; a development checkout and LONGSHOT_KEEP_BUILD_FILES=1 keep it).
Nothing a player keeps (saves, longshot_options.ini) changes.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A check under tests/ that reads changed source is updated, not weakened.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary A page mark is a shift, a mask into the soft write watch's table and the byte store, plus the table's address, which MinGW code loads from its `.refptr` slot, again after guest stores that might alias the slot. With `RX_MARK_DIRECT=1` the generated `a64.h` declares the table hidden (clang; it is the executable's own), so the lifted code and the host's files reach it directly. The index keeps its mask and the marks stay release stores: only the table's address changes. Unset or off, `a64.h` is main's byte for byte, so no lifted object's key changes. Compile only; the build's log names the mode, and `dev.py pack` builds the game when it is on. From 1bf5203e on a local performance branch, reworked in review: the switch was `RX_MARK_MASK=0` and also dropped the mask; the reviewers asked to keep the bounds, so that half is gone and the switch is renamed for what it still does. The bitmap lookup's plain load (`A64_GPU_PAGE`, which `hos_gpu_pages_set` updates atomically) is main's and this PR no longer touches it: making it atomic changes the default `a64.h` for every player, so it is left for a change of its own. Read first: `rt_header` in `nx2/build.py`, `MARK_DIRECT` in `nx2/rtgen.py`, `docs/dev/reference/page-marks.md`, `tests/test_mark_direct.py`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060, llvm-mingw clang 23.1.5, Python 3.14.7 (the Builder's); game-free checks only. This change has not run in the game on this PC yet, with the switch on or off (Still to test). Each check ran in a hidden console of its own, through the PC's job queue (job fb2-1979-probe), on this change's tree before its rebase onto `main` @ f1f2dc5b (whose one commit touches none of its files; head 327c4f8a after it), the test run with no arguments (its compiler found through `LONGSHOT_MINGW`, as `tests/run_all.py` gives it): - `python -I -B tests/test_mark_direct.py`: exit 0, "all passed": the header main's for every off value and different in the table's declaration alone for every on value; the table hidden in the IR with the switch on alone, release marks each after its guest store; `.refptr.a64_dirty` in main's MinGW assembly and none with the switch; seeds 1 to 3, O1 and O2, bitmap off and on, the same digest, polls and marks either way (e.g. "seed 3, O2, bitmap on: off 3b699ce7951861dc 7657 3701, on 3b699ce7951861dc 7657 3701"), the index inside the table at the window's edge. The same test with `tests/test_vk_pgo.py`, `tests/test_split_giants.py` and `tests/test_light_passes.py` is queued on 327c4f8a itself (job fb2-1979-final); its result goes in a comment. Earlier, at dd39b843 (the first form, `RX_MARK_MASK=0`), `python -I -B tests/run_all.py --strict --changed origin/main` on `main` @ 2e1f6c32: 98 passed, 17 failed (named in that run's comment as failing on `main` too; two of them, `test_pipeline_states.py` and `test_recorder_sync.py`, had found no Vulkan headers in that run (`RX_CPM` unset) and compiled nothing on either side), 7 skipped. That run was not repeated on this head. Those two, run again with the headers at 327c4f8a and on `main` @ f1f2dc5b: `test_pipeline_states.py` passes on both; `test_recorder_sync.py` fails on `main` too with the same lines. **Round 2** (`dev.py base` and the shared base): the base is keyed by its commit alone, and `dev.py base` passed the caller's environment to its build, so a base first built with `RX_MARK_DIRECT=1` became the off side of every later A/B. Now the shared base's build leaves the switch out (`scripts/devtools/build.py` `cmd_base`, `packs.GAME_COMPILE_SWITCHES`). The branch was rebased unchanged onto `main` @ 9dd41326 (`git range-diff`: "=" for the PR's commit). Through the PC's job queue (job fb2-lift-r2, `RX_CPM` set, each check in a hidden console of its own, 20 minutes at most): - `python -I -B tests/test_mark_direct.py` at the test commit dfa8408e: exit 1, "1 failed": "dev.py base, RX_MARK_DIRECT=1 then unset: its build leaves RX_MARK_DIRECT out, as main builds it"; at the fix, head de30a7f9: exit 0, "all passed" (both orders, one base build, without the switch). - `tests/test_vk_pgo.py` and `tests/test_light_passes.py` at de30a7f9: exit 0 each. **Round 3** (`dev.py packs base` and a base you name): round 2 left the switch out only in `dev.py base`. The documented `dev.py packs base` runs `packs.cmd_base` directly, whose build still took `RX_MARK_DIRECT` from the caller's environment, so a named base built with it on was what a later `dev.py packs rt` with it unset copied. Now `packs.cmd_base`, where both commands converge, builds with the game's compile-only experiments (`GAME_COMPILE_SWITCHES`) taken out of the environment after `packs.run` merges the caller's; the wrapper's own strip is gone. A pack with the switch on still builds its own game in that mode. On `main` @ 9dd41326 (main has not moved; the earlier commits are unchanged), through the PC's job queue (job fb2-lift-r3b, light, two checks at once at most, `RX_CPM` set, each check in a hidden console of its own, 20 minutes at most): - `python -I -B tests/test_mark_direct.py` at the test commit fa3212a5: exit 1, "2 failed": "dev.py packs base, RX_MARK_DIRECT=1 first, unset second: the base's build (make_game.py's environment) has no RX_MARK_DIRECT, as main builds it" and "...: dev.py packs rt with the switch unset and no pack runs a game built off (got on)"; at the fix, head 4388b51c: exit 0, "all passed" (`dev.py packs base` and `dev.py base`, each with the switch on first and unset second and the other way round, then `dev.py packs rt` with no pack, and `dev.py packs pack` and `rt` with the switch on). - `tests/test_dev_packs.py` and `tests/test_dev_build.py` at 4388b51c: one failure each, the same on `main` @ 9dd41326 in the same job ("every file the renderer keeps beside the game is classified (unknown: ['longshot_fsr4_pipelines.bin'])"; `Linux.test_folders`: `C:\Users\User\Games` against `C:\Users\User/Games`). In the game, on this PC (Windows 11, i7-9700, RTX 4060), 2026-11-22: Kokiri Forest 800x600, `RX_MARK_DIRECT` unset against set at build, both at 4388b51c, a warm-up a side then 8 interleaved runs a side, every run in the same scene state: median calm game time a frame 10.05 against 10.23 ms with it on, 1.8 % higher, no gain shown (leaning to a loss) (Measurements). **Still to test:** - Windows 11, this PC (i7-9700): a clean build of the game with `RX_MARK_DIRECT` unset and with it set, its build time and the executable's size, expecting figures for Measurements (by hand) - Linux: `python3 -I -B tests/test_mark_direct.py`, expecting it to pass; and the game built with the switch unset and set, played in Kokiri Forest, expecting no crash and frame times alike (by hand) - Steam Deck: the game with `RX_MARK_DIRECT` unset and set, Kokiri Forest or the Market, a few interleaved runs a side, expecting no crash and frame times alike (by hand) - Windows and Linux (Test Farm): the existing request, default lifted C and lift timing with the switch unset against `main` @ 9dd41326, expecting the same lifted code; held until the farm pairs this head with 9dd41326 (the Test Farm request already on this PR) - Windows 11, this PC, by hand: if the reviewer wants the lean settled, the same Kokiri Forest comparison again, 8 a side, expecting the medians within about 2 % of each other or the lean confirmed (by hand) ## Measurements **Verdict:** `RX_MARK_DIRECT` ships off (the maintainer's request, while the enabled and disabled results are gathered). In play, no gain: the median calm game time a frame is 1.8 % higher with the switch on (10.23 against 10.05 ms), with more long-tail runs on that side (five of eight with p99 above 24 ms, one at 24.3, against one of eight off); inside the spread, but leaning to a loss. Eight runs a side resolve roughly 1.5 to 2 % on the median (an estimate: identical runs of one build spread about 5 % end to end in calm game time). The switch stays off. **In play: Kokiri Forest, off against on at 4388b51c, 8 runs a side (the current reading)** Setup: Windows 11, i7-9700, RTX 4060, run alone on the PC on 2026-11-22, 06:48 to 07:33 UTC; Kokiri Forest (child), 800x600 (CPU-paced), frame cap off, 60 s a run; a warm-up a side, then 8 interleaved runs a side (off on on off, four times). The teleport into the scene checked as taken in every run, so every run is in the same (light) scene state; the GPU cooled to 70 °C before each run, its median clock over each measured stretch its full 2,745 to 2,760 MHz (the few dips are in the clk_min column). Calm game time: the measured 60 s in six 10 s windows, each window whose median game time is more than 20 % above the run's lowest dropped (a burst), then the median game time of the frames left; blocked: the game thread's wait over the same frames. Baseline: this PR with `RX_MARK_DIRECT` unset (off), which builds with `main`'s flags and header, so it stands for `main` @ 9dd41326 (the merge base); on = set at build. | Kokiri Forest, 8 runs a side, median (range) | Before, switch unset: this PR @ 4388b51c | After, RX_MARK_DIRECT=1: this PR @ 4388b51c | |---|---|---| | calm game ms a frame | 10.05 (9.96 to 10.79) | 10.23 (10.05 to 10.46) | | fps | 90.7 (78.0 to 91.6) | 85.2 (77.0 to 88.5) | | p99 frame ms | 16.6 (15.6 to 57.8) | 25.4 (17.1 to 64.9) | | blocked ms a frame | 8.13 (8.02 to 8.78) | 8.39 (8.27 to 8.55) | | burst windows dropped | none | 2 runs, 4 windows | Each run (the run's table; game and calm in ms a frame; clock in MHz over the measured stretch): ``` tag runtime fps med_ms p99_ms low1 class game calm bursts blocked gpu_start gpu_max slowed cool_wait clock clk_min idle_s md4-warm-off mdir-off 86.3 10.88 38.62 25.9 light 9.89 9.76 1 8.33 71 80 0.00 5 s 2760 1875 3 md4-warm-on mdir-on 90.6 10.66 16.69 59.9 light 10.00 10.00 0 8.16 73 81 0.00 10 s 2760 2760 0 md4-off1 mdir-off 91.4 10.54 16.71 59.8 light 10.01 10.01 0 8.02 74 79 0.00 10 s 2760 2760 0 md4-on2 mdir-on 88.1 10.84 24.33 41.1 light 10.05 10.05 0 8.29 74 81 0.00 10 s 2760 2760 0 md4-on3 mdir-on 78.6 11.26 43.57 22.9 light 10.61 10.23 2 8.55 74 84 0.00 10 s 2745 2685 1 md4-off4 mdir-off 90.2 10.72 15.99 62.5 light 10.14 10.14 0 8.15 73 81 0.00 5 s 2760 2760 0 md4-off5 mdir-off 78.0 10.99 57.75 17.3 light 10.14 10.14 0 8.45 74 84 0.00 10 s 2760 1350 2 md4-on6 mdir-on 77.0 11.02 64.86 15.4 light 10.27 10.27 0 8.42 71 75 0.00 5 s 2760 1860 2 md4-on7 mdir-on 83.4 11.12 26.37 37.9 light 10.46 10.46 0 8.39 70 83 0.00 0 s 2760 2760 0 md4-off8 mdir-off 91.6 10.54 15.72 63.6 light 10.02 10.02 0 8.09 73 81 0.00 5 s 2760 2760 0 md4-off9 mdir-off 91.1 10.59 15.63 64.0 light 10.07 10.07 0 8.04 74 84 0.00 10 s 2760 2760 0 md4-on10 mdir-on 88.0 10.85 20.36 49.1 light 10.24 10.24 0 8.27 73 82 0.00 5 s 2760 2760 0 md4-on11 mdir-on 87.0 10.96 17.72 56.5 light 10.23 10.23 0 8.43 72 84 0.00 5 s 2760 2760 0 md4-off12 mdir-off 91.1 10.58 16.56 60.4 light 9.96 9.96 0 8.11 73 77 0.00 5 s 2760 2760 0 md4-off13 mdir-off 88.1 10.86 16.55 60.4 light 10.03 10.03 0 8.36 74 85 0.00 10 s 2760 2760 0 md4-on14 mdir-on 82.3 11.01 34.41 29.1 light 10.35 10.09 2 8.38 74 81 0.00 5 s 2760 2760 0 md4-on15 mdir-on 88.5 10.83 17.09 58.5 light 10.13 10.13 0 8.32 73 84 0.01 5 s 2745 2745 0 md4-off16 mdir-off 84.2 11.45 18.66 53.6 light 10.79 10.79 0 8.78 73 82 0.00 5 s 2760 2760 0 ``` Clean build time and size, Linux and the Steam Deck are not measured. **Earlier: 4 runs a side, before the wrapper checked the teleport (2026-11-22, 00:15 to 01:03 UTC)** Reading then: not decided, no loss seen. Every run sits in one of two scene states, and both sides land in both. Setup: Windows 11, i7-9700, RTX 4060, run alone on the PC on 2026-11-22; Kokiri Forest (child), 800x600 (CPU-paced), frame cap off, 60 s a run; one warm-up a side, then 4 interleaved runs a side (off on on off off on on off). off = `RX_MARK_DIRECT` unset, whose lifted code, object keys and executable are those of `main` @ 9dd41326 (the merge base); on = set at build. | run | side | fps | median frame ms | p99 frame ms | state | |---|---|---|---|---|---| | off1 | off | 69.05 | 13.57 | 46.69 | heavy | | on2 | on | 81.69 | 11.98 | 19.85 | light | | on3 | on | 83.19 | 11.95 | 17.27 | light | | off4 | off | 70.45 | 13.35 | 34.72 | heavy | | off5 | off | 80.33 | 12.21 | 17.56 | light | | on6 | on | 69.43 | 13.47 | 33.52 | heavy | | on7 | on | 80.32 | 11.97 | 18.44 | light | | off8 | off | 78.29 | 12.47 | 25.07 | light | Range: off 69.1 to 80.3 fps, on 69.4 to 83.2. The two states: light about 1,100 textures a frame and 76 to 83 fps, heavy about 1,380 and 67 to 73. The run wrapper asks the Debug teleport into Kokiri Forest, where the save already puts Link; asked within about 22 frames of entering play, the game refuses it and nothing reloads (heavy), asked later it is taken (light). Within one state the two sides overlap (heavy: off 69.1 and 70.5 fps, on 69.4; light: off 78.3 and 80.3, on 80.3, 81.7 and 83.2): inside the spread. **Game-free checks** Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5; `tests/test_mark_direct.py` through the PC's job queue | | Before: main @ f1f2dc5b | After: this PR @ 327c4f8a | |---|---|---| | `tests/test_mark_direct.py` | not run (no `rt_header` on main) | all passed: seeds 1 to 3, O1 and O2, bitmap off and on, off and on alike; `.refptr.a64_dirty` in main's MinGW code, none with the switch | ## Risks Only the table's declaration changes: the index keeps its mask, the marks their release order. Off by default: unset, the build and its header are main's. Turning it off is unsetting it and building again: an executable keeps what it was built with. Changing it compiles the lifted code and the host's files again, and translates the game again where the translated C is gone (a player's build folder; a development checkout and `LONGSHOT_KEEP_BUILD_FILES=1` keep it). Nothing a player keeps (saves, `longshot_options.ini`) changes. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [ ] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A check under `tests/` that reads changed source is updated, not weakened. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
perf(lift): RX_MARK_MASK=0, page marks index the table unmasked and reach it directly (compile only, off)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
b4a0379028
A page mark (rtgen's A64_MARK) was a shift, a mask into the soft write watch's table and the byte store, plus the
table's address loaded from MinGW's .refptr slot, again after every guest store that might alias it. With
RX_MARK_MASK=0 nx2/build.py gives the lifted code and the host's files -DA64_MARK_UNMASKED: a64.h declares a64_dirty
hidden (clang; the table is the executable's own, rt/a64_rt.c), so code reaches it rip-relative, and indexes it by the
address over 4 KB without the mask. The table covers the whole 512 GB guest window; an address past it has already
taken the data store just before its mark out of the window, which the mask could not undo. Memory ordering is
unchanged: a mark is still the release store main has (on x86-64, TSO, the same MOV as a plain store), and the
compiler cannot move it above its data store, whose address is a guest integer made into a pointer that may alias
a64_dirty. Unset or any other value: the mask and the plain declaration, as main. Compile only, so a build that sets
it keeps the lift (make_game.py and dev.py's COMPILE_ONLY_ENV). docs/dev/reference/page-marks.md, with the branch's
static counts labelled as such.

tests/test_mark_mask.py: the flags and header text, a lifted-shaped loop's IR at -O1/-O2 (hidden and unmasked only at
0; no .refptr.a64_dirty load at 0 where the default has one), and the loop built into a program run from the same
random states with the switch unset and at 0: same registers, memory, marks and polls taken.

From 1bf5203e on exp/x86-asm-sync (where the unmasked index was the default and RX_MARK_MASK=1 put the mask back), as
split from the local trial 7326db9f/92123e22.

Assisted-by: Claude Code
BeanBot left a comment

Recommendation: Request changes. The opt-in path can index past both write-watch tables, and the generated header forces existing installations to recompile the lifted game even with the experiment disabled. The Windows fixture shows the intended code generation, but there is no measured gain in play.

Adds a developer build switch that removes the page-index mask and lets Clang address the dirty table directly. The reported MinGW fixture eliminates the .refptr reference at O1 and O2 and produces matching digests for three seeds. The switch couples direct addressing to a separate bounds change and adds a rebuild cost for players who leave it unset.

@fix will do (12)

  1. L1 nx2/rtgen.py:73: The raw page index is used for both a64_dirty and a64_gpu_pages. (more on the line)
  2. L2 nx2/rtgen.py:82: The bitmap lookup whose index changes here is a plain load, while hos_gpu_pages_set atomically updates the same bytes. A game thread storing to one page can read a byte while another thread registers a neighboring page in that byte. The race already exists on main and carries into the new path. Use an atomic byte load. Relaxed ordering suffices if this read only tests the bit and mapping publication is synchronized separately.
  3. L3 (its part an edit can do) nx2/rtgen.py:60: Keep the generated default header byte-for-byte compatible and select the experimental variant only when requested, or preserve proven-compatible default object keys.
  4. L4 docs/dev/reference/page-marks.md:9: The exact instruction and mark counts here come from the game's lifted output, and the weighted figures also use a sampled game profile. (more on the line)
  5. L5 tests/test_mark_mask.py:36: The suite always passes --cc clang, so find_cc never consults CC or its llvm-mingw fallback. (more on the line)
  6. L6 tests/test_mark_mask.py:33: Importing build also runs host discovery outside the fixture. (more on the line)
  7. L7 tests/test_mark_mask.py:159: The poll callback writes c.poll through cur_ while the loop reads and writes the same member through its restrict-qualified c parameter. (more on the line)
  8. L8 (minor) tests/test_mark_mask.py:137: The assembly check is skipped whenever the masked build lacks the literal .refptr name, even on MinGW. (more on the line)
  9. L9 (minor) tests/test_mark_mask.py:180: The fixture compares two digests over low addresses and aligned eight-byte stores. (more on the line)
  10. L10 (minor) docs/dev/reference/page-marks.md:3: The introduction says every guest store marks and implies one byte store per distinct page. (more on the line)
  11. L11 (minor): Risks says unsetting the variable turns the change off. A binary already built with RX_MARK_MASK=0 keeps the unmasked code. Say to unset it and rebuild, or to switch back to a masked executable. Also qualify lift reuse when only compiled objects remain.
  12. L12 (its part an edit can do): Isolate the visibility change from the bounds change and record the compiled mode in the build output so each run identifies the code it exercised.

For the author (2) (a run, a measurement or a decision)

  • L3 nx2/rtgen.py:60: The emitted a64.h changes even with RX_MARK_MASK unset. (more on the line)
  • L12 (minor): The PR's table reports a test failing on a missing helper and passing after the helper is added. It does not establish a player gain or choose a default. Compare the merge base with the final enabled and disabled builds in warmed, interleaved runs on the affected Tier 1 Windows and Linux paths, including Steam Deck. Report average FPS, 1% lows, median and p99 frame time, plus clean build time and size. The required in-game author run and Linux fixture run are still outstanding.

Checks: Still running: PR description / Description follows the template (pull_request_target), CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Unproven. The author reports matching digests and poll counts for three seeds at O1 and O2 on Windows, with .refptr.a64_dirty present in the masked assembly and absent in the unmasked assembly. Those checks ran on dd39b843 before the reported unchanged rebase to b4a03790. There are no interleaved gameplay or clean-build measurements against 6e1521a7.

Other systems: The changed header reaches lifted code, shared A64 helpers, native host stores and the emulator-hosted DLL build. The Vulkan renderer consumes marks through host callbacks, so GPU imports, mirrored buffers and captures can all see the effect of missing marks on either OS, including Steam Deck. The collector's existing non-atomic word-walk reads were outside this diff. Windows and Linux standalone links retain the image-span guard for RIP-relative references. The Launcher and AppImage can hit the object-cache invalidation through the normal update path. The diff changes no renderer API, shader, save format, audio or input code.

Switches: RX_MARK_MASK is compile-time only. Unset, empty or any value other than 0, off, false or no keeps the mask and default visibility. Those four values enable A64_MARK_UNMASKED, which also hides the dirty symbol under Clang. Keeping the masked path as the default preserves the current bounds protection. No measurements justify shipping the experiment. The build switch is documented in build.md, which is the repository's table for build-only switches. switches.def covers runtime switches. The test adds --cc and --cflags, defaulting to compiler discovery and no extra flags, but the suite overrides discovery with --cc clang.

Reviewed: b4a0379028 against 6e1521a70a: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md and the Forgejo pull request template in full. Reviewed every hunk in all seven changed files and read nx2/rtgen.py, docs/dev/reference/page-marks.md and tests/test_mark_mask.py in full. Read the relevant surrounding sections of the other four changed files and traced the generated header, store helpers, bitmap writer, collector, build and lift caches, pack routing, compiler selection and test runner. Other sections of those larger files were skimmed or outside the reviewed paths. docs/review-checklist.md and the sibling Tooling checkout were absent. Reviewed the supplied test report and CI status. No comments, reviews, raw run logs or generated assembly were available.; nothing run (a code review).

Checked and fine:

  • The new define is passed to lifted code, shared helpers and native host C files, and object keys distinguish both flag settings.
  • Both compile-only lists agree, and the switch does not change the lift key itself.
  • The dirty mark operations still use release stores and the collector still takes a byte mark with an acquire exchange.
  • The commit has the required Assisted-by trailer. No third-party files, shader sources, generated artifacts or versioned renderer API structs changed.

Testing: Read-only review. git diff --check passed. The author reported the earlier Windows fixture passed and named 17 broader failures that also occurred on its earlier base. The PR lists seven skips. The rebased head, Linux fixture, game runs and build timings have no reported results. The supplied CI checks are still running.

Overlaps and merge order: None found in the supplied Forgejo record or local history. The offline checkout did not provide the open pull request list.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The opt-in path can index past both write-watch tables, and the generated header forces existing installations to recompile the lifted game even with the experiment disabled. The Windows fixture shows the intended code generation, but there is no measured gain in play. Adds a developer build switch that removes the page-index mask and lets Clang address the dirty table directly. The reported MinGW fixture eliminates the `.refptr` reference at O1 and O2 and produces matching digests for three seeds. The switch couples direct addressing to a separate bounds change and adds a rebuild cost for players who leave it unset. **`@fix` will do (12)** 1. **L1** `nx2/rtgen.py:73`: The raw page index is used for both a64_dirty and a64_gpu_pages. (more on the line) 2. **L2** `nx2/rtgen.py:82`: The bitmap lookup whose index changes here is a plain load, while hos_gpu_pages_set atomically updates the same bytes. A game thread storing to one page can read a byte while another thread registers a neighboring page in that byte. The race already exists on main and carries into the new path. Use an atomic byte load. Relaxed ordering suffices if this read only tests the bit and mapping publication is synchronized separately. 3. **L3** (its part an edit can do) `nx2/rtgen.py:60`: Keep the generated default header byte-for-byte compatible and select the experimental variant only when requested, or preserve proven-compatible default object keys. 4. **L4** `docs/dev/reference/page-marks.md:9`: The exact instruction and mark counts here come from the game's lifted output, and the weighted figures also use a sampled game profile. (more on the line) 5. **L5** `tests/test_mark_mask.py:36`: The suite always passes --cc clang, so find_cc never consults CC or its llvm-mingw fallback. (more on the line) 6. **L6** `tests/test_mark_mask.py:33`: Importing build also runs host discovery outside the fixture. (more on the line) 7. **L7** `tests/test_mark_mask.py:159`: The poll callback writes c.poll through cur_ while the loop reads and writes the same member through its restrict-qualified c parameter. (more on the line) 8. **L8** (minor) `tests/test_mark_mask.py:137`: The assembly check is skipped whenever the masked build lacks the literal .refptr name, even on MinGW. (more on the line) 9. **L9** (minor) `tests/test_mark_mask.py:180`: The fixture compares two digests over low addresses and aligned eight-byte stores. (more on the line) 10. **L10** (minor) `docs/dev/reference/page-marks.md:3`: The introduction says every guest store marks and implies one byte store per distinct page. (more on the line) 11. **L11** (minor): Risks says unsetting the variable turns the change off. A binary already built with RX_MARK_MASK=0 keeps the unmasked code. Say to unset it and rebuild, or to switch back to a masked executable. Also qualify lift reuse when only compiled objects remain. 12. **L12** (its part an edit can do): Isolate the visibility change from the bounds change and record the compiled mode in the build output so each run identifies the code it exercised. **For the author (2)** (a run, a measurement or a decision) - **L3** `nx2/rtgen.py:60`: The emitted a64.h changes even with RX_MARK_MASK unset. (more on the line) - **L12** (minor): The PR's table reports a test failing on a missing helper and passing after the helper is added. It does not establish a player gain or choose a default. Compare the merge base with the final enabled and disabled builds in warmed, interleaved runs on the affected Tier 1 Windows and Linux paths, including Steam Deck. Report average FPS, 1% lows, median and p99 frame time, plus clean build time and size. The required in-game author run and Linux fixture run are still outstanding. **Checks:** Still running: `PR description / Description follows the template (pull_request_target)`, `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven. The author reports matching digests and poll counts for three seeds at O1 and O2 on Windows, with `.refptr.a64_dirty` present in the masked assembly and absent in the unmasked assembly. Those checks ran on dd39b843 before the reported unchanged rebase to b4a03790. There are no interleaved gameplay or clean-build measurements against 6e1521a7. **Other systems:** The changed header reaches lifted code, shared A64 helpers, native host stores and the emulator-hosted DLL build. The Vulkan renderer consumes marks through host callbacks, so GPU imports, mirrored buffers and captures can all see the effect of missing marks on either OS, including Steam Deck. The collector's existing non-atomic word-walk reads were outside this diff. Windows and Linux standalone links retain the image-span guard for RIP-relative references. The Launcher and AppImage can hit the object-cache invalidation through the normal update path. The diff changes no renderer API, shader, save format, audio or input code. **Switches:** RX_MARK_MASK is compile-time only. Unset, empty or any value other than 0, off, false or no keeps the mask and default visibility. Those four values enable A64_MARK_UNMASKED, which also hides the dirty symbol under Clang. Keeping the masked path as the default preserves the current bounds protection. No measurements justify shipping the experiment. The build switch is documented in build.md, which is the repository's table for build-only switches. switches.def covers runtime switches. The test adds --cc and --cflags, defaulting to compiler discovery and no extra flags, but the suite overrides discovery with --cc clang. **Reviewed:** b4a037902808 against 6e1521a70a32: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md and the Forgejo pull request template in full. Reviewed every hunk in all seven changed files and read nx2/rtgen.py, docs/dev/reference/page-marks.md and tests/test_mark_mask.py in full. Read the relevant surrounding sections of the other four changed files and traced the generated header, store helpers, bitmap writer, collector, build and lift caches, pack routing, compiler selection and test runner. Other sections of those larger files were skimmed or outside the reviewed paths. docs/review-checklist.md and the sibling Tooling checkout were absent. Reviewed the supplied test report and CI status. No comments, reviews, raw run logs or generated assembly were available.; nothing run (a code review). **Checked and fine:** - The new define is passed to lifted code, shared helpers and native host C files, and object keys distinguish both flag settings. - Both compile-only lists agree, and the switch does not change the lift key itself. - The dirty mark operations still use release stores and the collector still takes a byte mark with an acquire exchange. - The commit has the required Assisted-by trailer. No third-party files, shader sources, generated artifacts or versioned renderer API structs changed. **Testing:** Read-only review. git diff --check passed. The author reported the earlier Windows fixture passed and named 17 broader failures that also occurred on its earlier base. The PR lists seven skips. The rebased head, Linux fixture, game runs and build timings have no reported results. The supplied CI checks are still running. **Overlaps and merge order:** None found in the supplied Forgejo record or local history. The offline checkout did not provide the open pull request list. </details> --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -0,0 +1,74 @@
# Page marks: the index and the table's address (RX_MARK_MASK)
Every guest store the lifted code makes marks the 4 KB pages it touches in `a64_dirty`, the soft write watch's table
Member

L10 (minor): The introduction says every guest store marks and implies one byte store per distinct page. The bitmap skips stores on untracked pages, the lifter can coalesce stores or omit stack marks, and an ordinary mark stores both endpoints even when they share a page. Describe the default path before using its instruction counts to explain the change.

**L10** (minor): The introduction says every guest store marks and implies one byte store per distinct page. The bitmap skips stores on untracked pages, the lifter can coalesce stores or omit stack marks, and an ordinary mark stores both endpoints even when they share a page. Describe the default path before using its instruction counts to explain the change.
@ -0,0 +6,4 @@
## What a mark costs
Counted in the clang `-O3` code of the game's 20 hottest lifted functions, on the local performance branch this
Member

L4: The exact instruction and mark counts here come from the game's lifted output, and the weighted figures also use a sampled game profile. The repository's no-game-derived-output rule covers those results. Remove the game-derived counts and percentages throughout this page, including the discarded trials. Use results from an authored game-free fixture for committed examples.

**L4**: The exact instruction and mark counts here come from the game's lifted output, and the weighted figures also use a sampled game profile. The repository's no-game-derived-output rule covers those results. Remove the game-derived counts and percentages throughout this page, including the discarded trials. Use results from an authored game-free fixture for committed examples.
nx2/rtgen.py Outdated
@ -58,3 +58,3 @@
#define A64_MARK(a, n) (__atomic_store_n(&a64_dirty[(uint32_t)(a) >> 12], 1, __ATOMIC_RELEASE), \
__atomic_store_n(&a64_dirty[(uint32_t)((a) + (n) - 1) >> 12], 1, __ATOMIC_RELEASE))"""
MARK_NATIVE_64 = r"""extern uint8_t a64_dirty[A64_DIRTY_PAGES];
MARK_NATIVE_64 = r"""/* (nx2/rtgen.py) A64_MARK_UNMASKED (nx2/build.py RX_MARK_MASK=0): the table is declared hidden, as it is the
Member

L3: The emitted a64.h changes even with RX_MARK_MASK unset. build.py hashes the header into the lifted object keys, so an existing player's update loses the cached lifted objects. If cleanup already removed the lifted C, the kept-object check also forces a new translation unless the lift can be restored. Measure an update from the merge base as well as a clean build.

**L3**: The emitted a64.h changes even with RX_MARK_MASK unset. build.py hashes the header into the lifted object keys, so an existing player's update loses the cached lifted objects. If cleanup already removed the lifted C, the kept-object check also forces a new translation unless the lift can be restored. Measure an update from the merge base as well as a clean build.
nx2/rtgen.py Outdated
@ -61,2 +70,4 @@
/* (Tried 2026-10-21: reading a mark first and writing it only when clear, against cache lines moving between cores:
game threads +1.1 ms a frame, the code 18 MB bigger. Not kept.) */
#if defined(A64_MARK_UNMASKED)
#define A64_DIRTY_AT(a) ((uint64_t)(a) >> 12)
Member

L1: The raw page index is used for both a64_dirty and a64_gpu_pages. A bad guest address can resolve to writable host memory outside the reservation, so a completed data store does not prove that either metadata index is valid. The bitmap read or the two endpoint marks can then fault or access unrelated host memory. Keep bounded indexing on every path, or check both endpoints before touching either table. The hidden-symbol change can be tested independently.

**L1**: The raw page index is used for both a64_dirty and a64_gpu_pages. A bad guest address can resolve to writable host memory outside the reservation, so a completed data store does not prove that either metadata index is valid. The bitmap read or the two endpoint marks can then fault or access unrelated host memory. Keep bounded indexing on every path, or check both endpoints before touching either table. The hidden-symbol change can be tested independently.
@ -0,0 +30,4 @@
sys.path.insert(0, os.path.join(ROOT, "nx2"))
sys.path.insert(0, HERE)
import kit # noqa: E402,F401 (a64-kit on the path)
import build # noqa: E402
Member

L6: Importing build also runs host discovery outside the fixture. On Linux it can invoke pkg-config and probe another Clang, and inherited RX_TARGET, LONGSHOT_SYSROOT, RX_MARCH and RX_MARK_BITMAP change the test flags. For example, RX_TARGET=windows with LONGSHOT_MINGW set makes the Linux run build Windows executables, and RX_MARK_BITMAP=0 silently skips bitmap coverage. Keep this test's flag selection free of build startup probes, set a runnable CPU baseline, and choose the target and both bitmap modes explicitly.

**L6**: Importing build also runs host discovery outside the fixture. On Linux it can invoke pkg-config and probe another Clang, and inherited RX_TARGET, LONGSHOT_SYSROOT, RX_MARCH and RX_MARK_BITMAP change the test flags. For example, RX_TARGET=windows with LONGSHOT_MINGW set makes the Linux run build Windows executables, and RX_MARK_BITMAP=0 silently skips bitmap coverage. Keep this test's flag selection free of build startup probes, set a runnable CPU baseline, and choose the target and both bitmap modes explicitly.
@ -0,0 +33,4 @@
import build # noqa: E402
import rtgen # noqa: E402
RUN_ALL_ARGS = ["--cc", "clang"]
Member

L5: The suite always passes --cc clang, so find_cc never consults CC or its llvm-mingw fallback. A supported Windows setup with LONGSHOT_MINGW pointing to the Builder toolchain and no clang on PATH fails before any fixture compiles. Let the suite use the test's discovery and select the compiler and target together, honoring an explicit CC or --cc.

**L5**: The suite always passes --cc clang, so find_cc never consults CC or its llvm-mingw fallback. A supported Windows setup with LONGSHOT_MINGW pointing to the Builder toolchain and no clang on PATH fails before any fixture compiles. Let the suite use the test's discovery and select the compiler and target together, honoring an explicit CC or --cc.
@ -0,0 +134,4 @@
check(r.returncode == 0, "the loop's assembly (%s): %s" % (what, r.stderr[-1500:]))
if r.returncode == 0:
refptr[value] = ".refptr.a64_dirty" in open(base + ".s", encoding="utf-8").read()
if refptr.get(""): # (a MinGW target: the default reaches the table through the slot)
Member

L8 (minor): The assembly check is skipped whenever the masked build lacks the literal .refptr name, even on MinGW. A compiler or target change can therefore make both variants omit that spelling and still pass without proving direct access. Select the assertion by target, and verify the unmasked instruction addresses a64_dirty directly. The advertised Windows target example also needs an assembly-only mode on Linux because program_checks executes what it builds.

**L8** (minor): The assembly check is skipped whenever the masked build lacks the literal .refptr name, even on MinGW. A compiler or target change can therefore make both variants omit that spelling and still pass without proving direct access. Select the assertion by target, and verify the unmasked instruction addresses a64_dirty directly. The advertised Windows target example also needs an assembly-only mode on Linux because program_checks executes what it builds.
@ -0,0 +156,4 @@
void a64_poll_cur(void) {
polls_++;
(void)rand();
if (++npoll_ %% 3) __atomic_store_n(&cur_->poll, 1, __ATOMIC_RELAXED);
Member

L7: The poll callback writes c.poll through cur_ while the loop reads and writes the same member through its restrict-qualified c parameter. That violates the fixture's aliasing contract. The real native callback leaves the poll member alone. Have the fixture count polls without modifying the context through a second pointer, or arrange repeated polls from the driver between loop invocations.

**L7**: The poll callback writes c.poll through cur_ while the loop reads and writes the same member through its restrict-qualified c parameter. That violates the fixture's aliasing contract. The real native callback leaves the poll member alone. Have the fixture count polls without modifying the context through a second pointer, or arrange repeated polls from the driver between loop invocations.
@ -0,0 +177,4 @@
f_80000000(&c);
for (int i = 0; i < 31; i++) digest = mix(digest, c.x[i]);
digest = mix(digest, npoll_);
for (int i = 0; i < 17; i++) { digest = mix(digest, a64_dirty[i]); a64_dirty[i] = 0; }
Member

L9 (minor): The fixture compares two digests over low addresses and aligned eight-byte stores. Both sides could stop marking the last byte's page or lower the marks to relaxed stores and still agree. Add expected marks for a deliberate page-crossing store, exercise addresses near the window limit, and check for atomic release stores in the IR.

**L9** (minor): The fixture compares two digests over low addresses and aligned eight-byte stores. Both sides could stop marking the last byte's page or lower the marks to relaxed stores and still agree. Add expected marks for a deliberate page-crossing store, exercise addresses near the window limit, and check for atomic release stores in the IR.
futondestroyer requested changes 2026-11-21 03:01:26 +00:00
Dismissed
futondestroyer left a comment

Recommendation: Request changes. Keep page-mark indexes bounded, preserve the masked build's update path, and make sure a switch-only development pack actually contains the experimental executable.

The same-source A/B path can silently produce a masked test run. dev.py pack routes from the files changed against the selected base. COMPILE_ONLY_ENV here only changes the lift-key diagnostic, and packs.py has no route override for RX_MARK_MASK. A clean checkout matching its base produces an empty pack with RX_MARK_MASK=0, so dev.py rt uses the base executable. A renderer-only source change can also skip the game build. Route an enabled experiment to the game build and cover the clean-base case in the pack fixture. Pack routing

The first and last bytes of a mark must stay within the dirty table, and the first-page lookup must stay within the bitmap. Completing a guest store does not establish either bound. Keep the existing bounds protection and measure direct symbol addressing independently. Generating a different a64.h with the switch unset also invalidates existing lifted object keys and can send an installation that retained only compiled objects back through translation. Preserve the default object keys or measure the actual update cost. Replace the committed game-derived counts with measurements from an authored game-free fixture. Bounds · Update cache · Documentation

Does it help? Unproven. The author reports that the earlier Windows fixture removed the .refptr.a64_dirty reference at O1 and O2 and produced matching digests across three seeds. Its setup has compiler-selection and poll-aliasing issues, and its aligned stores leave page crossings untested. Add assertions for the expected first and last marks across a page boundary, exercise both bitmap settings, and check release stores in the generated IR. Once corrected, compare the base and final masked and enabled builds in warmed, interleaved gameplay. Report average FPS, 1% lows, median and p99 frame times, and compare clean-build and update cost. The author's Windows game run, a Linux fixture run, and affected Tier 1 gameplay are outstanding. The description check passed and the remaining source checks were pending on this head. Beanstalk shows no jobs or reports. Fixture discussion · Beanstalk

Checked and fine. The flag reaches the lifted chunks, shared A64 helpers and native host sources. With the generated C retained, toggling the flag leaves the lift key alone and separates compiled object keys by flag. The mark stores still use release ordering and the collector takes the byte with an acquire exchange. The link-size guard still applies to the standalone executable and the emulator-hosted DLL. The build-variable entry belongs in the build reference and the flag currently defaults to masked.

Merge order. The loop-store and poll-convention experiments conflict in shared build wiring. Each can be evaluated independently. If either lands first, rebase the shared hunks and recheck that each switch still selects its own compiled mode. Loop-store PR · Poll-convention PR

Reviewed b4a03790 against 6e1521a7. Read all seven changed files and traced the generated store helpers, mark bitmap, renderer collector, object-cache keys, kept-translation fallback and development pack routing with revision-matched Graphify maps and source. No PR code was run.

**Recommendation: Request changes.** Keep page-mark indexes bounded, preserve the masked build's update path, and make sure a switch-only development pack actually contains the experimental executable. The same-source A/B path can silently produce a masked test run. `dev.py pack` routes from the files changed against the selected base. `COMPILE_ONLY_ENV` here only changes the lift-key diagnostic, and `packs.py` has no route override for `RX_MARK_MASK`. A clean checkout matching its base produces an empty pack with `RX_MARK_MASK=0`, so `dev.py rt` uses the base executable. A renderer-only source change can also skip the game build. Route an enabled experiment to the game build and cover the clean-base case in the pack fixture. [Pack routing](https://git.magicbean.dev/MagicBeanDev/Longshot/src/commit/b4a03790280807ae7cf669fe729804b6a7ca3bef/scripts/devtools/packs.py#L440) The first and last bytes of a mark must stay within the dirty table, and the first-page lookup must stay within the bitmap. Completing a guest store does not establish either bound. Keep the existing bounds protection and measure direct symbol addressing independently. Generating a different `a64.h` with the switch unset also invalidates existing lifted object keys and can send an installation that retained only compiled objects back through translation. Preserve the default object keys or measure the actual update cost. Replace the committed game-derived counts with measurements from an authored game-free fixture. [Bounds](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62540) · [Update cache](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62541) · [Documentation](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62542) **Does it help?** Unproven. The author reports that the earlier Windows fixture removed the `.refptr.a64_dirty` reference at O1 and O2 and produced matching digests across three seeds. Its setup has compiler-selection and poll-aliasing issues, and its aligned stores leave page crossings untested. Add assertions for the expected first and last marks across a page boundary, exercise both bitmap settings, and check release stores in the generated IR. Once corrected, compare the base and final masked and enabled builds in warmed, interleaved gameplay. Report average FPS, 1% lows, median and p99 frame times, and compare clean-build and update cost. The author's Windows game run, a Linux fixture run, and affected Tier 1 gameplay are outstanding. The description check passed and the remaining source checks were pending on this head. Beanstalk shows no jobs or reports. [Fixture discussion](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549) · [Beanstalk](https://beanstalk.magicbean.dev/prs/1979) **Checked and fine.** The flag reaches the lifted chunks, shared A64 helpers and native host sources. With the generated C retained, toggling the flag leaves the lift key alone and separates compiled object keys by flag. The mark stores still use release ordering and the collector takes the byte with an acquire exchange. The link-size guard still applies to the standalone executable and the emulator-hosted DLL. The build-variable entry belongs in the build reference and the flag currently defaults to masked. **Merge order.** The loop-store and poll-convention experiments conflict in shared build wiring. Each can be evaluated independently. If either lands first, rebase the shared hunks and recheck that each switch still selects its own compiled mode. [Loop-store PR](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1978) · [Poll-convention PR](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1985) **Reviewed** `b4a03790` against `6e1521a7`. Read all seven changed files and traced the generated store helpers, mark bitmap, renderer collector, object-cache keys, kept-translation fallback and development pack routing with revision-matched Graphify maps and source. No PR code was run.
@ -49,1 +49,3 @@
COMPILE_ONLY_ENV = ("RX_HOT_OPT", "RX_PGO", "RX_VK_PGO", "RX_VK_PGO_FILE", "RX_A64_RT_O2", "RX_GAME_ONLY") # (make_game.py's: left out of the lift key)
# (make_game.py's: left out of the lift key)
COMPILE_ONLY_ENV = ("RX_HOT_OPT", "RX_PGO", "RX_VK_PGO", "RX_VK_PGO_FILE", "RX_A64_RT_O2", "RX_GAME_ONLY",
"RX_MARK_MASK")
Member

This entry only suppresses the lift-key diagnostic. On a clean checkout matching its shared base, RX_MARK_MASK=0 leaves pack_route at None because no source files changed, so dev.py pack creates an empty pack and dev.py rt still uses the masked base. On a renderer-only diff the game also stays at base. Route the enabled switch to a game build, as the pack path already does for RX_PGO, and add a clean-base routing case.

This entry only suppresses the lift-key diagnostic. On a clean checkout matching its shared base, `RX_MARK_MASK=0` leaves `pack_route` at `None` because no source files changed, so `dev.py pack` creates an empty pack and `dev.py rt` still uses the masked base. On a renderer-only diff the game also stays at base. Route the enabled switch to a game build, as the pack path already does for RX_PGO, and add a clean-base routing case.
siantm force-pushed perf/lift-mark-mask from b4a0379028
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 327c4f8a93
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 29s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-21 03:53:42 +00:00
Compare
siantm changed title from WIP: perf(lift): RX_MARK_MASK=0, page marks index the table unmasked and reach it directly (compile only, off) to WIP: perf(lift): RX_MARK_DIRECT=1, page marks reach their table directly, not through a .refptr slot (compile only, off) 2026-11-21 03:54:00 +00:00
Author
Contributor

Pushed 327c4f8a (one commit, on main @ f1f2dc5b; it replaces b4a03790). The reviews' bounds point decided the scope: the unmasked index is dropped, both tables keep main's masked index, and the switch is renamed RX_MARK_DIRECT=1 (title, docs, build reference, test) for what it still does, the hidden declaration of the marks' table. With the switch unset a64.h is main's byte for byte; dev.py pack builds the game when it is on; the build's log names the mode; the counts over the game's lifted functions are removed (the commit is squashed so that no pushed doc keeps them). The fixture stores across pages with exact expected marks, takes the marks at each poll, checks release marks in the IR, the assembly by target and the index at the window's edge. Tested: tests/test_mark_direct.py all passed on the tree before the rebase (main's new commit touches none of its files); the run on 327c4f8a itself is queued and its result follows here. Still to test: the in-game A/B and Linux.

Pushed 327c4f8a (one commit, on `main` @ f1f2dc5b; it replaces b4a03790). The reviews' bounds point decided the scope: the unmasked index is dropped, both tables keep main's masked index, and the switch is renamed `RX_MARK_DIRECT=1` (title, docs, build reference, test) for what it still does, the hidden declaration of the marks' table. With the switch unset `a64.h` is main's byte for byte; `dev.py pack` builds the game when it is on; the build's log names the mode; the counts over the game's lifted functions are removed (the commit is squashed so that no pushed doc keeps them). The fixture stores across pages with exact expected marks, takes the marks at each poll, checks release marks in the IR, the assembly by target and the index at the window's edge. Tested: `tests/test_mark_direct.py` all passed on the tree before the rebase (main's new commit touches none of its files); the run on 327c4f8a itself is queued and its result follows here. Still to test: the in-game A/B and Linux.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62850: fixed in 327c4f8a. scripts/devtools/packs.py pack_route now builds the game when the switch is on (nx2/kit.py env_on's rule), for a clean checkout matching its base and for a renderer-only change; tests/test_mark_direct.py covers both routes and the off values. The switch is now RX_MARK_DIRECT=1 (see the review reply), and the build's log prints build: RX_MARK_DIRECT=1: ... when the executable has it.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62850](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62850): fixed in 327c4f8a. `scripts/devtools/packs.py` `pack_route` now builds the game when the switch is on (nx2/kit.py `env_on`'s rule), for a clean checkout matching its base and for a renderer-only change; `tests/test_mark_direct.py` covers both routes and the off values. The switch is now `RX_MARK_DIRECT=1` (see the review reply), and the build's log prints `build: RX_MARK_DIRECT=1: ...` when the executable has it.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62851: addressed in 327c4f8a (one commit, rebased on main @ f1f2dc5b), and your bounds point decided the scope: the unmasked index is dropped, both tables keep main's bounded index, and the switch is renamed RX_MARK_DIRECT=1 for what it still does, the hidden table declaration alone (title, docs, build reference and test renamed; RX_MARK_MASK is left only in the doc's note on the dropped form). With the switch unset a64.h is main's byte for byte, so retained objects keep their keys; the pack route is fixed (62850); the game-derived counts are removed. The fixture now stores across pages with exact expected marks, takes the marks at each poll, checks release marks after their data in the IR and the index at the window's edge, both bitmap modes. The gameplay and build-cost comparisons are pending with the opener.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62851](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62851): addressed in 327c4f8a (one commit, rebased on `main` @ f1f2dc5b), and your bounds point decided the scope: the unmasked index is dropped, both tables keep main's bounded index, and the switch is renamed `RX_MARK_DIRECT=1` for what it still does, the hidden table declaration alone (title, docs, build reference and test renamed; `RX_MARK_MASK` is left only in the doc's note on the dropped form). With the switch unset `a64.h` is main's byte for byte, so retained objects keep their keys; the pack route is fixed ([62850](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62850)); the game-derived counts are removed. The fixture now stores across pages with exact expected marks, takes the marks at each poll, checks release marks after their data in the IR and the index at the window's edge, both bitmap modes. The gameplay and build-cost comparisons are pending with the opener.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62540 (L1): fixed in 327c4f8a by dropping the unmasked index, as the maintainer's review also asked: A64_DIRTY_AT is main's masked index on every path, for both tables. The test checks the index at the guest window's edge (the last byte, one past, far past) in every program. The switch now only declares the table hidden and is renamed RX_MARK_DIRECT=1.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62540](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62540) (L1): fixed in 327c4f8a by dropping the unmasked index, as the maintainer's review also asked: `A64_DIRTY_AT` is main's masked index on every path, for both tables. The test checks the index at the guest window's edge (the last byte, one past, far past) in every program. The switch now only declares the table hidden and is renamed `RX_MARK_DIRECT=1`.
Author
Contributor

(L2 in https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549): not changed here. The plain bitmap load is main's (A64_GPU_PAGE), and after this round the PR no longer touches it or the index; making it an atomic load changes the default a64.h, which is what L3 asks us not to do in this PR (every player's lifted objects would rebuild). It is left for a change of its own, and the description says so.

(L2 in [https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549)): not changed here. The plain bitmap load is main's (`A64_GPU_PAGE`), and after this round the PR no longer touches it or the index; making it an atomic load changes the default `a64.h`, which is what L3 asks us not to do in this PR (every player's lifted objects would rebuild). It is left for a change of its own, and the description says so.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62541 (L3): fixed in 327c4f8a. rtgen.write adds the hidden declaration only with mark_direct=True (nx2/build.py rt_header, switch on); unset or off, a64.h is main's byte for byte, so no object key changes and no update cost is added. The test compares the header with rtgen.write's default for every off value.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62541](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62541) (L3): fixed in 327c4f8a. `rtgen.write` adds the hidden declaration only with `mark_direct=True` (`nx2/build.py` `rt_header`, switch on); unset or off, `a64.h` is main's byte for byte, so no object key changes and no update cost is added. The test compares the header with `rtgen.write`'s default for every off value.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62542 (L4): removed in 327c4f8a as asked. The counts and weighted percentages over the game's lifted functions are gone from the page (the discarded trials too), the commit message and the description; the page says those counts were taken locally and are not published, and the test is the authored fixture for the claims kept. The synthetic probe's figure stays, labelled as belonging to the dropped unmasked form.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62542](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62542) (L4): removed in 327c4f8a as asked. The counts and weighted percentages over the game's lifted functions are gone from the page (the discarded trials too), the commit message and the description; the page says those counts were taken locally and are not published, and the test is the authored fixture for the claims kept. The synthetic probe's figure stays, labelled as belonging to the dropped unmasked form.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62543 (L5): fixed in 327c4f8a. RUN_ALL_ARGS is gone and the test resolves the compiler itself: --cc, then CC, clang on PATH, then LONGSHOT_MINGW's clang, with that compiler's own target. The final run found the Builder's clang through LONGSHOT_MINGW with no arguments.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62543](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62543) (L5): fixed in 327c4f8a. `RUN_ALL_ARGS` is gone and the test resolves the compiler itself: `--cc`, then `CC`, clang on PATH, then `LONGSHOT_MINGW`'s clang, with that compiler's own target. The final run found the Builder's clang through `LONGSHOT_MINGW` with no arguments.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62544 (L6): fixed in 327c4f8a. nx2/build.py is imported with RX_*, LONGSHOT_* and OOT_* cleared, RX_MARCH=x86-64 and no process allowed to start (pkg-config, a compiler probe); the target comes from the compiler and every program is built with the bitmap off and on explicitly.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62544](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62544) (L6): fixed in 327c4f8a. `nx2/build.py` is imported with `RX_*`, `LONGSHOT_*` and `OOT_*` cleared, `RX_MARCH=x86-64` and no process allowed to start (pkg-config, a compiler probe); the target comes from the compiler and every program is built with the bitmap off and on explicitly.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62545 (L7): fixed in 327c4f8a. The poll stand-in touches no context; the loop raises the request through its own c every eighth iteration and the driver checks the exact polls taken per call.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62545](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62545) (L7): fixed in 327c4f8a. The poll stand-in touches no context; the loop raises the request through its own `c` every eighth iteration and the driver checks the exact polls taken per call.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62546 (L8): fixed in 327c4f8a. The assembly check is chosen by the compiler's target (-dumpmachine): with the switch on the code must name a64_dirty itself with no .refptr slot or GOT entry, on any target; off, a MinGW target must load .refptr.a64_dirty. A --target in --cflags now runs the IR and assembly checks only, never a program.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62546](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62546) (L8): fixed in 327c4f8a. The assembly check is chosen by the compiler's target (`-dumpmachine`): with the switch on the code must name `a64_dirty` itself with no `.refptr` slot or GOT entry, on any target; off, a MinGW target must load `.refptr.a64_dirty`. A `--target` in `--cflags` now runs the IR and assembly checks only, never a program.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62547 (L9): fixed in 327c4f8a. Destinations are unaligned so stores cross pages, and each run checks the exact marks left (first and last byte's pages, where the bitmap says) after the last poll took the others; the index is checked at the window's limit; the IR check requires each guest store to be followed by two release mark stores before the next.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62547](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62547) (L9): fixed in 327c4f8a. Destinations are unaligned so stores cross pages, and each run checks the exact marks left (first and last byte's pages, where the bitmap says) after the last poll took the others; the index is checked at the window's limit; the IR check requires each guest store to be followed by two `release` mark stores before the next.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62548 (L10): fixed in 327c4f8a. The page now describes the default path first: only stores starting on a GPU page mark (the bitmap), two release marks a store even on one page, SP and frame-pointer stores unmarked, coalesced stores, the host's own marks.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62548](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62548) (L10): fixed in 327c4f8a. The page now describes the default path first: only stores starting on a GPU page mark (the bitmap), two release marks a store even on one page, SP and frame-pointer stores unmarked, coalesced stores, the host's own marks.
Author
Contributor

(L11 in https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549): fixed in 327c4f8a. Risks and the page now say turning it off is unsetting it and building again (an executable keeps what it was built with), and that lift reuse holds only where the translated C is kept (LONGSHOT_KEEP_BUILD_FILES=1 or a development checkout).

(L11 in [https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549)): fixed in 327c4f8a. Risks and the page now say turning it off is unsetting it and building again (an executable keeps what it was built with), and that lift reuse holds only where the translated C is kept (`LONGSHOT_KEEP_BUILD_FILES=1` or a development checkout).
Author
Contributor

(L12 in https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549): the edit part is fixed in 327c4f8a: the bounds change is gone, so the switch is the visibility change alone, and the build's log records the compiled mode. The A/B against the merge base in play, with clean build and update cost, is pending with the opener.

(L12 in [https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62549)): the edit part is fixed in 327c4f8a: the bounds change is gone, so the switch is the visibility change alone, and the build's log records the compiled mode. The A/B against the merge base in play, with clean build and update cost, is pending with the opener.
Author
Contributor

@Beanstalk This adds a compile switch, RX_MARK_DIRECT, off by default: with RX_MARK_DIRECT=1 the lifted code's page marks index the soft write watch's table without the mask and reach it directly (nx2/build.py, nx2/rtgen.py). Setting it keeps the lift; only the compile changes. The farm cannot build with it set, so only the default build is to test: with the switch unset, the lift and its C must be main's. Please time the lift against main's and check the lifted C is the same, on a Windows PC and on a Linux PC. No game scenario is needed: unset, the game's code is main's.

@Beanstalk This adds a compile switch, RX_MARK_DIRECT, off by default: with RX_MARK_DIRECT=1 the lifted code's page marks index the soft write watch's table without the mask and reach it directly (nx2/build.py, nx2/rtgen.py). Setting it keeps the lift; only the compile changes. The farm cannot build with it set, so only the default build is to test: with the switch unset, the lift and its C must be main's. Please time the lift against main's and check the lifted C is the same, on a Windows PC and on a Linux PC. No game scenario is needed: unset, the game's code is main's.
Member

@siantm only members of BigBeans can start the farm.

One of them can approve this request by reacting 👍 to it or to this comment. This request was for 327c4f8, and the PR has changed since (de30a7f), so a 👍 no longer approves it. Comment @Beanstalk again to ask for the new commit.

@siantm only members of `BigBeans` can start the farm. ~~One of them can approve this request by reacting 👍 to it or to this comment.~~ This request was for `327c4f8`, and the PR has changed since (`de30a7f`), so a 👍 no longer approves it. Comment `@Beanstalk` again to ask for the new commit.
Author
Contributor

A correction to 63220: "the final run" there is the queue run of this change's tree before its rebase onto main @ f1f2dc5b (job fb2-1979-probe, uncommitted; the rebase touched none of its files). The run on 327c4f8a itself (fb2-1979-final) is still queued; its result will follow here.

A correction to [63220](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-63220): "the final run" there is the queue run of this change's tree before its rebase onto `main` @ f1f2dc5b (job fb2-1979-probe, uncommitted; the rebase touched none of its files). The run on 327c4f8a itself (fb2-1979-final) is still queued; its result will follow here.
Author
Contributor

The queue run on 327c4f8a itself (job fb2-1979-final, Windows 11, i7-9700, llvm-mingw clang 23.1.5, each check in a hidden console of its own) ended: python -I -B tests/test_mark_direct.py (no arguments, the compiler through LONGSHOT_MINGW) "all passed", and tests/test_vk_pgo.py, tests/test_split_giants.py, tests/test_light_passes.py exit 0 each.

The queue run on 327c4f8a itself (job fb2-1979-final, Windows 11, i7-9700, llvm-mingw clang 23.1.5, each check in a hidden console of its own) ended: `python -I -B tests/test_mark_direct.py` (no arguments, the compiler through `LONGSHOT_MINGW`) "all passed", and `tests/test_vk_pgo.py`, `tests/test_split_giants.py`, `tests/test_light_passes.py` exit 0 each.
Author
Contributor

Tested corrected: an earlier run on this PC had no Vulkan headers for the renderer checks (RX_CPM unset), so the ones it listed as also failing on main had compiled nothing on either side. They ran again with the headers at 327c4f8a and on main (queue job fb2-vk-1); the description now gives those results. No check fails at this head alone; nothing in the branch changed.

Tested corrected: an earlier run on this PC had no Vulkan headers for the renderer checks (`RX_CPM` unset), so the ones it listed as also failing on `main` had compiled nothing on either side. They ran again with the headers at 327c4f8a and on `main` (queue job fb2-vk-1); the description now gives those results. No check fails at this head alone; nothing in the branch changed.
futondestroyer left a comment

Recommendation: Wait for the enabled gameplay and build comparison. The revision keeps both page-index masks and the release stores. With RX_MARK_DIRECT unset, the generator follows the original header path. Turning the switch on now routes an unchanged or renderer-only development pack through the game build. The new fixture checks cross-page stores, both bitmap modes and release stores in the generated IR. Earlier review

The author reports the new fixture and three related checks passed on exact head 327c4f8a in Windows. The change gets the intended direct symbol access in the fixture. The earlier instruction-count result came from a now-removed unmasked variant. Measure the retained direct declaration with a verified enabled executable and a default executable from the same revision, plus the merge-base build, in warmed, interleaved runs. Linux validation, author gameplay, build and update cost, and required CI remain open.

The existing farm request can check default lifted C and lift timing with a matching base. It still describes the removed unmasked path, and the dashboard lists 6e1521a7 as the base while this revision branches from f1f2dc5b. Hold it until the request and pair match. The enabled compile-time switch needs a separate build. Recheck the shared build and pack wiring if #1978 or #1985 lands first. Existing farm request · Farm status

Reviewed exact 327c4f8a and f1f2dc5b, the interdiff with b4a03790, and the affected build and page-mark paths using both revision maps and source.

**Recommendation: Wait for the enabled gameplay and build comparison.** The revision keeps both page-index masks and the release stores. With `RX_MARK_DIRECT` unset, the generator follows the original header path. Turning the switch on now routes an unchanged or renderer-only development pack through the game build. The new fixture checks cross-page stores, both bitmap modes and release stores in the generated IR. [Earlier review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-62851) The author reports the new fixture and three related checks passed on exact head `327c4f8a` in Windows. The change gets the intended direct symbol access in the fixture. The earlier instruction-count result came from a now-removed unmasked variant. Measure the retained direct declaration with a verified enabled executable and a default executable from the same revision, plus the merge-base build, in warmed, interleaved runs. Linux validation, author gameplay, build and update cost, and required CI remain open. The existing farm request can check default lifted C and lift timing with a matching base. It still describes the removed unmasked path, and the dashboard lists `6e1521a7` as the base while this revision branches from `f1f2dc5b`. Hold it until the request and pair match. The enabled compile-time switch needs a separate build. Recheck the shared build and pack wiring if #1978 or #1985 lands first. [Existing farm request](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-63228) · [Farm status](https://beanstalk.magicbean.dev/prs/1979) Reviewed exact `327c4f8a` and `f1f2dc5b`, the interdiff with `b4a03790`, and the affected build and page-mark paths using both revision maps and source.
futondestroyer left a comment

Recommendation: Request changes. I missed a shared-base cache issue in the earlier Wait review. dev.py base <rev> forwards RX_MARK_DIRECT=1 when it builds a base, but it keys that base by commit alone. If the enabled build populates the base first, a later invocation with the switch unset reuses it. An unchanged-tree pack then has no game executable to overlay, so both sides of the comparison can run with direct marks enabled. Earlier review · Shared base path

Build ordinary shared bases with the experimental switch off, or include the effective mode in the base identity and selection. Cover both build orders at the same revision and verify the executable that the default runtime actually starts. The bounds, default header and focused Windows fixture findings from the earlier review still apply. The farm request remains held for its baseline mismatch.

**Recommendation: Request changes.** I missed a shared-base cache issue in the earlier Wait review. `dev.py base <rev>` forwards `RX_MARK_DIRECT=1` when it builds a base, but it keys that base by commit alone. If the enabled build populates the base first, a later invocation with the switch unset reuses it. An unchanged-tree pack then has no game executable to overlay, so both sides of the comparison can run with direct marks enabled. [Earlier review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65478) · [Shared base path](https://git.magicbean.dev/MagicBeanDev/Longshot/src/commit/327c4f8a93abde400b3a816de6798af82649e47e/scripts/devtools/build.py#L386) Build ordinary shared bases with the experimental switch off, or include the effective mode in the base identity and selection. Cover both build orders at the same revision and verify the executable that the default runtime actually starts. The bounds, default header and focused Windows fixture findings from the earlier review still apply. The farm request remains held for its baseline mismatch.
siantm force-pushed perf/lift-mark-mask from 327c4f8a93
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 29s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to de30a7f9a6
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
beanstalk/approval waiting for a BigBean's 👍 on the request
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-21 10:31:48 +00:00
Compare
Author
Contributor

Pushed de30a7f9 (on main @ 9dd41326; the PR's commit rebased unchanged): futondestroyer's second look. dev.py base now builds the commit-keyed shared base without RX_MARK_DIRECT, so the base is the default executable whichever order the builds come in; the test commit dfa8408e fails on it and the fix de30a7f9 passes (job fb2-lift-r2). Testing has the round's results.

Pushed de30a7f9 (on `main` @ 9dd41326; the PR's commit rebased unchanged): futondestroyer's second look. `dev.py base` now builds the commit-keyed shared base without `RX_MARK_DIRECT`, so the base is the default executable whichever order the builds come in; the test commit dfa8408e fails on it and the fix de30a7f9 passes (job fb2-lift-r2). Testing has the round's results.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65506: fixed in de30a7f9 (the test first, failing, in dfa8408e), rebased unchanged onto main @ 9dd41326. dev.py base (cmd_base) now leaves RX_MARK_DIRECT (packs.GAME_COMPILE_SWITCHES) out of the shared base's build and says so, so the commit-keyed base is the default executable whichever order the builds come in, and a pack with the switch on builds its own game. tests/test_mark_direct.py runs cmd_base at one revision in both orders: one build, without the switch, and the default runtime of an unchanged-tree pack is that base's game. At the test commit the test fails ("dev.py base, RX_MARK_DIRECT=1 then unset: its build leaves RX_MARK_DIRECT out"); at de30a7f9 it passes (job fb2-lift-r2).

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65506](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65506): fixed in de30a7f9 (the test first, failing, in dfa8408e), rebased unchanged onto `main` @ 9dd41326. `dev.py base` (`cmd_base`) now leaves `RX_MARK_DIRECT` (`packs.GAME_COMPILE_SWITCHES`) out of the shared base's build and says so, so the commit-keyed base is the default executable whichever order the builds come in, and a pack with the switch on builds its own game. `tests/test_mark_direct.py` runs `cmd_base` at one revision in both orders: one build, without the switch, and the default runtime of an unchanged-tree pack is that base's game. At the test commit the test fails ("dev.py base, RX_MARK_DIRECT=1 then unset: its build leaves RX_MARK_DIRECT out"); at de30a7f9 it passes (job fb2-lift-r2).
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65478: the comparison you ask for is queued with the opener: two packs at one revision, mdir-off (RX_MARK_DIRECT unset) and mdir-on (RX_MARK_DIRECT=1), rebuilt on this round's head, plus main @ f1f2dc5b built by its own lift build (build time and executable size logged); gameplay as one held segment, warm-ups then main, off, on, on, off, main, in Market. A separate update-cost build is not planned. With de30a7f9 the shared base no longer takes the switch (65506), so the off arm is the default executable; the farm request will be redone for this head's merge base, describing the hidden declaration only.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65478](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65478): the comparison you ask for is queued with the opener: two packs at one revision, mdir-off (`RX_MARK_DIRECT` unset) and mdir-on (`RX_MARK_DIRECT=1`), rebuilt on this round's head, plus `main` @ f1f2dc5b built by its own lift build (build time and executable size logged); gameplay as one held segment, warm-ups then main, off, on, on, off, main, in Market. A separate update-cost build is not planned. With de30a7f9 the shared base no longer takes the switch ([65506](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65506)), so the off arm is the default executable; the farm request will be redone for this head's merge base, describing the hidden declaration only.
Author
Contributor

@Beanstalk This adds a compile switch, RX_MARK_DIRECT, off by default: with RX_MARK_DIRECT=1 the soft write watch's table is declared hidden (the executable's own), so the lifted code's page marks reach it directly instead of through an address slot (nx2/build.py, nx2/rtgen.py). The farm cannot build with it set, so only the default build is to test: with the switch unset, the lift and its C must be main's. This head is on main 9dd41326: please compare with main at that merge base. Please time the lift against main's and check the lifted C is the same, on a Windows PC and on a Linux PC. No game scenario is needed: unset, the game's code is main's.

@Beanstalk This adds a compile switch, RX_MARK_DIRECT, off by default: with RX_MARK_DIRECT=1 the soft write watch's table is declared hidden (the executable's own), so the lifted code's page marks reach it directly instead of through an address slot (nx2/build.py, nx2/rtgen.py). The farm cannot build with it set, so only the default build is to test: with the switch unset, the lift and its C must be main's. This head is on main 9dd41326: please compare with main at that merge base. Please time the lift against main's and check the lifted C is the same, on a Windows PC and on a Linux PC. No game scenario is needed: unset, the game's code is main's.
Member

@siantm only members of BigBeans can start the farm.

One of them can approve this request by reacting 👍 to it or to this comment. This request was for de30a7f, and the PR has changed since (4388b51), so a 👍 no longer approves it. Comment @Beanstalk again to ask for the new commit.

@siantm only members of `BigBeans` can start the farm. ~~One of them can approve this request by reacting 👍 to it or to this comment.~~ This request was for `de30a7f`, and the PR has changed since (`4388b51`), so a 👍 no longer approves it. Comment `@Beanstalk` again to ask for the new commit.
futondestroyer left a comment

Recommendation: Request changes. The high-level dev.py base command now strips RX_MARK_DIRECT, but the documented dev.py packs base command still inherits it. With the flag enabled when that named base is built, dev.py packs rt later copies its enabled game even with the flag unset. An unchanged off-mode pack also supplies no game override. Enforce the default in the common base builder, and cover the direct command’s environment in the fixture. Enabled packs should still build their own game. Documented A/B route

The new fixture covers both high-level construction orders at the same revision. The author reports the enabled-first case failing at dfa8408e and passing at de30a7f9. The test replaces the queue call, so it never checks the separate direct entry point. Previous review · Author’s regression result

The retained experiment changes how the table is addressed. It preserves both page-index masks and the acquire/release protocol. The author’s exact-head Windows fixture covers direct addressing, page-crossing stores and both bitmap settings. Gameplay benefit remains unproven. The earlier instruction-count result was for the removed unmasked variant. Compare the off and enabled executables at de30a7f9 and the actual merge-base build at 9dd41326, using warmed interleaved play and clean build time and size. Linux gameplay and source validation remain open.

The existing farm request now describes the direct declaration and asks for the right merge base. The dashboard still shows f1f2dc5b. Hold the off-only lift parity request until the farm pins 9dd41326. The enabled compile mode needs separately built executables. Farm request · Dashboard

Reviewed exact de30a7f9 and 9dd41326, the interdiff against 327c4f8a, and the affected build and page-mark paths with both revision maps and source. Required CI remains pending and the PR remains WIP.

**Recommendation: Request changes.** The high-level `dev.py base` command now strips `RX_MARK_DIRECT`, but the documented `dev.py packs base` command still inherits it. With the flag enabled when that named base is built, `dev.py packs rt` later copies its enabled game even with the flag unset. An unchanged off-mode pack also supplies no game override. Enforce the default in the common base builder, and cover the direct command’s environment in the fixture. Enabled packs should still build their own game. [Documented A/B route](https://git.magicbean.dev/MagicBeanDev/Longshot/src/commit/9dd41326e71094d8b774459dac5154eb5a3306d9/docs/dev/developing.md#test-a-change-against-a-base) The new fixture covers both high-level construction orders at the same revision. The author reports the enabled-first case failing at `dfa8408e` and passing at `de30a7f9`. The test replaces the queue call, so it never checks the separate direct entry point. [Previous review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65506) · [Author’s regression result](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65899) The retained experiment changes how the table is addressed. It preserves both page-index masks and the acquire/release protocol. The author’s exact-head Windows fixture covers direct addressing, page-crossing stores and both bitmap settings. Gameplay benefit remains unproven. The earlier instruction-count result was for the removed unmasked variant. Compare the off and enabled executables at `de30a7f9` and the actual merge-base build at `9dd41326`, using warmed interleaved play and clean build time and size. Linux gameplay and source validation remain open. The existing farm request now describes the direct declaration and asks for the right merge base. The dashboard still shows `f1f2dc5b`. Hold the off-only lift parity request until the farm pins `9dd41326`. The enabled compile mode needs separately built executables. [Farm request](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-65908) · [Dashboard](https://beanstalk.magicbean.dev/prs/1979) Reviewed exact `de30a7f9` and `9dd41326`, the interdiff against `327c4f8a`, and the affected build and page-mark paths with both revision maps and source. Required CI remains pending and the PR remains WIP.
@ -392,1 +394,4 @@
py, env = environment()
# (a shared base is keyed by its commit alone, and is the off side of every A/B on it: built as main builds it, without
# the game's compile-only experiments; a pack with one on builds its own game, packs.py pack_route)
left = [k for k in packs.GAME_COMPILE_SWITCHES if env.pop(k, None) is not None]
Member

This strips RX_MARK_DIRECT when invoked through dev.py base. The documented dev.py packs base command calls packs.cmd_base directly and still inherits RX_MARK_DIRECT=1 through make_game. A later packs rt with the flag unset copies that already-enabled executable. Enforce the default in the common base builder and test the direct entry point as well as the wrapper.

This strips RX_MARK_DIRECT when invoked through dev.py base. The documented dev.py packs base command calls packs.cmd_base directly and still inherits RX_MARK_DIRECT=1 through make_game. A later packs rt with the flag unset copies that already-enabled executable. Enforce the default in the common base builder and test the direct entry point as well as the wrapper.
Review of #1979 (futondestroyer): round 2 left the switch out only in the dev.py base wrapper. The documented dev.py packs
base runs packs.cmd_base directly, and its make_game.py build still took RX_MARK_DIRECT from the caller's environment, so a
named base built with it on was what a later dev.py packs rt with it unset copied. The base check now drives both entry
points through dev.py's own command line, each with the switch on first and unset second and the other way round, then
dev.py packs rt with no pack and dev.py packs pack and rt with the switch on. A stand-in for make_game.py records the
environment the build gets after packs.run's merge and writes executables saying the mode they were built in. Fails
here: the direct command's base build gets the switch when it comes first.

Assisted-by: Claude Code
fix(devtools): dev.py packs base builds the named base without RX_MARK_DIRECT
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 5s
4388b51c7b
Review of #1979 (futondestroyer): round 2 left the switch out only in the dev.py base wrapper (build.py cmd_base). The
documented dev.py packs base runs packs.cmd_base directly, whose make_game.py build took RX_MARK_DIRECT from the caller's
environment through packs.run, so a named base built with it on was the executable a later dev.py packs rt with it
unset copied, and an unchanged off-mode pack supplies none of its own. The rule now sits where both commands converge:
packs.cmd_base builds with the game's compile-only experiments (GAME_COMPILE_SWITCHES) taken out of the environment
after packs.run merges the caller's, and says so; the wrapper's own strip is gone. A pack with one on still builds its
own game in that mode (pack_route; its make_game.py call keeps the caller's environment).

Assisted-by: Claude Code
Author
Contributor

Pushed 4388b51c (on main @ 9dd41326; the earlier commits unchanged): futondestroyer's third look. dev.py packs base now builds a named base without RX_MARK_DIRECT too: the rule sits in packs.cmd_base, where it and dev.py base converge, and the wrapper's own strip is gone. The test commit fa3212a5 fails without it and the fix 4388b51c passes (job fb2-lift-r3b). Testing has the round's results; Measurements says the switch stays off by default at the maintainer's request.

Pushed 4388b51c (on `main` @ 9dd41326; the earlier commits unchanged): futondestroyer's third look. `dev.py packs base` now builds a named base without `RX_MARK_DIRECT` too: the rule sits in `packs.cmd_base`, where it and `dev.py base` converge, and the wrapper's own strip is gone. The test commit fa3212a5 fails without it and the fix 4388b51c passes (job fb2-lift-r3b). Testing has the round's results; Measurements says the switch stays off by default at the maintainer's request.
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67213: fixed in 4388b51c (the test first, failing, in fa3212a5). packs.cmd_base, which dev.py packs base runs directly and dev.py base through the queue, now builds the base with RX_MARK_DIRECT taken out of the environment after packs.run merges the caller's, and the wrapper's strip in scripts/devtools/build.py is gone; a pack with the switch on still builds its own game in that mode. tests/test_mark_direct.py now drives both entry points through dev.py's command line, enabled first then unset and the reverse, checking the base build's environment and that dev.py packs rt with no pack runs an off game while the enabled pack's runtime runs an on one: 2 failures at fa3212a5, all passed at 4388b51c (job fb2-lift-r3b).

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67213](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67213): fixed in 4388b51c (the test first, failing, in fa3212a5). `packs.cmd_base`, which `dev.py packs base` runs directly and `dev.py base` through the queue, now builds the base with `RX_MARK_DIRECT` taken out of the environment after `packs.run` merges the caller's, and the wrapper's strip in `scripts/devtools/build.py` is gone; a pack with the switch on still builds its own game in that mode. `tests/test_mark_direct.py` now drives both entry points through `dev.py`'s command line, enabled first then unset and the reverse, checking the base build's environment and that `dev.py packs rt` with no pack runs an off game while the enabled pack's runtime runs an on one: 2 failures at fa3212a5, all passed at 4388b51c (job fb2-lift-r3b).
Author
Contributor

https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67214: addressed in fa3212a5 (test) and 4388b51c (fix), on main @ 9dd41326 with the earlier commits unchanged. Round 2 covered only the dev.py base wrapper; this round covers the direct dev.py packs base command, with the default applied in packs.cmd_base where both converge (inline 67213). Testing in the description has the round's results (67295); the gameplay comparison and the Test Farm pairing are still open.

[https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67214](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67214): addressed in fa3212a5 (test) and 4388b51c (fix), on `main` @ 9dd41326 with the earlier commits unchanged. Round 2 covered only the `dev.py base` wrapper; this round covers the direct `dev.py packs base` command, with the default applied in `packs.cmd_base` where both converge (inline [67213](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67213)). Testing in the description has the round's results ([67295](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67295)); the gameplay comparison and the Test Farm pairing are still open.
Author
Contributor

@Beanstalk This adds a compile switch, RX_MARK_DIRECT, off by default: with RX_MARK_DIRECT=1 the soft write watch's table is declared hidden (the executable's own), so the lifted code's page marks reach it directly instead of through an address slot (nx2/build.py, nx2/rtgen.py). The farm cannot build with it set, so only the default build is to test: with the switch unset, the lift and its C must be main's. Please time the lift against main's and check the lifted C is the same, on a Windows PC and on a Linux PC. No game scenario is needed: unset, the game's code is main's.

@Beanstalk This adds a compile switch, RX_MARK_DIRECT, off by default: with RX_MARK_DIRECT=1 the soft write watch's table is declared hidden (the executable's own), so the lifted code's page marks reach it directly instead of through an address slot (nx2/build.py, nx2/rtgen.py). The farm cannot build with it set, so only the default build is to test: with the switch unset, the lift and its C must be main's. Please time the lift against main's and check the lifted C is the same, on a Windows PC and on a Linux PC. No game scenario is needed: unset, the game's code is main's.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

Re #1979 (comment) (gameplay benefit): the warmed, interleaved off/on comparison in play has run at 4388b51c, and Measurements now has it run by run. off is the switch unset, whose lifted code, object keys and executable are those of the merge base 9dd41326; on is set at build. Kokiri Forest, 800x600, 4 runs a side after a warm-up: off 69.1 to 80.3 fps, on 69.4 to 83.2. Reading: not decided, no loss seen. Every run sits in one of two scene states (a refused or a taken teleport at the run's start, about 1,380 against 1,100 textures a frame) and both sides land in both; within a state the two sides overlap. The run wrapper is fixed, and a rerun of 8 a side, classed by state, follows. Clean build time and size, Linux and Steam Deck are not measured yet. The switch stays off.

Re https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67214 (gameplay benefit): the warmed, interleaved off/on comparison in play has run at 4388b51c, and Measurements now has it run by run. off is the switch unset, whose lifted code, object keys and executable are those of the merge base 9dd41326; on is set at build. Kokiri Forest, 800x600, 4 runs a side after a warm-up: off 69.1 to 80.3 fps, on 69.4 to 83.2. Reading: not decided, no loss seen. Every run sits in one of two scene states (a refused or a taken teleport at the run's start, about 1,380 against 1,100 textures a frame) and both sides land in both; within a state the two sides overlap. The run wrapper is fixed, and a rerun of 8 a side, classed by state, follows. Clean build time and size, Linux and Steam Deck are not measured yet. The switch stays off.
Author
Contributor

Re #1979 (comment) (gameplay benefit): the warmed, interleaved in-game comparison has run at 4388b51c, 8 runs a side in Kokiri Forest (800x600, CPU-paced), every run checked in the same scene state, GPU cooled before each, on this PC alone; off is the switch unset (main's flags and header). Calm game time a frame, median (range): off 10.05 ms (9.96 to 10.79), on 10.23 (10.05 to 10.46), 1.8 % higher with the switch on; fps 90.7 against 85.2, and five of eight on-runs had p99 above 24 ms against one off-run. That is inside what eight a side resolve (roughly 1.5 to 2 %, an estimate) but leans to a loss: play shows no gain for it. The switch stays off; whether it stays as an off-by-default experiment or is dropped is your call and the author's. Measurements has every run; clean build time and size, Linux and the Steam Deck are not measured.

Re https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1979#issuecomment-67214 (gameplay benefit): the warmed, interleaved in-game comparison has run at 4388b51c, 8 runs a side in Kokiri Forest (800x600, CPU-paced), every run checked in the same scene state, GPU cooled before each, on this PC alone; off is the switch unset (main's flags and header). Calm game time a frame, median (range): off 10.05 ms (9.96 to 10.79), on 10.23 (10.05 to 10.46), 1.8 % higher with the switch on; fps 90.7 against 85.2, and five of eight on-runs had p99 above 24 ms against one off-run. That is inside what eight a side resolve (roughly 1.5 to 2 %, an estimate) but leans to a loss: play shows no gain for it. The switch stays off; whether it stays as an off-by-default experiment or is dropped is your call and the author's. Measurements has every run; clean build time and size, Linux and the Steam Deck are not measured.
Author
Contributor

Testing brought in line with Measurements: the Kokiri Forest off/on comparison of 2026-11-22 (8 a side at 4388b51c, no gain shown) is now under Tested; Still to test lists what is still open: clean build time and size, Linux, the Steam Deck and the Test Farm run, and a repeat of the Kokiri comparison only if the reviewer wants its lean settled.

Testing brought in line with Measurements: the Kokiri Forest off/on comparison of 2026-11-22 (8 a side at 4388b51c, no gain shown) is now under Tested; Still to test lists what is still open: clean build time and size, Linux, the Steam Deck and the Test Farm run, and a repeat of the Kokiri comparison only if the reviewer wants its lean settled.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 5s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/lift-mark-mask:siantm-perf/lift-mark-mask'
git switch 'siantm-perf/lift-mark-mask'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1979
No description provided.