MagicBeanDev/Hookshot
Archived
Watch
6
0
Fork
You've already forked Hookshot
 0
Code Issues 125 Pull requests 205 Activity

feat(dev): test a change against a base, as a pack of what it changes #2732

Open
github-sync wants to merge 20 commits from github/pr/2732 into main
pull from: github/pr/2732
merge into: MagicBeanDev:main
Conversation 4 Commits 20 Files changed 4 +1238
github-sync commented 2026-11-12 10:34:13 +00:00
Owner

@klimovich008 opened this pull request on GitHub · 2026-11-12 09:45 UTC · original

Testing a change in the game means a new full build: a new work folder translates the game again, and a new game folder prepares every shader from cold, 69 minutes in all on an RX 9070 XT. Most of that is the same for every change. This adds a development tool that builds the game once as a base and each change as a small pack over it.

What changed

  • scripts/dev_packs.py (new, development only):
    • base builds a checkout at a commit. --seed copies an older game folder's shader caches first, as an update keeps them.
    • pack builds a change. Commits, uncommitted edits and untracked files all count.
      • Renderer alone (RX_VK_ONLY, seconds): when every changed file is a renderer file the game isn't built from. Those files are found in the checkout from host/'s includes and build.py's dependencies.
      • Otherwise make_game.py, which builds both modules.
    • What a pack keeps:
      • A module when an object in its link list (objs.rsp, vk_objs.rsp) differs from the base's. Bytes can't decide this: nx2/version.py puts a fingerprint of every source into both modules.
      • Both modules when a build script, a non-translator data file under nx2/, or a non-C file changed.
      • Generated files that differ, such as the shader seed and the prompts. Files the base's own build made that the change no longer makes are removed.
    • rt makes a runtime: the base with romfs hard-linked and the other files copied, the packs' files over them. It runs a prepare when the shader translator or the seed changed. Two packs that bring the same file are refused, and so is a pack built on another base.
  • docs/development.md: a new section, Testing a change against a base.
  • AGENTS.md: one bullet pointing agents to it.
  • Nothing players build changes. It calls make_game.py and build.py as they are; no build or CI file is touched.

Testing

  • tests/test_dev_packs.py (no game):

    • the build route, and the renderer files the game reads;
    • that the translator list matches build.py's stamp, and that the version-stamped objects are build.py's;
    • that link-only inputs keep both modules;
    • the lift key's dependence on the folders;
    • generated files that change or are removed, and a clean output folder;
    • a runtime from packs, and every refusal by its own reason.
    • Negative controls: every rule above has a sabotaged variant of the script that fails the test.

  • Ten review rounds with gpt-6.1-sol, every finding fixed; the last found none.

  • End to end on Windows (Ryzen 7 5700X3D, RX 9070 XT), packs over a full build of 57ed3aa8:

    Pack Changed Kept Time
    host only host/nvn_native.c hookshot.exe 14 min first (one translation), 48 s later
    host + renderer host/nvdrv.c, vk_memory.cpp both 34 s
    renderer only vk_replay.cpp hookshot_vk.dll 16 s first, 2 s later
    runtime of two packs exe from one, DLL from the other 3 min 51 s

    Each pack's modules are byte-identical to that change's own earlier full build, apart from 2 PE header bytes. The renderer-only route gives the same DLL as make_game.py for the same tree.

  • Seeded base on main 46780503 from 57ed3aa8's caches:

    • 52.6 min, against 69 min for 57ed3aa8 cold.
    • Shader programs: 290 s against 822 s. main changed the translator, so the program cache was rebuilt, but the driver's pipeline cache carried over.
    • Precompile: 258 s against 602 s.
    • Predicted pipelines: 1,790 s against 2,150 s.

  • In the game: a runtime made by rt from two packs over 57ed3aa8 (the host-only pack's exe, with its new switches off, and the renderer-only pack's DLL):

    • The run: the standard Hyrule scene (32:9, unlocked frame rate, camera turning), 3 runs, RX 9070 XT, Windows 11.
    • Result: the game started and played the scene every time, at 40.4–41.9 fps. 57ed3aa8's own runtime measured 39.8 fps on the same test that morning.
    • Not a measurement: n=1 after two warm-ups, and the game clock dipped to 0.94–0.98 in these runs, as it did in the base's own runs that day.
    • No tested-configurations row: this was a development runtime with unmerged changes, not a player build.

  • Not tested: Linux. The module names there are hookshot and hookshot_vk.so, and RX_TARGET=windows uses the Windows names; neither pack nor rt has run there.

Related: the dev-loop proposal (#2636). This covers part of its "middle loop" without changing the build.

🤖 Generated with Claude Code

> [@klimovich008](https://github.com/klimovich008) opened this pull request on GitHub · 2026-11-12 09:45 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2732) Testing a change in the game means a new full build: a new work folder translates the game again, and a new game folder prepares every shader from cold, 69 minutes in all on an RX 9070 XT. Most of that is the same for every change. This adds a development tool that builds the game once as a base and each change as a small pack over it. ## What changed - **`scripts/dev_packs.py`** (new, development only): - `base` builds a checkout at a commit. `--seed` copies an older game folder's shader caches first, as an update keeps them. - `pack` builds a change. Commits, uncommitted edits and untracked files all count. - **Renderer alone** (`RX_VK_ONLY`, seconds): when every changed file is a renderer file the game isn't built from. Those files are found in the checkout from `host/`'s includes and `build.py`'s dependencies. - **Otherwise `make_game.py`**, which builds both modules. - **What a pack keeps:** - **A module** when an object in its link list (`objs.rsp`, `vk_objs.rsp`) differs from the base's. Bytes can't decide this: `nx2/version.py` puts a fingerprint of every source into both modules. - **Both modules** when a build script, a non-translator data file under `nx2/`, or a non-C file changed. - **Generated files that differ**, such as the shader seed and the prompts. Files the base's own build made that the change no longer makes are removed. - `rt` makes a runtime: the base with `romfs` hard-linked and the other files copied, the packs' files over them. It runs a prepare when the shader translator or the seed changed. Two packs that bring the same file are refused, and so is a pack built on another base. - **`docs/development.md`:** a new section, *Testing a change against a base*. - **`AGENTS.md`:** one bullet pointing agents to it. - **Nothing players build changes.** It calls `make_game.py` and `build.py` as they are; no build or CI file is touched. ## Testing - **`tests/test_dev_packs.py`** (no game): - the build route, and the renderer files the game reads; - that the translator list matches `build.py`'s stamp, and that the version-stamped objects are `build.py`'s; - that link-only inputs keep both modules; - the lift key's dependence on the folders; - generated files that change or are removed, and a clean output folder; - a runtime from packs, and every refusal by its own reason. - **Negative controls:** every rule above has a sabotaged variant of the script that fails the test. - **Ten review rounds** with gpt-6.1-sol, every finding fixed; the last found none. - **End to end on Windows** (Ryzen 7 5700X3D, RX 9070 XT), packs over a full build of 57ed3aa8: | Pack | Changed | Kept | Time | |---|---|---|---| | host only | `host/nvn_native.c` | `hookshot.exe` | 14 min first (one translation), 48 s later | | host + renderer | `host/nvdrv.c`, `vk_memory.cpp` | both | 34 s | | renderer only | `vk_replay.cpp` | `hookshot_vk.dll` | 16 s first, 2 s later | | runtime of two packs | | exe from one, DLL from the other | 3 min 51 s | Each pack's modules are byte-identical to that change's own earlier full build, apart from 2 PE header bytes. The renderer-only route gives the same DLL as `make_game.py` for the same tree. - **Seeded base** on main 46780503 from 57ed3aa8's caches: - 52.6 min, against 69 min for 57ed3aa8 cold. - **Shader programs:** 290 s against 822 s. main changed the translator, so the program cache was rebuilt, but the driver's pipeline cache carried over. - **Precompile:** 258 s against 602 s. - **Predicted pipelines:** 1,790 s against 2,150 s. - **In the game:** a runtime made by `rt` from two packs over 57ed3aa8 (the host-only pack's exe, with its new switches off, and the renderer-only pack's DLL): - **The run:** the standard Hyrule scene (32:9, unlocked frame rate, camera turning), 3 runs, RX 9070 XT, Windows 11. - **Result:** the game started and played the scene every time, at 40.4–41.9 fps. 57ed3aa8's own runtime measured 39.8 fps on the same test that morning. - **Not a measurement:** n=1 after two warm-ups, and the game clock dipped to 0.94–0.98 in these runs, as it did in the base's own runs that day. - **No tested-configurations row:** this was a development runtime with unmerged changes, not a player build. - **Not tested:** Linux. The module names there are `hookshot` and `hookshot_vk.so`, and `RX_TARGET=windows` uses the Windows names; neither `pack` nor `rt` has run there. Related: the dev-loop proposal (#2636). This covers part of its "middle loop" without changing the build. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- github-sync:item:2732 -->
scripts/dev_packs.py builds the game once as a base and each change as a
pack: the game for host/ and nx2/, the renderer alone for gpu/, a prepare
when the shader translator changed, chosen by the changed paths. A runtime
is the base with romfs hard-linked and the packs' modules over it. New
bases seed their shader caches from an older game folder. It calls
make_game.py and build.py as they are; nothing players build changes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A review found that choosing modules by path missed renderer files the game
is built from, non-gpu files the renderer is built from, generated files
(button prompts) and untracked sources. A pack now builds the renderer alone
only for gpu/vk files the game does not read (found from host/'s includes and
build.py's dependencies), and otherwise builds the game and keeps each module
whose linked objects differ from the base's, plus generated files that differ.
Module names follow RX_TARGET.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 2: a header change can reach only hos_main.c or vk_menu.cpp, the
objects that hold the version, so any header change now counts them; an nx2/
file other than the translator's, or a non-C file under host/, gpu/ or
third_party/, reaches the modules outside their objects (flags, link options,
the manifest), so both modules are kept; a generated prompt the change no
longer makes is removed from the runtime.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 3: a64-kit's toolchain.py is read by the lift and also sets the
link options, so the build's own scripts (build.py and what it imports) now
keep both modules whether or not the lift reads them; the output folder is
cleared but for romfs before each game pack, so an earlier pack's files are not
taken for this one's; any of the base's files outside romfs that the change's
build no longer makes (the shipped pipeline list) is removed from the runtime,
the files the game writes as it runs aside.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 4: make_game.py sets build.py's environment (the hot order, the
profile), so a change there keeps both modules; and a played base holds saves
and settings no build makes, so a pack now only removes files the base's own
build made, which `base` records (an adopted base records none).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 5: a pack name such as ../game would delete the base, so names
are plain; a base adopted from its own packs' folders would be rebuilt by them,
so that is refused; a base is built only into a new game folder, so a played
folder's saves are never recorded as build outputs. The test writes the map
only where the platform has one (Linux has none).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 6: a base folder inside a pack folder (or the reverse) is refused, not only the same folder; a
runtime inside the base or a pack folder is refused before anything is copied; the base records its language
(base --language, adopt reads language.txt) and its packs are built in it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 7: overlaps() resolves links and junctions (realpath), and folders on two drives never overlap
instead of raising; a game folder with its modules in bin/ (make_game.py's launcher layout) is refused with a
reason rather than failing on a missing module. The link case is tested with a junction where links need a
right Windows has not given.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 8: a base whose work or game folder is on another drive than its packs or a runtime failed at
the first hard link; the objects and romfs are copied there instead, as make_game.py does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review round 9: RX_TARGET=Windows chose the Linux names; and rt ran hookshot.exe on Linux for a prepare, which
make_game.py leaves to the game's first start on Windows.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
github-sync commented 2026-11-12 11:36:27 +00:00
Author
Owner

@hckrman101 reviewed on GitHub · 2026-11-12 11:25 UTC · original

Thanks, this is a careful tool, and the end-to-end table (byte-identical modules against full builds, the seeded base time, a played runtime) is the evidence I wanted for something like this. I read the script and the docs and did not run anything. CI is green. Two things I'd like before merge, both small.

1. Say plainly that packs and runtimes stay on the PC. A pack is not a patch of source: packs/<name>/ holds the built hookshot / hookshot_vk modules (the lifted game is linked into them) and files/, the generated files that differ from the base (shader seed, prompts, anything the build writes into romfs). A runtime is a whole game folder with the romfs hard-linked. "Pack" is a word people will hear as "something I can send a tester", and the new docs section never says no. CONTRIBUTING.md's main rule is that nothing built from the game leaves the PC, and the DLSS section there spells the same thing out in one sentence for its own files. Please add one to Testing a change against a base (and a few words to the AGENTS.md bullet): a base, a pack and a runtime are built from the game and are never attached, uploaded, or sent to a tester or the chat. What you share is the branch, and the tester builds their own.

2. rt trusts pack.json's paths. generated, deleted and files are joined onto the runtime folder with os.path.join(rt, rel) and then written, or removed with os.remove (the "replaced, never written through" branch), with no check that rel is relative and stays inside rt. Today the script writes those lists itself, so it is not reachable by accident. But a pack folder is a plain folder with a JSON file, and as soon as someone does send one around (point 1), an absolute path or a .. in it deletes or overwrites a file outside the runtime. A one-line guard in cmd_rt (reject absolute paths and anything whose realpath is not under rt) closes it, and it deserves a refusal case in test_guards like the others. Same for meta["files"] keys.

The rest of the safety side looks fine to me: every subprocess is a list (no shell) and the Python children run with -I; there is no network use; pack validates its name before its rmtree; rt refuses an existing folder and one that overlaps the base's folders; romfs is hard-linked and the pack's files replace the link, not the base's file; base refuses a non-empty game folder. Hard links with a copy fallback should behave on both NTFS and Linux; you say Linux wasn't run, and I agree that's the part to watch (hookshot / hookshot_vk.so, RX_TARGET), so please keep that line in the description.

Overlap with the dev loop (#2636). nx2/dev.py and tests/scenarios/ are only in that proposal, not on main, so nothing conflicts today. This lands squarely in its "middle loop" (build less, start warm) and takes a different route from it: a base plus a diff instead of a faster incremental build in one work folder. It sits in scripts/ as a standalone tool, which is where the proposal says standalone tools go, and it could later become dev.py base|pack|rt without a rewrite. I would not hold this for that. scripts/test_requests.py is unrelated (asks the team for testers). The existing RX_HOT_OPT section ends with "never use it for numbers"; this one is the opposite, for A/B numbers against a base, and the doc says so ("compare with a runtime made by rt from no pack"), which reads fine side by side.

AGENTS.md. The facts match the docs section, and the point about agents sharing one base is useful. It is a 7-line bullet in a list of one-liners, though: nit, cut it to the pointer plus the two rules that matter for agents (one shared base, agree before moving it; development and A/B only), and the new "never leave the PC" sentence. It does not collide with the "tested in the game, with measurements" rule from #2712: it is about how to build for a test, and the description is honest that its own 40.4-41.9 fps run is not a measurement.

Tests. They are meaningful without a game: they cut the real nx2/build.py and make_game.py text to check the translator list, the stamped objects and the renderer files the host reads, and build a fake base, packs and runtime in a temp folder, with the refusals checked by their own reason. Good, and as the repo's habit goes, they will need updating together with those scripts.

Description: it has a real end-to-end use, on a named GPU. Nothing else to ask there.

> [@hckrman101](https://github.com/hckrman101) reviewed on GitHub · 2026-11-12 11:25 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2732#pullrequestreview-5873552489) Thanks, this is a careful tool, and the end-to-end table (byte-identical modules against full builds, the seeded base time, a played runtime) is the evidence I wanted for something like this. I read the script and the docs and did not run anything. CI is green. Two things I'd like before merge, both small. **1. Say plainly that packs and runtimes stay on the PC.** A pack is not a patch of source: `packs/<name>/` holds the built `hookshot` / `hookshot_vk` modules (the lifted game is linked into them) and `files/`, the generated files that differ from the base (shader seed, prompts, anything the build writes into romfs). A `runtime` is a whole game folder with the romfs hard-linked. "Pack" is a word people will hear as "something I can send a tester", and the new docs section never says no. CONTRIBUTING.md's main rule is that nothing built from the game leaves the PC, and the DLSS section there spells the same thing out in one sentence for its own files. Please add one to *Testing a change against a base* (and a few words to the AGENTS.md bullet): a base, a pack and a runtime are built from the game and are never attached, uploaded, or sent to a tester or the chat. What you share is the branch, and the tester builds their own. **2. `rt` trusts `pack.json`'s paths.** `generated`, `deleted` and `files` are joined onto the runtime folder with `os.path.join(rt, rel)` and then written, or removed with `os.remove` (the "replaced, never written through" branch), with no check that `rel` is relative and stays inside `rt`. Today the script writes those lists itself, so it is not reachable by accident. But a pack folder is a plain folder with a JSON file, and as soon as someone does send one around (point 1), an absolute path or a `..` in it deletes or overwrites a file outside the runtime. A one-line guard in `cmd_rt` (reject absolute paths and anything whose `realpath` is not under `rt`) closes it, and it deserves a refusal case in `test_guards` like the others. Same for `meta["files"]` keys. The rest of the safety side looks fine to me: every subprocess is a list (no shell) and the Python children run with `-I`; there is no network use; `pack` validates its name before its `rmtree`; `rt` refuses an existing folder and one that overlaps the base's folders; `romfs` is hard-linked and the pack's files replace the link, not the base's file; `base` refuses a non-empty game folder. Hard links with a copy fallback should behave on both NTFS and Linux; you say Linux wasn't run, and I agree that's the part to watch (`hookshot` / `hookshot_vk.so`, `RX_TARGET`), so please keep that line in the description. **Overlap with the dev loop (#2636).** `nx2/dev.py` and `tests/scenarios/` are only in that proposal, not on main, so nothing conflicts today. This lands squarely in its "middle loop" (build less, start warm) and takes a different route from it: a base plus a diff instead of a faster incremental build in one work folder. It sits in `scripts/` as a standalone tool, which is where the proposal says standalone tools go, and it could later become `dev.py base|pack|rt` without a rewrite. I would not hold this for that. `scripts/test_requests.py` is unrelated (asks the team for testers). The existing `RX_HOT_OPT` section ends with "never use it for numbers"; this one is the opposite, for A/B numbers against a base, and the doc says so ("compare with a runtime made by `rt` from no pack"), which reads fine side by side. **AGENTS.md.** The facts match the docs section, and the point about agents sharing one base is useful. It is a 7-line bullet in a list of one-liners, though: nit, cut it to the pointer plus the two rules that matter for agents (one shared base, agree before moving it; development and A/B only), and the new "never leave the PC" sentence. It does not collide with the "tested in the game, with measurements" rule from #2712: it is about how to build for a test, and the description is honest that its own 40.4-41.9 fps run is not a measurement. **Tests.** They are meaningful without a game: they cut the real `nx2/build.py` and `make_game.py` text to check the translator list, the stamped objects and the renderer files the host reads, and build a fake base, packs and runtime in a temp folder, with the refusals checked by their own reason. Good, and as the repo's habit goes, they will need updating together with those scripts. Description: it has a real end-to-end use, on a named GPU. Nothing else to ask there. <!-- github-sync:r:5873552489 -->
Review (hckrman101): pack.json is a plain file, so rt now refuses a module name
that is not hookshot/hookshot_vk and any generated or deleted path that is
absolute or resolves outside the runtime, before anything is made; the docs say
that bases, packs and runtimes are built from the game and are never sent; the
AGENTS.md bullet is cut to the pointer and the rules agents need.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
github-sync commented 2026-11-12 12:37:37 +00:00
Author
Owner

@klimovich008 commented on GitHub · 2026-11-12 12:32 UTC · original

Thanks, all three are done in c3f666f6:

  1. Never leaves the PC. Testing a change against a base now says it plainly: a base, a pack and a runtime are built from the game (the pack's modules have the lifted game linked in, its files are generated from it, a runtime is a whole game folder), so they are never attached, uploaded or sent to a tester or the chat; what you share is the branch, and the tester builds their own. It links CONTRIBUTING.md's rule. The AGENTS.md bullet says the same in its last sentence.
  2. rt and pack.json. Before anything is made, rt now refuses a pack whose files names anything but the two modules, and any generated or deleted path that is absolute or resolves (realpath) outside the runtime. test_guards has four refusal cases: an absolute path, ../x, romfs/../../x, and a module key ../x. Each checks that nothing was made and that the file outside is untouched.
  3. AGENTS.md is down to three lines: the pointer, one shared base agreed before moving, development and A/B only, never send them.

The Linux line stays in the description.

> [@klimovich008](https://github.com/klimovich008) commented on GitHub · 2026-11-12 12:32 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2732#issuecomment-6823695890) Thanks, all three are done in c3f666f6: 1. **Never leaves the PC.** *Testing a change against a base* now says it plainly: a base, a pack and a runtime are built from the game (the pack's modules have the lifted game linked in, its files are generated from it, a runtime is a whole game folder), so they are never attached, uploaded or sent to a tester or the chat; what you share is the branch, and the tester builds their own. It links CONTRIBUTING.md's rule. The AGENTS.md bullet says the same in its last sentence. 2. **`rt` and `pack.json`.** Before anything is made, `rt` now refuses a pack whose `files` names anything but the two modules, and any `generated` or `deleted` path that is absolute or resolves (realpath) outside the runtime. `test_guards` has four refusal cases: an absolute path, `../x`, `romfs/../../x`, and a module key `../x`. Each checks that nothing was made and that the file outside is untouched. 3. **AGENTS.md** is down to three lines: the pointer, one shared base agreed before moving, development and A/B only, never send them. The Linux line stays in the description. <!-- github-sync:c:6823695890 -->
A second review of the guard (sol, on c3f666f6) found two ways past it:
- the check ran before the runtime existed, so a link in the base's romfs,
  cloned into the runtime, could carry a deleted path outside it;
- normcase let ../Runtime pass as the runtime in a case-sensitive folder.

in_runtime now judges a pack path by its text alone: relative, not rooted
("/x" is not isabs on Windows since Python 3.13), no "..", not ".". No name
comparison is left to fool. Where each file is written or removed, after the
runtime exists, resolves_inside checks that its folder, links followed, is
the runtime or below it, compared by file identity; a link itself is removed,
never written through.

Tests: the reviewer's cloned-link case (skipped where the account cannot make
a file link, as on Windows without developer mode; CI's Linux runs it),
resolves_inside with a folder link or junction, and in_runtime's text cases.
With resolves_inside disabled the junction case fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sol on ac5af7b0: romfs/foo:evil passed both checks and named a stream of the
hard link romfs/foo shares with the base, so removing or writing it changed
the base too. Pack paths never hold a ':'; in_runtime refuses one. The case
test now runs only where the temporary folder is case-insensitive.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
github-sync commented 2026-11-12 13:40:32 +00:00
Author
Owner

@klimovich008 commented on GitHub · 2026-11-12 12:51 UTC · original

Extra hardening of point 2 in ac5af7b0 and 13075852: rt now judges a pack path by its text first (no .., not rooted, no : stream names), then re-checks each destination where it is written, once the runtime exists, by file identity through links, so a link cloned from the base's romfs can't lead a write or delete outside the runtime; tests cover each case.

> [@klimovich008](https://github.com/klimovich008) commented on GitHub · 2026-11-12 12:51 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2732#issuecomment-6824107490) Extra hardening of point 2 in ac5af7b0 and 13075852: `rt` now judges a pack path by its text first (no `..`, not rooted, no `:` stream names), then re-checks each destination where it is written, once the runtime exists, by file identity through links, so a link cloned from the base's romfs can't lead a write or delete outside the runtime; tests cover each case. <!-- github-sync:c:6824107490 -->
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A development base prepares the whole game's ~51,000 predicted pipelines, about
30 of a base's 41-60 minutes, though a test draws a few scenes. base --predict-s N
puts RX_VK_PREDICT_S=N into make_game.py's environment (and rt's re-prepare):
the rest are built in play when first drawn (0, the default, builds them all; a
player's build never uses it). warm plays a runtime with no pack once per --run
command and copies the caches that grew back into the base, so later runtimes
start warm for the tested scenes. base, pack, rt and warm hold
<base>/dev_packs.busy, so a second job on one base is refused.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review: make_game.py's lift key hashes every RX_ variable, so a budgeted base
now builds with HOOKSHOT_NO_PREPARE=1 and runs the game's preparation itself
with RX_VK_PREDICT_S. warm copies back only plain cache files of the runtime
(never a link), through a new temporary file in the base's game folder.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
github-sync commented 2026-11-12 14:44:53 +00:00
Author
Owner

@Joey-Magnus approved on GitHub · 2026-11-12 14:34 UTC · original

I merged 13075852 into current main and ran tests/test_dev_packs.py, which passes. The two red jobs are not from this PR. One is test_release_scripts.py (test_nightly_two_runs_at_once_end_at_the_newer_commit), which is flaky. The other is test_pipeline_cache_save.py on Windows part 1, a timing wait on the in-play save. This PR only adds scripts/dev_packs.py, its test and docs, so a re-run should clear both.

hckrman101's points are all addressed. Docs and AGENTS.md now say plainly that bases, packs and runtimes never leave the PC. rt refuses absolute paths, .. paths and module keys other than the two modules, and re-checks each destination by file identity, and test_guards has a refusal case for each. The AGENTS.md bullet is down to three lines. I also read clear_output and the pack rmtree: both only work inside the dev folder, and the pack name is validated before the delete.

The one gap is Linux (hookshot/hookshot_vk.so), which the description already states. That's fine for a development tool.

> [@Joey-Magnus](https://github.com/Joey-Magnus) approved on GitHub · 2026-11-12 14:34 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2732#pullrequestreview-5876242465) I merged 13075852 into current main and ran tests/test_dev_packs.py, which passes. The two red jobs are not from this PR. One is test_release_scripts.py (`test_nightly_two_runs_at_once_end_at_the_newer_commit`), which is flaky. The other is test_pipeline_cache_save.py on Windows part 1, a timing wait on the in-play save. This PR only adds scripts/dev_packs.py, its test and docs, so a re-run should clear both. hckrman101's points are all addressed. Docs and AGENTS.md now say plainly that bases, packs and runtimes never leave the PC. `rt` refuses absolute paths, `..` paths and module keys other than the two modules, and re-checks each destination by file identity, and test_guards has a refusal case for each. The AGENTS.md bullet is down to three lines. I also read `clear_output` and the `pack` rmtree: both only work inside the dev folder, and the pack name is validated before the delete. The one gap is Linux (`hookshot`/`hookshot_vk.so`), which the description already states. That's fine for a development tool. <!-- github-sync:r:5876242465 -->
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Commenting is not possible because the repository is archived.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
autorelease: pending

   
autorelease: tagged

   
bug
Something isn't working

  
dependencies
Pull requests that update a dependency file

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
github_actions
Pull requests that update GitHub Actions code

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
testing wanted

   
wontfix
This will not be worked on

No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Hookshot!2732
No description provided.