perf(build): skip unchanged links, write compile_commands.json, key the hot pieces by the profile's contents #2893

Open

github-sync wants to merge 1 commit from github/pr/2893 into main

github-sync commented

2026-11-12 19:01:04 +00:00

Owner

@jordanella opened this pull request on GitHub · 2026-11-12 18:41 UTC · original

A build with nothing to compile still relinked the 440 MB executable and the renderer, a build from a second checkout of the same tree recompiled all eight hot pieces (~10-20 minutes) because their keys named the play profile by its path, and clangd had no compile database. Three small changes to nx2/build.py from the dev-loop proposal (#2636, A4 and A8); one PR because they touch the same lines, and I can split them if you prefer.

Change

nx2/build.py:

A4, no link when nothing changed: run_link skips a link whose inputs are the last successful one's. Its key is kept beside the output (<output>.linkkey) with the output's size:
- the command, the compiler and the work folder's system-headers stamp;
- each object's key (its .o.sha1; its size and time without one);
- the contents of the other files it names, inside options too (the hot symbol order, the libraries, the resources).
A failed link, or an output of another size or gone, links again. The executable's, the renderer's and the game DLL's links go through it.
A8, compile_commands.json: every compile as compile_one runs it, written to the build folder for clangd and clang-tidy (compile_commands.json at the checkout's top is git-ignored, for a copy there).
The hot pieces' keys name the profile by its contents: flag_text leaves -fprofile-instr-use=<path> out of the key, since -DHOOKSHOT_PROFILE_SHA1 beside it already says what the key needs of the profile. This changes their keys once.

Measured

Windows 11, Ryzen 9 3900X, a dev work folder:

A build with nothing to do: 16 s against 6 s, both links skipped.
One renderer edit: only the renderer's link runs.
A second checkout of the same tree building in the same work folder: the eight hot pieces from the cache, where main compiled them again.

In the game

Every build I ran in the game today used this build.py. The latest: Windows 11, RTX 3080 (NVIDIA 621.14), a dev-profile build, from a save, headless through the control bridge: in play in Kokiri Forest, 28-31 fps, nothing in the log about a lost device or a crash. A skipped link leaves the same binary in place.

Testing

tests/test_link_skip.py (new): a stand-in linker through the real run_link: the same inputs aren't linked again; a changed object, library or symbol order is; so is an output of another size or gone; a failed link forgets its key; the files inside options are inputs; the compile commands are the compiles; the build's links go through it.
tests/test_profile_path_key.py (new): two paths, one profile give one key; another profile, another key; every use of the profile's path names its contents too.
Every check that reads the changed files (37), through run_all.py, rebased on today's main, Windows: pass, apart from four that fail the same way on main here (test_driver_cache_folder, test_fpx_sticky, test_linux_update, nx2/test_dll.py). Rebasing found that main's build.py doesn't import re (#2706 adds it); this PR imports it too.
Not tested: Linux.

See #2636 (A4, A8).

🤖 Generated with Claude Code

> [@jordanella](https://github.com/jordanella) opened this pull request on GitHub · 2026-11-12 18:41 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2893) A build with nothing to compile still relinked the 440 MB executable and the renderer, a build from a second checkout of the same tree recompiled all eight hot pieces (~10-20 minutes) because their keys named the play profile by its path, and clangd had no compile database. Three small changes to `nx2/build.py` from the dev-loop proposal (#2636, A4 and A8); one PR because they touch the same lines, and I can split them if you prefer. ## Change `nx2/build.py`: - **A4, no link when nothing changed:** `run_link` skips a link whose inputs are the last successful one's. Its key is kept beside the output (`<output>.linkkey`) with the output's size: - the command, the compiler and the work folder's system-headers stamp; - each object's key (its `.o.sha1`; its size and time without one); - the contents of the other files it names, inside options too (the hot symbol order, the libraries, the resources). A failed link, or an output of another size or gone, links again. The executable's, the renderer's and the game DLL's links go through it. - **A8, `compile_commands.json`:** every compile as `compile_one` runs it, written to the build folder for clangd and clang-tidy (`compile_commands.json` at the checkout's top is git-ignored, for a copy there). - **The hot pieces' keys name the profile by its contents:** `flag_text` leaves `-fprofile-instr-use=<path>` out of the key, since `-DHOOKSHOT_PROFILE_SHA1` beside it already says what the key needs of the profile. This changes their keys once. ## Measured Windows 11, Ryzen 9 3900X, a dev work folder: - **A build with nothing to do:** 16 s against 6 s, both links skipped. - **One renderer edit:** only the renderer's link runs. - **A second checkout of the same tree** building in the same work folder: the eight hot pieces from the cache, where `main` compiled them again. ## In the game Every build I ran in the game today used this `build.py`. The latest: Windows 11, RTX 3080 (NVIDIA 621.14), a dev-profile build, from a save, headless through the control bridge: in play in Kokiri Forest, 28-31 fps, nothing in the log about a lost device or a crash. A skipped link leaves the same binary in place. ## Testing - **`tests/test_link_skip.py` (new):** a stand-in linker through the real `run_link`: the same inputs aren't linked again; a changed object, library or symbol order is; so is an output of another size or gone; a failed link forgets its key; the files inside options are inputs; the compile commands are the compiles; the build's links go through it. - **`tests/test_profile_path_key.py` (new):** two paths, one profile give one key; another profile, another key; every use of the profile's path names its contents too. - **Every check that reads the changed files (37), through `run_all.py`, rebased on today's `main`, Windows:** pass, apart from four that fail the same way on `main` here (`test_driver_cache_folder`, `test_fpx_sticky`, `test_linux_update`, `nx2/test_dll.py`). Rebasing found that `main`'s `build.py` doesn't import `re` (#2706 adds it); this PR imports it too. - **Not tested:** Linux. See #2636 (A4, A8). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

github-sync added 1 commit

2026-11-12 19:01:04 +00:00

perf(build): skip unchanged links, write compile_commands.json, key the hot pieces by the profile's contents 618fdd8832

Three changes to nx2/build.py:

- A4: a link whose inputs are the last successful one's is not run again
  (run_link). Its key, beside the output as <output>.linkkey: the command,
  the compiler and the work folder's system-headers stamp, each object's
  key (.o.sha1), and the contents of the other files it names, inside
  options too (the hot symbol order). A build with nothing to do went from
  16 s to 6 s on Windows. Players' builds let sa/build go, so it helps a
  work folder that keeps its files (the checkout's work/,
  HOOKSHOT_KEEP_BUILD_FILES=1).
- A8: <build>/compile_commands.json, every compile as compile_one runs it,
  for clangd and clang-tidy (git-ignored at the checkout's top).
- The hot pieces' keys name the play profile by its contents
  (-DHOOKSHOT_PROFILE_SHA1), not its path: a build from a checkout other
  than the work folder's compiled all eight again (~10-20 minutes). This
  changes their keys once.

tests/test_link_skip.py (a stand-in linker: skipped, relinked on a changed
object, library, symbol order or output, a failed link forgets the key;
the compile commands) and tests/test_profile_path_key.py.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

Commenting is not possible because the repository is archived.

No reviewers