WIP: build(asm): hand-written x86-64 kernels have a common header, a check kit and the build's jobs, and the GPU-visible ranges' sort is the first #1947
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1947
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:build/asm-kit"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Longshot has no hand-written assembly yet: every hot loop is C or C++ with intrinsics. A few loops of the capture, the
texture uploads, the hashes and the write marks run faster written by hand for x86-64 (the follow-up pull requests,
one per kernel, each with its own check and numbers). Each of them needs the same ground, so it comes first, with
the first kernel on it, so that the ground is exercised by a caller the game runs:
gpu/asm/lsasm.h: a kernel is written once, in Intel syntax against the System V registers (arguments in rdi,rsi, rdx, rcx, r8, r9). Its macros (
LSASM_FUNC,LSASM_ARGS,LSASM_PUSH,LSASM_SAVE_XMM,LSASM_ALLOC,LSASM_RET...) make the Win64 convention around it on Windows, with the SEH unwind data Windows needs to unwind afault inside a kernel (a capture copy's
guarded_copyingpu/vk/vk_memory.cppcatches one) and to walk a stackstopped in one (the samplers' walk in
host/mkw_glue.c), and the DWARF CFI on Linux. The epilogue is laid out asWindows' unwinder recognises one (the
vzeroupperand the frees before the pops), so a thread stopped in it unwindsto its caller too. The body runs with rsp 16-byte aligned (the macros count the pushes and pad where needed, in the
unwind data too); a frame over 4096 bytes stops the assembly, as the macros probe no stack page;
LSASM_ARGStakesthe kernel's count of integer or pointer arguments and reads only the stack slots a caller passed. In C and C++ it
gives
LSASM_ABIfor the declarations.tests/asm_kit.py: what every kernel's check shares. It builds the check's fixture (the C the kernel standsfor, as the reference) with the kernels at SSE2 and at AVX2 (where this CPU has it), on Windows also the System V
body (what Linux runs) called through
sysv_abi, so a Linux-only register mix-up fails on Windows too; and itassembles every kernel with clang for ELF and for COFF (the clang given, beside the compiler, or on PATH; none fails
the check). Its AVX2 run is gated on the whole x86-64-v3 level and the OS's ymm state, and each kernel is its own
object even when two folders hold the same name.
tests/test_asm_abi.pywithtests/asm_abi_probe.S: probes that use every part of a frame and clobber everyregister they may. Six arguments arrive; a one-argument call on a stack that ends at its home area reads no other
slot; every callee-saved register comes back; the body's rsp is 16-byte aligned; a table in
LSASM_RODATAis read.On Windows a fault inside a probe unwinds to its caller with the registers restored, by hand and through Windows'
own dispatch to a caller's
__except, and every single step of each probe (the trap flag) is unwound by Windows'own unwinder to its caller with the whole nonvolatile state, the epilogue's steps included. On Linux a probe stopped
in its body is unwound through its DWARF CFI by the system's unwinder.
plan_modulesandasm_jobsinnx2/build.py): every.Sinhost/asmgoes into the game, ingpu/vk/asminto the renderer, ingpu/asminto both (an object each), preprocessed with the build's CPU flags(which pick each kernel's AVX2 or SSE2 body) and keyed, like the host's and the renderer's own files, by the headers
its depfile says it read (every header of its folder and
gpu/asmbefore it has one). A shared kernel's secondjob leaves the cache entry the first published, and a kernel compiled again is written to a new file (
compile_oneremoves the object first: clang's assembler writes in place, and the object is a hard link into the cache);
gpu/asm's headers key the renderer's own files. No folder, nojob. The AppImage carries
gpu/asm;dev.py packtakes a.Sas a source of its module, ahost/asmone building the game alone (scripts/devtools/packs.py), and thesources' fingerprint (
nx2/version.py) counts it.lsasm_sort_ranges(host/asm/ranges.S,588159c4):hos_gpu_guest_ranges(
host/nvdrv.c) sorts every GPU mapping's and native NVN pool's guest range by start before it merges them, eachtime the mappings change (at boot, at area loads, when pools come and go). It sorted with
qsortand a comparisoncall per compare; on x86-64 it now sorts with a merge sort written on
lsasm.h(runs of 16 pairs by insertion, thenmerges of runs twice as long, two runs already in order copied, not merged; one body for both CPU levels). Other
CPUs keep
qsortwithcmp_range(#if defined(__x86_64__)). The sort is stable whereqsortis not; the mergeafter it makes the same ranges from any order of ranges that start together, so the result is the same. Both
lists now grow through
hos_xrealloc(whererealloc's NULL was used as the list before).scripts/profiling/cpu_functions.jsonnames the kernel. No switch: the ranges are the same.Read first:
gpu/asm/lsasm.h,tests/asm_kit.py,plan_modulesandasm_jobsinnx2/build.py, thenhost/asm/ranges.Sandhos_gpu_guest_rangesinhost/nvdrv.c.Testing
Tested: Windows 11, i7-9700, llvm-mingw clang 23.1.5. Every check ran through the PC's job queue. Head
db3d91f1,rebased onto main
7c23d2be. It has NOT run in the game on this PC at this head: the only game run was on df88c325,before the review's fixes and without a kernel (below).
e6196f0f(rebased since onto7c23d2bewith no conflict,git range-diffall=):tests/test_asm_end_to_end.py --cxx clang++(5 tests, pass). It writes one kernel of both modules (gpu/asm) andone kernel of each module alone (
host/asm,gpu/vk/asm) in a tree of its own, on the reallsasm.h. It runs thebuild's own code on them with clang:
plan_modules,plan_jobs,compile_oneon four threads,rekey_depfiles, andmain()'s link-list statements. It links and runs each module's link list with a caller. It covers:new name it compiles again, alone;
scripts/devtools/packs.py's decisions ascmd_packmakes them: thegame alone, the renderer alone, then both. The base's objects stay as they were;
At its test commit 1fefe953 (
b21224ffnow) it failed 4 checks: a reverted edit linked the edited kernel, and a pack rewrote thebase's objects. These pass with the fix,
db3d91f1(Summary, the build).The PR's own checks pass:
tests/test_asm_abi.py --cxx clang++(both levels, both conventions, the clangassemblies, the frame bound),
tests/test_asm_build.py(15),tests/test_asm_kit.py,tests/test_asm_ranges.py --cxx clang++(SSE2 and AVX2, both conventions),tests/test_appimage_source_headers.py(3) andtests/test_appimage_headers.py.tests/run_all.py --strict --changed origin/mainran every check, because a shared test file changed.RX_CPMwasset, so the renderer checks compiled against the Vulkan headers. 866 checks: 786 passed, 80 failed, 48 skipped; no check stopped for want of the Vulkan headers. 78 of the 80 fail on main
e6196f0ftoo, rerun by name there withRX_CPMset. Run again by name atdb3d91f1(fb3-1947-2), the same 79 fail andtests/test_present_semaphores.pypasses; it failed in the run above (no output, 10.9 s, 8 at once) and passes alone atdb3d91f1and on main7c23d2be.tests/test_vbytes.pytimes out (600 s) here and on main alike, alone too (db3d91f1, main7c23d2be).Four checks timed out under load in the whole-suite run on
8f5c08c6:tests/test_host_alloc_wait.py,test_mesh_args_cleanup.py,test_oot_wide_bubble.pyandtest_oot_wide_save_picture.py. Each passes when runalone, at 3bf51a69 and on main
e6196f0f.4cf7de04(main6e1521a7):tests/test_dev_packs.pywas run one check at a time, because its script stopsat the first failing check; the kernel routing passed. The earlier
--changedfigures are replaced by the run above:those runs had no Vulkan headers, so their renderer checks did not run.
dev.py packs, 56 s), started inKokiri Forest from the scene's save, 60 s standing and turning. It played (41 to 52 fps lines), did not crash, and
stopped cleanly.
Still to test:
python3 -I -B tests/test_asm_abi.py --cxx g++and--cxx clang++, expecting PASS at both levels(the System V body natively, the ELF unwind from a probe's body) and the clang assemblies (by hand: the Test Farm
plays the game and times builds, it runs no test file)
python3 -I -B tests/test_asm_ranges.pywith gcc and with clang, expecting PASS (the kernel'sSystem V body natively against
qsort) (by hand)7c23d2beon the same PC, and the update rebuild cost (by hand)lsasm_sort_ranges: a clean build against main (its time and output size), this head builtas an update over main's stores, and a rebuild after renaming
gpu/asm/lsasm.h; a realdev.py packspack of a kernelchange; a Linux AppImage build.
tests/test_asm_end_to_end.pyruns the build's code on authored kernels, not these(by hand: the build pair is queued on this PC)
main
7c23d2beagainst this head, warmed, interleaved, expecting the same play (no crash, no stale or flickeringtexture, the same frame times within their spread) (by hand)
mainon the same PC, started in Kokiri Forest for a minute, expecting it to play as onmain(no crash, no hang,frame time and VRAM as on
main); the build timed againstmain's, expecting the lifted C to be the same (the TestFarm, requested in a comment)
Measurements
Verdict: not measured: no micro-benchmark of the ported kernel and no run in the game on this PC at this head. On
the branch it comes from (971f8074,
exp/x86-asm-sync) a micro-benchmark on this PC found the kernel faster thanqsort, more so in near order; the sort runs only when the mappings change, so a frame-time difference in play is notexpected to show. The build's one cost, the runtime's C files compiled again once after updating (their key counts
the headers under
gpu/, andgpu/asm/lsasm.his a new one: 80 files, 26 s of compiling in the pack of df88c325), isin Risks; the clean build's time and size against main are in Still to test.
Risks
The game now runs a hand-written kernel on x86-64: a wrong sort would merge the GPU-visible ranges wrongly, and the
renderer would miss or over-track guest writes in GPU memory (stale or flickering textures, more CPU in change
tracking) from boot on; a fault in it is unwound as any kernel's (
lsasm.h). The test compares it withqsortonevery size around its runs and merges. Other CPUs keep
qsort. The build's flags are as before; the runtime's Cfiles compile once more after updating (Measurements). A revert of
588159c4gives backqsort; no switch.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). Not yet: 79 checks fail here and on main alike (Testing).main, or Measurements says why there is nothing to measure. Not yet: the kernel is not measured (Measurements).third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Tested in game: Windows 11, RTX 4060 (NVIDIA 610.88), i7-9700: this change (df88c325, the same commit before its rebase) built as a pack over main's base and started in Kokiri Forest from the scene's save, 60 s standing and turning: it played (41-52 fps lines), no crash, clean stop; the version line named the commit. It adds no kernel, so nothing the game runs differs.
@Beanstalk This adds the build's jobs for hand-written x86-64 assembly (
asm_jobsinnx2/build.py) and a shared header,gpu/asm/lsasm.h, with no kernel yet: no code the game runs should change, and it has not run in the game on Linux, AMD or Intel yet. Please build this PR and main and time the build against main's (a warm build), checking the lifted C is the same, then start the game in Kokiri Forest for a minute on each side. On Linux with NVIDIA, Linux with AMD, Windows with AMD, Windows with Intel, and the Steam Deck. It must not crash or hang, and frame time p99 and peak VRAM must be as on main.@siantm only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for10d5d0d, and the PR has changed since (4cf7de0), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Recommendation: Request changes. The shared assembly layer would reduce duplicated kernels, but it can misalign Win64 frames, reuse stale objects, and break AppImage builds as soon as a kernel is added. The new checks leave key unwind and build paths unverified.
Adds a common x86-64 ABI header, a game-free test kit, and build, pack, and version support for assembly sources. There are no game kernels in this PR. The author reported a one-time rebuild of 80 runtime C files taking 26 seconds in a 56-second Windows pack.
Does it help: Not a perf change. The author reports a passing 9.3-second Windows ABI check and a passing build helper check on df88c325 before rebase. The 60-second Kokiri Forest run at 41–52 fps is a smoke test. Clean build time and size against
8d56b59dand Linux ABI results are missing.Other systems: Windows and Linux share the job planner and object cache but use different unwind formats. The Linux AppImage stages a narrower source tree than the Windows Builder. Developer packs can choose to rebuild either module, so dependency tracking matters independently of the full build. No kernel enters capture, graphics, audio, input, or save paths in this PR. The only reported game run was Windows on NVIDIA. Linux, Steam Deck, Windows AMD and Intel, and the full suite on this head remain unverified.
Switches: LSASM_TEST_SYSV is a test-only preprocessor define, unset by default. The Windows fixture sets it for its second ABI variant. That default is right for shipping builds. Assembly jobs inherit the existing RX_MARCH choice and its automatic CPU default. No player switch or switch-table change.
Reviewed:
10d5d0de15against8d56b59d47: Read all ten changed-file diffs and the full new header, assembly probe, three Python files, build reference, and version module. Read the changed sections and relevant callers in nx2/build.py, packs.py, and test_dev_packs.py. Read AGENTS.md, CONTRIBUTING.md, the Forgejo template, and docs/dev/standards.md. There is no docs/review-checklist.md in the checkout. Traced the relevant compile, link, cache, package, CPU detection, sampler, guarded-copy, lint, and test-runner paths. Read the author’s testing report and sole comment. Unrelated sections of the large modules and historical recorded runs were outside the review scope. No generated files, translations, or recorded-run files changed.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Waiting on the author (a run, a measurement or a decision; no edit does it)
Testing: Read-only review. git diff --check passed. No project scripts, builds, or tests were run. The author reported the Windows ABI and build helper passes and a clean game exit on df88c325. The changed-test run stopped after 356 checks and the pack test was last tried on the older base.
Overlaps and merge order: No overlapping PR found in local history. The planned kernel PRs depend on this work. Open Forgejo PRs could not be checked from the offline checkout.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -0,0 +19,4 @@LSASM_ENDFUNC lsasm_exampleThe rules a kernel keeps:- a leaf: it calls nothing, so the stack needs no alignment and nothing above it needs to know its frame;Win64 frame functions need aligned RSP in the body. The six-argument probe pushes eight registers and stays misaligned. Account for padding in the unwind data and epilogue.
@ -0,0 +83,4 @@pop \reg.endm.macro LSASM_ALLOC nsub rsp, (\n + 15) / 16 * 16 /* (16s: Windows' xmm save offsets above it must stay multiples of 16) */A large scratch allocation can skip the Windows stack guard page. Bound the total frame or probe it.
@ -0,0 +168,4 @@mov rsi, rdxmov rdx, r8mov rcx, r9mov r8, [rsp + 56] /* (the fifth and sixth: above the two pushes, the return address and the 32-byte home area) */A short call owes only the home area. Use an argument count so LSASM_ARGS reads stack slots only when the caller supplied them.
@ -659,0 +663,4 @@if not os.path.isdir(folder):return []common = os.path.join(GPU, "asm")deps = b"".join(open(os.path.join(d, f), "rb").read() for d in sorted({common, folder}) for f in sorted(os.listdir(d))Renaming an included header can leave this key unchanged, and .inc edits are invisible. Key the assembly object's actual include paths and contents.
@ -659,0 +666,4 @@deps = b"".join(open(os.path.join(d, f), "rb").read() for d in sorted({common, folder}) for f in sorted(os.listdir(d))if f.endswith(".h"))return [(cc, os.path.join(folder, f), os.path.join(obj_dir, prefix + f[:-2] + ".o"),target() + CPU + ["-x", "assembler-with-cpp"], deps)The AppImage omits gpu/asm. A private kernel triggers a missing-directory error here, and shared kernels are omitted. Stage the common sources and cover them in the package check.
@ -2113,6 +2130,8 @@ def main():vk_jobs.append((cc, zstd_src, os.path.join(obj_dir, "vk_zstd.o"),target() + THIRD_PARTY_CPU + (["-fPIC"] if LINUX else []) +["-O2", "-g0", "-DNDEBUG", "-Wno-everything", "-fvisibility=hidden", "-I" + zbic], zstd_deps + tp_deps))vk_jobs += asm_jobs(cc, os.path.join(vk, "asm"), obj_dir, "vk_asm_") + \The renderer's C++ dependency roots omit gpu/asm. A shared declaration edit can leave a Vulkan caller using the old signature.
@ -2114,2 +2131,4 @@target() + THIRD_PARTY_CPU + (["-fPIC"] if LINUX else []) +["-O2", "-g0", "-DNDEBUG", "-Wno-everything", "-fvisibility=hidden", "-I" + zbic], zstd_deps + tp_deps))vk_jobs += asm_jobs(cc, os.path.join(vk, "asm"), obj_dir, "vk_asm_") + \asm_jobs(cc, os.path.join(GPU, "asm"), obj_dir, "vk_lsasm_") # (the renderer's hand-written x86-64 kernels)Both jobs for a shared kernel have the same cache key. Concurrent cache_put calls can copy over an already linked object. Compile once or publish atomically.
@ -101,3 +101,3 @@"scripts/check_windows_files.py", "scripts/run-vk.ps1", "scripts/fetch_commit.sh", "scripts/release_keys.json")HEADERS = (".h", ".hpp", ".hh", ".inc", ".inl")SOURCES = (".c", ".cc", ".cpp") + HEADERSSOURCES = (".c", ".cc", ".cpp", ".S") + HEADERS # (.S: the hand-written x86-64 kernels, nx2/build.py asm_jobs)A host/asm-only edit still rebuilds the renderer. Teach renderer_untouched about host assembly and test routing for all three folders.
@ -0,0 +11,4 @@import asm_kitasm_kit.run_fixture(cxx, "name", fixture_cpp_text, [ROOT / "gpu/vk/asm/hash.S"], work_dir)"""import osRemove the unused os import. Ruff F401 is enabled.
@ -0,0 +35,4 @@def has_avx2(cxx, work):"""Whether this CPU runs x86-64-v3 code (AVX2, BMI2, FMA)"""probe = Path(work) / "cpu_probe.cpp"exe = Path(work) / ("cpu_probe" + (".exe" if WINDOWS else ""))Gate the optimized fixture on the full x86-64-v3 level, including OS AVX support. Fail if the CPU probe itself crashes.
@ -0,0 +63,4 @@out = Path(out)objs = []for s in sources:obj = out.with_name(out.stem + "_" + s.stem + ".o")Same-stem sources from different folders overwrite the fixture object. Give each source its own output name.
@ -0,0 +79,4 @@work.mkdir(parents=True, exist_ok=True)fixture = work / (name + ".cpp")fixture.write_text(fixture_text, encoding="utf-8")clang = clang if clang is not None else shutil.which("clang")Strict runs can skip both cross targets silently and still claim AVX2 was assembled. Resolve Clang explicitly and report the checks that ran.
@ -0,0 +78,4 @@DWORD64 frame = 0;RtlVirtualUnwind(UNW_FLAG_NHANDLER, base, u.Rip, f, &u, &data, &frame, nullptr);const uint64_t r[6] = {u.Rbx, u.Rbp, u.R12, u.R13, u.R14, u.R15};ok = u.Rip == step_ret && u.Rsp == step_rsp && !std::memcmp(r, step_regs, sizeof r);Check RDI, RSI, and XMM6–15 during Win64 stepping. Add a fault through a real __except caller to check Windows exception dispatch.
@ -0,0 +121,4 @@const uint64_t changed = lsasm_probe_harness(lsasm_probe_clobber, 41, &r);if (changed) std::fprintf(stderr, "callee-saved registers changed: %#llx\n", (unsigned long long)changed);CHECK(changed == 0 && r == 42);#ifdef _WIN32Deleting the CFA adjustment leaves the Linux fixture green. Exercise ELF unwinding from the scratch frame and use LSASM_RODATA in a probe.
@ -0,0 +20,4 @@sys.path[:0] = [str(ROOT / "nx2"), str(ROOT / "third_party" / "a64-kit" / "tools")]spec = importlib.util.spec_from_file_location("build_asm_jobs", ROOT / "nx2" / "build.py")build = importlib.util.module_from_spec(spec)spec.loader.exec_module(build)Importing build.py inherits RX_MARCH and can invoke native compiler discovery before the tests run. Isolate the test settings and stub discovery.
@ -0,0 +71,4 @@def test_no_folder_no_job(self):self.assertEqual(self.jobs(self.kernels.parent / "missing"), [])def test_the_modules_take_their_folders(self):These string checks can pass with missing link inputs. Assert the production planner's outputs in full, game-only, and renderer-only builds.
Recommendation: Request changes. The assembly kit needs a correct Win64 frame contract and an exercised path from source packaging through both link lists before kernels can use it. I independently confirmed the central blockers in the existing same-head review. Existing review
Does it help: Build infrastructure. The author reports a 9.3 s Windows ABI pass and a passing job-planner fixture on
df88c325before rebase. The reported 56 s development pack included 26 s recompiling 80 runtime C files. Those results show the proposed checks and the one-time update cost on that setup. Neither the gameplay smoke nor that pack compiled a new production kernel. Author's smokeReviewed:
10d5d0de159563c22b11a13cf7e6a8bd4bd91409against merge base8d56b59d4766034b42ffe306d19a5eaf0487c989. Read all ten file diffs, the new assembly macros and probes, and the compile, object-cache, pack, AppImage, Windows Builder and exception callers. Both exact Graphify maps validated. Graphify did not extract symbols from the assembly macro header, which I read directly. No PR source was executed.LSASM_TEST_SYSVis fixture-only. Production continues to use the existing CPU selection.Checked and fine: The Win64 offsets for six supplied integer or pointer arguments account for both saved registers and the shadow space. The shared and private jobs have separate output names and the production linker separates game and renderer inputs. Windows Builder stages tracked
gpu/andhost/files, which includes the new sources. The AppImage uses a separate staging list.Blocking:
.Sreaches both concurrent job lists with the same object key. If both cache publishers observe the key missing, the losing hard-link attempt can fall back to copying over the winner's cache file. Also, the AppImage leaves outgpu/asm. A private kernel then reachesos.listdir(common)on a missing directory, while a shared kernel disappears from the packaged inputs. I would check one shared kernel and one private kernel per module end to end through a cold build, a cached rebuild after an included-file rename, developer packs, and the Linux AppImage staging.Testing and farm: Full exact-head suite and native Linux GCC/Clang ABI and DWARF unwind results are outstanding. The existing author request is pending. Its Kokiri smoke can check basic compatibility once the above is fixed. The current head has no production kernel, so that smoke cannot cover the disputed ABI, rebuild or packaging paths. Do the ABI fixtures by hand on Windows and Linux, including an application-style SEH handler and saved-register checks. Reuse the author's request after the revised head is pinned and the plan says which real build path will exercise
.S. Author's requestOverlap: The sampler is a consumer of the unwind metadata. The sampler change can be reviewed independently. This kit needs to satisfy the OS ABI used by both the sampler and guarded copy. Build-profile changes in the same review batch also edit
nx2/build.pyand should be rechecked when rebased.10d5d0de154cf7de0482Thank you for the review. The branch is now
4cf7de04, rebased onto main6e1521a7with no conflicts;git range-diffshows the PR's own commit as=. Each behaviour fix is two commits: the test first, then the fix. Every check below ran through this PC's job queue (Windows 11, i7-9700, llvm-mingw clang 23.1.5, jobs fb-1947-b3, -c1, -d1 and -e1). Every fix's test failed at its test commit and passed at its fix. The game has not run at this head.7ff1ed9b(test5eb2e659). The macros count the pushes. When rsp would be 8 modulo 16 after them, they take 8 more bytes before the first xmm save or scratch, recorded in both the SEH codes and the CFI. The epilogue gives the 8 bytes back in its single add. Three probes return rsp % 16 from their bodies: scratch only, an odd count of pushes and an even count. On Windows each one is also single-stepped and unwound. At5eb2e659the check failed (rsp % 16was 8); at7ff1ed9bit passes.61011b0b(test52672716).LSASM_ARGSnow takes the kernel's argument count, 0 to 6; any other count stops the assembly. It moves only those registers, and it loads the fifth and sixth from the stack only when they were passed. The header states that the mapping covers integer and pointer arguments. The test calls one-argument probes on a stack that ends at their 32-byte home area, with the page above it unreadable. At52672716the fixture crashed with an access violation (0xC0000005); at61011b0bit passes.7ff1ed9b(test5eb2e659).LSASM_END_PROLOGUEstops the assembly when a frame is larger than 4096 bytes (pushes, padding, xmm saves and scratch together). In both conventions, 3072 bytes of scratch assemble and 4096 bytes are rejected. At the test commit, the run stopped at point 1's failure before it reached this check.bd9564da(test95432bfd).cache_putno longer writes over an entry that is already there (FileExistsError). A copy across volumes is written under a temporary name and renamed into place. The test replays the race. At the test commit the second publication wrote into the first job's object; at the fix both the first object and the cache entry stay intact.bd9564da(test95432bfd). A kernel is now compiled with-MDand keyed the same way as the host's and the renderer's own files (renderer_deps,DEPFILE_JOBS,rekey_depfiles): by the names and contents of the headers its depfile lists,.incincluded. Until a kernel has a depfile, every header in its folder and ingpu/asmkeys it. A rename, a removal and a.incedit each change the key, and an edit to a header the kernel does not read keeps it.bd9564da(test95432bfd).gpu/asmis now one of the renderer's dependency roots. The test edits agpu/asmdeclaration that a renderer file's depfile lists, and that file's key changes.8fcc2519(testc8f8692b). The AppImage script now stagesgpu/asmwhole.tests/test_appimage_source_headers.pyruns the script's copy block and checks two things: that every file ofgpu/asmis staged, and that every include of the kernels inhost/asm,gpu/asmandgpu/vk/asmis staged. At the test commit,gpu/asm/lsasm.hwas missing.8fcc2519(testc8f8692b). Ahost/asm.Sfile now builds the game alone.gpu/vk/asmandgpu/asmkernels, and a header next to ahost/asmkernel, still build the renderer.tests/test_dev_packs.pystops at a check that also fails on main6e1521a7(longshot_fsr4_pipelines.binis unclassified), so I ran each of its checks on its own.test_renderer_untouchedfailed atc8f8692band passes at8fcc2519and at the head. Three other checks fail on main alike:test_caches_named_in_renderer,test_inputs_beside_the_objectsandtest_stamp_matches_build. Becausetest_inputs_beside_the_objectsstops on main's failure, its new kernel line has not run.b9536c12(test896a42d6). The CPU probe is now built for the baseline. It reads the whole x86-64-v3 level from CPUID (leaves 1 and 7, and the extended leaf for LZCNT) and the OS's ymm state through xgetbv. Any exit other than 0 or 1 now fails the check, so a crashed probe no longer counts as a CPU without AVX2.b9536c12(test896a42d6). Each object name now includes its source's index, in the native build and in the cross assembly. Two kernels with the same stem become two objects, each linked once.b9536c12(test896a42d6). The cross assembly uses--clangif given, otherwise the clang next to the compiler, otherwise the one on PATH. With no clang at all the check fails, and the PASS line names the clang it used.tests/test_asm_abi.pynow readsCXXwhen--cxxis not given.b9536c12; the import is removed. I did not run Ruff, because it is not installed in this PC's Builder Python.fa359ec4. A Linux probe with scratch and padding calls out from its body, and the fixture unwinds from there with_Unwind_Backtrace. It checks the probe's and the harness's return addresses, and rbx, rbp and r12-r15. A probe also reads a table inLSASM_RODATARIP-relative. That probe is assembled for ELF and COFF and runs on Windows, where it passes. The Linux run has not been done on this PC (see below).__except): test added infa359ec4. Each single step's unwind is now compared with the whole nonvolatile state at entry: rsi, rdi and xmm6-xmm15 too, in Win64. A fault in a probe is also caught through Windows' own dispatch to a caller's__except, built with-fms-extensionsasguarded_copyis. It passes atfa359ec4with the macros as they were, so no fix was needed.4cf7de04.tests/test_asm_build.pynow importsnx2/build.pywithRX_*,LONGSHOT_*andOOT_*cleared andRX_MARCH=x86-64-v3set. While the import runs,subprocess.Popenandos.systemraise, so any process the import starts fails it. I ran it withRX_MARCH=nativeand no clang reachable: at5853d097the import aborted before unittest started; at4cf7de04all 15 tests pass.5853d097.main()now plans through a new function,plan_modules, innx2/build.py. It joins each module's kernels to its jobs, appliesRX_VK_ONLY, and returns the jobs plus the two lists thatobjs.rspandvk_objs.rspare written from. The test runs it on synthetic kernels: two inhost/asm, one ingpu/asm, and one with the same name ingpu/vk/asm. It checks the compiled jobs and both modules' full link inputs for a full build, a game-only build and a renderer-only build. Three tests anchored on lines that no longer exist, and now anchor on the planner's line.At the head:
tests/test_asm_abi.pypasses at both levels and in both conventions, including the clang assemblies and the frame bound.tests/test_asm_build.py(15 tests),tests/test_asm_kit.py(6),tests/test_appimage_source_headers.pyandtests/test_appimage_headers.pypass, andgit diff --checkis clean. The--strict --changedrun covered every check, because a shared test file changed. I reran the checks it failed at main6e1521a7and at this head: the same 126 fail on both. The one difference wastests/test_audio_wave_buffer_release.py, which hit a compile timeout here and passed when run alone.Waiting on the author (no code change, listed for the owner):
6e1521a7, and the update rebuild cost: not measured yet.tests/test_asm_abi.py(including point 13's ELF unwind), the AppImage, the hardware runs, the full strict suite and CI on this head: not run yet.@futondestroyer thank you for the review (#issuecomment-62705). Everything below is in head
4cf7de04, rebased onto main6e1521a7. The point-by-point reply to BeanBot's review (#issuecomment-63062) lists the test runs. Each test failed at its test commit and passed at its fix.7ff1ed9b(test5eb2e659). The body now runs with rsp 16-byte aligned for both an odd and an even count of pushes. The padding is recorded in the SEH codes and the CFI and released in the epilogue's single add. A frame over 4096 bytes stops the assembly.LSASM_ARGSnow takes the declared arity (61011b0b, test52672716). Your guard-boundary case is the test: one-argument probes run on a stack that ends at their 32-byte home area, with the page above it unreadable. Before the fix, that test crashed with an access violation.bd9564da(test95432bfd).cache_putleaves an entry that is already there, and a cross-volume copy is renamed into place. A kernel is now keyed by the headers its depfile lists, so renaming or removing one changes the key.8fcc2519(testc8f8692b) stagesgpu/asmin the AppImage and routes ahost/asmpack to the game alone.5853d097runs the production planner on one private kernel per module plus a shared one, and checks both link lists for full, game-only and renderer-only builds. These are unit-level checks. The end-to-end run you describe (cold build, cached rebuild after an include rename, a pack, AppImage staging) needs a real kernel. The first kernel joins this PR as a later commit, and that run is listed as pending.fa359ec4adds a fault caught by a caller's__except, built with-fms-extensionsasguarded_copyis. It also checks the whole Win64 nonvolatile state (rsi, rdi, xmm6-xmm15) at every single step. Both pass on Windows. The native ELF unwind from a probe's body is in the fixture but has not run here. The Linux GCC/Clang runs are pending, along with the full suite on this head.WIP: build(asm): hand-written x86-64 kernels have a common header, a check kit and the build's jobsto WIP: build(asm): hand-written x86-64 kernels have a common header, a check kit and the build's jobs, and the GPU-visible ranges' sort is the firstPushed
8f5c08c6, one commit on4cf7de04(a fast-forward): the first kernel joins this pull request, as BeanBot's review asked (the ground and its first caller together).lsasm_sort_ranges(host/asm/ranges.S) sorts the GPU-visible ranges inhos_gpu_guest_ranges(host/nvdrv.c) on x86-64; other CPUs keepqsort.tests/test_asm_ranges.pyand the kit's checks pass on this head through the queue;tests/test_host_guest_writes.pyfails here and on main75317e1falike. It has not run in the game on this PC yet; the clean build's time and size against main, the cold and rename rebuilds and the whole suite are queued. Title and description updated.@Beanstalk This adds the build's jobs for hand-written x86-64 assembly (asm_jobs in nx2/build.py, the shared header gpu/asm/lsasm.h) and the first kernel, lsasm_sort_ranges, which replaces the qsort of the GPU-visible guest ranges in host/nvdrv.c (hos_gpu_guest_ranges): it runs at boot, at each area load and whenever the game maps or unmaps GPU memory. It has not run in the game on Linux, AMD or Intel yet. Please build this PR and main, time the build against main's (a warm build) and check the lifted C is the same; then start a new game in Kokiri Forest, play a minute, and go through three area loads by teleport (Hyrule Field, Castle Town Market, Kakariko; if a teleport does not arrive, the three area loads from Kokiri Forest's own exits), on each side. On Linux with NVIDIA, Linux with AMD, Windows with AMD, Windows with Intel, and the Steam Deck. It must not crash or hang at a load; frame time p99, load times and peak VRAM must be within main's run-to-run spread.
@siantm only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for8f5c08c, and the PR has changed since (db3d91f), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.8f5c08c6ecdb3d91f148@Beanstalk This adds the build's jobs for hand-written x86-64 assembly (asm_jobs in nx2/build.py, the shared header gpu/asm/lsasm.h) and the first kernel, lsasm_sort_ranges, which replaces the qsort of the GPU-visible guest ranges in host/nvdrv.c (hos_gpu_guest_ranges): it runs at boot, at each area load and whenever the game maps or unmaps GPU memory. It has not run in the game on Linux, AMD or Intel yet. Please build this PR and main, time the build against main's (a warm build) and check the lifted C is the same; then start a new game in Kokiri Forest, play a minute, and go through three area loads by teleport (Hyrule Field, Castle Town Market, Kakariko; if a teleport does not arrive, the three area loads from Kokiri Forest's own exits), on each side. On Linux with NVIDIA, Linux with AMD, Windows with AMD, Windows with Intel, and the Steam Deck. It must not crash or hang at a load; frame time p99, load times and peak VRAM must be within main's run-to-run spread.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.@futondestroyer on the last open part of your review (#issuecomment-62705, Blocking, second point: one shared kernel and one private kernel per module end to end through a cold build, a cached rebuild after an included-file rename, developer packs and the Linux AppImage staging). The branch is now
db3d91f1, rebased onto main7c23d2be.git range-diffshows every earlier commit as=, except that5653f5daand5853d097differ only in a hunk header ofdocs/dev/reference/build.md. There were no conflicts. Two commits are new:b21224ff,tests/test_asm_end_to_end.py). It writes three kernels in a temporary tree on the realgpu/asm/lsasm.h: a shared one ingpu/asm, one inhost/asmand one ingpu/vk/asm, each with its own include. It runs the build's own code on them with clang:plan_modules,plan_jobswith the hash record,compile_oneon four threads (so the shared kernel's two jobs run at once),rekey_depfiles, andmain()'s own statements that writeobjs.rspandvk_objs.rsp. Each module's link list is then linked with a caller and run.route,link_inputs_changed,renderer_untouched,link_objectsandmodule_changed, ascmd_packruns them. A game kernel builds the game alone, a renderer kernel the renderer alone, and the shared kernel both. Each pack compiles what changed, its modules differ from the base's, and the base's objects and cache must not change.scripts/linux/appimage.sh's source-copy block runs with bash on the tree, and a cold build of the staged tree compiles and links the three kernels.db3d91f1). The test found a defect that a real kernel would hit. clang's assembler writes its object in place: cc1as opens the-opath itself, while clang's C and C++ compiles write a temporary file and rename it.compile_onehard-links each object into the cache, so assembling a kernel again wrote through the link. The cache entry under the old key became the new object, and in a pack's work folder the base's cache entry and object changed too. At the test commit, a reverted include edit linked the edited kernel: the caller returned 1055 where the cold build returned 1022. A shared kernel's pack also rewrote the base's objects, so its game module compared equal to the base's. The renderer pack that followed then compiled the shared kernel again, though that pack had not changed it.compile_onenow removes the object before it starts the compiler, so the compiler writes a new file and the old entries keep their objects.Test runs, all in one queue job (fb3-1947-1; Windows 11, i7-9700, llvm-mingw clang 23.1.5). The job ran on 3bf51a69, these same commits on main
e6196f0f, before main moved; the rebase onto7c23d2bechanged none of them:tests/test_asm_end_to_end.py --cxx clang++failed at the test commit (1fefe953 then) with 4 failures (the edit-and-revert test and three of the pack checks). At the head all 5 tests pass, including the AppImage staging through Git for Windows' bash.tests/test_asm_abi.py,test_asm_build.py(15 tests),test_asm_kit.py,test_asm_ranges.py,test_appimage_source_headers.pyandtest_appimage_headers.py.tests/run_all.py --strict --changed origin/mainran every check, withRX_CPMset so the renderer checks compile against the Vulkan headers: 866 checks: 786 passed, 80 failed, 48 skipped, and none stopped for want of the headers. Rerun by name on maine6196f0fwithRX_CPM, 78 of the 80 fail there too.tests/test_present_semaphores.pyfailed here with no output in 10.9 s (8 checks at once) and passed on main.tests/test_vbytes.pytimed out here at 600 s and was not rerun on main. Neither one reads a file this PR changes. Both are queued to run alone, at the head and on main (fb3-1947-2), and I will post the result.8f5c08c6: those checks pass when run alone, at 3bf51a69 and on maine6196f0f.Still pending (not covered by this test, which runs the build's code on authored kernels): the real build of this head with
lsasm_sort_ranges. That means a clean build against main (its time and size), the head built as an update, and a rebuild after renaminggpu/asm/lsasm.h; this job is queued on this PC. Also pending: a realdev.py packspack, a Linux AppImage build, and the Linux GCC/Clang runs oftests/test_asm_abi.pyandtests/test_asm_ranges.py. The game has not run at this head.The description now names head
db3d91f1on main7c23d2be. Testing covers this round's job: the new end-to-end test, which failed at its test commit and passes with the fix, the PR's own checks, and the--changedrun with the Vulkan headers. That run's 78 failures that fail on main too replace the earlier figures, whose runs had no Vulkan headers. Two checks are being rerun alone. The Summary's build bullet names the fix:compile_onegives a recompiled object a new file. Still to test now lists the real build pair, a real pack and a Linux AppImage build.The two checks from the
--changedrun have been rerun (job fb3-1947-2,RX_CPMset).tests/test_present_semaphores.pypasses alone, atdb3d91f1and on main7c23d2be.tests/test_vbytes.pytimes out at 600 s when run alone, atdb3d91f1and on main7c23d2bealike. The 80 checks that failed in the--changedrun were rerun by name atdb3d91f1: the same 79 fail as on main, andtests/test_present_semaphores.pypasses. No check fails at this head alone. Testing is updated to match.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.