test(lift): scripts/lift_parity.py, the translation of two checkouts compared byte for byte #1798
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1798
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:test/lift-parity"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: test · Risk: low · Tested on: Windows 11, RTX 3080, Ryzen 9 3900X · Needs: a run on Linux
TL;DR
A refactor of the translator is only safe when it can be shown not to change the translated game. This adds a tool that translates the game from two checkouts the way a real build does, and compares the two byte for byte, naming any functions that differ without printing their code. It is the Modding API Plan's first parity gate (M1, gate 1), and it is what the hook compiler's pull requests use to show which functions they change. Two translations of main came out identical (91 files, 1.73 GB), so the lift is stable enough to gate on.
Breakdown
nx2/, the hook compiler first (milestone M2): a refactor shows "the same", a converted feature shows exactly the functions it changes.scripts/lift_parity.py(lift,compare) and its game-free test.Why
The Modding API Plan (wiki, Modding-API-Plan, section 4, M1) asks for parity gates before the first refactor of the core: gate 1 is code parity, the translation unchanged with a hook-site table present and empty. The plan names
nx2/test_leaf.pyfor it, which checks a sample of functions against Unicorn; a byte-for-byte comparison of the whole translation is a stronger check for a change that must change nothing, and the only one that says which functions a deliberate change touched.What changed
A script with two commands, and a test of the second.
scripts/lift_parity.py lift [--out DIR] [--exefs DIR] [--checkout DIR]: a checkout's translation, written asnx2/make_game.pywrites it:nx2/lift.pywith make_game's environment (RX_RELEASE=1,A64_HOT_CHUNK=1, the hot listhot_list_forgives), thennx2/split_hot.py; through the job queue as a build, withscripts/dev_build.py's Python and tools (on Windows the Builder's Python and llvm-mingw: what a base or a pack lifts with). It writes its settings into the folder (lift_parity.txt: the Python, and a file setting by its contents' hash).scripts/lift_parity.py compare A B: every file undersa/andimage/byte for byte (keys, timings and logs left out); a differing C file compared function by function (nx2/lifted_c.py's definitions), the functions named; different settings said. Exit 0: the same.tests/test_lift_parity.py(new): two made-up translations: equal ones the same; a changed function named, and only it; files only on one side, a changed image file and different settings said; keys and logs not compared; no function's code printed.Measurements
Verdict: nothing to measure: a tool; what it found is in Testing.
Flags and settings
None.
Testing
Tested:
python -I -B tests/test_lift_parity.pypasses.ad9fa48elifted twice, ~4 minutes each: 91 files, 1.73 GB, the same.ab68fc1a) against a branch adding the hook compiler with OoT's site table present and empty: the same.f_80b1e140), plus the new site table file and the chunk cost estimates.python -I -B tests/run_all.py(the Builder's llvm-mingw first onPATH,LONGSHOT_MINGWset): 693 passed / 41 failed on the branch before its rebase onto maina433db32; the same 41 checks fail ona433db32, rerun there, and none fails on the branch only. After the rebase (which touches none of its files) its own test passes again.Not tested: Linux.
Must be tested:
lifttwice from one checkout andcompare, expecting "the same" (by hand, or the Test Farm on a Linux machine).Risks
What could break: nothing the game runs; a wrong comparison would show in its test.
Turning it off: delete the script.
Compatibility: none.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.🤖 Generated with Claude Code
Possibly related, for context:
Verdict: Request changes. “Parity” can currently be reported for incomplete or non-independent outputs or for lifts with different effective inputs.
Reviewed
c271dd2bagainsta433db32.scripts/lift_parity.py:82–114,198: require both sides to produce the complete expected output set and independently own their outputs before comparing. Missing artifacts on both sides, stale outputs and shared output roots need to fail explicitly.scripts/lift_parity.py:57–77: preserve or validate the effective lift inputs on both sides, including settings and inherited overrides. Report the mismatch before claiming byte parity.scripts/lift_parity.py:129,169: retain the file identity in function reporting. A function moving between files needs a correct per-file report even when its body text is unchanged.Add game-free fixtures for those failure modes and a positive comparison from two independent complete runs. Existing comparison evidence applies to the author’s tested configuration.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.