WIP: test(lift): the instruction check without the game, over encodings drawn from the instruction set #1570
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1570
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:test/insn-free"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: test · Risk: low · Tested on: Windows 11, Ryzen 9 3900X · Needs: a run on Linux (Capstone's form names there)
TL;DR
The check that runs every instruction form in an emulator and as translated C, and compares them, only works with the game's own files, so the automatic checks always skip it. This adds a version that draws its instructions from the instruction set instead, with nothing from the game: 1,629 forms and 16,998 cases in about 50 seconds. Forms that already fail on main are listed, so only a new failure fails it. On the way it showed that the emulator, not our translation, gets the signed atomic min and max wrong.
Breakdown
tests/run_all.py: the instruction check now runs there.nx2/test_insn.pysamples the game's code, sorun_all.pyskips it on every system; translator changes were checked by hand on a PC with the game, or not at all.tests/test_insn_free.pydraws seeded random encodings in the load/store, data-processing and SIMD/FP classes and runs a64-kit'sinsn_testunchanged over them, withnx2/test_insn.py's fixes.nx2/test_insn.py.Why
The translator is changed often (#1469, #1507, #1202, #1185 and others are open), and its only instruction-level check needs a dump, so
tests/run_all.pynever runs it. The wiki's CPU rework plan makes a game-free version its first tool, so that every later change to what the lifter emits is checked on every PC and on the forge.What changed
tests/test_insn_free.py(new): draws up to--per(2) encodings of each form, grouped by mnemonic and vector arrangement asinsn_test.samplegroups the game's, from--tries(400,000) seeded random words kept whena64_lift.classifytakes them as plain ops, Capstone decodes them and they are not one ofnx2/insn.py's identity forms (pointer authentication, MTE); theninsn_test.mainover them with--states(6) register states each, on Unicorn's ARMv8 "max" CPU withnx2/test_insn.py's memory-operand fix. A failing form not in the known list fails the check; a listed form that passes is named.--write-knownwrites the list.tests/insn_free_known.txt(new): the 263 forms that fail on main. Most are forms the translator leaves unhandled (halted 7), which the game does not use. The signed LSE min/max forms (ldsmax*,ldsmin*,stsmax*,stsmin*) are Unicorn's: version 2.1.4 compares them unsigned (memory -1, register 1: itsLDSMAXwrites -1, where the answer is 1);nx2/insn.py's C is right.Measurements
Verdict: nothing to measure in the game (a check only); it runs in about 50 s on a Ryzen 9 3900X.
Flags and settings
None.
Testing
Tested: on Windows 11 with the Builder's Python 3.14, Unicorn 2.1.4 and Capstone 5.0.7: passes on
310e7c9d. LDEOR's lowering innx2/insn.pychanged by hand to an OR:FAIL (12 new failing forms); put back: passes. Unicorn's signed min/max checked on its own (LDSMAXwith memory -1 and register 1 writes -1).Not tested: Linux;
tests/run_all.pyin full.Must be tested:
python -I -B tests/test_insn_free.py, expectingok(Capstone's names for a form may differ: then the known list needs those names too).Risks
What could break: nothing in the game. The check could fail on a PC whose Capstone or Unicorn names or runs a form differently.
Turning it off: revert the commit.
Compatibility: nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main. On310e7c9d.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). This check passes; the full suite not run yet.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.🤖 Generated with Claude Code
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.