test: the two vector rewrite checks run 10,000 random states by default #2139
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2139
Loading…
Reference in a new issue
No description provided.
Delete branch "Veix/Longshot:test/vector-tests-random-10000"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The two vector rewrite checks from #1576 run 10,000 random states by default instead of 1,000,000 and 2,000,000: with the old counts the suite's run took about 28 minutes on my PC because of them.
tests/test_vbytes.pyran 1,000,000 random states for each of 130 guest words in four builds, andtests/test_vfpx_elem.py2,000,000 for each of 6 guest words in three builds, every timetests/run_all.pyran. I chose those counts in #1576 and they are far more than the checks need. The default of--randomis now 10,000 in both; a longer run is still one flag away (--random 1000000) for someone changing the helpers innx2/fastvec.py. The fixed sweep oftest_vfpx_elem.pyand both timing parts are unchanged.Read first: the
--randomdefault in the two files; nothing else changes.See #1576
Testing
Tested: Windows 11, Ryzen 9 9900X, llvm-mingw clang. Both checks pass at the new default:
test_vbytes.pyin 28 s (1,300,000 checks a build, 0 mismatches),test_vfpx_elem.pyin 5 s.That 10,000 states still catch a fault: nine deliberate faults put into the helpers in
nx2/fastvec.py, one at a time, each followed by the check at its new default. Eight are caught: TBL reading the table for an index of 16 to 31 (53,898 mismatches of 1,300,000), TBL's 8-byte form leaving the top half (49,164), the fallback's rule at index 16 exactly (10,659), EXT's sources swapped, EXT's 8-byte shift a byte short (189,923), the multiply's smallest-normal constant off by one bit (1 in the random part, 88 in the sweep), a tiny product under FZ keeping its value (873 and 326), a non-finite product of a zero kept (273 and 1,201). The ninth (a product exactly the smallest normal no longer sent to the scalar helper) passes at 10,000 and passes the same way at the old 2,000,000, so the lower count loses nothing there.The checks:
tests/run_all.py --changed(the checks that read the changed files) passes, 3 of 3. The whole suite does not run to its end on my PC (its runner is interrupted partway, onmainas well): of the 364 checks it reached, 323 pass and 41 fail, and those 41 fail the same way run by name onmain@c7bd8f80; none of them reads these two files.Still to test: None
Measurements
Nothing to measure: two checks' default counts, no code the game runs.
Risks
A fault in the helpers that shows only on an input rarer than about one in 10,000 random states is no longer found by the default run;
--randomwith a larger count finds it as before. Nothing changes for players. A revert is enough.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Written with AI assistance (Claude Code); run and checked on my own PC.
🤖 Generated with Claude Code
Possibly related, for context: