fix(rules): a picture is judged against main's own noise, not the limit alone #87
No reviewers
Labels
No labels
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
priority
critical
priority
high
priority
low
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Beanstalk!87
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Beanstalk:fix/screenshot-noise"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
rp_c64831cd35a6deae (Longshot #1458) failed only on
screenshot.max_diff: 0.0506 against a 0.05 limit, in a Hyrule Field shot taken after a 40 s measure. It wasn't the PR:The farm had no measure of how much main differs from itself, so a fixed limit failed PRs by chance.
What changed
ScenarioRunner.PairScreenshots): diffs main's own pictures of each shot against each other. That's two or more runs a side, which the ABBA order gives. The worst pair is the picture'sNoise.Rules.Evaluate,screenshot.max_diff): a picture fails only when its diff is over the limit and overNoiseMargin(1.5) × that noise.baselineis now the noise.notesays so.noise. A failure's kept pictures (#85) follow the same judgement.screenshot.max_diff) and report-schema.md (noise, the rule's baseline).Tests