fix(video): a decoder refuses submissions while 16 wait, so a movie's first seconds are not skipped #110
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!110
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:fix/video-input-limit"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
The first seconds of every movie stutter: the frames are on time (60 a second), but the movie shows about ten pictures a second, for about 5 s at the start of the first movie of a new game and about 2 s at the start of the second. A third of the first movie's pictures are skipped there.
What changed
dec_submit()inhost/nn_video.crefuses a submission withRESULT_INPUT_FULLwhile 16 submissions wait in the decoder (input_full(),submit_refused()), as the console's decoder does when it has no room; the stream's last submission is always taken. Until now every submission was taken, so the game's movie player put a whole movie in at once, on the thread that also takes the pictures: while it was busy with that it took a picture only about every 105 ms, and the catch-up (the movie's clock) dropped the 4 to 6 pictures that had come due meanwhile.ready_event(),input_room()), clear while the decoder is full and signalled again when an Acquire, a Flush or a Stop makes room. It was one event for all decoders, signalled for ever. A decoder that ends gives its event back for the next one (ready_event_give_back()), so none is left behind per movie.HOS_VIDEO_LOG=1a third gives those waits by length (the first 5 s and after) and how far the last picture was from the movie's clock.hos_system_event_clear()inhost/nn_os_sync.c, for the event above.tests/test_video_input_limit.py(new) plays a 600-picture movie through the decoder's own code with a stand-in for libavcodec;tests/test_vcore_video_acquire.pygets a stand-in forinput_room().docs/development.md, a row indocs/tested-configurations.md.Flags and settings
HOS_VIDEO_INPUT_LIMIT=<n>0: no limit, as beforeMeasurements
Baseline: this PR @
61d4f4f8,HOS_VIDEO_INPUT_LIMIT=0(three runs; median)main:
4dd972a8(2026-11-12)this PR: fix/video-input-limit @
61d4f4f8(merged with mainb9cdc432asa81848e5)Why main wasn't the comparison: with the limit at 0 the build takes every submission, as main does, so one build gives both sides in the same folder with the same caches. A build of main
4dd972a8itself gave the same counts in two earlier runs (first movie left to play in full: 246 of 5,947 pictures skipped, about 229 of them in its first 4.8 s). The measured build is the commit's code; only comment text and the docs changed after it was built.Hardware: RTX 3090 24 GB, NVIDIA 620.70.06, Ryzen 9 7900X, 30 GB, Linux (CachyOS, kernel 7.3.3)
Scene: a new game from an empty data folder: the first movie (an A press 11 s in ends it after about 820 pictures), then the second movie (9,254 pictures, 154 s) to its end. 2048x1152, default options, frame rate cap 60, no sound device (
HOS_AUDIO=0), a private headless swayMethod: three runs each, interleaved (off, on, on, off, off, on), the whole session each time. Counts from the movie's lines in the log (
HOS_VIDEO_LOG=1); frame times fromRX_VK_FRAME_LOG, from 25 s after the first frame to 8 s before the last (the second movie without its first seconds and the scene's load at its end). The PC was short of memory and swapping in every run (110,000 to 590,000 pages swapped in a run): the picture counts of the clean runs do not depend on it, the frame times do. In the second run of each side it swapped hard enough to make frames late, and the pictures skipped there are the late frames' on both sides.How to reproduce: an empty data folder (
XDG_DATA_HOME),HOS_VIDEO_LOG=1, start the game and press nothing until the first movie has played 11 s, then A once a second until the second movie starts; let it play out; read the twovideo: a movie of ...lines. Again withHOS_VIDEO_INPUT_LIMIT=0.61d4f4f8,HOS_VIDEO_INPUT_LIMIT=0(three runs; median)61d4f4f8(three runs; median)The middle run of each side is the one with late frames (the PC swapping): there the limit cannot help, and does not hurt.
Limits of 4, 8 and 12 at the first movie's start (one run each, 14 s): 8, 9 and 3 pictures skipped, no Acquire left waiting. 16 is kept for the margin over what the AV1 decoder's frame threads can hold back.
Testing
Tested: On the PC above (Linux, RTX 3090, NVIDIA 620.70.06, libdav1d), in the game: both movies of a new game, six full sessions as measured, plus the first movie left to play in full (5,947 pictures) with the limit on: 3 pictures skipped in its first 10 s (229 on main), the movie ends, the second one starts. Checked in those runs: a movie stopped early by a button (a Flush with submissions still waiting: the next movie starts and plays), the end of a stream (the last picture marked, the movie stops, the game goes on into the scene behind it), two movies in a row (the second decoder made while the first is torn down), and a refusal on the thread that also takes the pictures (the game goes on to its Acquire, which makes room: 6,935 refusals in the second movie, one a frame).
python -I -B tests/run_all.py: 505 passed, 2 failed, 24 skipped on the merged tree; the two failures aretests/test_bake_portability.py(nozstandardmodule on this PC) andtests/test_builder_log_read_only.py(no Tk library on this PC), neither touches this change and both fail the same way without it.tests/test_video_input_limit.pyfails on main and passes here.Not tested: The merged tree (
a81848e5) in the game: the game runs were made from61d4f4f8, whosehost/nn_video.cis the same apart from the rename's spelling. Windows, with dav1d or with Media Foundation's AV1 decoder. With a sound device (the runs had none; the movie's clock is the host's, not the sound's). Movies later in the game (cutscenes with a movie, the title's clips when they loop), a movie paused or sought. A frame rate cap above 60. Games other than OoT that use these decoders (VP8, VP9): the limit is on for every codec.Needs other testers:
video: a movie of ...lines say 0 to about 10 pictures skipped andsubmissions put offabove 0 (test request T-58KT)Screenshots
Nothing on screen (a movie's pace; no picture changes).
Risk and rollback
A game whose movie player does not expect a refused submission would stop at its first movie; OoT's takes it and submits again. If a movie hangs or the game stops at a movie's start,
HOS_VIDEO_INPUT_LIMIT=0gives main's behaviour back; reverting the commit is enough too. The refusal itself waits for nothing, so it cannot block a thread; room is made by Acquire, Flush and Stop, which the game calls whether or not it was refused.Reviewer notes
RESULT_INPUT_FULLinhost/nn_video.c, thensubmit_refused(),input_room()andready_event().Issues
See #73
Checklist
For everyone
type(scope): description).main. (mainb9cdc432merged in:a81848e5)python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.docs/tested-configurations.mdfor each measured run.mainand passing here.tests/that reads changed source updated in this PR, not weakened.b9cdc432) into the video decoder's input limitSelf-review of the change, and three test requests opened (T-58KT Windows, T-BW29 AMD, Intel and Steam Deck, T-JJF9 Windows without an AV1 decoder).
Read again line by line; nothing found that blocks it, and these are the points a reviewer should weigh:
HOS_VIDEO_INPUT_LIMIT=0is the way back; limiting it to AV1 would be a one-line change if a reviewer prefers that.submit_refusedtakes the decoder's lock, counts and returns; room is made only by Acquire, Flush and Stop. A game that submits and acquires on one thread therefore cannot block in it. A game that waits on the ready event without ever acquiring would wait for ever, as it would on the console.input_limit()'s are unguarded, like the file's other lazy statics.License box ticked: the change is our own code under GPL-3.0-or-later, with nothing copied in and no game code or generated output in the diff.
WIP: fix(video): a decoder refuses submissions while 16 wait, so a movie's first seconds are not skippedto fix(video): a decoder refuses submissions while 16 wait, so a movie's first seconds are not skippedChecklist tidied: the three boxes that do not apply (screenshots, API version, SPIR-V headers) are deleted instead of marked n/a. The Windows build box stays, unticked: host code changed and Windows has not been built here.
Measured on a Steam Deck LCD (VanGogh: AMD Custom GPU 0405, RADV VANGOGH driver 0x6801063, 4 CPU cores, 15 GB RAM), Linux, Longshot
fabcd2c0(v3.0.0-rc.2-3-gfabcd2c0, source a6589a6e), which includes this PR (ae7b1165is an ancestor), default options withHOS_VIDEO_LOG=1, game audio confirmed open:INPUT_LIMIT=16): the first endcard movie, 5947 pictures: 4702-4766 skipped to keep up with its sound, 917-960 submissions put off while 16 waited. Two further 1800-picture movies: 230/271/199 skipped, 1158-1330 put off.HOS_VIDEO_INPUT_LIMIT=0(this PR's off state): same movie: 1082-1227 skipped, decode 4.8-5.3 ms/picture.c356bef4era) showed ~1134 skipped on the same endcard movie, i.e. the=0state matches the old behaviour; the default limit roughly 4x the skips on this hardware.The 1800-picture movies stay mild on both arms (51-259 vs 65-82 skips); the effect is concentrated on the long endcard movie. dav1d decode alone is fast (2.3-9.3 ms against the 16.7 ms budget, 8 worker threads ~72% of one core in aggregate), so this looks like submit/decode interleaving rather than decoder speed: on 4 cores, refusing submissions while 16 wait appears to idle the decoder between the acquire thread's drains. The PR notes Steam Deck as "untested"; on this Deck the limit is a clear regression, while
HOS_VIDEO_INPUT_LIMIT=0behaves as before the PR. Numbers are per-run ranges over several interleaved runs; happy to run a matched A/B with more repetitions or aRX_VK_FRAME_LOGcapture if useful.T-BW29 (Steam Deck): fails with the limit on. On a Steam Deck the opening movie freezes about 14 s in and ends 7.5 s behind its sound. With
HOS_VIDEO_INPUT_LIMIT=0it stays in time.Setup: Steam Deck OLED, SteamOS 3.9.5 (kernel 7.2.7-valve2), RADV Mesa 26.2.0-devel, libdav1d. Build: Longshot
e7ed2f7a(with #110) plus Deck patches that don't touchhost/nn_video.c. A copy of the user's install, started to the title, so the first movie plays (~3,200-3,700 pictures, 1920x1080, one every 16.7 ms).HOS_VIDEO_LOG=1. Two runs each, interleaved: on, off, on, off.Run 1 timeline with the limit (
acquire N: picture at ..., wall clock ...): on time until acquire 420 (14.30 s / 14.31 s). The next logged one is acquire 600 at 34.73 s against 40.35 s on the wall clock (5.62 s behind), then 42.65 s / 49.56 s and 50.52 s / 58.35 s. Between 14 s and 40 s only ~180 pictures were taken, so the picture sticks. Time between pictures after the first 5 s: under 20 ms 62, 20-50 ms 209, 50-90 ms 35, over 90 ms 156.Run 2 timeline without it: within 0.06 s of the clock at every logged acquire up to 1440 (59.52 s / 59.53 s). Time between pictures after the first 5 s: 319, 775, 173, 123.
The user reported the same: "movies play ~10 s, then stick on one frame", on his installed build. The Hookshot build before #110 played them through.
Not decode speed: dav1d needs 6.7-9.7 ms a picture against a 16.7 ms budget. The limit's refusals (550+ a movie) seem to leave the decoder starved once it falls behind. The ~2,200 pictures still skipped without the limit come from the game's own rate during the movie on the Deck: about 24 acquires a second for a 60-picture-a-second movie. That's the frame rate, not a freeze, so "0 to about 10 skipped" can't happen on a Deck.
Workaround on the Deck:
HOS_VIDEO_INPUT_LIMIT=0inlongshot_env.txt(now set on the user's Deck). Not tested: later movies, Windows. #252 (a slow movie shows its pictures) is being tested next on the Deck with the limit on.