Idea: switch for the plain movie upscale on small GPUs (threshold on frame period, not 40 ms) #336

Closed

opened 2026-11-07 21:29:34 +00:00 by GuiFlam · 1 comment

GuiFlam commented

2026-11-07 21:29:34 +00:00

(Migrated from github.com)

The idea

A way to run the movie upscaler as the plain linear upscale (the path that is already used on AMD) on cards where the real networks are too slow, and a trigger for it that also catches laptop GPUs that are slow but not "pathologically" slow.

What I see

RTX 3050 Ti Laptop (4 GB, about 35 W), Fedora 43 (Linux build), release foliage-linux-updater (code from less-stutter + movie-sync's nn_video.c). Title screen and intro movie, same log in several sessions:

vk: native matrix multiplies on (cooperative matrices, the console's fragment layout checked)
vk: 840 presents, 31.8 fps; a frame: 13 draws (0 skipped), 14 dispatches, 3 passes
vk: GPU time 30.3 ms a frame (the longest submission 29.7 ms)

Movie: 14 network dispatches a frame, about 30 ms of GPU time, so about 32 fps. The picture stutters; the sound is fine since movie-sync.
Same card on the title menu: 3 to 5 ms a frame, 60 fps, so the cost is the networks and nothing else.
Not pipelines: I ran a full lap of Kokiri Forest first (pipeline record grew from 26 to 33 MB, 1252 pipelines) and the movie stuttered exactly as before. Not the decoder either (same dav1d, and the GPU time is the whole frame).

Why the planned auto-switch will probably not catch this

In #109 the maintainer says the next release will notice "a movie frame over 40 ms of GPU time" and switch the movie to a plain linear upscale. A 3050 Ti Laptop sits at 30 to 35 ms: not over 40, but already about twice the 16.7 ms a 60 fps movie needs, so it would keep stuttering. Suggestion: base the threshold on the display period (for example the movie's GPU time over about 18 ms for several frames), not on a fixed 40 ms.

Also wanted: a manual switch

On NVIDIA the networks are always kept (vk_shader.cpp: they are left out only if mma_software or the vendor is not NVIDIA). RX_VK_NETWORK_KEEP can keep networks on AMD but cannot leave them out on NVIDIA, and RX_VK_NO_F16 throws for every cooperative-matrix program, the gameplay TAAU included. Something like RX_VK_NETWORK_KEEP=none (and a row in the Insert menu, e.g. "Movie upscaler: Auto / Networks / Plain") would let players on small GPUs choose smooth over sharp. HOS_NO_MOVIES=1 is not a way out, since the title screen waits on its movie (#52).

Notes

Everything above is on the Linux build; I have no Windows numbers.
The gameplay TAAU network should stay as it is (the in-world frame is another problem: 6 to 9 fps at 1300 to 2100 draws on this card).
Happy to test a build with the plain upscale on this card and report the movie's GPU time and fps.

### The idea A way to run the movie upscaler as the plain linear upscale (the path that is already used on AMD) on cards where the real networks are too slow, and a trigger for it that also catches **laptop GPUs that are slow but not "pathologically" slow**. ### What I see RTX 3050 Ti Laptop (4 GB, about 35 W), Fedora 43 (Linux build), release `foliage-linux-updater` (code from `less-stutter` + `movie-sync`'s `nn_video.c`). Title screen and intro movie, same log in several sessions: ``` vk: native matrix multiplies on (cooperative matrices, the console's fragment layout checked) vk: 840 presents, 31.8 fps; a frame: 13 draws (0 skipped), 14 dispatches, 3 passes vk: GPU time 30.3 ms a frame (the longest submission 29.7 ms) ``` - Movie: 14 network dispatches a frame, **about 30 ms of GPU time, so about 32 fps**. The picture stutters; the sound is fine since `movie-sync`. - Same card on the title menu: 3 to 5 ms a frame, 60 fps, so the cost is the networks and nothing else. - Not pipelines: I ran a full lap of Kokiri Forest first (pipeline record grew from 26 to 33 MB, 1252 pipelines) and the movie stuttered exactly as before. Not the decoder either (same dav1d, and the GPU time is the whole frame). ### Why the planned auto-switch will probably not catch this In #109 the maintainer says the next release will notice "a movie frame over 40 ms of GPU time" and switch the movie to a plain linear upscale. A 3050 Ti Laptop sits at 30 to 35 ms: not over 40, but already about twice the 16.7 ms a 60 fps movie needs, so it would keep stuttering. Suggestion: base the threshold on the display period (for example the movie's GPU time over about 18 ms for several frames), not on a fixed 40 ms. ### Also wanted: a manual switch On NVIDIA the networks are always kept (`vk_shader.cpp`: they are left out only if `mma_software` or the vendor is not NVIDIA). `RX_VK_NETWORK_KEEP` can keep networks on AMD but cannot leave them out on NVIDIA, and `RX_VK_NO_F16` throws for every cooperative-matrix program, the gameplay TAAU included. Something like `RX_VK_NETWORK_KEEP=none` (and a row in the Insert menu, e.g. "Movie upscaler: Auto / Networks / Plain") would let players on small GPUs choose smooth over sharp. `HOS_NO_MOVIES=1` is not a way out, since the title screen waits on its movie (#52). ### Notes - Everything above is on the Linux build; I have no Windows numbers. - The gameplay TAAU network should stay as it is (the in-world frame is another problem: 6 to 9 fps at 1300 to 2100 draws on this card). - Happy to test a build with the plain upscale on this card and report the movie's GPU time and fps.

NotRexed commented

2026-11-09 05:46:55 +00:00

(Migrated from github.com)

Thanks for the idea. Feature requests are now collected in #1007, which lists this one with a link back here, so closing it in favour of that list (this is not a decision against the idea).