fix(dlss): a render size a pixel past a mode's range is that mode's, and a refused size keeps the feature #2193

Merged

pmwork-personal merged 1 commit from fix/dlss-near-mode into main

2026-11-11 16:41:28 +00:00

pmwork-personal commented

2026-11-11 06:58:23 +00:00

(Migrated from github.com)

I found these while looking into reports in #dev-work of lower DLSS presets running worse than DLAA. Those turned out to be mostly something else: DLSS model M on an RTX 3060, and VRAM filling up over a session. So this PR doesn't claim to fix them; it fixes the two bugs above.

What happened (RTX 5090, 4416x1864 output: upstream's 3840x2160-pixel widescreen picture on a 21:9 screen)

Ultra Performance's range at that output is 1472x621 to 1472x621, and the game's third of 1864 came out 622. The log said NVIDIA DLSS: the game's upscale runs (no DLSS mode takes this render size).
prepare let the feature go before looking for a mode. So after a refused size the feature was made again on the next frame DLSS could take (feature N made (again after it was let go ...), 10–25 ms of CPU each).

What changed

dlss_checks.h mode_for picks the mode as a plain, testable function:
- It takes the first mode whose range holds the render size.
- If none does, it takes the first mode the size misses by a little: at most 1/64 of the range's largest size per axis, and 2 pixels at least. That range is widened to the size, and the widening is logged when the feature is made.
- A size is widened only if DLSS can scale it at all (scene_size_fits: the output 1–3× the size). A size between two modes' ranges is still refused, for example 42% at 5120x2160, between Ultra Performance's third and the other modes' half.
vk_dlss.cpp prepare finds the mode first and lets the old feature go only once a mode takes the size. A refused size keeps the feature, and the log names the size (no DLSS mode takes the render size WxH for OWxOH).

Tested

In game, RTX 5090, driver 621.14, Windows 11, 4416x1864 output, preset M, frame generation off:

Render resolution	before	after
Native (DLAA)	DLSS, ~60–64 fps	same
Performance 2208x932	DLSS, ~66–76 fps	same
Ultra Performance 1472x622	the game's upscale, ~85–95 fps	DLSS (`a little outside quality mode 3's range (1472x621 to 1472x621): taken by it`), ~76–78 fps

On this GPU the game's network is faster than DLSS at a third, so Ultra Performance goes down here; either way the player now gets DLSS, which is what they picked.

tests/test_dlss_checks.py / .cpp cover Performance's half, Ultra Performance's exact and rounded third, a size between the modes, one below a third, a 2560x1440 output's Ultra Performance at the menu's own size, and no widening into a range NGX didn't give. The source check confirms prepare uses mode_for and lets the old feature go after it. Both fail on main.
tests/run_all.py (Windows, llvm-mingw): the DLSS, upscaler and temporal checks pass. The only failure is test_recorder_sync's exit case, which fails on main too.

Not tested

Other output sizes in game (the test covers 2560x1440's rounding).
Linux.

Relation to #2145. That PR (Arkex) makes the switch to Native wait for the native scene and touches the same prepare. This PR leaves the Native transition to it and changes only how the mode is found and when the old feature is let go, so whichever lands second needs a small rebase there.

🤖 Generated with Claude Code

At output sizes that don't divide evenly by 3, Ultra Performance never runs DLSS. The game's third of the output rounds a pixel past DLSS's Ultra Performance size, no mode takes it, and the game's own upscale network runs in DLSS's place; only a log line shows it. And when a size is refused, `prepare` has already let its feature go, so DLSS rebuilds it a frame later. I found these while looking into reports in #dev-work of lower DLSS presets running worse than DLAA. Those turned out to be mostly something else: DLSS model M on an RTX 3060, and VRAM filling up over a session. So this PR doesn't claim to fix them; it fixes the two bugs above. **What happened** (RTX 5090, 4416x1864 output: upstream's 3840x2160-pixel widescreen picture on a 21:9 screen) - Ultra Performance's range at that output is 1472x621 to 1472x621, and the game's third of 1864 came out 622. The log said `NVIDIA DLSS: the game's upscale runs (no DLSS mode takes this render size)`. - `prepare` let the feature go *before* looking for a mode. So after a refused size the feature was made again on the next frame DLSS could take (`feature N made (again after it was let go ...)`, 10–25 ms of CPU each). **What changed** - `dlss_checks.h` `mode_for` picks the mode as a plain, testable function: - It takes the first mode whose range holds the render size. - If none does, it takes the first mode the size misses by a little: at most 1/64 of the range's largest size per axis, and 2 pixels at least. That range is widened to the size, and the widening is logged when the feature is made. - A size is widened only if DLSS can scale it at all (`scene_size_fits`: the output 1–3× the size). A size between two modes' ranges is still refused, for example 42% at 5120x2160, between Ultra Performance's third and the other modes' half. - `vk_dlss.cpp` `prepare` finds the mode first and lets the old feature go only once a mode takes the size. A refused size keeps the feature, and the log names the size (`no DLSS mode takes the render size WxH for OWxOH`). **Tested** - In game, RTX 5090, driver 621.14, Windows 11, 4416x1864 output, preset M, frame generation off: | Render resolution | before | after | |---|---|---| | Native (DLAA) | DLSS, ~60–64 fps | same | | Performance 2208x932 | DLSS, ~66–76 fps | same | | Ultra Performance 1472x622 | **the game's upscale**, ~85–95 fps | **DLSS** (`a little outside quality mode 3's range (1472x621 to 1472x621): taken by it`), ~76–78 fps | On this GPU the game's network is faster than DLSS at a third, so Ultra Performance goes down here; either way the player now gets DLSS, which is what they picked. - `tests/test_dlss_checks.py` / `.cpp` cover Performance's half, Ultra Performance's exact and rounded third, a size between the modes, one below a third, a 2560x1440 output's Ultra Performance at the menu's own size, and no widening into a range NGX didn't give. The source check confirms `prepare` uses `mode_for` and lets the old feature go after it. **Both fail on main.** - `tests/run_all.py` (Windows, llvm-mingw): the DLSS, upscaler and temporal checks pass. The only failure is `test_recorder_sync`'s exit case, which fails on main too. **Not tested** - Other output sizes in game (the test covers 2560x1440's rounding). - Linux. **Relation to #2145.** That PR (Arkex) makes the switch to Native wait for the native scene and touches the same `prepare`. This PR leaves the Native transition to it and changes only how the mode is found and when the old feature is let go, so whichever lands second needs a small rebase there. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 (Migrated from github.com) approved these changes

2026-11-11 16:30:43 +00:00

hckrman101 (Migrated from github.com) left a comment

Looks good to me. Finding the mode first and only then letting the old feature go is the right order, and pulling the choice out into mode_for makes the rounded-third case testable. I went through it against vk_temporal.cpp:

Keeping the feature across a refused size is safe for the history: if DLSS comes back to a size the kept feature takes, prepare returns 1 rather than 2, but temporal_upscale already resets when more than 2 frames passed since the last evaluation (now - last > 2), so no stale history gets used.
The DLAA path is unchanged in effect (the old same[] list was never searched, since found = aa).
The widened range is what gets stored in min_rw/feature_rw, so the early-out at the top of prepare keeps taking 1472x622 on later frames instead of making the feature again each frame. Good.
The 1/64 tolerance only ever widens toward sizes scene_size_fits accepts, so it can't push DLSS below a third. The "no range" and "below a third" cases cover that.

Nits only, none blocking:

vk_dlss.cpp around line 300: the comment says a size "a pixel or two" outside a range is taken, but the tolerance is 1/64 of the range's largest size. That's 23 px wide for Ultra Performance at 4416, and up to 69 px for the 50-100 % range. The header comment in dlss_checks.h gets this right; I'd say the same here or just point to it.
See the inline note on the refused message and fail()'s dedup.

CI is all green, including the DLSS link-check build and both Windows/Linux source checks. No other discussion on the PR yet.

Overlaps: #2145 changes prepare's signature (adds native_aa) and adds an else if to the why chain plus a feature_aa store after creation. None of that touches the lines this PR rewrites, so I expect at most a trivial rebase. The new source check here keys on int prepare(VkCommandBuffer cmd, which still matches after #2145. #2118 changes vk_dlss.cpp only in ngx_init and the frame-generation code below dlss_probe, not prepare, so there's no conflict.

Looks good to me. Finding the mode first and only then letting the old feature go is the right order, and pulling the choice out into `mode_for` makes the rounded-third case testable. I went through it against `vk_temporal.cpp`: - Keeping the feature across a refused size is safe for the history: if DLSS comes back to a size the kept feature takes, `prepare` returns 1 rather than 2, but `temporal_upscale` already resets when more than 2 frames passed since the last evaluation (`now - last > 2`), so no stale history gets used. - The DLAA path is unchanged in effect (the old `same[]` list was never searched, since `found = aa`). - The widened range is what gets stored in `min_rw`/`feature_rw`, so the early-out at the top of `prepare` keeps taking 1472x622 on later frames instead of making the feature again each frame. Good. - The 1/64 tolerance only ever widens toward sizes `scene_size_fits` accepts, so it can't push DLSS below a third. The "no range" and "below a third" cases cover that. Nits only, none blocking: - `vk_dlss.cpp` around line 300: the comment says a size "a pixel or two" outside a range is taken, but the tolerance is 1/64 of the range's largest size. That's 23 px wide for Ultra Performance at 4416, and up to 69 px for the 50-100 % range. The header comment in `dlss_checks.h` gets this right; I'd say the same here or just point to it. - See the inline note on the refused message and `fail()`'s dedup. CI is all green, including the DLSS link-check build and both Windows/Linux source checks. No other discussion on the PR yet. Overlaps: #2145 changes `prepare`'s signature (adds `native_aa`) and adds an `else if` to the `why` chain plus a `feature_aa` store after creation. None of that touches the lines this PR rewrites, so I expect at most a trivial rebase. The new source check here keys on `int prepare(VkCommandBuffer cmd`, which still matches after #2145. #2118 changes `vk_dlss.cpp` only in `ngx_init` and the frame-generation code below `dlss_probe`, not `prepare`, so there's no conflict.

gpu/vk/vk_dlss.cpp

Nit: fail() in vk_temporal.cpp only logs again when the why pointer changes (or every 600th repeat). refused is one static buffer, so if a second, different size is refused right after the first, it isn't logged until the 600th repeat, and by then the buffer holds whatever size came last. It's fine for the case this fixes. If you want each refused size logged, you could log it here when rw/rh/ow/oh change.

Nit: `fail()` in `vk_temporal.cpp` only logs again when the `why` pointer changes (or every 600th repeat). `refused` is one static buffer, so if a second, different size is refused right after the first, it isn't logged until the 600th repeat, and by then the buffer holds whatever size came last. It's fine for the case this fixes. If you want each refused size logged, you could log it here when `rw`/`rh`/`ow`/`oh` change.

tests/test_dlss_checks.py

Nit: there's a long run of spaces before the message string here, probably left over from joining two lines. Wrapping it onto two lines like the other asserts would read better.

pmwork-personal (Migrated from github.com) reviewed

2026-11-11 17:36:36 +00:00

gpu/vk/vk_dlss.cpp

Done in 89b21647, through fail() itself: the refused message now has two buffers, and a new size goes into the other one, so fail() sees a new pointer and logs each refused size once, in its usual format. (Logging it in prepare as well could print a size twice when another reason came between two refusals, since fail()'s last reason isn't reset.) The comment near line 300 now says the tolerance as dlss_checks.h does (at most 1/64 of the range's largest size on each axis, 2 pixels at the least).

Done in 89b21647, through `fail()` itself: the refused message now has two buffers, and a new size goes into the other one, so `fail()` sees a new pointer and logs each refused size once, in its usual format. (Logging it in `prepare` as well could print a size twice when another reason came between two refusals, since `fail()`'s last reason isn't reset.) The comment near line 300 now says the tolerance as `dlss_checks.h` does (at most 1/64 of the range's largest size on each axis, 2 pixels at the least).