WIP: feat(dlss): port regular NGX neural rendering controls #619

Draft
TheDevil wants to merge 2 commits from TheDevil/Longshot:feat/ngx-neural-rendering into main
First-time contributor

Type: feat · Risk: high · Tested on: Windows renderer builds and game-free fixtures · Needs: fresh compatible unmodified NGX runtime/game and Linux validation

Why

Port experimental Neural Rendering separately from input and pacing so it can be reviewed and reverted on its own. Replaces the NR portion of #173. Based on HypeZ85's Hookshot PR #1590, commit a1f730de8c8e7b2adfbca4e0b8ac4cfbdf72834c. HypeZ85 is valerio85 on Discord; credit belongs to him for the initial NR work.

What changed

Regular NGX feature creation/evaluation, real scene depth/motion, deferred resource lifecycle and locked retry/failure recovery. An optional NR settings page includes model controls, Compare Save/Cancel transactions, original-chromaticity blending, translations and persistent diagnostics. Non-NGX builds retain ordinary rendering.

This initial port is SDR-only and runs before DLSS/DLAA/FSR. Native HDR composition, stage choice and NR scaling are a dependent follow-up. API remains 22. No input, item-wheel or FPS-cap changes.

The patched-runtime bridge has been removed completely: no alternate runtime loader, RX_VK_DLSS_NR_DIRECT, nr_bridge, or hardcoded registered application ID. Existing main NGX initialization is reused. Supply a compatible unmodified runtime; this project does not ship or download it. Patch details and the old AppID's provenance are unknown, so this PR makes no claims about them.

Measurements

Verdict: Windows Vulkan renderer builds with and without NGX. Game-free fixtures exercise settings, menus, routing and lifecycle. No fresh live GPU/visual/performance claim.

Before: main @ 2b1d38c682
After: 95126d5729
Hardware: Windows x64, i7-14700F, 16 GB; GPU/driver not exercised by CPU checks.
Scene: synthetic settings, menu and Vulkan/NGX call fixtures; no gameplay rendered.
Method: renderer-only SDK-on/off builds and production-code fixture execution.
How to reproduce: python -I -B tests/run_all.py --jobs 5, with official llvm-mingw tools and Vulkan headers available. Renderer-only RX_VK_ONLY=1 python -I -B nx2/build.py --standalone --jobs 4, first with LONGSHOT_NGX_SDK, then without it.

Full suite: 541 passed, 13 failed, 34 skipped (178 s). All failures also fail on main @ 2b1d38c6: tests/test_appimage_player_build.py, tests/test_cpm_discovery.py, tests/test_crash_dialog.py, tests/test_env_on.py, tests/test_lift_cleanup_error.py, tests/test_linux_preflight.py, tests/test_make_game_cache.py, tests/test_oot_state_tap.py, tests/test_readback_late.py, tests/test_tested_configurations.py, tests/test_vk_pgo.py, tests/test_vram_budget_row.py, tests/test_walk_key.py. Main had 530 passed, 14 failed, 34 skipped; test_finish_output passed in this run. No new failing checks.

Check Before: main @ 2b1d38c6 After: this PR @ 95126d57
Game-free suite 530 pass, 14 fail, 34 skip 541 pass, 13 baseline failures, 34 skip
NR menu/lifecycle/adapter fixtures absent pass
Windows renderer SDK on/off no NR feature both link

Flags and settings

dlss5=0 by default. No automatic runtime download. Native Linux is opt-in with RX_VK_DLSS_NR_LINUX=1, which only probes native NGX capability; Windows DLLs cannot be used natively. Existing RX_VK_DLSS=0 disables this path. RX_VK_DLSS_NR_DEFAULT defaults enable only without a saved key, RX_VK_DLSS_NR overrides enable for a run, and RX_VK_DLSS_NR_INTENSITY overrides finite intensity 0.0–2.0; overrides preserve saved values. Model/color settings are documented in docs/dlss5-neural-rendering.md. Debug views are session-only; Compare previews are saved only by Save. The test-only rejection switch is compiled only with LONGSHOT_NR_TESTING.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
dlss5 menu/ini absent 0 request NR opt-in; live GPU/device behavior pending default unless testing
dlss5_profile menu/ini absent Original when off; derived when enabled select tuning profile opt-in; live GPU/device behavior pending default unless testing
dlss5_style menu/ini absent 0 native style opt-in; live GPU/device behavior pending default unless testing
dlss5_intensity menu/ini absent 35 model intensity percent opt-in; live GPU/device behavior pending default unless testing
dlss5_preset menu/ini absent 0 model preset opt-in; live GPU/device behavior pending default unless testing
dlss5_structure menu/ini absent 100 local structure percent opt-in; live GPU/device behavior pending default unless testing
dlss5_tone menu/ini absent 100 local tone percent opt-in; live GPU/device behavior pending default unless testing
dlss5_skin menu/ini absent 100 skin structure percent opt-in; live GPU/device behavior pending default unless testing
dlss5_color menu/ini absent 100 NR color contribution percent opt-in; live GPU/device behavior pending default unless testing
dlss5_mask menu/ini absent 1 auto skin mask opt-in; live GPU/device behavior pending default unless testing
dlss5_ui menu/ini absent 1 native UI correction opt-in; live GPU/device behavior pending default unless testing
dlss5_compare_top menu/ini absent 0 comparison toolbar placement opt-in; live GPU/device behavior pending default unless testing
dlss5_history menu/ini absent 0 (Auto) NR reset policy opt-in; live GPU/device behavior pending default unless testing
dlss5_debug session menu absent 0 (Normal) actual buffers/debug view opt-in; live GPU/device behavior pending default unless testing
RX_VK_DLSS_NR_LINUX environment absent unset (off) native experimental capability gate opt-in; live GPU/device behavior pending default unless testing
RX_VK_DLSS_NR_DEFAULT environment absent unset default request only if no saved key opt-in; live GPU/device behavior pending default unless testing
RX_VK_DLSS_NR environment absent unset request override for run opt-in; live GPU/device behavior pending default unless testing
RX_VK_DLSS_NR_INTENSITY environment absent unset finite strength override 0.0–2.0 opt-in; live GPU/device behavior pending default unless testing
LONGSHOT_NR_TESTING test build flag absent undefined compile test-only rejection switch opt-in; live GPU/device behavior pending default unless testing
RX_VK_NR_TEST_REJECT_INTENSITY test environment absent unset reject requested intensity in test builds only opt-in; live GPU/device behavior pending default unless testing

Testing

Tested: Windows SDK-on/off renderer compilation, production-code model clamping/edit coalescing, Compare/persistence/navigation, translation lookup, adapter inputs/fallback, retry locking and deferred destruction. Source/SPIR-V correspondence and recorder boundaries checked. Baseline failures are named in the results.

Not tested: fresh running game/host, compatible unmodified NR runtime, GPU timing/quality/VRAM, native Linux build/runtime. Previous combined patched-runtime results are not validation of this regular NGX port. User installation untouched.

Must be tested:

  • Windows, supported GPU/runtime: build host/renderer and compare Off/On, scene transitions, menu changes, Save/Cancel, upscaler switches and failures, expecting correct colors, temporal stability and recoverable fallback (by hand).
  • Linux: build and native capability-gate checks, expecting successful compilation and graceful opt-in/unavailable behavior (by hand).

Risks

Experimental runtime contract and model/resource lifecycle can fail or increase GPU time/VRAM. Enable is optional; disable to use ordinary rendering. Runtime/GPU/driver support must be advertised by NGX, never inferred from SR support.

Compatibility: API/save/cache formats unchanged. New optional ini keys preserve preferences during unavailable states. No vendor binary, SDK, game asset or lifted code is distributed. Only source shader companions are committed. Native HDR falls back unchanged in this slice.

Initial full-suite measurements use main @ 2b1d38c6. The current head incorporates main @ 6c9edffe by non-rewriting merge; focused post-merge results are recorded below.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by trailer.
  • Upstream NR author credited by linked PR and commit.
  • Optional settings, defaults and runtime requirements documented.
  • Regression fixtures and source/SPIR-V checks included.
  • Fresh unmodified-runtime game measurements and docs/tested fragment, pending WIP.
  • Linux host/renderer build and live validation, pending WIP.

Upstream refresh

Current head: d6a6ba99152333a189def02c126105c74a01dd7c. Main snapshot 6c9edffe is incorporated by merge, preserving published history. 18 focused checks passed after the merge; zero failures/skips. Earlier full-suite measurements remain tied to their named feature commits above. The literal rebase checklist remains unticked because history was merged, not rewritten. Fresh game/Linux checks remain WIP.

**Type:** feat · **Risk:** high · **Tested on:** Windows renderer builds and game-free fixtures · **Needs:** fresh compatible unmodified NGX runtime/game and Linux validation ## Why Port experimental Neural Rendering separately from input and pacing so it can be reviewed and reverted on its own. Replaces the NR portion of #173. Based on HypeZ85's Hookshot [PR #1590](https://github.com/NotRexed/Hookshot/pull/1590), commit a1f730de8c8e7b2adfbca4e0b8ac4cfbdf72834c. HypeZ85 is **valerio85 on Discord**; credit belongs to him for the initial NR work. ## What changed Regular NGX feature creation/evaluation, real scene depth/motion, deferred resource lifecycle and locked retry/failure recovery. An optional NR settings page includes model controls, Compare Save/Cancel transactions, original-chromaticity blending, translations and persistent diagnostics. Non-NGX builds retain ordinary rendering. This initial port is SDR-only and runs before DLSS/DLAA/FSR. Native HDR composition, stage choice and NR scaling are a dependent follow-up. API remains 22. No input, item-wheel or FPS-cap changes. The patched-runtime bridge has been removed completely: no alternate runtime loader, `RX_VK_DLSS_NR_DIRECT`, `nr_bridge`, or hardcoded registered application ID. Existing main NGX initialization is reused. Supply a compatible unmodified runtime; this project does not ship or download it. Patch details and the old AppID's provenance are unknown, so this PR makes no claims about them. ## Measurements **Verdict:** Windows Vulkan renderer builds with and without NGX. Game-free fixtures exercise settings, menus, routing and lifecycle. No fresh live GPU/visual/performance claim. Before: main @ 2b1d38c682baaa4d52f2cacc75fffb9505c1f34f After: 95126d572975dc2c55839bb9533780af64fc7f35 Hardware: Windows x64, i7-14700F, 16 GB; GPU/driver not exercised by CPU checks. Scene: synthetic settings, menu and Vulkan/NGX call fixtures; no gameplay rendered. Method: renderer-only SDK-on/off builds and production-code fixture execution. How to reproduce: `python -I -B tests/run_all.py --jobs 5`, with official llvm-mingw tools and Vulkan headers available. Renderer-only `RX_VK_ONLY=1 python -I -B nx2/build.py --standalone --jobs 4`, first with LONGSHOT_NGX_SDK, then without it. Full suite: **541 passed, 13 failed, 34 skipped** (178 s). All failures also fail on main @ 2b1d38c6: `tests/test_appimage_player_build.py`, `tests/test_cpm_discovery.py`, `tests/test_crash_dialog.py`, `tests/test_env_on.py`, `tests/test_lift_cleanup_error.py`, `tests/test_linux_preflight.py`, `tests/test_make_game_cache.py`, `tests/test_oot_state_tap.py`, `tests/test_readback_late.py`, `tests/test_tested_configurations.py`, `tests/test_vk_pgo.py`, `tests/test_vram_budget_row.py`, `tests/test_walk_key.py`. Main had 530 passed, 14 failed, 34 skipped; test_finish_output passed in this run. No new failing checks. | Check | Before: main @ 2b1d38c6 | After: this PR @ 95126d57 | |---|---|---| | Game-free suite | 530 pass, 14 fail, 34 skip | 541 pass, 13 baseline failures, 34 skip | | NR menu/lifecycle/adapter fixtures | absent | pass | | Windows renderer SDK on/off | no NR feature | both link | ### Flags and settings `dlss5=0` by default. No automatic runtime download. Native Linux is opt-in with `RX_VK_DLSS_NR_LINUX=1`, which only probes native NGX capability; Windows DLLs cannot be used natively. Existing `RX_VK_DLSS=0` disables this path. `RX_VK_DLSS_NR_DEFAULT` defaults enable only without a saved key, `RX_VK_DLSS_NR` overrides enable for a run, and `RX_VK_DLSS_NR_INTENSITY` overrides finite intensity 0.0–2.0; overrides preserve saved values. Model/color settings are documented in docs/dlss5-neural-rendering.md. Debug views are session-only; Compare previews are saved only by Save. The test-only rejection switch is compiled only with LONGSHOT_NR_TESTING. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | dlss5 | menu/ini | absent | 0 | request NR | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_profile | menu/ini | absent | Original when off; derived when enabled | select tuning profile | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_style | menu/ini | absent | 0 | native style | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_intensity | menu/ini | absent | 35 | model intensity percent | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_preset | menu/ini | absent | 0 | model preset | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_structure | menu/ini | absent | 100 | local structure percent | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_tone | menu/ini | absent | 100 | local tone percent | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_skin | menu/ini | absent | 100 | skin structure percent | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_color | menu/ini | absent | 100 | NR color contribution percent | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_mask | menu/ini | absent | 1 | auto skin mask | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_ui | menu/ini | absent | 1 | native UI correction | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_compare_top | menu/ini | absent | 0 | comparison toolbar placement | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_history | menu/ini | absent | 0 (Auto) | NR reset policy | opt-in; live GPU/device behavior pending | default unless testing | | dlss5_debug | session menu | absent | 0 (Normal) | actual buffers/debug view | opt-in; live GPU/device behavior pending | default unless testing | | RX_VK_DLSS_NR_LINUX | environment | absent | unset (off) | native experimental capability gate | opt-in; live GPU/device behavior pending | default unless testing | | RX_VK_DLSS_NR_DEFAULT | environment | absent | unset | default request only if no saved key | opt-in; live GPU/device behavior pending | default unless testing | | RX_VK_DLSS_NR | environment | absent | unset | request override for run | opt-in; live GPU/device behavior pending | default unless testing | | RX_VK_DLSS_NR_INTENSITY | environment | absent | unset | finite strength override 0.0–2.0 | opt-in; live GPU/device behavior pending | default unless testing | | LONGSHOT_NR_TESTING | test build flag | absent | undefined | compile test-only rejection switch | opt-in; live GPU/device behavior pending | default unless testing | | RX_VK_NR_TEST_REJECT_INTENSITY | test environment | absent | unset | reject requested intensity in test builds only | opt-in; live GPU/device behavior pending | default unless testing | ## Testing **Tested:** Windows SDK-on/off renderer compilation, production-code model clamping/edit coalescing, Compare/persistence/navigation, translation lookup, adapter inputs/fallback, retry locking and deferred destruction. Source/SPIR-V correspondence and recorder boundaries checked. Baseline failures are named in the results. **Not tested:** fresh running game/host, compatible unmodified NR runtime, GPU timing/quality/VRAM, native Linux build/runtime. Previous combined patched-runtime results are not validation of this regular NGX port. User installation untouched. **Must be tested:** - Windows, supported GPU/runtime: build host/renderer and compare Off/On, scene transitions, menu changes, Save/Cancel, upscaler switches and failures, expecting correct colors, temporal stability and recoverable fallback (by hand). - Linux: build and native capability-gate checks, expecting successful compilation and graceful opt-in/unavailable behavior (by hand). ## Risks Experimental runtime contract and model/resource lifecycle can fail or increase GPU time/VRAM. Enable is optional; disable to use ordinary rendering. Runtime/GPU/driver support must be advertised by NGX, never inferred from SR support. **Compatibility:** API/save/cache formats unchanged. New optional ini keys preserve preferences during unavailable states. No vendor binary, SDK, game asset or lifted code is distributed. Only source shader companions are committed. Native HDR falls back unchanged in this slice. Initial full-suite measurements use main @ 2b1d38c6. The current head incorporates main @ 6c9edffe by non-rewriting merge; focused post-merge results are recorded below. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [ ] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with Assisted-by trailer. - [x] Upstream NR author credited by linked PR and commit. - [x] Optional settings, defaults and runtime requirements documented. - [x] Regression fixtures and source/SPIR-V checks included. - [ ] Fresh unmodified-runtime game measurements and docs/tested fragment, pending WIP. - [ ] Linux host/renderer build and live validation, pending WIP. ## Upstream refresh Current head: `d6a6ba99152333a189def02c126105c74a01dd7c`. Main snapshot `6c9edffe` is incorporated by merge, preserving published history. 18 focused checks passed after the merge; zero failures/skips. Earlier full-suite measurements remain tied to their named feature commits above. The literal rebase checklist remains unticked because history was merged, not rewritten. Fresh game/Linux checks remain WIP.
feat(dlss): port regular NGX neural rendering controls
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
95126d5729
Based on HypeZ85 (Discord valerio85), Hookshot PR #1590 at a1f730de8c8e7b2adfbca4e0b8ac4cfbdf72834c. Uses the regular NGX path; excludes the patched runtime bridge.

Assisted-by: Codex
Merge current main into regular NGX port
Some checks are pending
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 6s
Website / Website on Cloudflare (pull_request_target) Has been skipped
d6a6ba9915
Assisted-by: Codex
Author
First-time contributor

Updated the branch to include main @ 6c9edffe by non-rewriting merge. Current head d6a6ba9915; 18 focused post-merge checks passed, 0 failed/skipped. Description preserves commit-specific full-suite results and remaining live/Linux validation. Still WIP.

Updated the branch to include main @ 6c9edffe by non-rewriting merge. Current head d6a6ba99152333a189def02c126105c74a01dd7c; 18 focused post-merge checks passed, 0 failed/skipped. Description preserves commit-specific full-suite results and remaining live/Linux validation. Still WIP.
Owner

This pull request now conflicts with main: gpu/vk/menu_i18n.h, gpu/vk/vk_menu.cpp and three tests (test_framegen_preset.py, test_menu_keys.py, test_upscaler_round_trip.py), touched on main by #500, #550, #729, #735, #736. Could you rebase it onto the current main? Thank you.

This pull request now conflicts with `main`: gpu/vk/menu_i18n.h, gpu/vk/vk_menu.cpp and three tests (test_framegen_preset.py, test_menu_keys.py, test_upscaler_round_trip.py), touched on `main` by https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/500, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/550, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/729, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/735, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/736. Could you rebase it onto the current `main`? Thank you.
Some checks are pending
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 6s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request has changes conflicting with the target branch.
  • gpu/vk/menu_i18n.h
  • gpu/vk/vk_menu.cpp
  • tests/test_framegen_preset.py
  • tests/test_menu_keys.py
  • tests/test_upscaler_round_trip.py
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'feat/ngx-neural-rendering:TheDevil-feat/ngx-neural-rendering'
git switch 'TheDevil-feat/ngx-neural-rendering'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!619
No description provided.