WIP: fix(vk): leave out the depth draws of a mesh the game no longer shows: no ghosting with the camera inside Link (#1370) #1447
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1447
Loading…
Reference in a new issue
No description provided.
Delete branch "pmwork-personal/Longshot:fix/hidden-mesh-depth"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11, RTX 5090 · Needs: Linux (build and a quick run), AMD and Intel (any GPU with a temporal upscaler)
Why
When the camera is pushed into Link (the top of a vine wall, a crawl tunnel), the game stops drawing his colour but goes on
drawing his body into the scene's depth. DLSS, DLAA and frame generation take that depth and the actor motion drawn over it,
so they follow an invisible body: the picture smears, with dark blotches and copies that move with the camera. The reporter
of #1370 found the cause and posted a patch; this is that patch, made into a PR with their credit.
How to reproduce
Seen on: Windows 11 (build 26300), RTX 5090, driver 621.24
Settings and switches: Upscaler DLAA (5120x2160), DLSS frame generation on; no switches
Steps: Inside the Deku Tree, climb the vine wall on 1F to its top and turn the camera so that it goes into Link.
Expected: the room behind Link, sharp.
Actual: on main, dark blotches, flat watercolour patches and a veil over the picture that move with the camera.
What changed
A skinned mesh (a vertex buffer) that had colour, and then for 3 frames in a row only depth draws into the scene and no
colour anywhere, is hidden: its depth draws into the scene are left out until its colour comes back. Only the game's own
draws are left out; nothing is added.
gpu/vk/hidden_mesh.h(new, no Vulkan calls): the rule,hidden_mesh::Tracker, and the reasons for each part of it.gpu/vk/vk_draw.cpp:hidden_depth_draw()gives each draw to the rule right afterskin_check(), before actor motiontakes it and before its descriptors are built;
skin_check()now fills aSkinnedDrawanddraw()gives it to actormotion only if it was not left out. Never left out: a draw the game asks about with an occlusion query, a draw into
another depth target (shadows, other views), a mesh that never had colour.
gpu/vk/vk_temporal.cpp:temporal_hidden_mesh_mode()(the switches, read once, off with the parallel replay) andtemporal_scene_depth_is().so with 1 a mesh that blinks would be lost for good, and a colour lost for one frame would be lost for two); the decision
comes before actor motion; one table for every drawing thread (the game submits from several); a stream-0 check on the key;
logging.
Measurements
Verdict: the smear is gone at the spot in both runs with this PR, and present in both runs of main (interleaved). No
numbers: this is a picture fix. The rule adds one table lookup to each skinned draw into the scene (a few dozen a frame) and
leaves out up to 30 draws a frame while the camera is in Link.
Hardware: Windows 11 Pro (build 26300), RTX 5090, driver 621.24, Ryzen 7 9800X3D
Scene: Inside the Deku Tree, the top of the 1F vine wall; DLAA at 5120x2160, DLSS frame generation on (~55 fps base, ~165 shown)
Method: 4 runs interleaved (main, PR, main, PR), the same save and settings, the camera turned into Link; NVIDIA clips
reviewed frame by frame; the PR's runs with
RX_VK_DLSS_FADED_LOG=1.How to reproduce: a save in the Deku Tree (child Link); Upscaler DLAA or DLSS, any frame generation; climb the 1F vine
wall to its top and turn the camera into Link;
RX_VK_DLSS_FADED_LOG=1inlongshot_env.txtfor thevk: upscaler: faded:lines,RX_VK_DLSS_FADED=0for main's picture with this build.544229a2544229a2; rebased since)The same spot (the top of the 1F vine wall, the camera in Link), DLAA and frame generation on; frames from NVIDIA clips at 1920x810:
544229a2Flags and settings
Two new switches, the rule shipping on;
RX_VK_PARALLELis read by it too.RX_VK_DLSS_FADED=0draws hidden meshes' depth againRX_VK_DLSS_FADED_LOG=1logs the first frames' counts and why a hidden mesh's draws were drawnRX_VK_PARALLELTesting
Tested: Windows 11 (build 26300), RTX 5090, driver 621.24: the A/B above (DLAA, frame generation on). Game-free:
tests/test_hidden_mesh_depth.pyfails on main and passes here (the rule over frames modelled on the game's: the camera inLink, blinks, short losses, the query, other targets, a full table, a skipped frame; and where
draw()asks it); it failsagain with the wait set to 1 frame, the query check turned around, or the scene check taken out.
tests/run_all.py(Windows, llvm-mingw): 628 passed,48 failed, 47 skipped; the same 48 fail on a clean main @
50170f81(listed below).Not tested: DLSS Quality and the other scales (the reporter's own run was DLSS 2560x1440 to 3840x2160 with their
patch); AMD FSR 4, FSR 3.1 and Intel XeSS; the first-person look (C-up); Linux; AMD and Intel GPUs. The parallel replay
(
RX_VK_PARALLEL=2and more) turns the rule off.Must be tested:
@Beanstalk linux, the Deku Tree vine wall top with the camera in Link).@Beanstalk intel).Risks
What could break: a mesh whose colour stops for 3 frames while it is still meant to be seen: its depth draws are left
out until its colour comes back, and it comes back one frame late (the logs count those). The game's own picture changes
too where it reads that depth (later depth-tested draws, effects made from the depth). Draws the game asks about with an
occlusion query are never left out, so its culling is unchanged.
Turning it off:
RX_VK_DLSS_FADED=0.Compatibility: Nothing changes.
Issues
Fixes #1370
The 48 checks that fail here and on main @
50170f81(Windows, llvm-mingw)test_auto_handheld, test_bug_report, test_bug_report_defaults, test_cpu_report, test_crash_dialog, test_crash_groups, test_crt_options, test_damage_debug_menu, test_display_startup_failure, test_dlss_feature_size, test_failed_compile_keeps_objects, test_fastfp_fma, test_fpx_sticky, test_framegen_preset, test_fsr_uniform_ownership, test_guest_imported, test_hardware_report, test_hud_layout_row, test_import_reach, test_invalidate_vcore, test_job_queue, test_linux_preflight, test_linux_pressure, test_mirror_budget, test_missing_nup, test_pads_after_menu, test_parallel_init_order, test_pending_guest_import, test_perf_report, test_pipeline_play_cache, test_portable_builder_startup, test_present_thread, test_profiling_mode, test_readback_timeout_source, test_recorder_sync, test_replay_write_order, test_rumble_menu, test_run_all_changed, test_slow_frames, test_spawn_emit, test_staging_burst, test_target_image_pool, test_tested_configurations, test_upscaler_round_trip, test_upscaler_rows, test_vcore_escape, test_vk_occlusion_queries, test_walk_key
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: Measurements adds a per-draw table lookup with no numbers, so that cost is unmeasured.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened. (The newtests/test_hidden_mesh_depth.pyreads them; no existing check needed a change.)🤖 Generated with Claude Code
Tested in game: RTX 5090, driver 621.24, Windows 11 (build 26300): Inside the Deku Tree, the top of the 1F vine wall with the camera pushed into Link; DLAA at 5120x2160, DLSS frame generation on. Four runs interleaved with main
544229a2(this change at 6b3232b8, before the rebase): main smeared both times (dark blotches, flat watercolour patches, a veil moving with the camera), this change neither time. Its log: at most 30 depth draws a frame left out, only while the camera is in Link, none with a stencil test; one draw kept for an occlusion query; Link comes back with the game's own fade-in and no missing frame in a 60 fps clip. No errors.@Beanstalk please run this branch at
02cc4682against main on Linux (any GPU: build it there too) and on Windows with an Intel GPU, with a temporal upscaler on (DLSS or DLAA on NVIDIA, FSR 4 or FSR 3.1 on AMD, XeSS on Intel) and frame generation where the GPU has it. The spot: inside the Deku Tree as child Link, climb the 1F vine wall to its top and turn the camera into Link for about 30 s, then let it come back out; after that a short walk with Link in view. What to look at: on main the picture smears there (dark blotches, flat patches and a veil that move with the camera); with this branch it should be sharp, Link should come back with no missing frame, and Link and his shadow should look as on main on the walk. Run the branch withRX_VK_DLSS_FADED_LOG=1: itsvk: upscaler: faded:lines should show at most 30 depth draws a frame left out, only while the camera is in Link.@pmwork-personal only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.@Beanstalk
Beanstalk test farm
Commit
02cc468· tier reviewed · approved by llun · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On any available PC, the plan compares main and the PR in the Deku Tree with FSR 3.1 enabled. It moves Link and the camera, captures images for visual regressions, and measures frame times over roughly 13 minutes. Crashes, substantial image differences, or a 20% p99 frame-time regression fail the plan; the scripted route does not guarantee reaching the vine-wall reproduction spot.
Done (started 18:37 UTC, last report 20:13 UTC). Comment
@Beanstalkto run it again.unknown, unknown; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 3080 Ti, 620.70.6Measurements · any
Verdict: fail on any (2/1 report):
deku-camera crashfailed.Hardware: 1: linux+other
unknown, unknownAMD Ryzen 7 5700X3D 8-Core Processor, 128 GB RAM; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 3080 Ti, 620.70.6AMD Ryzen 9 3900X 12-Core Processor, 48 GB RAMScene: deku-camera, from a new save (dlss=3); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); deku-camera: 1 game start a side, tree 30 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.