vk: bring back #640 with mesh stages kept on mesh-shader GPUs #792
Closed
hckrman101 wants to merge 7 commits from
reland-640-mesh-stages into main
pull from: reland-640-mesh-stages
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2866
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2814
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!792
Loading…
Reference in a new issue
No description provided.
Delete branch "reland-640-mesh-stages"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Brings back
ea4eced(#640, by @shayderrr: task and mesh programs run as compute shaders on GPUs withoutVK_EXT_mesh_shader), reverted in806c566/b11058c, with the fix for what made GPUs that do have mesh shaders lose the device.Why it was reverted
With #640 merged, an RTX 4060 Ti lost the device at the title (mesh shaders present, the fallback off). The same happened here on an RTX 3090 (driver 615.78.08). Every run of a build containing #640 hit, within 20-40 s of start:
Xid 109(CTX SWITCH TIMEOUT), thenXid 31MMU faults (GPCCLIENT_GCC faulted @ 0x1070_1200f000 ... FAULT_PDE ACCESS_TYPE_VIRT_READ,UNRECOGNIZED_CLIENT faulted @ 0x0_00000040)vk: device lost; presents stop, the replay thread spins in the driver and memory use keeps growingCause
build_graphics(gpu/vk/vk_draw.cpp) left out every stage with the compute, task or mesh bit, to keep the fallback's dispatches out of the graphics pipeline:On the fallback, the task and mesh stages are
VK_SHADER_STAGE_COMPUTE_BITalready. On a GPU with mesh shaders, though, the task and mesh stages are a mesh program's pre-rasterization stages. So every native mesh pipeline was made with a fragment stage alone, and its draws faulted the GPU. The rest of #640 is gated ondev.mesh_fallbackand doesn't change anything on such a GPU.Fix
build_graphicsleaves out onlyVK_SHADER_STAGE_COMPUTE_BITstages (the fallback's), so native mesh pipelines keep their task and mesh stages.RX_VK_FORCE_MESH_FALLBACK=1, a test switch: leavesVK_EXT_mesh_shaderoff, so a GPU that has mesh shaders runs the fallback.The first commit is #640 as merged (its author kept). The second commit is the fix.
Evidence (RTX 3090, Linux, driver 615.78.08)
Same save and scripted start each run (Kokiri Forest), with the same game executable for every Vulkan back-end variant. A run counts as clean after 300 s in play with no Xid.
d09dacd)40c0042(CAS loop ceiling)RX_VK_FORCE_MESH_FALLBACK=1Tested
RX_VK_FORCE_MESH_FALLBACK=1(the log saysissue 640 fallback): the game reaches play, then crashes in the driver's SPIR-V compiler (libnvidia-glvkspirv.so+0xfc0d5, a write to0xffffffff00000008) at the first mesh draw. That is #640's fallback path, which this PR doesn't change. It may be SPIR-V that NVIDIA's compiler doesn't accept, or an NVIDIA-only problem. Either way, the fallback is not verified here.tests/run_all.py --strict: 101 passed, includingtest_mesh_fallback.pyandtest_cas_loop_ceiling.py. 2 failed, the same way without this change (test_chunk_split.py, fixed by #735;test_cpp_preflight.pyexpects a Visual Studio message on a PC without the fake toolchain on PATH). 4 skipped (Windows/Unicorn-only).Hi! I'm the author of issue #640.
I have an AMD Radeon RX 5500 XT 8 GB (RDNA1), and I'm willing to test the compute mesh shader fallback on real hardware.
In previous Hookshot builds, I experienced missing geometry, including buildings, terrain, foliage, and a chest/box. I also tested Lichii, where some of that missing geometry renders correctly on the same GPU.
If there's a Windows test build containing PR #792, I'd be happy to try it and report FPS, missing geometry, crashes, and logs.
Thank you for continuing to work on this!
You can build from source, just clone the repository from this PR's branch. Build instructions are in CONTRIBUTING.md
I have an older Pascal Nvidia GPU (doesn't have mesh shader support) which I will test later today when I get home.
Thanks for the information! I'm on Windows with an AMD Radeon RX 5500 XT 8 GB (RDNA1).
Would it be possible to provide a Windows test build of PR #792? I'd be happy to test it and share screenshots, FPS results, and hookshot_log.txt.
I'm also looking forward to hearing how the Pascal GPU test goes!
Tested this PR on an RDNA1-class GPU without
VK_EXT_mesh_shader: a PS5 running Linux (Bazzite, kernel 7.2.3), RADV GFX1013 (Oberon), Mesa 26.2.1, built from3eb85bcwith the v2.0.5 AppImage toolchain. The fallback turns on as expected (vk: no mesh shaders: task and mesh programs run as compute shaders (issue 640 fallback)), and the title screen runs at 57-60 fps.Two bugs in
emit_spirv_mesh.cppthat crash RADVBoth show up only on GPUs that take the fallback, which is probably why nobody hit them yet.
PushU64is given byte offsets instead of struct member indices. It doesOpAccessChain(rescaling_push_constants, Const(member)), but the payload calls passMESH_FALLBACK_PAYLOAD_PUSH_OFFSET(64).spirv-valon the fallback compute shaders:OpAccessChain cannot find index 64 into the structure ResolutionInfo. This structure has 6 members.A debug RADV asserts (glsl_get_struct_field: index < t->length); the release build segfaults insidelibvulkan_radeon.soat the first task/mesh draw. Fix: passctx.mesh_fallback_payload_member_index.TaskEpilogue.startedis anOpULessThanresult, stored straight into au32device-address word.spirv-val:OpStore Pointer's type does not match Object's type. Debug RADV:SPIR-V parsing FAILED: Source and destination types of SpvOpStore do not match: uint vs. bool. Fix:OpSelect(U32, started, 1, 0)(the reader inMeshFallbackEpiloguealready testsstarted == 0).With both fixes the title runs, and the save loads into gameplay without the driver crashing.
Still broken: GPU page fault while Hyrule Field loads
About 15-20 s after Load, every run (stock RADV) ends with:
It's a scalar read of what looks like a raw guest GPU address (
0x80_xxxx_xxxx) that was never translated, from a fallback task/mesh draw (the log just before shows program1bd1: stage 6 with a 256-byte payload,mesh draw kind 0: 0 4 512).memory_guest_pages()and the fallback addresses are both pushed (push_constants,p.uses_pushis true for fallback programs), so it seems to be a pointer path in the converted task/mesh code that skips the guest page table.spirv-valalso flags the fallback's generated vertex stage:Memory accesses with PhysicalStorageBuffer must use Aligned(the record loads). RADV accepts that, but it may be worth addingAligned 4.I can test further builds on this hardware.
@AmiriisCustoms thank you, this was a great report! Both of your fixes are in, in a commit that credits you: the payload address now comes from the push block's member index, and the task's started flag is stored as a u32 (1 or 0). The fallback's PhysicalStorageBuffer loads and stores now have
Aligned 4as well.After that we dumped every stage the fallback generates (task and mesh as compute, the generated vertex stage, and the fragment stage) and ran spirv-val on them. One more error came up in every task and mesh stage: the strided copy loops declared a function variable in the middle of the function. The loops are unrolled now, and all 5243 stages from a Kokiri Forest run validate.
On an RTX 3090 with the fallback forced on, the driver's shader compiler no longer crashes at the first mesh draw (that was the same invalid SPIR-V). The game reaches gameplay at about 50 fps and geometry and foliage draw. There were no GPU faults, including while running around the village. Native mesh shaders on the same card still work as before. One known gap: some grass that the native path draws is still missing with the fallback.
About the page fault: we couldn't reproduce it here, but we found and fixed two fallback bugs that could cause it, or at least wrong geometry:
Could you retest at the new commit (
b9d6c6f), ideally the same Hyrule Field load? If it still faults, the log lines just before it would help a lot.@joao0205-lab thanks for offering to test! We can't build or distribute a Windows test build from here: a build contains game files, so it has to be built from source with your own game files, following CONTRIBUTING.md. If you build this branch on your RX 5500 XT, reports with screenshots and
hookshot_log.txtare very welcome.BTW the CI builds Windows builder, that actually is probably what you are looking for ^
Hey! Thanks for the information!
I'm already building Hookshot from source on Windows using the PR #792 branch, at commit
b9d6c6f, following the CONTRIBUTING.md instructions.My PC specs:
The game version check passed successfully, and the full build is currently running with LLVM-MinGW.
Once the build finishes, I'll test the mesh shader fallback, check whether the missing geometry and foliage appear, and report back with screenshots, FPS results, and
hookshot_log.txt.Really excited to help test this on RDNA1 hardware! Thanks for all your work on the fallback!
Thank you for the careful investigation (the Xid traces were very useful)! #779 brings back the same #640 fallback (
ea4eced) with the same fix for mesh-shader GPUs: build_graphics leaves out task/mesh stages only for fallback programs, so native mesh programs keep theirs. #779 has been taken into the maintainer's tree for the next release after a game test, so I'm closing this one as superseded. If testing turns up something #779 misses that this branch handles, please say so here or open a follow-up.Well, I had already tested it before I realized it was closed, so might as well share:
Tested on both, a GTX 1660 Super and a 1080 Ti. Game runs well and the visual defects seem fixed on Pascal: foliage in the trees finally renders, as well as some walls in the Kokiri Village houses that were not rendering before. The 1660 does support mesh shaders, and it didn't crash, so that works too
The PR that superseded this one makes the game crash on the menus after selecting load game, so this one works better.
GPU: GTX 1080 Ti
Driver: 582.78
OS: Windows 11
Does it start and reach gameplay on in this PR? yes, and the shaders for foliage, walls, etc are now working
Logs: hookshot_log.txt
Before this mesh shader fix:

After this fix:

#779 is still crashing the game for graphics cards with no mesh shader support. This branch fixes it, verified by me (GTX 1080 ti, details posted above) and one other person running an AMD Radeon RX 580, see #1001
Requesting that this PR be reopened and re-evaluated @NotRexed