fix(vk): the compute fallback's records area holds the floor grass at 4x #2121
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2121
Loading…
Reference in a new issue
No description provided.
Delete branch "Jiro113/Longshot:fix/fallback-white-grass"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
On the compute fallback of mesh shaders (RADV, so the Steam Deck and RX 6000-9000 on Linux; RX 6000 on Windows), the
floor grass at #1877's Grass distance 4x is drawn with white blades. The fallback's mesh programs take their records from
one fixed area of 16M words, so a program fits 16M / its record's words. The grass draws through two programs on the
same workgroups: its blades' records are 22 words (fit 762600), its partner's 7 words (fit 2.4M). At 4x both draw ~1.02M
records a frame: the partner's all fit, the blades' past 762600 are dropped (MeshFallbackEpilogue drops a workgroup that
doesn't fit, silently). The white is where the partner drew and the blades did not: that link is inferred, the counts and
the white are measured.
The area is now 32M words (128 MB), which fits 1.52M records of 22 words, above the ~1.29M #1877's blade cap lets
through (extrapolated from the measured load, not measured).
fallback_buffers_ofrefused a records buffer past 128 MB, which every layout now passes (the plain one by its32-byte header, the one with the attribute pool, Intel Mesa's default, at ~160 MB): the limit is a named constant,
MESH_FALLBACK_RECORDS_LIMIT(192 MB), with a static_assert that the largest layout fits it and one that a word's byteoffset still fits BufferWordAt's 32 bits.
On
mainalone the bug doesn't show: on9dd41326eat the game's load, of the mesh programs with a draw-stats line (39of 49), none goes past a quarter of its fit at Hyrule Field, Kokiri or Kakariko; on
971f77f52with #1382's own Grassdistance 4x (merged as
d2f0e8dc4), the most is 38 % (Kokiri's short blades, 43ec86f35fd937b9) and there is no white. Itshows with #1877 (on #1382), which this unblocks on the fallback. It doesn't explain #1710, #1934, #1799 or #1760
(white or flickering foliage on AMD and the Deck at the game's own Grass distance), and doesn't claim them.
To see it: #1877 on
main,RX_VK_FORCE_MESH_FALLBACK=1 RX_VK_GRASS_DISTANCE=4, Hyrule Field: white blades across thefield, moving from frame to frame.
Read first:
gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv.h(MESH_FALLBACK_RECORD_WORDS,MESH_FALLBACK_RECORDS_LIMIT),gpu/vk/vk_draw.cppfallback_buffers_of, thentests/mesh_fallback_capacity_test.cpp.See #1877. Touches the same constants as #1561 (packed record layout, #1477 BC-1a): whichever merges second rebases on the
other's fit.
Testing
Tested: Windows 11, RTX 4090, NVIDIA 621.24, the fallback forced (
RX_VK_FORCE_MESH_FALLBACK=1) and native, fromthe same Hyrule Field save, standing still, 6 screenshots and the
RX_VK_DRAW_STATSvertex counts per run:partner in every frame (1.00-1.02M, 67 % of the new fit at most);
main;main9dd41326e; rebased since onto971f77f52without a conflict (main's changesdon't touch the records area or
fallback_buffers_of's limit);2194d263bon971f77f52againstmain971f77f52, the fallback forced,main's own Grass distance 4x (#1382): thefield save and Kokiri Forest twice a side, the field save at the game's load once a side. No white either side
(near-white 0.00-0.12 %), the same counts for every grass program (739a ~169k at Hyrule Field, 43ec ~290k at
Kokiri), no
vk: fallbackrefusal; other buffers 217-219 MB onmain, 271-281 MB here;test_mesh_fallback_capacityfails onmain(the 22-word program draws 762336 of 1023009records) and passes here;
test_mesh_fallback_global_attrs,test_fallback_buffer_cleanup,test_mesh_fallback_shared_warpand themesh_fallback_layoutmodel (62 cases) pass.run_all.pyon this PC: 771 passed, 96 failed here; 770 passed, 96 failed onmain9dd41326e. 95 of the failing checksfail on both, the same list on
main(cause not looked into for each: among themtest_mesh_convstops on a-Wunused-const-variableinmesh_conv_real_test.cppunder llvm-mingw, and source-reading checks that no longer matchmain);test_longshot_ctlfailed here only andtest_work_lockonmainonly, and both pass when run again alone on this branch.The whole suite above ran at f6058b234 on
9dd41326e. At the head,167aa2de3onmain8aa749834(rebased from2194d263b; ofmain's four newer commits only #2103 touches the fallback, its stages' global loads through the guestpage table, not their outputs, which make the records, nor the records area or
fallback_buffers_of, so the in-gameruns above at
2194d263bhold; not run in the game at the head):run_all.py --strict --changed(the checks that reada file this PR changes) on Windows: 148 passed, 19 failed, 5 skipped (at
2194d263b: 147, 19, 5, the same failures).14 of the failures are in the list below; the other 5 (
test_device_policy,test_framegen_preset,test_intel_prepare_side_by_side,test_prepare_stall,test_rumble_menu) came withmain's newer commits and failon
main971f77f52and8aa749834too (grass_distance_nowundeclared in the menu cut-outs,IntelPrepareLaneunknown,
acquire_aheadon RADV). Themesh_fallback_layoutmodel (62 cases, built with llvm-mingw's clang) passesat the head. Linux, at the head (Ubuntu 22.04 in WSL, g++ 13.4, Python 3.10):
test_mesh_fallback_capacity(its 6cases),
test_fallback_buffer_cleanup,test_mesh_fallback_global_attrs,test_mesh_fallback_shared_warpandtest_grass_distancepass;test_mesh_fallback_layoutskipped there (no SPIR-V headers or spirv-val in thatinstall). CI runs only the light checks on a pull request (the suite is Actions > CI > Run workflow), so it doesn't
check this.
Failing on both main and this PR on this PC
test_appimage_player_build test_auto_handheld test_bug_report test_bug_report_defaults test_capture_read_once
test_capture_readback_visibility test_compile_retry_jobs test_compile_stops_on_exception test_crash_dialog
test_crash_record test_crt_options test_damage_debug_menu test_dev_ab test_dev_build test_dev_migrate test_dev_packs
test_device_lost_exit test_display_startup_failure test_dlss_feature_size test_draw_binds test_draw_stream_visibility
test_failed_compile_keeps_objects test_fastfp_fma test_flush_nothing_held test_fma_soft test_fpx_sticky test_fpxkeep
test_frame_overlap test_fsr3fi_hud_layer test_fsr3fi_hudless_colour test_fsr_image_retirement test_fsr_uniform_ownership
test_game_menu_mouse_buttons test_gpu_wait_stats test_guest_imported test_high_fps_fixes test_host_guest_writes
test_input_background test_job_queue test_libc_hooks test_lift_cache test_lift_cleanup_error test_linux_preflight
test_linux_pressure test_low_vram_texture_mips test_make_game_ngx test_menu_i18n test_menu_keys test_menu_text_field
test_mesh_conv test_mise_build test_missing_nup test_mods_list test_ngx_sdk test_oot_shadow_fit test_pads_after_menu
test_pass_barrier test_pending_guest_import test_perf_report test_pipeline_play_cache test_portable_builder_home
test_portable_builder_startup test_prepare_cache_save test_present_queue_default test_present_source_crop
test_profiling_mode test_psychov35_fsr3_fg test_readback_timeout_source test_recorder_own_idle test_recorder_sync
test_recording_admission test_reflex_swapchain_lifetime test_renderer_new_handler test_renhdrum test_replay_write_order
test_run_all_mingw test_run_all_output test_sampler_admission test_sdiv_overflow test_slow_frames test_song_note_cue
test_sparse_mirror test_spawn_emit test_staging_burst test_target_reuse_memory test_texel_buffer_ownership
test_ub_update_batch test_ui_record test_upscaler_rows test_vbytes test_vcore_escape test_vcore_mask0 test_vecfp
test_vfpx_elem test_vk_occlusion_queries
Still to test:
vk: fallbackrefusal in the log, the picture and fps as on
main,vk: budgetother buffers ~54-74 MB higher (Test Farm, SteamDeck: Hyrule Field and Kokiri Forest against main, fps, video memory, no fallback refusal in the log)
main, fps, video memory, no fallback refusal in the log)
on the Test Farm there)
vk: fallbackrefusal (by hand, someone with the hardware)
Measurements
Verdict: the white blades are gone: near-white pixels 4.33-7.99 % of the frame with #1877 at 4x on the fallback,
0.03-0.12 % with this PR, as native (0.01-0.09 %); at the game's load nothing changes. Cost: +54 to +74 MB of video memory on
the fallback only.
Setup: Windows 11, RTX 4090, NVIDIA 621.24; the Hyrule Field save, standing still, fallback forced with
RX_VK_FORCE_MESH_FALLBACK=1; 6 screenshots a run, the share of near-white pixels and the change between frames;RX_VK_DRAW_STATSvertex counts of the two grass programs over 5 frames; video memory from thevk: budgetline.Commits: on current
main,main971f77f52against this PR at2194d263b(first table); before the rebase,main9dd41326eagainst this PR at f6058b234 (second table). Baseline for the #1877 rows:main9dd41326ewith #1877 atdefe02170against this PR with the same #1877; the bug needs #1877 to show.main's own Grass distance 4x (#1382), the fallback forced; the most a program draws of its fit, over 5 frames a run:971f77f522194d263bBefore the rebase, the #1877 rows with #1877 on top:
9dd41326e(*) The scene and its grass are drawn, without white; the six frames of a run were identical on both sides (a still
picture on the fallback here, as on
main: not looked into).Frame rate not measured: per frame only the records' header is cleared, whatever the area's size; an fps A/B on the
fallback is still to run.
Risks
in all above, so one thread here), nothing on the
native path. On the Steam Deck's shared memory and 4 GB cards on the fallback that is the main cost; to be checked on
the Deck (Still to test).
fallback_buffers_ofrefuses one today. Measured only on NVIDIA.today (the widest, 28 words, ~13.5k a frame). The overflow stays silent: a one-time log line when a workgroup is
dropped would make the next case visible (a follow-up).
longshot_options.inior caches.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). (its command,run_all.py --strict: the wholesuite at f6058b234 on
main9dd41326e,--changedat the head, as Testing says; every failure here also fails onmain, named there)main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here.tests/that reads changed source is updated, not weakened.changed code compiled with g++ 13 in the new check, as Testing says)
mainand here, for a change that shows on screen. (in a comment: #1877 at 4x onthe fallback, before and after)
Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Tested in game: Windows 11, RTX 4090, NVIDIA 621.24, fallback forced (RX_VK_FORCE_MESH_FALLBACK=1) and native, from a Hyrule Field save and at Kokiri and Kakariko, at f6058b234 on main
9dd41326e(rebased since onto971f77f52without a conflict). With #1877 at Grass distance 4x on the fallback the white blades are gone (near-white 4.33-7.99 % -> 0.03-0.12 % of the frame) and the blades' program draws as many records as its partner every frame. Without #1877 the picture is as on main on both paths. Other buffers +64 MB on the fallback, unchanged native.Screenshots, the same spot (the Hyrule Field save, standing still), #1877 at Grass distance 4x, the fallback forced (
RX_VK_FORCE_MESH_FALLBACK=1), RTX 4090, NVIDIA 621.24:Before,
main9dd41326ewith #1877 (near-white 4.3-8.0 % of the frame):After, this PR at f6058b234 with the same #1877 (0.03-0.12 %, as native):
(Taken before the rebase onto
971f77f52; the run on currentmainis in Still to test.)2194d263b6167aa2de30Rebased onto main
8aa749834(167aa2de3), no conflict. The in-game numbers at2194d263bon971f77f52still hold: main's four newer commits (#2123 CI, #2125 NVIDIA's shader cache file, #2103 global loads in shaders, #2124 XInput thread) don't touch the fallback's records area, its programs' record sizes or fallback_buffers_of. At the head, run_all.py --strict --changed: 148 passed, 19 failed, the same 19 as before the rebase, all failing on main too (named in Testing); the fallback and Grass distance checks pass.WIP: fix(vk): the compute fallback's records area holds the floor grass at 4xto fix(vk): the compute fallback's records area holds the floor grass at 4x@review
Recommendation: Approve. The forced-fallback run shows that the larger area removes the reported white grass with #1877, but the change increases fallback memory use for players who do not have that change. Its effects on the Steam Deck and RADV, and the first-start cost of invalidating shader caches, still need measurement. Nothing blocking is left: what remains is for the author before merging (below). Recommend approval; not approved on the forge.
Doubles the compute fallback's records area and raises its allocation limit so the larger buffer can be created. This fixes the observed capacity shortfall in the tested #1877 scene at a cost of 64 MiB per fallback recording thread and a shader-cache rebuild.
@fixwill do (2)gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv.h:69: Allocate the larger area only when the workload requires it, or otherwise reduce the permanent per-thread allocation.Before merging, for the author (2) (a run, a measurement or a decision; it doesn't hold the recommendation)
gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv.h:69: The larger allocation applies to every fallback recording thread, including at the game's normal grass load where the reported runs show no visual benefit. (more on the line)gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv.h:69: This header is included in the translator stamp, so changing the constant invalidates the program cache and prepare record even on native-mesh systems. (more on the line)Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request),Website / Website on Cloudflare (pull_request).More: does it help, other systems, switches, what was read
Does it help: Yes, for the tested setup: with #1877 at 4x on a forced NVIDIA fallback, near-white pixels fell from 4.33–7.99% to 0.03–0.12%, and the two grass programs drew matching record counts. The before side had one run, the after side two; the head and the affected AMD hardware have not been tested in game.
Other systems: The native path does not allocate the larger fallback buffer, but all players receive a new translator stamp and must rebuild affected caches. RADV and the Steam Deck use the fallback by default; Intel Mesa also uses the attribute pool, producing a roughly 160 MiB records buffer. Allocation, image and performance behavior on those systems remain untested.
Switches: None added, changed or removed.
Reviewed:
167aa2de30against8aa7498345: Read AGENTS.md, CONTRIBUTING.md, the PR template, standards, the commits and all seven changed-file diffs; read the new capacity test and harness, changed constants, allocation function, layout model and global-attributes check with their surrounding code. Traced shader emission, buffer reuse and the translator cache stamp. Skimmed the reported runs and related callers. docs/review-checklist.md and the adjacent Tooling checkout were absent; screenshots and generated files were not independently inspected.; nothing run (a code review).Checked and fine:
Testing: Read-only review; no builds, tests or project scripts were run. The author's reported game and test results are described above. CI is still running.
Overlaps and merge order: #1561 touches the same constants; whichever change lands second needs to reconcile the record fit.
Automated code review by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
@ -66,0 +66,4 @@// the floor grass at the Grass distance's 4x (#1877) lost its blades: their program's records are 22 words and fit 762600,// its partner's 7 words, and each drew ~1.02M records a frame, so the blades past the fit were left out and drawn white.// 32M words fit 1.52M of 22 words)constexpr u32 MESH_FALLBACK_RECORD_WORDS = 32u << 20;L1 (minor): The larger allocation applies to every fallback recording thread, including at the game's normal grass load where the reported runs show no visual benefit. Measure average and 1% low fps, frame times and memory against main on the Deck and RADV, and check that Intel Mesa accepts the pool layout; these are the systems that use or can use this path.
L2 (minor): This header is included in the translator stamp, so changing the constant invalidates the program cache and prepare record even on native-mesh systems. Measure the first-start or rebuild cost against main, including the native path, so players' one-time cost is known.