perf(vk): the vegetation's indirect mesh draws keep their passes: their arguments converted at the game's barriers (RX_VK_MESH_CONV) #1846
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1846
Loading…
Reference in a new issue
No description provided.
Delete branch "GummiNaomi/Longshot:perf/mesh-conv-batch"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Most of the game's grass, ferns and leaves are indirect mesh draws (41 a frame in Kokiri Forest, 49 in Hyrule Field), and each one ended its render pass, recorded four barriers (three of them drains), a fill and a copy, and began its pass again. This fills the argument ring with 1s once, so a draw's copy writes only its counts, and converts the draws after each barrier of the game's together, right there, so they keep their passes (
RX_VK_MESH_CONV). In in-run A/Bs on my PC the old way's mean frame time was 0.166 ms longer in Hyrule Field (+1.7 %), 0.113 ms in the Market and 0.094 ms in Kokiri Forest's stand, with 0.12-0.17 ms less GPU time a frame; two interleaved runs a side in Hyrule Field went 10.08 -> 9.87 ms. On by default;RX_VK_MESH_CONV=0is the old way.Why: NVN's indirect mesh records are {count, first} and Vulkan's {x, y, z}, so each count is copied into a ring of {count, 1, 1} on the GPU before its draw (
mesh_indirect_convert,gpu/vk/vk_draw.cpp). That copy was recorded at the draw itself:end_pass, a drain to transfer, a fill of 1s, a barrier, the copy, a barrier to indirect reads, then the pass begun again with its own full barrier. The game's culling writes the counts before one of its barriers, so they can be read at that barrier instead, as long as nothing between it and the draw may write buffer memory. The ring's slots were 16 bytes apart, which put a draw's x words on an earlier draw's y and z words, so a ring filled once needs slots 48 bytes apart (the least common multiple of 12 and 16).What changes: the ring is filled with 1s once, by a submission of its own that is waited for, and its slots are 48-byte aligned (
gpu/vk/mesh_conv.htake); a slot is used only while the ring has not come round onto it (still_holds). WithRX_VK_MESH_CONV=2(the default), right after each barrier of the game's the replay scans the stream up to the next barrier, the first op that may write buffer memory (a dispatch, a copy, a buffer or texture clear, an event, a counter's report, any op not known) or the segment's end, and converts the indirect mesh draws it finds there together, one full barrier after their copies (mesh_conv_at_barrier,gpu/vk/vk_replay.cpp;mesh_conv_batch). The scan starts again after the op that cut it. A dispatch, or a draw whose program writes memory, drops the rest of the batch, and those draws convert one by one as with1: the ring's fill alone, each draw's copy between two full barriers. No conversion on a GPU without mesh shaders, whose compute fallback reads the game's own records. The switch is read at each recording's start through a newMkwVkHost.ab_value(host API version 36,host/rx_backend.c), soHOS_AB=RX_VK_MESH_CONV=0compares the two ways within one run, and the log counts the conversions and each A/B side's GPU time. Unchanged: what is drawn,RX_VK_MESH_ARGS's rewrite, the compute fallback.Read first:
gpu/vk/mesh_conv.h, thenmesh_conv_batchandmesh_indirect_convertingpu/vk/vk_draw.cpp,mesh_conv_at_barrieringpu/vk/vk_replay.cpp, andtests/test_mesh_conv.py.Testing
Tested: Windows 11, RTX 4070 SUPER (NVIDIA 617.42), Ryzen 9 9950X3D with SMT off, 32 GB, on a build of main @
f505340bwith this change: in-run A/Bs in Kokiri Forest, Hyrule Field and the Market, and four separate runs in Hyrule Field; the log's line counts every indirect mesh draw converted from a batch in Kokiri Forest (41 a frame from 25 batches) and Hyrule Field (49 from 30), none dropped after a write and the ring never came round onto one; the grass, ferns and leaves drawn in every shot, and the shots with it on against off differing no more than two shots of the same side (mean pixel difference 3.5-5.4 against 4.0-7.0).At this head
27dc8499(on main @1780fbe0), on the same PC: the builder's Windows (llvm-mingw) build, and one headless run of it fromkokiri-startwith every switch at its default (the control tool'stestprofile, 1920x1080, uncapped) and the in-run A/BHOS_AB=RX_VK_MESH_CONV=0,3,60: it started, stood in Kokiri Forest for 30 s, teleported to the Market (40 s) and to Hyrule Field (150 s), and quit normally (exit code 0); no FATAL, no device loss and no crash in the log; the grass, ferns and leaves drawn in its shots. The log's line with the default 2: 47.8 indirect mesh draws a frame converted from 29.5 batches at the game's barriers, 0.8 one by one, none dropped after a write, the ring never came round. In Hyrule Field the old way (0) was 0.163 ms a frame slower (+1.67 %), as onf505340b(Measurements, the second table).No Linux build was done: on Linux only the checks below ran, in WSL.
Checks:
tests/test_mesh_conv.py(mesh_conv.hon a model of the stream, slots, scan and batch, with four broken variants it catches; the wiring in the source) passes on this head on Windows (clang) and Linux (g++ 13.3, WSL Ubuntu 24.04) and fails on main;tests/test_hos_ab.py(the host's table withhost_ab_valueafter the Mods tab's functions, the version check),tests/test_switch_table.py,tests/test_draw_side_effects.py,tests/test_mesh_args_cleanup.py,tests/test_mesh_fallback.pyandtests/test_split_before_draw.pypass.Suite:
tests/run_all.pywith every check on main @068fb8ebin WSL Ubuntu 24.04 (740 pass, 61 fail, 69 skipped: no clang or spirv-val there), then this head's--changedchecks in WSL (265 pass, 23 fail, 8 skipped) and on Windows (291 run, 52 fail). Every failure also fails on main @068fb8ebon the same OS: in WSL and on Windowstest_capture_readback_visibility,test_crash_dialog,test_dev_ab,test_dev_packs,test_draw_stream_visibility,test_frame_overlap,test_fsr_image_retirement,test_fsr_uniform_ownership,test_host_guest_writes,test_hud_layout_row,test_linux_pressure,test_parallel_init_order,test_pipeline_play_cache,test_prepare_cache_save,test_present_queue_default,test_present_source_crop,test_present_thread,test_profiling_mode,test_replay_write_order,test_slow_frames,test_upscaler_rows,test_vk_occlusion_queries,test_walk_key; on Windows onlytest_audio_output_devices,test_audio_volume,test_collect_wide,test_crash_record,test_dev_build,test_draw_binds,test_driver_env_choose,test_hud_layout,test_image_arrays,test_invalidate_vcore,test_linux_driver_fault_guard,test_mesh_fallback_layout,test_oot_fov,test_pads_after_menu,test_pass_hazard,test_pipeline_tour,test_precompile_compute_guard,test_present_aspect_vulkan,test_prompt_font_bind,test_push_state,test_readback_wait_bounded,test_same_draw,test_same_dyn,test_shader_module_guard,test_table_track,test_text_size,test_tracy_funcs,test_vertex_bind,test_wide_ui. (On Windows the changed checks ran on the head before its last fixes:host_ab_valuemoved above the text fields' code inhost/rx_backend.c, whichtests/test_menu_text_field.pycompiles as C, and two tests' fixtures;test_menu_text_field,test_readback_early,test_recording_admissionwere run again on this head and passed, as in WSL.) (On Windows most of them are compile errors with this PC's compiler wrapper, on main as here.) These ran on the head as it was on main @068fb8eb. It has since been rebased onto main @1780fbe0(after the renderer's threads rewrite: the rescan no longer checks the parallel replay's dry pass, which is gone); there its own checks pass on Windows and in WSL, and every one of its--changedfailures in WSL (257 pass, 30 fail) also fails on that main buttests/test_prepare_stall.py, a timed watchdog check that failed once among eight at once and passed twice run alone. The 29 that fail on main @1780fbe0in WSL as here:test_capture_readback_visibility,test_crash_dialog,test_dev_ab,test_dev_packs,test_draw_binds,test_draw_stream_visibility,test_early_split_before_draw,test_frame_overlap,test_fsr_image_retirement,test_fsr_uniform_ownership,test_hold_gpu,test_host_guest_writes,test_hud_layout_row,test_linux_pressure,test_pipeline_play_cache,test_prepare_cache_save,test_present_queue_default,test_present_source_crop,test_profiling_mode,test_readback_early,test_readback_late,test_replay_write_order,test_sampler_admission,test_slow_frames,test_split_before_draw,test_ub_update_batch,test_upscaler_rows,test_vk_occlusion_queries,test_walk_key.Still to test:
HOS_AB=RX_VK_MESH_CONV=0,3,60 RX_VK_FRAME_TRACE=1, thenscripts/perf.py ab, expecting a larger gain than on this PC (a drain costs more there) and the vegetation drawn (@Beanstalk steamdeck, Hyrule Field 3 minutes with HOS_AB=RX_VK_MESH_CONV=0,3,60 and RX_VK_FRAME_TRACE=1, and a screenshot)Measurements
Rebased: this head,
2125e5b5, is27dc8499rebased onto main89c3d21ewith no conflicts; the numbers below are from27dc8499on its old base. At2125e5b5,tests/test_mesh_conv.py,test_hos_ab.pyandtest_switch_table.pypass on Windows (clang);test_readback_early.pyfails the same way on main89c3d21e;vk_draw.cpp,vk_replay.cppandhost/rx_backend.ccompile syntax-only with the builder's flags.Verdict: a small, steady gain where the vegetation is: the old way was 1.7 % slower in Hyrule Field, 1.0 % in the Market and about 1 % in Kokiri Forest's stand, by mean frame time, with 0.12-0.17 ms less GPU time a frame; two interleaved runs a side in Hyrule Field agree (median 10.07 / 9.98 -> 9.86 / 9.80 ms). These are from a build on main @
f505340b; at this head27dc8499, one in-run A/B with default settings agrees: the old way 1.67 % slower in Hyrule Field and 1.33 % in the Market.Setup: Windows 11 Pro 26200, RTX 4070 SUPER 12 GB (NVIDIA 617.42), Ryzen 9 9950X3D with SMT off, 32 GB; from
kokiri-start, a stand in Kokiri Forest, and a teleport to Hyrule Field and to the Market; 1920x1080, no window (HOS_HEADLESS=1), uncapped, 16 cores; the in-run A/BHOS_AB=RX_VK_MESH_CONV=0,3,60(3 s periods after 60 s of warm-up,scripts/perf.py abon the frame trace for each side's numbers and the interval), and the separate runs 100 s each, in the order off, on, on, off; in all runsRX_VK_LOAD=create RX_VK_ASTC_BC_ALL=1 RX_VK_IMAGE_BLOCK_MB=32 RX_VK_BC_LAYERS=8 RX_VK_LATE_FIX=1 RX_VK_TEX_MOVE=1 RX_VK_FRAME_TRACE=1(and two switches this build does not have).Baseline: one build of main @
f505340b(before the tooling revamp068fb8eb, which changes none of this code) with this change:RX_VK_MESH_CONV=0, main's conversion, against the default 2, in the same run (A/B) or in separate runs; the host API's number on that build was 35.RX_VK_MESH_CONV=0(as main) @f505340bf505340bThe A/B intervals of the mean's difference (old less new): Hyrule Field +0.137 to +0.197 ms, the Market +0.061 to +0.167 ms. Kokiri Forest's A/B covered its teleport's frames too: over the whole trace the interval holds 0 (-2.37 to +2.60 ms, the means +1.2 % apart), so its row is the stand alone.
The host's A/B lines and the log's conversion counts (Kokiri Forest, Hyrule Field, the Market):
At this head,
27dc8499: one build, one run, the in-run A/B as above but with every other switch at its default (1920x1080, uncapped, 16 cores),scripts/perf.py abon the frame trace cut to each scene's frames (from 300 frames after the arrival):RX_VK_MESH_CONV=0(as main) @27dc849927dc8499Over the whole run (Kokiri Forest's high frame rates, the two teleports and both scenes mixed) the host's
ab:line says side 1 -0.24 % and the interval holds 0, and the log's GPU time line, also over the whole run, says 5.912 ms a frame for0and 5.985 ms for 2: neither is split by scene, so the scene rows above are the comparison. The GPU time by scene was not measured at this head.RX_VK_MESH_CONV(new, a kill switch) ships at 2, chosen by the table above;0is the conversion as on main,1the ring's fill alone.Risks
What could break: the batch reads a draw's counts at the barrier before it instead of at the draw, so a write to them between the two that the scan does not know of would be missed; every op not known to record no buffer write ends the scan, a dispatch or a draw whose program writes memory drops the batch, and the shots matched in all three scenes, but the scan's list of ops is the thing to review. Not run here: RADV, a GPU without mesh shaders. (The parallel workers,
RX_VK_PARALLEL, are gone from main since the threads rewrite, so there is no such case left to run.) The host API goes to version 36, which #1034, #1358 and #1811 also take: whichever lands second renumbers. #1763 (the renderer's threads rewritten) touches the same neighbourhood ofvk_replay.cpp(the frame stats' log,do_present,gpu_time_end) without overlapping lines, and its lighter pass-start barrier agrees with this change: the copies end with a full barrier, so the pass that starts after them adds none under either. Turning it off:RX_VK_MESH_CONV=0. After updating, a player sees slightly faster frames where there is grass and foliage; nothing changes for saves,longshot_options.inior the caches.Checklist
Ready for review
main; the title is a Conventional Commit. (Rebased onto main89c3d21eas2125e5b5, no conflicts.)mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.MKWVK_API_VERSIONbumped (MkwVkHostorMkwVkRendereringpu/vk/vk_api.hchanged).27dc8499, run in the game as Testing says, and of the measured build onf505340b. On Linux only checks:tests/test_mesh_conv.py(mesh_conv.hwith g++ 13.3) and the--changedsuite in WSL Ubuntu 24.04. No Linux build was done.)Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Possibly related, for context:
e992631ed327dc849964@Beanstalk let's test widely
Beanstalk test farm · an earlier run: commit
27dc849(the newer results are in a comment below)Commit
27dc849· tier reviewed · approved by HumanBean · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)The plan tests vegetation rendering in Kokiri Forest, Hyrule Field and the Market on Linux, Windows and Steam Deck. It compares main with the PR with parallel replay both off and on, because batching must also work across worker segments. Each target takes roughly 22 minutes; screenshots, frame times, GPU time and load times are compared. Crashes, device loss, substantial visual differences or performance regressions fail the plan.
0or1.kokiri-start, then teleport to Kokiri Forest, measure forest for 20 s (after 5 s warm-up), take the picture forest to compare with main, teleport to Hyrule Field, measure field for 20 s (after 5 s warm-up), take the picture field to compare with main, teleport to Market, measure market for 20 s (after 5 s warm-up), take the picture market to compare with main; 1 run on each side for each of 2 cells (2 game starts a run).Started at 08:01 UTC.
Estimate: done in about 2 h 45 min, from the queue and the testers online now; some jobs can't start yet.
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4080, 621.1.0RX_VK_PARALLEL=0RX_VK_PARALLEL=0RX_VK_PARALLEL=0RX_VK_PARALLEL=1RX_VK_PARALLEL=1RX_VK_PARALLEL=1Measurements · windows
Verdict: could not be judged on windows (2/2 reports).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4080, 621.1.0AMD Ryzen 7 9800X3D 8-Core Processor, 32 GB RAMScene: vegetation-tour, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); vegetation-tour: 1 game start a side, forest 20 s after 5 s warm-up, field 20 s after 5 s warm-up, market 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
RX_VK_PARALLEL0,1Markdown for the PR description
WIP: perf(vk): the vegetation's indirect mesh draws keep their passes: their arguments converted at the game's barriers (RX_VK_MESH_CONV)to perf(vk): the vegetation's indirect mesh draws keep their passes: their arguments converted at the game's barriers (RX_VK_MESH_CONV)27dc8499642125e5b5a9Beanstalk test farm
Commit
2125e5b· tier reviewed · approved by GummiNaomi · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)From kokiri-start, this plan runs the in-game mesh-conversion A/B for two minutes each in Hyrule Field and the Market on Steam Deck, Linux, and Windows with NVIDIA. It compares vegetation screenshots and frame times with main, failing on crashes, device loss, substantial visual differences, or a 15% p99 frame-time regression. Each tester’s game runs take about 15 minutes. The farm cannot set the requested 2560x1440 output or target GPUs by mesh-shader capability, so the no-mesh fallback and its log cannot be guaranteed; the requested frame-trace switch is also unavailable to plans.
kokiri-start(switchesHOS_AB=RX_VK_MESH_CONV=0,3,60), then teleport to Hyrule Field, take the picture field-vegetation to compare with main, measure field for 2 min (after 10 s warm-up), teleport to Market, take the picture market-vegetation to compare with main, measure market for 2 min (after 10 s warm-up); 1 run on each side.Done (started 14:36 UTC, last report 14:57 UTC). Comment
@Beanstalkto run it again.unknown, unknownNVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070 Ti, 621.14.0Measurements · linux
Verdict: could not be judged on linux (1/1 report).
Hardware: 1: linux+other
unknown, unknownAMD Ryzen 7 5700X3D 8-Core Processor, 128 GB RAMScene: vegetation-ab, from the repository's save kokiri-start (HOS_AB=RX_VK_MESH_CONV=0,3,60); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); vegetation-ab: 1 game start a side, field 120 s after 10 s warm-up, market 120 s after 10 s warm-up; each number the median of the runs, then of 1 report
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Measurements · windows+nvidia
Verdict: could not be judged on windows+nvidia (2/1 report).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070 Ti, 621.14.0AMD Ryzen 9 5900X 12-Core Processor, 64 GB RAMScene: vegetation-ab, from the repository's save kokiri-start (HOS_AB=RX_VK_MESH_CONV=0,3,60); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); vegetation-ab: 1 game start a side, field 120 s after 10 s warm-up, market 120 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
@Beanstalk steamdeck, linux and windows (an NVIDIA card at 2560x1440, and a GPU without mesh shaders if there is one): from kokiri-start, Hyrule Field and Castle Town's market, 2 minutes each,
HOS_AB=RX_VK_MESH_CONV=0,3,60 RX_VK_FRAME_TRACE=1. Expecting the old way (side 1) slower, as here (+1.7 % in Hyrule Field, +1.3 % in the Market at 2125e5b5's code), the vegetation drawn, and on a GPU without mesh shaders "left as they are (no mesh shaders)" in the log with no change in frame time. This replaces today's run, which ended inerrorbefore building (the farm's checkout of main failedgit rev-parse --verify HEAD^{commit}in packs.py's cmd_base) and whose plan also testedRX_VK_PARALLEL, which main has removed.