WIP: perf(vk): write-backs left out only in pools the CPU reads after an invalidate (the pools' NVN flags reach the renderer) #799
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8015c9a
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!799
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:perf/vk-writeback-pool-flags"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Windows 11 (the game-free checks; the game A/B is next) · Needs: a game run on NVIDIA, AMD and Intel
Why
Copying the GPU's writes back to the game's memory is up to 60-68 MB a frame in Kokiri Forest and Hyrule Field, and
RX_VK_WB_MAX_KBcan leave most of it out (2.5 ms of GPU time on an RX 5600 XT in #547). It is off for everyone because some ranges are read by the CPU with no warning and were found stale. Those are the game's uncached pools: in its CPU-cached pools the CPU must invalidate before it reads what the GPU wrote, and the renderer hears every invalidate. The renderer never knew a pool's flags. In a Kokiri run 3,368 of the game's 4,588 pools were CPU-cached, and the import that took most of the write-back was 126 MB of cached pools and 0.4 MB of uncached ones.What changed
nvnMemoryPoolInitializenow passes the pool's NVN flags to the renderer, which keeps them by guest address, andRX_VK_WB_MAX_KB's leave-out applies only where a binding lies wholly in CPU-cached pools or pools with no CPU access. A log line splits the write-back by pool kind with the cap on or off. The cap itself stays off by default in this PR; the numbers below decide whether a follow-up turns it on.rx_nvn_pool_created(gpu, size, flags)fromnvnMemoryPoolInitialize(host/nvn_native.c,host/rx_backend.c,gpu/render/rx_nvn.h);MkwVkRenderer::nvn_pool_createdtakes the flags,MKWVK_API_VERSION24 (gpu/vk/vk_api.h)gpu/vk/wb_pool_flags.h(the map and the rule),pool_flagsinmemory_pool_created, the check inwb_leave_out,wb_access_note/wb_access_report(gpu/vk/vk_memory.cpp)RX_VK_WB_MAX_KBparagraphtests/test_wb_pool_flags.pyMeasurements
Verdict: not measured yet: the measurements are pending from the RX_VK_BENCH A/B of this PR @
506ac7a4against main @b9e95173(under way), which will replace this section. First numbers (Kokiri Forest, RTX 3080) are in the comment of 18:33: withRX_VK_WB_MAX_KB=1024the write-back fell from 68.9 to 16.5 MB a frame, the same as withRX_VK_WB_ANY_POOL=1.Flags and settings
Switches:
RX_VK_WB_ANY_POOL=1RX_VK_WB_MAX_KB: leave out in any pool, as before=1only to compareRX_VK_WB_MAX_KB=<KB>(unchanged)Testing
Tested: Windows 11 (Ryzen 9 3900X), llvm-mingw and the Vulkan headers of the builder's v2.0.14 tools: rebased onto main @
b9e95173(this PR @506ac7a4; it stands alone on main, on no other PR). The 124 checks that read this diff (listed below: those under tests/ and nx2/ that name a changed file, and the ones this PR adds or changes), not the whole suite:python -I -B tests/run_all.py --jobs 6 <those checks>withRX_CPMat the builder's CPM cache: 117 passed, 7 failed; each failure (tests/test_appimage_player_build.py,tests/test_crash_dialog.py,tests/test_longshot_saves.py,tests/test_pipeline_tour.py,tests/test_tested_configurations.py,tests/test_vk_pgo.py,tests/test_walk_key.py) also fails on main @b9e95173in the same checks, on the same PC and the same day.The 124 checks
nx2/test_import_windows.pytests/test_amd_subgroup_profiles.pytests/test_appimage_player_build.pytests/test_audio_in.pytests/test_audio_mono.pytests/test_audio_volume.pytests/test_auto_settings.pytests/test_bake_ds.pytests/test_bind_memo.pytests/test_bound_keys.pytests/test_camera_speed.pytests/test_capture_copy_helpers.pytests/test_capture_copy_nt.pytests/test_capture_copy_runs.pytests/test_capture_wait_log.pytests/test_check_pr_description.pytests/test_collect_wide.pytests/test_crash_dialog.pytests/test_dev_packs.pytests/test_device_lost_exit.pytests/test_direct_buffer_writes.pytests/test_dlss_math.pytests/test_drawn_page_map.pytests/test_forge_pr.pytests/test_fps_stats.pytests/test_frame_rate_cap_step.pytests/test_frame_trace_waits.pytests/test_fsr_uniform_ownership.pytests/test_hle_stats.pytests/test_hud_layout.pytests/test_import_buffer_binding.pytests/test_import_chunk_default.pytests/test_import_span.pytests/test_import_write_watch.pytests/test_input_background.pytests/test_intel_compiler_gate.pytests/test_intel_compiles_auto.pytests/test_invalidate_vcore.pytests/test_jobs_failure.pytests/test_jobs_wait_runs_queued.pytests/test_late_fix_build.pytests/test_late_fix_shaders.pytests/test_launcher_capture.pytests/test_linux_pressure.pytests/test_log_levels.pytests/test_longshot_ctl.pytests/test_longshot_saves.pytests/test_low_vram_texture_mips.pytests/test_mem_trim.pytests/test_menu_hit.pytests/test_menu_keys.pytests/test_menu_text_field.pytests/test_met_wait_capture.pytests/test_mirror_budget.pytests/test_mirror_gpu_only.pytests/test_mirror_inherit.pytests/test_mirror_trim.pytests/test_mouse_settings.pytests/test_no_webgpu.pytests/test_nvn_buffer_map.pytests/test_nvn_host_tables.pytests/test_nvn_null_pool.pytests/test_nvn_tex_builder_guard.pytests/test_nvn_texture_handle.pytests/test_ocarina_note_bindings.pytests/test_ocarina_notes.pytests/test_oot_damage.pytests/test_oot_fov.pytests/test_oot_menu_mouse.pytests/test_opening_skip.pytests/test_option_ids.pytests/test_pad_layout.pytests/test_parallel_mirror_make.pytests/test_parallel_rb_refresh.pytests/test_pipeline_tour.pytests/test_pool_span.pytests/test_prepare_stall.pytests/test_present_trace.pytests/test_prof_clock.pytests/test_prof_sampling.pytests/test_program_cache_format.pytests/test_prompt_auto_face.pytests/test_prompt_font_bind.pytests/test_prompts_live.pytests/test_readback_part.pytests/test_readme_menu_rows.pytests/test_registration_host_copies.pytests/test_renderer_load_error.pytests/test_report_words.pytests/test_retired_give_back.pytests/test_rumble_menu.pytests/test_sampler_admission.pytests/test_scene_format.pytests/test_shader_stamp_cache.pytests/test_shader_stamp_inputs.pytests/test_skip_title.pytests/test_sparse_mirror.pytests/test_split_ub_flush.pytests/test_stick_dead_zone.pytests/test_submitted_page.pytests/test_switches_removed.pytests/test_temporal_jitter.pytests/test_tested_configurations.pytests/test_tex_spans_once_a_present.pytests/test_texel_buffer_ownership.pytests/test_text_size.pytests/test_texture_layout_sizes.pytests/test_threaded_rendering.pytests/test_touchpad_menu.pytests/test_vertex_bind.pytests/test_vk_pgo.pytests/test_vk_range_walk.pytests/test_vram_pressure.pytests/test_walk_key.pytests/test_wb_diff_cpu_writes.pytests/test_wb_diff_split.pytests/test_wb_diff_stale_shadow.pytests/test_wb_learn.pytests/test_wb_observation_lifetime.pytests/test_wb_pages.pytests/test_wb_pool_flags.pytests/test_wb_shadow_follows_sync.pytests/test_wb_shadow_remake.pytests/test_wide_ui.pyNot tested: the game yet. This PR is WIP until its A/B is in: the RX_VK_BENCH A/B against main is running on an RTX 3080 with a Ryzen 9 3900X, Windows 11; AMD and Intel need other testers.
Must be tested:
RX_VK_WB_MAX_KB=1024: Kokiri Forest, Hyrule Field and Castle Town a few minutes each, expecting no flicker, wrong exposure or popping (the stale reads the cap had), and the log'swritten back by the pools' CPU accessandwrite-backs left outlines (Test Farm: @Beanstalk any, those scenes with RX_VK_WB_MAX_KB=1024)Risks
What could break: With the cap on: a range the CPU reads rarely (less often than every 8 frames) in a cached pool still finds the stale copy once, as before. Off (the default) only the new log line differs.
Turning it off: leave
RX_VK_WB_MAX_KBunset (the default); revert the commitCompatibility:
MKWVK_API_VERSION23 -> 24: the game and the renderer must come from the same build (they always do in a release). No caches or settings change.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).mainand passing here.🤖 Generated with Claude Code
The checks on this PC (Windows 11, RTX 3080, Ryzen 9 3900X, llvm-mingw from the builder's v2.0.14 tools):
python -I -B tests/run_all.pyon main @aa95d35ffails these 19 here, and so does this branch:test_appimage_player_build.py,test_bake_portability.py,test_cpm_discovery.py,test_cpp_preflight.py,test_crash_dialog.py,test_env_on.py,test_fpx_sticky.py,test_lift_cleanup_error.py,test_linux_preflight.py,test_make_game_cache.py,test_make_game_prepare.py,test_missing_nup.py,test_oot_state_tap.py,test_options_budget_once.py,test_readback_late.py,test_spawn_emit.py,test_tested_configurations.py,test_vk_pgo.py,test_walk_key.py. Any other failure in a full run on this branch was rerun on its own and passed (the full runs shared the PC with dev_packs builds: timing checks).Next: an interleaved A/B in the game against main on one dev_packs base of
aa95d35f; the numbers will replace the Measurements section.First in-game numbers (RTX 3080, Ryzen 9 3900X, Windows 11; dev_packs runtimes of this PR and main @
aa95d35fon one base; Kokiri Forest, 3 interleaved runs, camera turning): with RX_VK_WB_MAX_KB=1024 the write-back went from 68.9 MB a frame (main) to 16.5 MB, and with RX_VK_WB_ANY_POOL=1 (the cap in any pool, as before this PR) also 16.5 MB: in Kokiri every binding the cap leaves out lies in CPU-cached or no-access pools, so keeping uncached pools whole costs nothing here. On this fast card fps and GPU time did not move beyond the runs' spread (main 30.5-32.2 fps, this PR with the cap 32.7-35.0; GPU 11.6-13.7 against 11.1-11.8 ms), as #753 found on an RX 9070 XT; the gain the cap gave on an RX 5600 XT (#547) is where to look. Not checked yet: pictures (stale reads), other scenes. Stays WIP.2181a1c793506ac7a4e0Rebased onto main @
b9e95173(now506ac7a4; stands alone on main, not stacked); the 124 checks that read this diff, not the whole suite: 117 passed, 7 failed, each failure also failing on main @b9e95173in the same checks. Stays WIP: measurements pending from the RX_VK_BENCH A/B.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.