perf(vk): the pipeline cache file not read on Windows with NVIDIA's driver and 20 GB of RAM or less, as on Linux #1845
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1845
Loading…
Reference in a new issue
No description provided.
Delete branch "GummiNaomi/Longshot:perf/pcache-windows-small-ram"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The pipeline cache's file (
cache/longshot_vk_pipelines.bin, 1.2 GB after a preparation) is handed whole tovkCreatePipelineCacheat every start, and NVIDIA's Windows driver keeps its copy in the game's private memory for the session: 1,563 MB in a Market stand on my PC. Linux has skipped the file since #416 with NVIDIA's own driver, 20 GB of RAM or less, programs loaded at create and NVIDIA's own shader cache warm, where that cache answers the pipelines. This applies the same rule on Windows, with the folder NVIDIA's Windows driver files its Vulkan pipelines in,%LOCALAPPDATA%\NVIDIA\GLCache. On my 32 GB PC, with the file left unread on purpose: working set 1.0 GB and private memory 1.2 GB lower, steady frames the same, but each load at create took 3.4-3.8 ms instead of 0.9 and draws ran about 40 loads themselves instead of 14. So, as on Linux, only with 20 GB of RAM or less, where that gigabyte is what makes the game page; this PC still reads the file, and no PC of 20 GB or less has run it yet (WIP).RX_VK_PCACHE_LOAD=1reads it whatever the rule says.Why: a temporary census of the process's allocations by call stack (not in this PR) put 1,563 MB under
vkCreatePipelineCache<-pcache_init<-precompile_run: the driver's copy of the file. NVIDIA's own disk cache holds the same pipelines (4.7 GB on this PC), and with programs loaded at create every pipeline the load builds is asked of it then. Not reading the file moves that work from a memory copy to NVIDIA's disk cache: cheaper in RAM, dearer per load.What changes:
pcache_load::choose(gpu/vk/pcache_load.h) gets a Windows-with-NVIDIA arm: not read with 20 GB of RAM or less (GlobalMemoryStatusEx), programs loaded at create, and NVIDIA's folder holding 512 MB or more (nvidia_cache_windows); read where the RAM is more or not known, the folder nearly empty, missing or not readable, programs loaded before the title, and in a--prepare-shadersrun.needs_cachewalks the folder only where everything else already says not to read.pcache_load_off(gpu/vk/vk_draw.cpp) asks Windows for the RAM and the folder, and the log'svk: pipeline cache file read/not readline names the reason. Unchanged: Linux's rule, AMD and Intel (read),RX_VK_PCACHE_LOAD, and what happens when the file is not read (as on Linux).Read first:
chooseandnvidia_cache_windowsingpu/vk/pcache_load.h,pcache_load_offingpu/vk/vk_draw.cpp, thentests/test_pcache_load.py.Testing
Tested: Windows 11, RTX 4070 SUPER (NVIDIA 617.42), Ryzen 9 9950X3D with SMT off, 32 GB: on a build of main @
f505340bwith #1810-#1813's changes, this one and #1844, two Market stands from thekokiri-startsave, whose log saidpipeline cache file read: more than 20 GB of RAM, or the RAM not known (31620 MB of RAM), the rule's answer on this PC; the unread side on a build whose Windows arm had no RAM limit yet: two Market stands and a 7-area teleport tour, the log sayingnot read: Windows with NVIDIA's own driver, ...with the folder's 4,729 MB, every area drawn, the tour's pictures alike in all 7 areas against a tour with the file read and the modules kept.Checks:
tests/test_pcache_load.py(the rule's table with the Windows cases;pcache_load_offrun on Windows against a LOCALAPPDATA folder of each kind: warm, 32 GB, RAM not known, nearly empty, empty, DirectX's cache only, no LOCALAPPDATA, AMD, programs loaded before the title, a preparation, the switch either way) passes on this head on Windows (clang), and on Linux (g++ 13.3, WSL Ubuntu 24.04) with Linux's folder cases;tests/test_switch_table.pypasses.Suite:
tests/run_all.pywith every check on main @068fb8ebin WSL Ubuntu 24.04 (740 pass, 61 fail, 69 skipped: no clang or spirv-val there), then this head's--changedchecks in WSL (156 pass, 12 fail, 4 skipped) and on Windows (167 run, 27 fail). Every failure also fails on main @068fb8ebon the same OS: in WSL and on Windowstest_crash_dialog,test_dev_ab,test_dev_packs,test_draw_stream_visibility,test_frame_overlap,test_fsr_uniform_ownership,test_linux_pressure,test_pipeline_play_cache,test_prepare_cache_save,test_present_queue_default,test_upscaler_rows,test_vk_occlusion_queries; on Windows onlytest_draw_binds,test_driver_env_choose,test_image_arrays,test_linux_driver_fault_guard,test_mesh_fallback_layout,test_pass_hazard,test_pipeline_tour,test_precompile_compute_guard,test_push_state,test_same_draw,test_same_dyn,test_shader_module_guard,test_table_track,test_tracy_funcs,test_vertex_bind. (On Windows most of them are compile errors with this PC's compiler wrapper, on main as here.) These ran on the head as it was on main @068fb8eb; it was then rebased onto main @3f2c9b3c, whose two new commits change only the NFC menu (gpu/vk/menu/nfc.cppand its test), none of this PR's files.Still to test:
RX_VK_PCACHE_LOAD=1, expecting the log'snot readline, about 1 GB less private memory and fewer hard page faults unset, and the loads' and long frames' cost against that (by hand)nearly emptyin the log and the file read (by hand)Measurements
Verdict: on this 32 GB PC, where the rule keeps reading the file: leaving it unread took 1.0 GB of working set and 1.2 GB of private memory off and kept the steady frames, but made the loads at create about 4 times slower, sent about three times as many draws to run a load themselves, and gave the tour more frames of 33 ms or more (71 against 62). A trade only for a PC where the gigabyte is what makes the game page; not measured on one yet.
Setup: Windows 11 Pro 26200, RTX 4070 SUPER 12 GB (NVIDIA 617.42), Ryzen 9 9950X3D with SMT off, 32 GB; from
kokiri-start, a teleport to the Market and 150 s there (the working set's median over the last third of the log's memory lines, frames over the last 100 s), and a teleport tour of 7 areas, 20 s each; 2560x1440, no window (HOS_HEADLESS=1), uncapped, 16 cores; in both armsRX_VK_LOAD=create RX_VK_ASTC_BC_ALL=1 RX_VK_IMAGE_BLOCK_MB=32 RX_VK_BC_LAYERS=8 RX_VK_LATE_FIX=1 RX_VK_TEX_MOVE=1 RX_VK_TEX_AHEAD=2 HOS_NVN_FRAME_OVERLAP=1 RX_VK_FRAME_TRACE=1, and the loaded modules let go (#1844) on.Baseline: builds of main @
f505340b(before the tooling revamp068fb8eb, which changes none of this code) with #1810-#1813's changes, #1844 and this change. The stands: one build whose Windows arm had no RAM limit, so it left the file unread on this PC (After), against the same build withRX_VK_PCACHE_LOAD=1(Before, one stand); that build also held the temporary census, off. The tour: that build unset (After) against the final build, which reads the file here (Before). This PR's own default on this PC is the Before column.f505340bf505340bRX_VK_PCACHE_LOAD(a hw switch) keeps reading the file on every PC of more than 20 GB, as these numbers say for this one; at 20 GB or less with NVIDIA's own driver it follows Linux's rule, which was measured on Linux (#416), not yet on Windows.Risks
What could break: on a Windows PC with NVIDIA, 20 GB of RAM or less and programs loaded at create, every load at create builds its pipelines from NVIDIA's disk cache instead of the file's copy in memory: more draws wait on a load, and an area's first moments may hitch more, in exchange for about 1 GB of RAM. NVIDIA's folder is shared by every Vulkan and OpenGL program on the PC: 512 MB of another game's pipelines reads as warm, and then the pipelines Longshot needs compile in play, as on Linux. A driver update empties it, and then the file is read again. Turning it off:
RX_VK_PCACHE_LOAD=1. After updating, a player with such a PC has about 1 GB less of the game in RAM and may see slower first moments in an area; nothing changes for saves,longshot_options.inior the caches (the file is kept for a start that reads it).Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure. (Not on a PC of 20 GB or less, the only one it changes; on this 32 GB PC with the rule's limit lifted: one stand read, two not, and one tour a side.)third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def. (Its row is changed; the default at 20 GB or less follows Linux's measured rule and is not measured on Windows yet: Still to test.)tests/that reads changed source is updated, not weakened.gpu/vk/vk_draw.cppwas built on Windows for the measured builds, not yet on this head or on Linux; the test compilespcache_load.handpcache_load_offwith clang on Windows and g++ 13.3 on WSL Ubuntu 24.04. On this head the changed files pass a syntax-only compile with the builder's own Windows flags.)Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
WIP: perf(vk): the pipeline cache file not read on Windows with NVIDIA's driver and 20 GB of RAM or less, as on Linuxto perf(vk): the pipeline cache file not read on Windows with NVIDIA's driver and 20 GB of RAM or less, as on Linux