Shader programs, modules and pipelines are never released while the game runs, and the rule that would release a program has no caller #1509

Open
opened 2026-11-18 01:45:59 +00:00 by mooki · 1 comment
Member

Type: performance · Area: renderer, shaders, RAM · Part of: #1493

TL;DR

Everything the renderer loads for shaders (programs, shader modules, pipelines) stays in memory until the game exits, and the rule that would let a whole program unit go has no caller on main. In today's default start, on a 16 GB PC, private memory grows from 2.9 GB at the save load to 5.1 GB after eight areas (#388). Letting programs go at the game's free call was tried for that change and dropped (at most 8.5% of loaded units, for over 2,000 reloads). I propose to count what a release would give back first, then release only under memory pressure.

Breakdown

  • Who it affects: PCs with 16 GB of RAM and the Steam Deck, where the game's memory and the port's together outgrow RAM and swap fills (#388, #223). PCs with plenty of RAM do not feel it.
  • What is wrong: loaded programs, their modules and their pipelines are only ever added. The loader's table, the program lists and the pipeline map have no erase for a loaded unit. Modules are let go once, after the precompile, on Linux with 20 GB or less or under pressure. The release rule's only callers are its tests.
  • What we propose: (1) a log-only count at each census line: how many loaded units, and how many bytes, the rule would release, and how many an "unused for N presents" rule would. (2) If that is large, one release pass that runs only under memory pressure, like #223's module drop, as a pool in #1339's arbiter. (3) A released unit loads again through the path that exists (create call, then first use).
  • What does not change: the default start mode, the cache files and their stamp, and what the game draws. A draw that already holds a pipeline keeps it until the GPU is past it.
  • How we would prove it: a game-free test of the release pass on stand-in lists, then paired tours on a 16 GB PC: memory at the end, reloads, first-use waits, draws left out (0 today).

Evidence

Code, on main (310e7c9df):

  • gpu/vk/program_life.h:91-111: release_candidates(keep) (units that are Ready, have no live program, and were drawn; only the excess above keep, longest freed first) and released(unit). The header comment at :16-18 calls this "phase C". git grep finds callers only in tests/program_life_test.cpp.
  • The rule skips a unit that was loaded and never drawn (:96; "the game made it for something it has not shown"). In #388's tour only 3,859 of 10,686 built pipelines were taken by draws, so part of the stock is outside the rule by design. How much, the count in step 1 below says.
  • gpu/vk/vk_programs.cpp:595-615: the game's free call reaches table.free() (:601), which only lowers a count and stamps an order (program_life.h:84-90, 125-129). Nothing is erased.
  • vk_programs.cpp:285-297: a finished load is appended to programs. The only other places that add to it are vk_shader.cpp:1819, 2085, 2537 and vk_progcache.cpp:832. No code removes a loaded program.
  • vk_shader.cpp:2622-2626: the program-released call erases a small code record and nothing else.
  • Pipelines: vk_draw.cpp:11529-11540 keeps a pipeline made at a create call in kept_ready; an entry leaves only when a draw takes it (:2636-2643), never by destroy. The pipeline map's entries "live as long as the cache" and per-thread maps hold raw pointers to them (:2628). A release pass must handle those, so this is not a one-line change.
  • Modules: vk_draw.cpp:11422-11456 calls program_cache_drop_modules() (vk_progcache.cpp:985) once, on Linux, with 20 GB of RAM or less or under pressure. #388 measured 1.39 GB of module code "never let go" in the default mode.
  • The default: gpu/vk/load_default.h:23-32 loads at create calls, except with the compressed program cache, after a safety-net marker, or with RX_VK_LOAD=start.

Measurements (each from the source named):

  • Start mode, 5.2 of 6.8 GB: RTX 4090, Linux, Kokiri Forest, headless, 220 s, main bdfe09c (11 Nov), every listed program loaded before the title. 14,354 MB resident: 7,096 MB the game's window, 6,845 MB the port, of which 5,242 MB was shader stock: pipeline cache 1,015 MB, 23,641 modules 2,197 MB, 16,574 pipelines 2,030 MB. Allocator checkpoints in a diagnostic build. RX_VK_PRECOMPILE=0 took the process from 14,312 to 10,457 MB. This was before load-at-create became the default (86a9cda23, 12 Nov), so it is not what a default start holds today.
  • Growth with the list: three builds had 13.3, 15.1 and 16.8 GB resident, with 8,497, 13,173 and 17,858 programs and 10,775, 17,949 and 25,337 pipelines (start mode).
  • Default mode, 16 GB: #388, RTX 3070 Ti, main 0caf0618, 8-area tour of about 7 minutes. Private memory 2.9 GB at the save load, 5.1 to 5.2 GB after eight areas. At the end 7,391 programs were loaded, 10,686 graphics pipelines built, 3,859 taken by draws, plus a 1.2 GB pipeline-cache copy.
  • Default mode, RTX 3090: the load-at-create description (86a9cda23), 11-minute tour of five area visits, programs a list names only. Heap in use about 3.6 GB a minute in and 4.3 to 4.4 GB at the end, against 7.9 and 8.1 GB in start mode.
  • Releasing was tried: the same description lists "Releasing programs when the game frees them" under "Tried and dropped": at most 8.5% of the loaded units, for over 2,000 reloads, because the game frees and makes the same programs again within a second. So an always-on release looks like the wrong trigger, and the rule alone is a small win.

Not measured: a default start's memory on a PC with 24 GB or more at today's main; what each part is worth on Windows.

Proposed change

  1. Add the count: at each census line, call release_candidates(keep) under life_mu (it is const), add an "unused for N presents" count beside it, and log units and bytes (186 KB a program and 125 KB a pipeline were measured in one run; use the allocator's own sizes if they are at hand). No behaviour change.
  2. If the count is large, write the release pass in vk_programs.cpp: erase the unit's programs and pipelines from the lists, retire them by the GPU's timeline as the other dead lists do, bump a generation that the per-thread pipeline maps check, then call released().
  3. Call it only under pressure (Linux MemAvailable and swap rules, Windows commit headroom), and register it in the arbiter of #1339 so that one place decides who gives memory back first.
  4. Keep the first-use path as the way back: a draw that meets a released unit waits for its load, as for a unit that was never loaded.

How we would test it

  • Game-free, fails today: extend tests/program_load_wait_test.cpp (it already runs the loader on stand-in lists) with a release pass. It must remove a freed unit's programs from the stand-in list, leave a unit whose program is alive, and load the unit again on a new create call. The test cannot be written against main, because no such pass exists.
  • Game-free: with the pressure trigger, a unit is never released while the trigger is off.
  • In the game: a 16 GB Linux PC and a Steam Deck, the same tour as #388, main against the branch, interleaved, after a null main-against-main pair to show the run-to-run spread. Read private memory at the end, units reloaded, first-use waits, draws left out and the log's memory-pressure lines. Any speed claim needs paired timing under the project's quiet rule. This is a memory change and makes no speed claim.
  • #388 (open): the 16 GB reproduction this builds on. Its follow-ups name the modules, not the unit rule.
  • #223 (merged): Linux memory pressure and the module drop after the precompile. #416 (merged): pipeline cache file unread on Linux NVIDIA with 20 GB or less.
  • #1474 (open idea) and #1488 (open, work in progress): the module release and the cache-file rule on Windows. Neither touches programs or pipelines.
  • #1339 (open, work in progress): the budget line and an arbiter for memory given back under pressure. A program pool would register there.
  • #975 (merged) and #805 (open): load at create on the compute fallback; #805 measured 5 GB for loading everything before the title.
  • #1443 (open): memory growth in long sessions. #1493: the umbrella. Its line "5.2 GB of shader stock made before the title" describes start mode at bdfe09c; the default has loaded at create since 12 Nov.

New on this forge; not opened on GitHub.

**Type:** performance · **Area:** renderer, shaders, RAM · **Part of:** #1493 ## TL;DR Everything the renderer loads for shaders (programs, shader modules, pipelines) stays in memory until the game exits, and the rule that would let a whole program unit go has no caller on main. In today's default start, on a 16 GB PC, private memory grows from 2.9 GB at the save load to 5.1 GB after eight areas (#388). Letting programs go at the game's free call was tried for that change and dropped (at most 8.5% of loaded units, for over 2,000 reloads). I propose to count what a release would give back first, then release only under memory pressure. ## Breakdown - **Who it affects:** PCs with 16 GB of RAM and the Steam Deck, where the game's memory and the port's together outgrow RAM and swap fills (#388, #223). PCs with plenty of RAM do not feel it. - **What is wrong:** loaded programs, their modules and their pipelines are only ever added. The loader's table, the program lists and the pipeline map have no erase for a loaded unit. Modules are let go once, after the precompile, on Linux with 20 GB or less or under pressure. The release rule's only callers are its tests. - **What we propose:** (1) a log-only count at each census line: how many loaded units, and how many bytes, the rule would release, and how many an "unused for N presents" rule would. (2) If that is large, one release pass that runs only under memory pressure, like #223's module drop, as a pool in #1339's arbiter. (3) A released unit loads again through the path that exists (create call, then first use). - **What does not change:** the default start mode, the cache files and their stamp, and what the game draws. A draw that already holds a pipeline keeps it until the GPU is past it. - **How we would prove it:** a game-free test of the release pass on stand-in lists, then paired tours on a 16 GB PC: memory at the end, reloads, first-use waits, draws left out (0 today). ## Evidence Code, on main (310e7c9df): - `gpu/vk/program_life.h:91-111`: `release_candidates(keep)` (units that are Ready, have no live program, and were drawn; only the excess above `keep`, longest freed first) and `released(unit)`. The header comment at `:16-18` calls this "phase C". `git grep` finds callers only in `tests/program_life_test.cpp`. - The rule skips a unit that was loaded and never drawn (`:96`; "the game made it for something it has not shown"). In #388's tour only 3,859 of 10,686 built pipelines were taken by draws, so part of the stock is outside the rule by design. How much, the count in step 1 below says. - `gpu/vk/vk_programs.cpp:595-615`: the game's free call reaches `table.free()` (`:601`), which only lowers a count and stamps an order (`program_life.h:84-90, 125-129`). Nothing is erased. - `vk_programs.cpp:285-297`: a finished load is appended to `programs`. The only other places that add to it are `vk_shader.cpp:1819, 2085, 2537` and `vk_progcache.cpp:832`. No code removes a loaded program. - `vk_shader.cpp:2622-2626`: the program-released call erases a small code record and nothing else. - Pipelines: `vk_draw.cpp:11529-11540` keeps a pipeline made at a create call in `kept_ready`; an entry leaves only when a draw takes it (`:2636-2643`), never by destroy. The pipeline map's entries "live as long as the cache" and per-thread maps hold raw pointers to them (`:2628`). A release pass must handle those, so this is not a one-line change. - Modules: `vk_draw.cpp:11422-11456` calls `program_cache_drop_modules()` (`vk_progcache.cpp:985`) once, on Linux, with 20 GB of RAM or less or under pressure. #388 measured 1.39 GB of module code "never let go" in the default mode. - The default: `gpu/vk/load_default.h:23-32` loads at create calls, except with the compressed program cache, after a safety-net marker, or with `RX_VK_LOAD=start`. Measurements (each from the source named): - Start mode, 5.2 of 6.8 GB: RTX 4090, Linux, Kokiri Forest, headless, 220 s, main bdfe09c (11 Nov), every listed program loaded before the title. 14,354 MB resident: 7,096 MB the game's window, 6,845 MB the port, of which 5,242 MB was shader stock: pipeline cache 1,015 MB, 23,641 modules 2,197 MB, 16,574 pipelines 2,030 MB. Allocator checkpoints in a diagnostic build. `RX_VK_PRECOMPILE=0` took the process from 14,312 to 10,457 MB. This was before load-at-create became the default (86a9cda23, 12 Nov), so it is not what a default start holds today. - Growth with the list: three builds had 13.3, 15.1 and 16.8 GB resident, with 8,497, 13,173 and 17,858 programs and 10,775, 17,949 and 25,337 pipelines (start mode). - Default mode, 16 GB: #388, RTX 3070 Ti, main 0caf0618, 8-area tour of about 7 minutes. Private memory 2.9 GB at the save load, 5.1 to 5.2 GB after eight areas. At the end 7,391 programs were loaded, 10,686 graphics pipelines built, 3,859 taken by draws, plus a 1.2 GB pipeline-cache copy. - Default mode, RTX 3090: the load-at-create description (86a9cda23), 11-minute tour of five area visits, programs a list names only. Heap in use about 3.6 GB a minute in and 4.3 to 4.4 GB at the end, against 7.9 and 8.1 GB in start mode. - Releasing was tried: the same description lists "Releasing programs when the game frees them" under "Tried and dropped": at most 8.5% of the loaded units, for over 2,000 reloads, because the game frees and makes the same programs again within a second. So an always-on release looks like the wrong trigger, and the rule alone is a small win. Not measured: a default start's memory on a PC with 24 GB or more at today's main; what each part is worth on Windows. ## Proposed change 1. Add the count: at each census line, call `release_candidates(keep)` under `life_mu` (it is `const`), add an "unused for N presents" count beside it, and log units and bytes (186 KB a program and 125 KB a pipeline were measured in one run; use the allocator's own sizes if they are at hand). No behaviour change. 2. If the count is large, write the release pass in `vk_programs.cpp`: erase the unit's programs and pipelines from the lists, retire them by the GPU's timeline as the other dead lists do, bump a generation that the per-thread pipeline maps check, then call `released()`. 3. Call it only under pressure (Linux `MemAvailable` and swap rules, Windows commit headroom), and register it in the arbiter of #1339 so that one place decides who gives memory back first. 4. Keep the first-use path as the way back: a draw that meets a released unit waits for its load, as for a unit that was never loaded. ## How we would test it - Game-free, fails today: extend `tests/program_load_wait_test.cpp` (it already runs the loader on stand-in lists) with a release pass. It must remove a freed unit's programs from the stand-in list, leave a unit whose program is alive, and load the unit again on a new create call. The test cannot be written against main, because no such pass exists. - Game-free: with the pressure trigger, a unit is never released while the trigger is off. - In the game: a 16 GB Linux PC and a Steam Deck, the same tour as #388, main against the branch, interleaved, after a null main-against-main pair to show the run-to-run spread. Read private memory at the end, units reloaded, first-use waits, draws left out and the log's memory-pressure lines. Any speed claim needs paired timing under the project's quiet rule. This is a memory change and makes no speed claim. ## Related - #388 (open): the 16 GB reproduction this builds on. Its follow-ups name the modules, not the unit rule. - #223 (merged): Linux memory pressure and the module drop after the precompile. #416 (merged): pipeline cache file unread on Linux NVIDIA with 20 GB or less. - #1474 (open idea) and #1488 (open, work in progress): the module release and the cache-file rule on Windows. Neither touches programs or pipelines. - #1339 (open, work in progress): the budget line and an arbiter for memory given back under pressure. A program pool would register there. - #975 (merged) and #805 (open): load at create on the compute fallback; #805 measured 5 GB for loading everything before the title. - #1443 (open): memory growth in long sessions. #1493: the umbrella. Its line "5.2 GB of shader stock made before the title" describes start mode at bdfe09c; the default has loaded at create since 12 Nov. New on this forge; not opened on GitHub.
Collaborator

Possibly related, for context:

  • issue #388 Game: on 16 GB of RAM the game's memory outgrows RAM within a minute of loading a save, then swap fills on area changes until RAM is at 0 — Same 16 GB Linux memory growth context, but this proposes reclaiming shader programs and pipelines specifically
  • issue #1474 Idea: on Windows, hold less memory for pipelines after the start: the cache file rule (#416) and the module release (#223) Linux has, once measured — Related Windows proposal covers cache-file and module release, not program/pipeline eviction
  • issue #1493 Renderer rewrite — Parent renderer rewrite issue; this is a specific resource-lifetime proposal
  • PR #223 fix(vk): give memory back under Linux memory pressure and after the precompile — Merged Linux memory-pressure/module release mechanism that the proposal would extend
  • PR #1488 WIP: perf(vk): on Windows, the shader modules of ready programs are let go after the start on PCs with 20 GB of RAM or less — Open Windows module-release work, distinct from releasing programs and pipelines
Possibly related, for context: - issue #388 Game: on 16 GB of RAM the game's memory outgrows RAM within a minute of loading a save, then swap fills on area changes until RAM is at 0 — Same 16 GB Linux memory growth context, but this proposes reclaiming shader programs and pipelines specifically - issue #1474 Idea: on Windows, hold less memory for pipelines after the start: the cache file rule (#416) and the module release (#223) Linux has, once measured — Related Windows proposal covers cache-file and module release, not program/pipeline eviction - issue #1493 Renderer rewrite — Parent renderer rewrite issue; this is a specific resource-lifetime proposal - PR #223 fix(vk): give memory back under Linux memory pressure and after the precompile — Merged Linux memory-pressure/module release mechanism that the proposal would extend - PR #1488 WIP: perf(vk): on Windows, the shader modules of ready programs are let go after the start on PCs with 20 GB of RAM or less — Open Windows module-release work, distinct from releasing programs and pipelines
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1509
No description provided.