Idea: the game's 12288 x 12288 D16 shadow map takes 288 MB of VRAM on every GPU; a half-size one on cards with little VRAM #1295

Open
opened 2026-11-17 01:20:58 +00:00 by GrimVisage · 3 comments
Contributor

The idea

The game makes one 12288 x 12288 depth image, D16 (NVN 0x32, VK_FORMAT_D16_UNORM): 288 MB of VRAM, on every GPU, made once per area and kept while you are in it. On an 8 GB card that is about 4% of what the renderer is allowed to use (an RX 6600 in #1066 was allowed about 6,500-7,400 MB). The idea: on cards with little VRAM, draw and keep it at half the size, 6144 x 6144, 72 MB, 216 MB less, as an Auto choice by VRAM with a switch to turn it off.

What it is, from two diagnostic runs on main 1764fed2 (a local build only, not proposed for merging), RX 9070 XT, Linux, RADV, with the task + mesh compute fallback, child Link, Market, Kokiri Forest, Hyrule Field, Kakariko Village and Death Mountain Trail:

  • Not the sun's cascades. Those are 1600 to 3072 a side and follow the Shadow resolution row (host/oot_shadow.c). This one is fixed at 12288, a static shadow map of the area: about 90-170 draws when an area loads, then a few more while you stay, none in between.
  • Its draws cover the whole image. Every draw's viewport and scissor is 0,0 to 12288,12288 (475 draws in one run, one rectangle).
  • Most of it holds depth. Read back after the area's draws: 93% of the texels drawn in Kokiri Forest (95% of 256-pixel tiles), about 56% in the Market (65% of tiles, the same in five reads), 35% on Death Mountain Trail (37% of tiles). Hyrule Field and Kakariko Village gave no read: nothing sampled the image in the frame read. So backing only the drawn part with memory (sparse residency) would save little.
  • It is sampled as a depth comparison through normalized coordinates. Three programs read it (d8180bd8c6ecc77b, 489438d734e2463a, 531432c36205e5ce), each in one slot, as a 2D depth-compare texture: not by integer texel, not from a light table. That is what makes a smaller image possible: the same coordinates land on the same place in a 6144 image drawn with its viewport and scissor halved.
vk: texture 0x1147c16000 12288x12288x1 x1 levels 1 NVN 0x32 -> vk 124 (478 textures, 1821 MB)
vk: upload: 12288x12288 NVN 0x32 (288 MB) is all zero: cleared on the GPU (checked in 46.5 ms)
vk: atlas-diag: depth 0x1147c16000 12288x12288: 475 draws so far, drawn box 0,0..12288,12288, 2304 of 2304 tiles of 256 px touched (100.0%), 1 distinct rects
vk: atlas-diag: coverage sampled by 531432c36205e5ce #485: 12288x12288 clear 0xffff (0: 55, 0xffff: 9920894): 141074050 texels drawn (93.43%), box 0,0..12288,12288, 2188 of 2304 tiles of 256 px (95.0%)
vk: atlas-diag: coverage sampled by 489438d734e2463a #485: 12288x12288 clear 0xffff (0: 0, 0xffff: 66981398): 84013546 texels drawn (55.64%), box 0,0..12288,12288, 1504 of 2304 tiles of 256 px (65.3%)
vk: atlas-diag: program d8180bd8c6ecc77b samples #485 12288x12288 (slot 0, is_depth 1, type 2, integer 0, table 0, count 1)

The same image is in players' rc.5 logs: an RX 6600's in #1066 (288 MB: 0x1145a7e000 12288x12288x1 x1 levels 1 NVN 0x32 rendered), and #1182, #1193, #1217, #1287, and a GTX 1660 SUPER's (6 GB) in #1278; in #1183 it could not be made (image 12288x12288x1 x1 levels 1 format 124 (NVN 0x32) not made).

Score (docs/workflow.md): S2 (3), it is one part of a large frame-rate loss on 8 GB cards (#1066: 6.1-6.3 GB of about 6.5 GB used, textures over their budget, GPU time 59.4 ms a frame at 15 fps); reach 0.3, a guess for cards with 8 GB or less (RX 6000 alone is 16% of the issues that name hardware), not yet counted with scripts/hardware_report.py; confidence 1.0 that the image is there and its size (reproduced with logs here and in players' logs), but the fps it gives back is not measured. Score 0.9. Alone it won't fix #1066: it is 216 MB of the gap.

Not known yet, and checked before any pull request:

  • whether anything reads this image back to the game's memory, copies or blits it, or asks its size, since those would see the smaller image;
  • how much softer the static shadows look at half size: Before and After screenshots in the same places;
  • what it gives back in VRAM and fps on an 8 GB-class card on Windows.

Anything else

Claimed: GrimVisage will try this: a switch for the half-size image, then Auto by VRAM. Tested on an RX 9070 XT (Linux, RADV, native and compute fallback) and an RX 6800 XT (Windows, compute fallback by default, with RX_VK_TEST_VRAM_MB / RX_VK_TEST_ALLOWED_MB to play an 8 GB card's budget).

Prior work searched (forge search, merged commits, team chat, worklist): 12288, 12288x12288, shadow map, shadow atlas, D16, 288 MB, VRAM short, texture budget. Nothing does this. Related, other parts of the same VRAM problem: #1158, #1159, #1104, #518 (render targets and VRAM given back), #1256 (the characters' own shadow maps, NVN 0x34, a different image), #790 (made this image's zero upload cheaper), #637 (merged, VRAM management for cards under 8 GB; it doesn't touch this image). #211 (TrashPanda, WIP) is about this same image from the other side: it keeps it through texture evictions so the shadows don't go missing, capped at an eighth of the texture budget, which leaves out the 288 MB map on small budgets. A 72 MB one would fit that cap from a budget of about 576 MB, so the two would work together; this one doesn't change #211's rule. See #1066, #933, #975.

### The idea The game makes one 12288 x 12288 depth image, D16 (NVN 0x32, `VK_FORMAT_D16_UNORM`): **288 MB of VRAM**, on every GPU, made once per area and kept while you are in it. On an 8 GB card that is about 4% of what the renderer is allowed to use (an RX 6600 in #1066 was allowed about 6,500-7,400 MB). The idea: on cards with little VRAM, draw and keep it at half the size, 6144 x 6144, **72 MB, 216 MB less**, as an Auto choice by VRAM with a switch to turn it off. What it is, from two diagnostic runs on main 1764fed2 (a local build only, not proposed for merging), RX 9070 XT, Linux, RADV, with the task + mesh compute fallback, child Link, Market, Kokiri Forest, Hyrule Field, Kakariko Village and Death Mountain Trail: - **Not the sun's cascades.** Those are 1600 to 3072 a side and follow the Shadow resolution row (`host/oot_shadow.c`). This one is fixed at 12288, a static shadow map of the area: about 90-170 draws when an area loads, then a few more while you stay, none in between. - **Its draws cover the whole image.** Every draw's viewport and scissor is 0,0 to 12288,12288 (475 draws in one run, one rectangle). - **Most of it holds depth.** Read back after the area's draws: 93% of the texels drawn in Kokiri Forest (95% of 256-pixel tiles), about 56% in the Market (65% of tiles, the same in five reads), 35% on Death Mountain Trail (37% of tiles). Hyrule Field and Kakariko Village gave no read: nothing sampled the image in the frame read. So backing only the drawn part with memory (sparse residency) would save little. - **It is sampled as a depth comparison through normalized coordinates.** Three programs read it (`d8180bd8c6ecc77b`, `489438d734e2463a`, `531432c36205e5ce`), each in one slot, as a 2D depth-compare texture: not by integer texel, not from a light table. That is what makes a smaller image possible: the same coordinates land on the same place in a 6144 image drawn with its viewport and scissor halved. ``` vk: texture 0x1147c16000 12288x12288x1 x1 levels 1 NVN 0x32 -> vk 124 (478 textures, 1821 MB) vk: upload: 12288x12288 NVN 0x32 (288 MB) is all zero: cleared on the GPU (checked in 46.5 ms) vk: atlas-diag: depth 0x1147c16000 12288x12288: 475 draws so far, drawn box 0,0..12288,12288, 2304 of 2304 tiles of 256 px touched (100.0%), 1 distinct rects vk: atlas-diag: coverage sampled by 531432c36205e5ce #485: 12288x12288 clear 0xffff (0: 55, 0xffff: 9920894): 141074050 texels drawn (93.43%), box 0,0..12288,12288, 2188 of 2304 tiles of 256 px (95.0%) vk: atlas-diag: coverage sampled by 489438d734e2463a #485: 12288x12288 clear 0xffff (0: 0, 0xffff: 66981398): 84013546 texels drawn (55.64%), box 0,0..12288,12288, 1504 of 2304 tiles of 256 px (65.3%) vk: atlas-diag: program d8180bd8c6ecc77b samples #485 12288x12288 (slot 0, is_depth 1, type 2, integer 0, table 0, count 1) ``` The same image is in players' rc.5 logs: an RX 6600's in #1066 (`288 MB: 0x1145a7e000 12288x12288x1 x1 levels 1 NVN 0x32 rendered`), and #1182, #1193, #1217, #1287, and a GTX 1660 SUPER's (6 GB) in #1278; in #1183 it could not be made (`image 12288x12288x1 x1 levels 1 format 124 (NVN 0x32) not made`). **Score** (docs/workflow.md): S2 (3), it is one part of a large frame-rate loss on 8 GB cards (#1066: 6.1-6.3 GB of about 6.5 GB used, textures over their budget, `GPU time 59.4 ms a frame` at 15 fps); reach 0.3, a guess for cards with 8 GB or less (RX 6000 alone is 16% of the issues that name hardware), not yet counted with `scripts/hardware_report.py`; confidence 1.0 that the image is there and its size (reproduced with logs here and in players' logs), but the fps it gives back is not measured. Score 0.9. Alone it won't fix #1066: it is 216 MB of the gap. **Not known yet**, and checked before any pull request: - whether anything reads this image back to the game's memory, copies or blits it, or asks its size, since those would see the smaller image; - how much softer the static shadows look at half size: Before and After screenshots in the same places; - what it gives back in VRAM and fps on an 8 GB-class card on Windows. ### Anything else **Claimed:** GrimVisage will try this: a switch for the half-size image, then Auto by VRAM. Tested on an RX 9070 XT (Linux, RADV, native and compute fallback) and an RX 6800 XT (Windows, compute fallback by default, with `RX_VK_TEST_VRAM_MB` / `RX_VK_TEST_ALLOWED_MB` to play an 8 GB card's budget). Prior work searched (forge search, merged commits, team chat, worklist): `12288`, `12288x12288`, `shadow map`, `shadow atlas`, `D16`, `288 MB`, `VRAM short`, `texture budget`. Nothing does this. Related, other parts of the same VRAM problem: #1158, #1159, #1104, #518 (render targets and VRAM given back), #1256 (the characters' own shadow maps, NVN 0x34, a different image), #790 (made this image's zero upload cheaper), #637 (merged, VRAM management for cards under 8 GB; it doesn't touch this image). #211 (TrashPanda, WIP) is about this same image from the other side: it keeps it through texture evictions so the shadows don't go missing, capped at an eighth of the texture budget, which leaves out the 288 MB map on small budgets. A 72 MB one would fit that cap from a budget of about 576 MB, so the two would work together; this one doesn't change #211's rule. See #1066, #933, #975.
Collaborator

Possibly related, for context:

  • issue #586 Idea: Smaller Texture Size for 4GB VRAM — Prior feature request explicitly mentions halving the same 12288 shadow target, but lacks current diagnostics and VRAM-based proposal.
  • issue #939 Game: on a 6 GB card VRAM goes over the driver's allowance after a few area changes, Hyrule Field falls from 31 to 14 fps until a restart (GTX 1060, nightly-189) — Related low-VRAM performance symptoms on a 6 GB GPU.
  • PR #1256 perf(host): a character's shadow map reserves stepped storage, so the frame's other render targets keep their addresses (HOS_SHADOW_BUCKET) — Character shadow-map memory work; a different image and feature.
Possibly related, for context: - issue #586 Idea: Smaller Texture Size for 4GB VRAM — Prior feature request explicitly mentions halving the same 12288 shadow target, but lacks current diagnostics and VRAM-based proposal. - issue #939 Game: on a 6 GB card VRAM goes over the driver's allowance after a few area changes, Hyrule Field falls from 31 to 14 fps until a restart (GTX 1060, nightly-189) — Related low-VRAM performance symptoms on a 6 GB GPU. - PR #1256 perf(host): a character's shadow map reserves stepped storage, so the frame's other render targets keep their addresses (HOS_SHADOW_BUCKET) — Character shadow-map memory work; a different image and feature.
Author
Contributor

A correction to the issue: integer 0 in the atlas-diag: program line is eden's is_integer, whether the texture's format is an integer one, not whether the program reads by texel. The conclusion stands for another reason: eden gives a compare sample and a texel read of the same texture separate slots (TextureDescriptors differ by is_depth, ir_opt/texture_pass.cpp), and each of the three programs had one slot bound to this image, a depth-compare one. A size query (TXQ) shares the sampling slot, so the diagnostic can't rule one out; that stays under "Not known yet" and gets checked in the programs' SPIR-V before a pull request.

A correction to the issue: `integer 0` in the `atlas-diag: program` line is eden's `is_integer`, whether the texture's *format* is an integer one, not whether the program reads by texel. The conclusion stands for another reason: eden gives a compare sample and a texel read of the same texture separate slots (`TextureDescriptor`s differ by `is_depth`, `ir_opt/texture_pass.cpp`), and each of the three programs had one slot bound to this image, a depth-compare one. A size query (TXQ) shares the sampling slot, so the diagnostic can't rule one out; that stays under "Not known yet" and gets checked in the programs' SPIR-V before a pull request.
Author
Contributor

An update on the plan: half size is not the default for 8 GB cards any more.

The branch is up: perf/half-size-static-shadow on GrimVisage's fork, 7e080be9 on main 6d72ca93. The map can be made at 75 % or 50 % of its side, with its draws' viewport, scissor and clears scaled by as much (RX_VK_SHADOW_MAP_SIZE=100, 75 or 50). Auto, by what the driver allows the process:

Allowed Map VRAM back
under 6 GB 6144 x 6144 (50 %) 216 MB
under 8.5 GB, or an integrated GPU 9216 x 9216 (75 %) 126 MB
more, or unknown 12288 x 12288, as now none

Why not half everywhere: these shadows aren't only far away. The grass's and branches' shadows on the ground near Link come from this map. At 50 % they are visibly coarser and stepped (RX 9070 XT, Linux, main 1764fed2 with the change, 2560x1440, the same spots at 100, 75 and 50; Death Mountain Trail shows it best). At 75 % they are close to the game's. Every size drew the shadows in the same places in the Market, Kokiri Forest, Hyrule Field, Kakariko Village and Death Mountain Trail, so nothing that reads the map is thrown off by the smaller image, and nothing copied it, uploaded into it or wrote it back in those runs (the branch logs it if anything does).

Still to come before a pull request: VRAM and fps on main against the branch, here and on an RX 6800 XT (Windows, compute fallback) with an 8 GB card's allowance played.

An update on the plan: half size is not the default for 8 GB cards any more. The branch is up: `perf/half-size-static-shadow` on GrimVisage's fork, 7e080be9 on main 6d72ca93. The map can be made at 75 % or 50 % of its side, with its draws' viewport, scissor and clears scaled by as much (`RX_VK_SHADOW_MAP_SIZE=100`, `75` or `50`). Auto, by what the driver allows the process: | Allowed | Map | VRAM back | |---|---|---| | under 6 GB | 6144 x 6144 (50 %) | 216 MB | | under 8.5 GB, or an integrated GPU | 9216 x 9216 (75 %) | 126 MB | | more, or unknown | 12288 x 12288, as now | none | Why not half everywhere: these shadows aren't only far away. The grass's and branches' shadows on the ground near Link come from this map. At 50 % they are visibly coarser and stepped (RX 9070 XT, Linux, main 1764fed2 with the change, 2560x1440, the same spots at 100, 75 and 50; Death Mountain Trail shows it best). At 75 % they are close to the game's. Every size drew the shadows in the same places in the Market, Kokiri Forest, Hyrule Field, Kakariko Village and Death Mountain Trail, so nothing that reads the map is thrown off by the smaller image, and nothing copied it, uploaded into it or wrote it back in those runs (the branch logs it if anything does). Still to come before a pull request: VRAM and fps on main against the branch, here and on an RX 6800 XT (Windows, compute fallback) with an 8 GB card's allowance played.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1295
No description provided.