WIP: perf(vk): a BC3-kept ASTC texture's levels re-encode in one run, and a GPU-drawn target is not uploaded again for a buffer write (test switches) #1987
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1987
Loading…
Reference in a new issue
No description provided.
Delete branch "Veix/Longshot:perf/bc-levels-one-run"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Work in progress, for others to try: opt-in BC3 mip batching and texture diagnostics. The batching switch is off unless set. Nothing here is ready for review; I'd like to know whether it holds on other GPUs.
RX_VK_BC_LEVELS=1: a BC3-kept ASTC texture's levels are re-encoded in groups. Today each level of such a texture is its own run: decode into the scratch, full barrier, encode into the output buffer, full barrier, copy into the image, full barrier. With the switch, a single-layer texture's levels are decoded into up to eight small scratch layers and one big scratch, then encoded to separate offsets and copied together. Each group has three full barriers and is also limited by output-buffer capacity. It is #1843's idea applied to the levels of one texture.Not covered: layer arrays (they keep #1843's path), BC7, resized textures (
bc_resized), and a card where the small scratch is not made.Also in the diff are the frame trace's texture step columns (
create_ms,reuse_ms,astc_stage_ms,astc_record_ms,astc_view_ms,astc_flush_ms,astc_flushes,create_n) and render target pool columns (pool_reused,pool_new,pool_resized,pool_put,pool_idle,pool_destroyed,pool_holds,dead_waiting,compact_pass), plusRX_VK_LOG_REWRITES,RX_VK_LOG_MADE_BURSTandRX_VK_LOG_POOL.What I test with. The numbers and play below are with these set, beside the batching switch:
RX_VK_TARGET_REUSE_MEMORY=0(see the finding below)RX_VK_COMPACT_PCT=60, to play a smaller card on a 12 GB one (off for the 4K DLAA sessions where said)RX_VK_READBACK_LATE=3980,9b040,8b040,44b040,1b580,RX_VK_FRAME_GATE=0,HOS_FRAME_AHEAD=0HOS_KEEP_FREED_PAGES=1,RX_VK_LOAD=start,RX_VK_CAPTURE_NT=1,HOS_NATIVE_ZSTD=1HOS_FPS_MAX=1000(uncapped), V-Sync off;RX_VK_FRAME_TRACE=1andHOS_FPS_LOG=1for the numbersA64_LOCAL_FUNCS=all,LONGSHOT_CHUNK_OPT=all, and play profiles of my own (RX_PGO,RX_VK_PGO, recorded ona104182b)RX_VK_POOL_IDLE=32(frames that make 40 or more render targets new 12 -> 4) andRX_VK_DEAD_MIN_UNUSED=8(MB uploaded a second 40.8 -> 32.3); neither changed the frames over 20 msA finding on #1681 on the way (
RX_VK_TARGET_REUSE_MEMORY, on by default since it merged). On this PC it makes render target creation slower, not faster, in the market. Same build (this PR's code on80bb1ccc,RX_VK_BC_LEVELS=1), the second market stay, one run a side:RX_VK_TARGET_REUSE_MEMORYunset (1)=0a104182b, before #1681create()over the stayIt does what it says (about 30 % fewer images made new), but a take by size still makes a new image on the pooled memory, and the status line of that run reads
3373 reused by size ... 692 missed after their image was taken by size. The reason for the misses: in the market the game makes its whole set of about 65 targets again not every frame but every 1 to 24 frames (the two alternating layouts above), so a shape that waits more thankSizeTakeIdle(1 present) is often one that is coming back. The same pattern empties the pool by its 8-present idle rule before the set returns. On an integrated GPU, where an allocation is far dearer, the trade may still pay; I have not measured one.Read first:
upload_astc_gpuingpu/vk/vk_texture.cpp(group,group_flush,bc_small_layer).To try it: put
RX_VK_BC_LEVELS=1inlongshot_env.txt, turn the camera in Kokiri Forest and Castle Town, and walk through an area entry. What I'd most like to hear is whether anything is wrong on far surfaces (the small levels).See #1843, See #1833, See #1720, See #1681, See #1256
Testing
Tested on earlier builds, not this head: Windows 11, Ryzen 9 9900X, RTX 4070 SUPER 12 GB (NVIDIA 621.24), on PR code based on main @
80bb1cccand before that ona104182b, with the switches listed under “What I test with” above, includingRX_VK_BC_LEVELS=1. Those earlier builds also contained the since-removedRX_VK_TEST_KEEP_DRAWNexperiment. A recorded walk (Castle Town market, Hyrule Castle, the market again, with camera turns) at 3840x2160 with DLSS Quality, uncapped, V-Sync off,RX_VK_COMPACT_PCT=60; and played by hand in Kokiri Forest, the market, Hyrule Castle and Hyrule Field at DLSS Quality and at DLAA, about half an hour in all. No wrong texture seen by eye; one pair of screenshots from the same spot in the market looks the same (the moving things aside).tests/test_switch_table.pypasses. Nothing else of the suite was run.Still to test:
vk_texture.cppabout grid lines on an RX 6600 with multi-layer runs): Kokiri Forest and Hyrule Field withRX_VK_BC_LEVELS=1, expecting far ground and foliage as on main (by hand)RX_VK_BC_LEVELS=1, expecting no error and a shorter entry (by hand)Measurements
Verdict: on this PC the GPU time of an ordinary BC3-kept texture falls from about 0.5 ms to 0.1 ms, the market walk's frames over 20 ms about halve, and area entries by hand stall for 0.1-0.4 s where they stalled for 0.6-1.0 s. One or two runs a side, not interleaved: a first look, not a measurement to the standard.
Setup (the runs were made on this PR's code before it was squashed into
365c7a1c; the squash only added the switches' rows): Windows 11, Ryzen 9 9900X, RTX 4070 SUPER 12 GB (NVIDIA 621.24), 64 GB; 3840x2160 with DLSS Quality, uncapped, V-Sync off,RX_VK_COMPACT_PCT=60,HOS_NATIVE_ZSTD=1; the recorded walk, the frame trace; the window: the second market stay from 5 s after arriving (about 55 s, 4,000 frames).80bb1ccc365c7a1c,RX_VK_BC_LEVELS=1On
a104182b, three runs a side: frames over 16.7 ms 139-164 against 85-99, over 20 ms 17-24 against 11.GPU time a texture, by
RX_VK_PASS_TIMING=1(its timestamps slow the GPU: read the ratio), two short runs:RX_VK_BC_LEVELS=1Risks
What could break: with
RX_VK_BC_LEVELS=1, a texture's small levels could be wrong on a GPU where runs over several scratch layers misbehave (the RX 6600 note; this version writes each layer through its own view, one dispatch a level). To turn batching off, leave the switch unset, the default. After updating: nothing changes for saves, options or the caches.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Tested in game: Windows 11, Ryzen 9 9900X, RTX 4070 SUPER (NVIDIA 621.24): the Castle Town market walk and about half an hour by hand in Kokiri Forest, the market, Hyrule Castle and Hyrule Field, at 4K DLSS Quality and DLAA, with both switches; no wrong texture seen.
Possibly related, for context:
Recommendation: Request changes. Mip batching has a promising timing signal, but a failed mip upload can leave scratch writes unordered. The separate upload-suppression rule can discard valid buffer writes, and the new frame trace can race with texture readback.
The BC3 change reduces barriers during single-layer ASTC uploads and could shorten texture hitches without growing the scratch images or output buffer. Keep the batching change focused. The separate buffer-write experiment needs a storage-lifetime rule, and the diagnostics add CPU work to ordinary uploads even with tracing off.
@fixwill do (21)gpu/vk/vk_texture.cpp:1510: A late mip failure can abandon a group after its decoder dispatches are recorded. If aligned staging fails on mip 1 after mip 0 writes small scratch layer 0, another texture in the same upload command buffer can write layer 0 without an intervening barrier. The upload retry protects the destination texture only. Validate and stage each group before recording, or order its outstanding scratch writes on every failure exit.gpu/vk/vk_texture.cpp:952: Failure to create one optional mip-batching view destroys the valid small scratch and its array view. (more on the line)gpu/vk/vk_texture.cpp:3591: A prior draw or a pool key does not establish who owns the storage now. (more on the line)gpu/vk/vk_replay.cpp:4921: The trace snapshots plain pool counters and the dead vector before do_present takes replay_mu. (more on the line)gpu/vk/vk_texture.cpp:3454: RX_VK_LOG_REWRITES=2 hashes live guest memory after the upload. (more on the line)gpu/vk/vk_texture.cpp:3472: Rewrite history combines distinct textures that reuse an address, dimensions and format. (more on the line)gpu/vk/vk_texture.cpp:3594: The kept count increments before the dirty scan, so an overlapping CPU write can still trigger an upload, including a whole upload for a released target. (more on the line)gpu/vk/vk_texture.cpp:1315: The new counters always take clock readings and update atomics in texture creation, reuse and ASTC recording, including each decode run. (more on the line)gpu/vk/vk_texture.cpp:1523: A capacity-triggered group flush happens inside the view timer. (more on the line)gpu/vk/vk_texture.cpp:5272: The pool telemetry cannot distinguish image creation from allocation reuse. (more on the line)gpu/vk/vk_texture.cpp:3152: Burst logging waits for the next successful texture creation to report the preceding burst. (more on the line)gpu/vk/vk_replay.cpp:5224: Skipping texture_frame also skips the frame increment used by other systems. (more on the line)gpu/vk/vk_texture.cpp:3213: A negative arrival cap such as -1 can keep eligible textures on the placeholder indefinitely because replenishment never makes the balance positive. (more on the line)gpu/vk/vk_texture.cpp:5688: RX_VK_POOL_IN_PASS can hand an image to the pool and immediately trim it while compaction is open. (more on the line)tests/test_astc_upload_chunking.py:541: The fixture extracts the whole production upload function without the new timing helpers, group scratch state or size constant. Its spy also expects one mip copy immediately after each encode. Update the fixture to build on Windows and Linux with the selected compiler and exercise both switch states in a clean environment. Assert per-mip scratch layers and BC3 ranges, the grouped copy and barriers, chunk splits, and a later mip failing before scratch is reused.tests/test_texture_staging_publication.py:34: The extracted gather prefix now calls step_now and updates step_ns, neither of which is in the fixture. Keep the instrumentation outside the extracted prefix or supply its dependencies while preserving the noncoherent publication check.tests/test_frame_trace_waits.py:42: The test requires the header to end at ab_side, so it rejects the new CSV. If the columns remain, extend the expected header while retaining the existing-order and row-width checks.tests/test_image_reaper.py:39: This check and tests/test_vram_compact.py still require the original unconditional compaction break. Removing RX_VK_POOL_IN_PASS restores both. If the experiment stays, cover completed pooled entries, unfinished entries and entries that may free allocations with a check that enforces the lifetime rule in both modes.gpu/vk/switches.def:280: The row and PR describe a maximum of eight levels per group. (more on the line)For the author (1) (a run, a measurement or a decision)
80bb1cccwith matched builds and at least two interleaved runs per side. Include a normal preset and default reuse policy, average fps, 1% low, median and p99 frame time, area-entry stalls and RAM/VRAM. Check Tier 1 drivers and Steam Deck, including a low-memory configuration that falls back without small scratch. The Windows hand-play result leaves small-mip image correctness on AMD and Linux open. A replacement refill rule needs its own matched image and upload comparison.Checks: Still running:
PR description / Description follows the template (pull_request_target),CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The reported Windows RTX 4070 SUPER market run had 24 → 11 frames over 20 ms, a 48.5 → 21.9 ms worst frame, and 72.9 → 73.7 fps. Short instrumented runs reported 461–529 → 102 µs for single-layer ASTC-to-BC3 uploads up to 1024. The market comparison used pre-squash code and one or two non-interleaved runs per side. The older three-run comparison used a different base. Keep-drawn reported 454 → 162 MB/s in hand-played 4K DLAA windows with different present counts. Neither result establishes the gain for the final head at normal settings.
Other systems: Arrays, BC7, resized BC and configurations without the small scratch retain the per-level encoding path. All ASTC GPU upload routes still incur the new instrumentation. The texture clock also changes frame identity for temporal inputs, persistent-target detection and capture-span aging. Pool diagnostics touch synchronous readback and invalidation state. The CSV keeps existing columns in order, and the in-tree A/B readers select columns by name. Low-VRAM fallback, AMD Windows and RADV, NVIDIA Linux and Steam Deck lack results. The diff has no direct save, audio, input, launcher, build, shader or cache-format changes.
Switches: RX_VK_BC_LEVELS defaults off. Its shipping default needs same-head measurements by device and driver. RX_VK_TEST_KEEP_DRAWN defaults off, which avoids the stale-content case until the rule is replaced. RX_VK_ARRIVE_MB defaults off, RX_VK_ARRIVE_ALL defaults off and only applies with the arrival budget, RX_VK_POOL_IN_PASS defaults off, RX_VK_TEX_CLOCK_HZ defaults to 0 (one tick per present), and RX_VK_TEST_BC_SKIP_BELOW defaults off. Remove these unsuccessful experiments. RX_VK_LOG_REWRITES, RX_VK_LOG_MADE_BURST and RX_VK_LOG_POOL default off, and RX_VK_LOG_MADE_BURST_FROM defaults to 0. Those are appropriate diagnostic defaults once the logging is accurate and safe. Existing RX_VK_FRAME_TRACE remains off by default and gains 16 columns. RX_VK_TARGET_REUSE_MEMORY is unchanged. Its existing default is 2 on integrated GPUs and 1 elsewhere.
Reviewed:
365c7a1c28against80bb1ccc18: Read the complete four-file diff and its single commit, AGENTS.md, CONTRIBUTING.md, the Forgejo template, and the standards. No review-checklist file was present. Read the changed functions and their relevant scratch, barrier, recorder, capture, texture lifetime, pool and readback code, the BC encoder, the ASTC dispatch entry point, and the affected test assertions and fixture assembly. Read the supplied comment, measurement tables and CI state. Skimmed the unaffected parts of the renderer files, switch table, reporting scripts, and temporal and capture consumers. No raw runs or screenshots were supplied. No translations or generated files changed.; nothing run (a code review).Checked and fine:
Testing: Read-only review. The author reports Windows gameplay and a switch-table pass. Full-suite and Tier 1 coverage are pending. CI was still running with no reported failures. The fixture and assertion issues above come from source inspection.
Overlaps and merge order: #1843 and #1681 are prerequisites already present in the base. The #1681 reuse measurements concern existing policy and can be followed up separately. No pending sibling or merge-order dependency was identified in the supplied material.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -275,6 +277,7 @@ SW(RX_VK_BATCH_MAX, tune, int, "32 command buffers", "The most command buffers hSW(RX_VK_BATCH_POLL_US, tune, int, "2000 us", "A progress poll sends held submits only once the oldest has waited this many microseconds (0: always).")SW(RX_VK_BC_IDLE_S, tune, int, "0: the BC scratch images are kept once made", "Seconds of disuse after which the ASTC-to-BC scratch images are given back; 0 never.")SW(RX_VK_BC_LAYERS, tune, int, "1 (clamped 1-8)", "How many layers the 2048 x 2048 ASTC-to-BC scratch images have, which is how many one run decodes and encodes at a level over 1024.")SW(RX_VK_BC_LEVELS, attempt, bool, "off: each level of a BC-kept ASTC texture is its own decode, encode and copy, three full barriers each", "=1: a single-layer BC3-kept ASTC texture's levels are decoded one into each layer of the small scratch (a level over 1024 into the big one), encoded each to its own place in the output buffer and copied with one command: three full barriers a group of up to eight levels.")L20 (minor): The row and PR describe a maximum of eight levels per group. The code allows one big level plus eight small ones when bc_out has room. State both limits and the buffer cap, or enforce an eight-level total. If the trace columns stay, document their units and which are cumulative deltas or snapshots.
@ -4914,0 +4918,4 @@texture_step_times(sp);static int64_t l_pc[8]; // (TEST, local: the render target pool's counts, vk_texture.cpp)int64_t pc[8];texture_pool_counts(pc);L4: The trace snapshots plain pool counters and the dead vector before do_present takes replay_mu. A game-thread invalidation or virtual remap can hold replay_mu and append a timed-out writeback buffer to dead while the trace calls dead.size(). The ReadTexels diagnostic can also create a target and change the pool counters under that mutex. Take the snapshot under replay_mu or publish a coherent snapshot from the protected path.
@ -5188,0 +5221,4 @@logf("vk: (TEST) texture clock %d Hz: %llu ticks in %llu presents", clock_hz, (unsigned long long)clock_ticks,(unsigned long long)clock_presents);}if (tick) texture_frame(completed());L12: Skipping texture_frame also skips the frame increment used by other systems. At 120 fps with a 60 Hz texture clock, two presents can share a frame number. For example, temporal camera checks can accept a previous present's camera for the same depth image, and persistent-target detection can treat the previous present as the current one. Remove the experiment or keep present identity and submission-completion bookkeeping per present and throttle only the intended maintenance.
@ -940,1 +949,4 @@if (vkCreateImageView(dev.device, &sv, nullptr, &bc_small_view) != VK_SUCCESS) why = "has no view";for (uint32_t l = 0; !why && bc_levels_on() && l < bc_small_layers(); l++) { // (TEST, local, RX_VK_BC_LEVELS)sv.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, l, 1};if (vkCreateImageView(dev.device, &sv, nullptr, &bc_small_layer[l]) != VK_SUCCESS) why = "has no view of a layer";L2: Failure to create one optional mip-batching view destroys the valid small scratch and its array view. An array upload then falls back to the big scratch and, at the default layer counts, goes from up to eight layers per run to one. On partial view failure, destroy only the new views and disable mip batching for that scratch allocation.
@ -1284,0 +1312,4 @@// 0 create(), 1 the reuse hash and copy, 2 the ASTC blocks staged, 3 the decode's recording (without 4 and 5), 4 the level views// made, 5 the upload batch handed over mid-frame (replay_flush_upload), 6 how many such hand-overs, 7 how many create()std::atomic<int64_t> step_ns[8];int64_t step_now() { return std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now().time_since_epoch()).count(); }L8: The new counters always take clock readings and update atomics in texture creation, reuse and ASTC recording, including each decode run. An area entry with frame tracing disabled still pays for these diagnostics. Gate collection on the trace switch and reuse the existing reuse-upload timing where possible.
@ -1429,0 +1520,4 @@const bool in_group = grouped && (small || (g.w <= kBcDim && g.h <= kBcDim));if (in_group) {const uint64_t need = (uint64_t)((std::max(1u, tex.width >> level) + 3) / 4) * ((std::max(1u, tex.height >> level) + 3) / 4) * 2;if ((!small && group_big) || (!group.empty() && group_off + need > group_cap)) group_flush();L9 (minor): A capacity-triggered group flush happens inside the view timer. A big mip that fills bc_out can make the next level's astc_view_ms include encoder dispatches, barriers and a copy while subtracting that time from astc_record_ms. Time the view selection after any capacity flush.
@ -3024,0 +3149,4 @@static uint64_t burst_frame = 0;static std::vector<std::string> burst_lines;static int bursts_logged = 0;if (burst_frame != frame_no) {L11 (minor): Burst logging waits for the next successful texture creation to report the preceding burst. An area entry followed by steady play can leave the burst unreported for the rest of the run. It also formats and stores rows past the 160-row output cap and after the 40-report quota. Close the report at present, count all creations separately, and stop collecting rows at the documented limits.
@ -5347,6 +5686,34 @@ void texture_frame(uint64_t completed) {it = dead.erase(it);}pool_trim();L14: RX_VK_POOL_IN_PASS can hand an image to the pool and immediately trim it while compaction is open. At critical pressure, the pool cap is zero. With RX_VK_REAPER=0, trimming destroys the image synchronously despite the experiment's promise to only move it to the pool. Drop the experiment or enforce the no-destruction rule through both pool insertion and trimming.
@ -3064,0 +3210,4 @@// A new texture met with the allowance used up waits (the draw samples the placeholder: the hard stand-in) and is looked at// again the next frame. It goes over the ms budget below. RX_VK_ARRIVE_ALL=1: layer arrays and the big ones wait too ----int64_t arrive_cap() {static const int64_t v = getenv("RX_VK_ARRIVE_MB") ? (int64_t)(atof(getenv("RX_VK_ARRIVE_MB")) * 1048576.0) : 0;L13: A negative arrival cap such as -1 can keep eligible textures on the placeholder indefinitely because replenishment never makes the balance positive. NaN and out-of-range values reach an invalid floating-to-integer conversion, and large positive caps can overflow the addition before it is clamped. Remove the experiment or validate a finite positive range and saturate the balance before adding.
@ -3219,0 +3451,4 @@sig = mix(sig, off), sig = mix(sig, n);b.resize((size_t)((n + 7) & ~7ull));if (n & 7) memset(b.data() + (n & ~7ull), 0, 8);if (!host->gpu_read(tex.gpu + off, b.data(), (size_t)n)) return;L5: RX_VK_LOG_REWRITES=2 hashes live guest memory after the upload. With replay behind the game, this can hash the next submission's bytes instead of the bytes uploaded. The direct read also checks the guest window rather than commitment, so decommitted backing can fault. Remove mode 2 or hash the actual staged input and mark failed or unavailable reads as unknown.
@ -3219,0 +3469,4 @@return;}rewrites_win.again_n++, rewrites_win.again_bytes += bytes;RewriteStat& s = rewrites[{tex.gpu, tex.width, tex.height, tex.nvn_format}];L6 (minor): Rewrite history combines distinct textures that reuse an address, dimensions and format. A replacement with more layers inherits the old dimensions, hash history and streak. Every new key also stays in the map for the session. Track texture identity, retire inactive entries, and keep any aggregate history bounded.
@ -3285,0 +3588,4 @@{ // (TEST, local, RX_VK_TEST_KEEP_DRAWN=1: above)static const bool keep_drawn = env_on("RX_VK_TEST_KEEP_DRAWN");static uint64_t kept_n, kept_bytes, kept_said;if (keep_drawn && gpu_dirty_found && (tex->gpu_written || tex->first_render_frame || tex->pool_key)) { // (drawn now or ever, or made as a target: a refill makes it 'not GPU-written', and the next one would pass)L3: A prior draw or a pool key does not establish who owns the storage now. After a target is finalized and a GPU buffer legitimately fills the same range, sampling a matching texture can return the old Vulkan image and permanently mark the real write seen. Preserve refills unless registration lifetime and storage ownership prove that the overlap belongs to a different resource. Remove the blanket rule from the mip-batching PR.
@ -3285,0 +3591,4 @@if (keep_drawn && gpu_dirty_found && (tex->gpu_written || tex->first_render_frame || tex->pool_key)) { // (drawn now or ever, or made as a target: a refill makes it 'not GPU-written', and the next one would pass)tex->gpu_wrote_seen = gpu_dirty_found; // (seen, and not taken)gpu_dirty_found = 0;kept_n++, kept_bytes += tex->guest_size;L7 (minor): The kept count increments before the dirty scan, so an overlapping CPU write can still trigger an upload, including a whole upload for a released target. gpu_wrote_into also logs that the texture was uploaded before the caller decides. Log and count the outcome after the decision, with separate counts for whole uploads, partial uploads and uploads actually avoided.
@ -4941,0 +5269,4 @@/// (TEST, local: the frame trace's pool columns) taken from the pool, made new, put back, gone idle, destroyed in all (running/// counts); what the pool holds now, what waits for the GPU, whether a compaction pass is openvoid texture_pool_counts(int64_t out[8]) {out[0] = (int64_t)st_pool_reused, out[1] = (int64_t)st_pool_made, out[2] = (int64_t)st_pool_put, out[3] = (int64_t)st_pool_idle_out;L10 (minor): The pool telemetry cannot distinguish image creation from allocation reuse. A take by size creates a new image but increments neither pool_reused nor pool_new, and an outright allocation failure can increment pool_new. The burst logger also calls a take by size 'new (not pooled)'. Report exact reuse, new images on reused memory, and new allocations separately, and count successful creation.
@fix
Let's make it so it doesn't need flag and works correctly for everyone
I changed this pull request as the review asked: the new head is
623507146b. I rebased it ontomain(ed602b9468).Changed
Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
365c7a1c28623507146b@Beanstalk
Beanstalk test farm · an earlier run: commit
6235071(the newer results are in a comment below)Commit
6235071· approved by HumanBean (reviewed) · dashboard⚠️ Note: The latest diff removes the keep-drawn switch, so its proposed upload-burst comparison is not possible; the farm also cannot set 4K output or use DLAA.
⚠️ Note: A run may not encounter an eligible BC3 texture or exercise scratch-allocation failure, so a passing result does not prove those paths were covered.
Started at 06:09 UTC.
Estimate: done in about 30 min, from the queue and the testers online now; some jobs can't start yet.
amd+gpu-rx-6*(GPU "rx 6*")vram8gb(8 GB VRAM)steamdeckWhy it failed
vram8gb:texture-turns-and-entries · RX_VK_BC_LEVELS=1 (PR only)screenshot.max_diff:limit 0.1What was tested
What will be tested · AI plan (
gpt-6.1-terra)Compare main with the pull request in Kokiri Forest, Hyrule Field and the Market, with BC3 mip batching off and on. Camera turns and area entries exercise texture uploads; screenshots, frame hitches and load times catch visual or performance regressions. Each tester needs roughly 25 minutes. The RDNA 2 target will wait for a matching tester. No tester online now matches amd+gpu-rx-6*: that target waits for one.
1(PR only).Measurements on `vram8gb`, for the PR description
Verdict: fail on vram8gb (1 report):
texture-turns-and-entries screenshot.max_difffailed.Hardware: 1: linux+amd+radv+dgpu
AMD Radeon RX 580 Series (RADV POLARIS10), 26.2.7AMD Ryzen 5 5500F 6-Core Processor, 32 GB RAMScene: texture-turns-and-entries, from a new save; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); texture-turns-and-entries: 1 game start a side, market 20 s after 5 s warm-up; each number the median of the runs, then of 1 report
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)ed602b96235071RX_VK_BC_LEVELS=1 (PR only)· longest load (s)RX_VK_BC_LEVELS=1 (PR only)· market · frame time (p99) (ms)RX_VK_BC_LEVELS=1 (PR only)· worst frame time in the whole run (ms)Flags and settings
RX_VK_BC_LEVELS1As Markdown for the PR description:
The testers' hardware
vram8gbAMD Radeon RX 580 Series (RADV POLARIS10), 26.2.7AMD Ryzen 5 5500F 6-Core Processor, 32 GB RAMRecommendation: Request changes. The refill suppression is gone and failed mip uploads now order their scratch writes, but the ASTC regression fixture still cannot compile and the mip-skip experiment remains. The older NVIDIA runs show promise, while the gain and a safe default for the rebased build remain unproven.
For eligible single-layer ASTC textures kept as BC3, the patch batches mip levels to reduce GPU barriers and copy commands. It uses existing scratch allocations, adds per-layer views and group bookkeeping, and retains optional texture and pool diagnostics. The blanket suppression of buffer-triggered refills was removed.
@fixwill do (10)gpu/vk/vk_texture.cpp:3414: A replacement texture at the same address with matching width, height and format but more layers still inherits the old texture's dimensions and streak. Key the history by texture identity, retire inactive entries and bound retained history.gpu/vk/vk_texture.cpp:4646: After a GPU buffer overlap, an upload that fails staging can still produce an 'uploaded again' message. Move that message to the successful upload path and use the actual whole or partial outcome there.gpu/vk/vk_texture.cpp:1523: A capacity-triggered group flush happens inside the view timer. A big mip that fills bc_out can make the next level's astc_view_ms include encoder dispatches, barriers and a copy while subtracting that time from astc_record_ms. Time the view selection after any capacity flush.gpu/vk/vk_texture.cpp:5272: The pool telemetry cannot distinguish image creation from allocation reuse. A take by size creates a new image but increments neither pool_reused nor pool_new, and an outright allocation failure can increment pool_new. The burst logger also calls a take by size 'new (not pooled)'. Report exact reuse, new images on reused memory, and new allocations separately, and count successful creation.gpu/vk/vk_texture.cpp:3152: Burst logging waits for the next successful texture creation to report the preceding burst. An area entry followed by steady play can leave the burst unreported for the rest of the run. It also formats and stores rows past the 160-row output cap and after the 40-report quota. Close the report at present, count all creations separately, and stop collecting rows at the documented limits.tests/test_astc_upload_chunking.py:66: The generated C++ extracts the grouped copy at vk_texture.cpp:1463, which casts its offset to VkDeviceSize. The header-free fixture never declares that type, so the selected compiler fails before any case runs on Windows or Linux. Add a uint64_t VkDeviceSize alias to the fixture's Vulkan types.gpu/vk/vk_texture.cpp:1480: The batching PR still carries a test switch that can leave distant foliage mips unwritten. Remove RX_VK_TEST_BC_SKIP_BELOW and its switch row, and update the title, summary, testing and risks to describe the batching and diagnostics that remain at HEAD.gpu/vk/switches.def:280: The row and PR describe a maximum of eight levels per group. The code allows one big level plus eight small ones when bc_out has room. State both limits and the buffer cap, or enforce an eight-level total. If the trace columns stay, document their units and which are cumulative deltas or snapshots.Guide screenshots / Guide images and local UI source review (pull_request)fails on this head and passes on the base branch: Failing after 39s.For the author (1) (a run, a measurement or a decision)
ed602b94with matched builds and at least two interleaved runs per side. Include a normal preset and default reuse policy, average fps, 1% low, median and p99 frame time, area-entry stalls and RAM/VRAM. Verify small-mip rendering across affected Tier 1 drivers and Steam Deck, including the low-memory fallback without small scratch.Since the last review (on
365c7a1c28)Resolved: L1 (A late mip failure can abandon a group after its decoder dispatches are recorded. If align…); L2 (Failure to create one optional mip-batching view destroys the valid small scratch and its…); L3 (A prior draw or a pool key does not establish who owns the storage now. After a target is…); L4 (The trace snapshots plain pool counters and the dead vector before do_present takes replay…); L5 (RX_VK_LOG_REWRITES=2 hashes live guest memory after the upload. With replay behind the gam…); L8 (The new counters always take clock readings and update atomics in texture creation, reuse…); L12 (Skipping texture_frame also skips the frame increment used by other systems. At 120 fps wi…); L13 (A negative arrival cap such as -1 can keep eligible textures on the placeholder indefinite…); L14 (RX_VK_POOL_IN_PASS can hand an image to the pool and immediately trim it while compaction…); L16 (The extracted gather prefix now calls step_now and updates step_ns, neither of which is in…); L17 (The test requires the header to end at ab_side, so it rejects the new CSV. If the columns…); L18 (This check and tests/test_vram_compact.py still require the original unconditional compact…)
Still open: L6 (A replacement texture at the same address with matching width, height and format but more…); L7 (After a GPU buffer overlap, an upload that fails staging can still produce an 'uploaded ag…); L9 (A capacity-triggered group flush happens inside the view timer. A big mip that fills bc_ou…); L10 (The pool telemetry cannot distinguish image creation from allocation reuse. A take by size…); L11 (Burst logging waits for the next successful texture creation to report the preceding burst…); L15 (The generated C++ extracts the grouped copy at vk_texture.cpp:1463, which casts its offset…); L19 (The batching PR still carries a test switch that can leave distant foliage mips unwritten.…); L20 (The row and PR describe a maximum of eight levels per group. The code allows one big level…); L21 (Measure final mip batching against rebased main
ed602b94with matched builds and at least…)New: L22 (The check
Guide screenshots / Guide images and local UI source review (pull_request)fai…)Checks: Still running:
CI / Website build and checks (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven at HEAD. The earlier Windows 11 RTX 4070 SUPER market trace against
80bb1cccreported 24 to 11 frames over 20 ms, 48.5 to 21.9 ms worst frame, and 72.9 to 73.7 average fps. Short timestamp runs reported 461–529 µs down to 102 µs per eligible texture. The reported runs used custom settings and builds and were not interleaved. HEAD62350714versus baseed602b94still needs matched runs with a normal preset and default target reuse, plus 1% low, median and p99 frame times, area-entry stalls, and RAM/VRAM.Other systems: Layer arrays retain their array path, including when optional mip views fail to initialize. BC7, resized textures and devices without small scratch bypass grouping. No final-head runs cover AMD Windows, Linux or Steam Deck. The current base adds explicit BC copy pitches for an AMD Windows terrain-array bug, so the earlier NVIDIA play does not establish correctness on that driver. The CSV preserves existing columns in order and appends 16, and the new pool snapshot takes replay_mu. No save, audio, input, shader, public renderer API or cache-format changes appear in the diff.
Switches: Relative to base, RX_VK_BC_LEVELS defaults off. That is appropriate for a developer A/B, but the PR has yet to establish the automatic policy the maintainer requested for players. RX_VK_LOG_MADE_BURST, RX_VK_LOG_POOL and RX_VK_LOG_REWRITES default to no logging, and RX_VK_LOG_MADE_BURST_FROM defaults to 0, meaning from the start. Those defaults are appropriate. Rewrite logging no longer hashes in mode 2. RX_VK_TEST_BC_SKIP_BELOW defaults off and should be dropped because enabling it leaves visible mips unwritten. RX_VK_FRAME_TRACE retains its off default and gains columns. Since the ledger head, RX_VK_TEST_KEEP_DRAWN, RX_VK_TEX_CLOCK_HZ, RX_VK_ARRIVE_MB, RX_VK_ARRIVE_ALL and RX_VK_POOL_IN_PASS have been removed.
Reviewed:
623507146bagainsted602b9468: Read AGENTS.md, CONTRIBUTING.md, the Forgejo template, docs/dev/standards.md, the profiling trace reference, the commit history, and the changed sections of all seven files in the base-to-HEAD diff. Read the relevant surrounding scratch, upload, refill, pool, present and fixture code. Unrelated portions of the renderer and full test suite were outside this re-check. No local docs/review-checklist.md or sibling Tooling checkout was available.; nothing run (a code review).Checked and fine:
Testing: Read-only review. No builds or tests were run. git diff --check reported no whitespace issues. The author reported earlier Windows 11 NVIDIA game play and a switch-table pass. The supplied CI shows the guide screenshot source review failing on this head and the website build still running.
Overlaps and merge order: No sibling merge dependency established from the checkout and supplied discussion. The related AMD BC copy-pitch fix is already in the merge base.
Automated code review by BeanBot (gpt-6.2-sol), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@fix
I changed this pull request where an edit could (the rest needs runs, measurements or decisions): the new head is
5b8879040d. I rebased it ontomain(40bfceb055). I edited the pull request's description too (10 edits).Changed
Changed in part (the rest is for the author)
perf(vk): batch BC3 mip uploads and add texture diagnostics; the title cannot be changed through description_edits.Left for the author
ed602b94and Tier 1, Steam Deck and low-memory rendering checks; report the requested performance and memory metrics, then select automatic batching conditions from those results. No hardware results are available to support a choice.Still for the author (no edit does these)
Guide screenshots / Guide images and local UI source review (pull_request)fails on this head and passes on the base branch: Failing after 39s.Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
623507146b5b8879040dBeanstalk test farm
Commit
08ed4d5· approved by nobody yet · dashboardA member of
BigBeanscan review this commit and comment@Beanstalkto test it.Recommendation: Request changes. The upload-log fix breaks the existing write-retry check, and the PR title still advertises the removed render-target upload suppression. The batching gains also need confirmation on this head before choosing an automatic default.
Batches eligible single-layer BC3 mip uploads to reduce GPU barriers during area entry. This revision removes the experiment that skipped small mips and improves upload, burst and pool diagnostics. Batching still requires opt-in and adds per-layer scratch views and command-recording work when enabled.
@fixwill do (4)gpu/vk/vk_texture.cpp: The switch removal is complete, but the title still promises that a GPU-drawn target avoids re-upload after a buffer write. Rename the PR for BC3 mip batching and diagnostics, and identify the earlier builds and switches in Testing so the play report is not attributed to this head.tests/test_wrote_regions.py:51: The check requires a literal one-line gpu_wrote_seen assignment under if (up_ok). HEAD wraps the assignment and the success log in a block, so check_source raises before the C++ test runs. Update the guard to verify that the assignment remains under both success and gpu_dirty and that the failure path still leaves the write available for retry.gpu/vk/vk_texture.cpp:3185: The burst threshold counts exact pool hits as images made. (more on the line)For the author (1) (a run, a measurement or a decision)
5b8879040dwith current main at40bfceb055using matched builds, a normal preset, default target reuse and at least two interleaved runs per side. Report average FPS, 1% low, median and p99 frame time, area-entry stalls and RAM/VRAM. Check small mips on affected Tier 1 drivers and Steam Deck, including the fallback without small scratch.Since the last review (on
623507146b)Resolved: L6 (A replacement texture at the same address with matching width, height and format but more…); L7 (After a GPU buffer overlap, an upload that fails staging can still produce an 'uploaded ag…); L9 (A capacity-triggered group flush happens inside the view timer. A big mip that fills bc_ou…); L10 (The pool telemetry cannot distinguish image creation from allocation reuse. A take by size…); L11 (Burst logging waits for the next successful texture creation to report the preceding burst…); L15 (The generated C++ extracts the grouped copy at vk_texture.cpp:1463, which casts its offset…); L20 (The row and PR describe a maximum of eight levels per group. The code allows one big level…); L22 (The check
Guide screenshots / Guide images and local UI source review (pull_request)fai…)Still open: L19 (The switch removal is complete, but the title still promises that a GPU-drawn target avoid…); L21 (Compare
5b8879040dwith current main at40bfceb055using matched builds, a normal pres…)New: L23 (The check requires a literal one-line gpu_wrote_seen assignment under if (up_ok). HEAD wra…); L24 (The burst threshold counts exact pool hits as images made. A present that takes 40 existin…)
Checks: Still running:
CI / Website build and checks (pull_request),PR description / Description follows the template (pull_request_target),CI / Conventional Commit title (pull_request),CI / Workflow lint (actionlint) (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The author’s earlier RTX 4070 SUPER runs report 461–529 µs versus 102 µs per eligible upload and 24 versus 11 market-walk frames over 20 ms. Those runs predate this head, were not interleaved and used a tuned setup. L21 still needs matched runs against
40bfceb055with the normal preset and reuse policy, including frame-time tails, area-entry stalls and RAM/VRAM.Other systems: The batching guard leaves BC7, resized textures, layer arrays and devices without usable small scratch on the existing path. Windows NVIDIA has earlier play evidence. Windows AMD, Linux NVIDIA/RADV, Steam Deck and low-memory fallback still need coverage. The trace appends pool_resized without shifting existing columns. No save, options, cache format, public renderer API, audio, input or build-path edits appear in the diff.
Switches: RX_VK_BC_LEVELS defaults off. It is suitable for an opt-in trial, and the evidence does not yet establish an automatic default. RX_VK_LOG_REWRITES, RX_VK_LOG_MADE_BURST and RX_VK_LOG_POOL default off, which suits diagnostics. RX_VK_LOG_MADE_BURST_FROM defaults to 0, meaning from the start when burst logging is enabled. The switch table matches those defaults. RX_VK_TEST_BC_SKIP_BELOW was removed from the branch, including its table row. RX_VK_FRAME_TRACE keeps its existing default and gains texture and pool columns.
Reviewed:
5b8879040dagainst40bfceb055: Read the base AGENTS.md, CONTRIBUTING.md and Forgejo PR template, plus the relevant standards and renderer architecture rules. No review checklist was present at the base. Read the changed hunks in all nine files, the latest fix, and the relevant callers, pool paths, scratch retirement and test fixtures at HEAD. Unrelated regions of the renderer files were not read in full. Skimmed the supplied run reports and CI status. The external Tooling checkout was unavailable.; nothing run (a code review).Checked and fine:
Testing: Read-only review. git diff --check passed. No build, test or game run was performed. The earlier Windows NVIDIA play report and switch-table pass are author evidence. Six CI checks are pending in the supplied snapshot, with no reported head-only failures.
Overlaps and merge order: #1843 supplies the small scratch path, #1681 supplies target memory reuse and #2015 supplies explicit BC layer pitch for the older path. All three are already in the base. No additional merge-order dependency was found in the checkout.
Automated code review by BeanBot (gpt-6.2-sol), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@ -3027,2 +3182,4 @@const auto create_t0 = collect ? std::chrono::steady_clock::now() : std::chrono::steady_clock::time_point{};tex = create(k, t, fi.format, fi.astc, bw, bh, bpb, !for_sampling);if (!tex) return nullptr;if (burst_minimum() > 0) burst_count++;L24 (minor): The burst threshold counts exact pool hits as images made. A present that takes 40 existing VkImages from the pool can emit a report for 40 images made and use one of the 40 report slots without creating an image. Exclude exact hits from the creation threshold and label any retained reuse rows separately.
@fix
I didn't fix this pull request: it failed (Codex didn't finish with an answer (exit 1)). Nothing was pushed.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
@fix
The ChatGPT usage limit is reached (it resets Nov 25th, 2026 1:04 AM), so this fix is paused. It runs by itself once the limit lifts (tried again every 30 minutes); nothing needs asking again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
I changed this pull request where an edit could (the rest needs runs, measurements or decisions): the new head is
08ed4d580a. I couldn't bring it up to date withmain: resolving the conflict in docs/user/images/guide/manifest.json needs the author: The manifest records two different reviewed source revisions, each with its own updated file hashes. The author must decide how to represent the combined review without attributing either side’s hashes to the wrong revision. The file was left unchanged. I edited the pull request's description too (1 edit).Changed
Already done
Left for the author
Still for the author (no edit does these)
gpu/vk/vk_texture.cpp: The PR title still promises that GPU-drawn targets avoid re-upload after a buffer write, although that experiment was removed. Rename it to describe BC3 mip batching and texture diagnostics.Nothing was built or run: check the change. Its review follows.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
Recommendation: Request changes. The title still promises a removed render-target upload optimization. The BC3 batching gain and its behavior across supported hardware also remain unproven on the current code.
The change batches eligible BC3 mip uploads and adds texture and pool diagnostics. It may reduce upload stalls, at the cost of a more complex upload path and hardware-dependent scratch usage.
@fixwill do (2)Guide screenshots / Guide images and local UI source review (pull_request)fails on this head and passes on the base branch: Failing after 13s.For the author (2) (a run, a measurement or a decision)
gpu/vk/vk_texture.cpp: The PR title still promises that GPU-drawn targets avoid re-upload after a buffer write, although that experiment was removed. Rename it to describe BC3 mip batching and texture diagnostics. (@fixcouldn't: Rename the PR title to describe BC3 mip batching and texture diagnostics; the title is not in the supplied description or editable checkout.)5b8879040dwith current main at40bfceb055using matched builds, a normal preset, default target reuse and at least two interleaved runs per side. Report average FPS, 1% low, median and p99 frame time, area-entry stalls and RAM/VRAM. Check small mips on affected Tier 1 drivers and Steam Deck, including the fallback without small scratch.Since the last review (on
5b8879040d)Resolved: L23 (The check requires a literal one-line gpu_wrote_seen assignment under if (up_ok). HEAD wra…); L24 (The burst threshold counts exact pool hits as images made. A present that takes 40 existin…)
Still open: L19 (The PR title still promises that GPU-drawn targets avoid re-upload after a buffer write, a…); L21 (Compare
5b8879040dwith current main at40bfceb055using matched builds, a normal pres…)New: L25 (The check
Guide screenshots / Guide images and local UI source review (pull_request)fai…)Checks: Still running:
CI / Website build and checks (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. Earlier Windows RTX 4070 SUPER runs reported 24 versus 11 frames over 20 ms and 72.9 versus 73.7 FPS, but were not interleaved, used non-default settings and included a removed experiment. Current-head measurements against the stated base are missing.
Other systems: The batching path can affect small mip rendering on other Vulkan drivers, including AMD RDNA 2, Linux and Steam Deck. The per-mip fallback still needs hardware coverage, including configurations without small scratch. The fix also changes a source check and diagnostic counts; no runtime verification was supplied.
Switches: RX_VK_BC_LEVELS defaults off; its automatic eligibility and eventual default remain unproven. RX_VK_LOG_MADE_BURST, RX_VK_LOG_POOL and RX_VK_LOG_REWRITES default to no logging; RX_VK_LOG_MADE_BURST_FROM defaults to 0. These diagnostic defaults are appropriate, and the switches have table entries.
Reviewed:
08ed4d580aagainst40bfceb055: Read the repository agent and contribution rules, PR template, full tests/test_wrote_regions.py, the changed burst-count code and its surrounding caller, the changes since the ledger, and the switch and profiling diffs. Skimmed the guide manifest changes and the overall diff summary; did not read the full earlier PR diff or all changed code files. docs/review-checklist.md was absent.; nothing run (a code review).Checked and fine:
Testing: No commands from the project were run. The supplied CI reports a head-only guide screenshot check failure and a website check still running; the bot accounts for the failing check.
Overlaps and merge order: Not checked; network access is closed.
Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.