perf(vk): the retirement sweep's texture write-backs in batches, one submission and one wait each #840
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!840
Loading…
Reference in a new issue
No description provided.
Delete branch "perf/batched-eviction-writebacks"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
flush_texture is split into a GPU half (flush_record: the image's copies into a staging buffer, into rb_cmd) and a CPU half (flush_finish: the texels into the texture's memory), and flush_textures does a batch with one submission and one fence wait. The sweep chooses the textures to go first, writes back the flushable ones in batches of 64 or ~512 MB, then destroys them; its budget test counts the chosen ones as gone. Each write-back was a submit and a wait on its fence, a drain of the queue: a sweep of 500-600 textures at a scene load stalled for seconds (hookshot_log: 517 and 635 retired at once, 160 write-backs, submits of 4.7 and 7.7 s).
Tested: RTX 4070 SUPER, Windows 11, Kokiri Forest and the scene loads after it. Before, sweeps retiring 517 and 635 textures at once made single submits of 4.7 and 7.7 s (hookshot_log: 160 write-backs); after, the worst submit of a session with the same loads was 0.48 s, with no device loss.
Data point for this PR from an AMD card, in case it helps the rebase: RX 7800 XT + Ryzen 7 5800X3D, Windows 11, v2.0.10, 2560x1440, frame generation x2, about 1.5 hours of normal play.
The retirement sweep is very busy on this card. From
hookshot_log.txtof that one session:textures retiredevents, 108255 textures retired in all (about 50 a sweep, up to 130 in one:vk: 132 textures retired (1083 left, 3065 MB))So the sweep runs constantly here, not just at a scene load. Each retired texture the GPU wrote goes through
flush_texture, one submission and one fence wait each, so batching them as this PR does should matter on this card too. I have not measured it: I could not apply the PR as it is, because the eviction loop it changes was reworked in v2.0.8 (the dead-target handling:need_dead,drawn_over_shifted, then_dead/b_deadcounters). The merge conflicts are in exactly that loop.Could the PR be rebased on v2.0.10 (or main)? I can test a rebased version on this machine (AMD, Windows) and report the effect on the frame times.
Related stutter data from the same card, for context: #995 (comment with the replay CPU times, v2.0.7 vs v2.0.10).
Nice one, the split into flush_record / flush_finish is clean and the numbers in the description are convincing.
This conflicts with main and needs a rebase. The flush helpers themselves haven't changed on main, so that half should apply as is. The conflict is in the
texture_frameeviction loop, which main has reworked since:frame_no - last < need_dead, followed by thedrawn_over_shifted/need_unusedcheck (dead transient targets)continue(issue 993)destroy_later:n_dead/b_deadfor shifted targets, and theevicted_from_memoryrecord for uploaded texturesWhen you rebase, those last two should go into the destroy loop. The easiest way is probably to carry
shiftedinEvict. Thepending_byteschange also needs to go onto main's new break line. Side note: main now runs the dead-target sweep every 60 presents instead of every 600, so batching helps there too. The failingtest_mem_trimcheck ("a BC3 upload uses a scratch it did not ask for") comes from main's newer test running against this old base, not from this change.What I checked and think is right:
full_barrierbefore all copies covers every image (they're all in GENERAL)flush_finishflush_recordfails, only that texture is skipped, and a failedreadback_beginis handledflush_finishruns in oldest-first order, so overlapping write-backs land in the same order as before, and thememory_written_runschecks see the same earlier writesdestroy_later/forget_gpuuntil after the selection doesn't change anydrawn_over/keep_finalizeddecision, because a texture evicted earlier inordercan't havelast_writtenafter a later one'slast_framepending_bytesuses the samebytes + padthatdestroy_latersubtractslast_value > completedis still checked, image destruction is still deferred, and it's still the same thread and the samerb_cmduse as beforeTwo smaller things inline. With a rebase I'd be happy with this.
@ -2525,0 +2558,4 @@// that failed. The copies may still be writing into the staging buffers, so they are not freed here: they go the way a// destroyed image does, once a submission made after this one is done. The batch's write-backs are lost, as a single// one's was before)static int logged;Already true before this PR, but it matters more now. If this 5 s wait times out (it also covers whatever
queue_flush_lockedjust submitted, and at a scene load that can be a lot),flush_finishfrees up to 64 staging buffers that the copies may still be writing into, and the whole batch's write-backs are lost at once instead of one. On a non-success result, could the buffers be kept alive (wait again / wait for idle, or hand them to the deferred-destroy list) instead of freed straight away?Small robustness point:
destroy_laterclamps withstd::min(total_bytes, ...), which suggeststotal_bytescan drift below the sum of the textures' sizes. If it does,total_bytes - pending_byteswraps around, the budget test never breaks, and everything old enough goes. Maybepending_bytes >= total_bytes || total_bytes - pending_bytes <= budget / 8 * 7? (This line is also where the rebase conflicts with main'sneed_deadchange.)Nit:
tex->bytesis the image's size. The staging buffer isf.total, which is the guest-format size, so the 512 MB cap is only approximate. WithkFlushMaxat 16 MB the 64-texture cap bounds it anyway (1 GB worst case), so this is fine. Countinge.tex->guest_sizewould just make the cap mean what the comment says.@hckrman101 thanks for the careful read. Main is merged into the branch now (
6ff5fc8), resolved the way you laid out:total_bytes - pending_bytes);shiftedis carried inEvict, andn_dead/b_dead, theevicted_from_memoryrecord and the remade log run in the destroy loop, after the batches;continuestays where main has it, before a texture is chosen;flush_texturehad not changed on main, so the split intoflush_record/flush_finish/flush_texturesapplies as it was.GitHub shows it mergeable, and
test_mem_trimpasses on the merge. It is a merge commit rather than a rebase because I could not force-push from my setup.@lucasmazzoni45-del it applies to main now, so it builds on top of v2.0.11. If you can run a session with it, the lines to compare are the
textures retiredsweeps and theslow submitlines around them.One honest note from my side: on current main I have not re-measured the effect. A ten-minute run here (RTX 4070 SUPER, Windows 11, main at
7dbdcdcwith this branch) had 84 sweeps, the largest retiring 553 textures at once, and none of them had anything to write back, so the batches were not exercised; no device loss. The multi-second sweeps in the description were measured on the v2.0.5 base. A card where the sweep writes back often, like yours, is the better test.Thanks for the merge, this now applies cleanly and I read the resolved eviction loop against main's: selection is unchanged,
shiftedrides inEvict, andn_dead/b_dead, theevicted_from_memoryrecord and the remade log run in the destroy loop after the batches. The keep-unflushablecontinuestill comes before a texture is chosen. CI is green. Theflush_record/flush_finish/flush_texturescode is the same as I reviewed before, and I still think it's correct (also checked: if everyflush_recordfails,rb_cmdis left recording but the nextreadback_beginresets the pool, so that's fine).Two small points from my first review are still open, neither blocking. Replying on the old threads:
total_bytes - pending_bytescan wrap iftotal_byteshas drifted below the sum (destroy_laterclamps withstd::min, so the code already expects that)The
tex->bytesvsguest_sizething is only a nit. I'm fine with this going in as is.@ -2525,0 +2558,4 @@// that failed. The copies may still be writing into the staging buffers, so they are not freed here: they go the way a// destroyed image does, once a submission made after this one is done. The batch's write-backs are lost, as a single// one's was before)static int logged;Still open at
6ff5fc8: on a non-successreadback_submit_wait,flush_finishstill destroys the staging buffers straight away. The risk is only a 5 s timeout, so a follow-up is fine.Still there on the new break line (
total_bytes - pending_bytes). A guard likepending_bytes >= total_bytes ||would cover it. Cheap, but not a blocker.Thanks @SadTransGirl, and a correction to my own first comment here, because I ran this branch's idea on my machine and the data does not support what I wrote.
I wrote that each retired texture the GPU wrote goes through
flush_textureone at a time, so batching "should matter on this card too". That was an assumption from the sweep counts, not something I had checked in the log. I now have four sessions on an RX 7800 XT / Windows 11 / v2.0.11 with a hand port of this PR in my build (flush_record/flush_finish/flush_textures, theEvictstruct carryingshifted, then_dead/b_deadbookkeeping in the destroy loop, the review's two points below), about 18 to 60 minutes each:textures retiredlines, 163574 textures in all, the largest 1365 in one sweep (the others: 374 sweeps / 9607 textures / max 614; 158 / 4594 / max 709; 115 / 15863 / max 824);textures the GPU drew, written back into their memory before eviction: N so farline, which prints at the 1st write-back and every 50th, never appears in any of the four logs. So, like your 4070 SUPER run, the batches were never exercised here either, and I cannot say anything about their effect. My guess that this sweep was behind my stutter was wrong.did not complete in timebatch timeout (the log line I added for the first review point), no device lost.I also have no
slow submitlines to compare, because I did not run withRX_VK_SLOW_LOG; I can do that if it helps.On the two points @hckrman101 says are still open, in case my port is useful as a reference (small, builds and runs here, though as said the batch path itself has not been hit):
if ((!sweep && (pending_bytes >= total_bytes || total_bytes - pending_bytes <= budget / 8 * 7)) || frame_no - last < need_dead) break;flush_textures, afterconst bool gpu_ok = any && readback_submit_wait();, ifany && !gpu_okI log once and setf.buf = VK_NULL_HANDLE, f.alloc = nullptrfor everyFlushin the batch beforeflush_finish, so the staging buffers are not freed while the copies may still write into them (they stay allocated, a rare small leak).I would not spend more of your time on this one on my account; if a card that writes back often shows up, that is the test.
@hckrman101 the two open points and the nit are in
8a453c4:flush_finish. They go on the deferred-destroy list (dead, withrec_value + 1), so they are freed once a submission made after this one is done, the way a destroyed image is. A log line says the batch was not written back.pending_bytes >= total_bytes || total_bytes - pending_bytes <= budget / 8 * 7.guest_size.vk_texture.cppcompiles andtest_mem_trim,test_nvn_texture_handleandtest_texture_init_barrierpass here.@lucasmazzoni45-del thank you for running four sessions with a port and for writing up that the batches never ran. That matches my run, and I agree: the measured benefit is from the v2.0.5 base, and on current main neither of us has a sweep that writes back. Your two snippets are what went in, except that the buffers are handed to the deferred-destroy list rather than leaked.
Thanks, both open points and the nit are fixed in
8a453c4.deadwithrec_value + 1is sound.texture_frameruns inside the present recording, sorec_valueis a recording that hasn't been submitted yet. It goes onto the same queue after the read-back, andrec_value + 1comes after that. A signal operation's first scope covers everything earlier in submission order, so the timeline reaching that value means the read-back's copies are done. If the read-back is really hung, nothing later ever signals and the buffers just stay allocated. That's a leak, not a use-after-free, which is the right way to fail. Nothing else is left half-done on timeout: the copies don't change layouts (GENERAL in, GENERAL out), andflush_finishwithgpu_ok == falsedoesn't invalidate, unswizzle orgpu_writeanything, so guest memory and the written-page state are untouched. The batch's textures are lost, same as a single one before.pending_bytes >= total_bytes ||covers it.guest_sizenow.One related gap (it was there before this PR too, so it doesn't block). The image the timed-out copies read from isn't deferred the same way. The sweep only picks textures with
last_value <= completed, anddestroy_laterqueues the image attex.last_value, so the nexttexture_framedestroys it while the hung copy may still be reading it. Inline comment with a one-liner below.Also pre-existing: after a timeout, the next
readback_beginresetsrb_poolandrb_fencewhile that submission may still be pending, which the spec doesn't allow. Same 5 s case, so it's for a follow-up.On coverage: no test reaches
flush_texturesor the timeout branch, and from the comments neither the 4070 SUPER run nor lucasmazzoni45-del's four RX 7800 XT sessions wrote anything back during a sweep. So the batch path is reviewed but hasn't really been run on current main. CI is green. Approving.Suggestion for a follow-up (or here if it's easy): the textures in this batch also need to outlive the copies. They go through
destroy_laterwithtex.last_value, which the sweep already checked is<= completed, so their images (and memory) get destroyed at the nexttexture_framewhile the timed-out copy may still read them. Bumping it here would defer them to the same point as the buffers:(
texture_flush_to_guestkeeps its texture, so a higherlast_valuethere just makes it look recently used.)@ -2525,0 +2558,4 @@// that failed. The copies may still be writing into the staging buffers, so they are not freed here: they go the way a// destroyed image does, once a submission made after this one is done. The batch's write-backs are lost, as a single// one's was before)static int logged;Fixed in
8a453c4: the buffers go todeadatrec_value + 1, which is safe given queue order. The source image is still not deferred, see the new inline comment.Fixed in
8a453c4, thanks.