WIP: fix(vk): a texture the GPU wrote, or whose upload failed, is uploaded again instead of kept stale #1162
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1162
Loading…
Reference in a new issue
No description provided.
Delete branch "TrashPanda/Longshot:fix/texture-upload-fixes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: medium · Tested on: Linux, RTX 4070 Ti (game-free checks) · Needs: any
Why
Two ways a texture kept old or empty texels: a GPU write into its storage was missed when a later, smaller write landed inside an earlier one, and a texture whose upload failed was left as it was instead of tried again.
Restored from Hookshot 3002, Hookshot 3003, by Joey Magnus, jarynclouatre, and split out of #287 so it can be reviewed and merged on its own.
What changed
texture_gpu_wrotekeeps its write records apart, sogpu_wrote_intofinds every write under a texture, and a failed upload marks the texture to be uploaded again.Commits, each with its original author and message:
611955e5fix(vk): a GPU buffer write into a texture's storage is found under a later, smaller write (jarynclouatre)texture_gpu_wroteingpu/vk/vk_texture.cppnotes each GPU bufferwrite into guest memory as a record, start to end plus the submission
that wrote it.
gpu_wrote_intofinds the finished ones a texture wasnot uploaded after. It starts at the one record before the texture's
address and walks forward, which is right only when records do not
overlap. The writer erased only the records a new write covered whole,
so a later, smaller write inside an older one left two records that
overlap.
For example, take
[0x100000, 0x140000)at submission 10, then[0x110000, 0x120000)at 11. For a texture at0x130000, the recordbefore it is the second one, which ends before the texture, and the
first is never looked at. The texture kept its old texels after the GPU
wrote them.
Change
The writer keeps the records apart:
each with its own submission. This applies to the one record that starts
before the write, and to one that starts inside it and runs past its
end.
inside a record adds two records.
gpu_wrote_intois unchanged.Testing
tests/test_gpu_write_intervals.py: both functions and the mapare cut from the source and run on such writes:
before it 10.
waits.
with the old submission.
llvm-mingw for Windows.
hookshot_vk.so).on main on this machine (
test_bake_portability:zstandardis notinstalled;
nx2/test_texture_init_barrier.py:clang++is notinstalled).
Restored-from: https://github.com/NotRexed/Hookshot/pull/3002
cadff79efix(vk): a texture whose upload failed is tried again, not left empty or stale (jarynclouatre)When
upload()fails intexture_touch(gpu/vk/vk_texture.cpp), forexample for want of staging memory with the VRAM or RAM full, a BC3
texture is tried again 30 frames on (https://github.com/NotRexed/Hookshot/pull/1505). Every other kind was marked
uploaded with
upload_genmoved on ("not retried every draw; a write toits memory tries again"):
the game wrote that memory again, which for most textures is never.
last upload. It also dropped the GPU buffer write
gpu_wrote_intohadjust marked as seen.
This is the "textures degrade until a restart" kind of report on cards
with little memory (https://github.com/NotRexed/Hookshot/pull/993's family).
Change
Texture::bc_retry_framebecomesretry_frame.uploadedandupload_genstay as they were.gpu_wrote_seengoes back to its value from before this touch.retry_frameis set 30 frames on.other kinds get one like it.
gpu_wrote_intoand the dirty scan. A waiting texture costs a compareand loses nothing.
retry_frame.its
evicted_from_memoryentry taken before the upload. If the uploadfails, the entry is put back. Otherwise the retry would take the texture
for rendered storage and mark it uploaded, empty.
big, packed depth-stencil) still fails at once, every time. It is still
marked uploaded, so it is not tried forever. Those checks move from the
top of
upload_innerintoupload_never, used by both.Testing
tests/test_upload_retry.pychecks the source, becausetexture_touchcannot run outside the renderer:gpu_wrote_seenback and logs.gpu_wrote_intoand the dirty scan, andgpu_wrote_seen's old value is taken before it.upload_neveris shared withupload_innerand still marks thoseimages uploaded.
evicted_from_memoryentry is put back on a failure.tests/test_bc_upload_retry.pyfollows the rename.hookshot_vk.so).on main on this machine (
test_bake_portability:zstandardis notinstalled;
nx2/test_texture_init_barrier.py:clang++is notinstalled).
memory on demand, and no game ran.
1 << 40), the dirty scan has already taken a remap (
page_map,map_sig) when the upload fails. The retry uploads what the CPU wrote,but not pages that were only remapped. That is the same as before;
replaying it would mean a whole re-upload of a sparse array while memory
is short.
Restored-from: https://github.com/NotRexed/Hookshot/pull/3003
c51e9d95test: the GPU write intervals check builds with main's wrote_regions.h, which the reader asks first (Joey Magnus)Measurements
Verdict: not measured yet: the writer splits records where it erased them before, and a failed upload is tried again, both on the frame path; a before/after in a texture-heavy area is needed before this is marked ready.
Flags and settings
None.
Testing
Tested: On Linux (kernel 7.0), RTX 4070 Ti,
main@71b1edd8, this PR @c51e9d95:the checks that read the changed files (tests/run_all.py --changed origin/main)pass. The full suite (tests/run_all.py) has not been run on this branch yet; a comment will follow when it has.Not tested: The game. Opened as work in progress until it runs in the game.
Must be tested:
Risks
What could break: texture uploads (an extra upload costs time).
Turning it off: revert the commit.
Compatibility: Nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.main.third_party/untouched (a new folder with aPROVENANCE.mdinstead).python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).When it applies
mainand passing here.Tested in game: Linux (Ubuntu 26.04.1, kernel 7.0.0-43), RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D: this branch built together with the other 18 split from #218 and #284-#288, merged onto main @
71b1edd8(make_game.py --update, build v3.0.0-rc.5-82-g5952bb3d), then played by a scripted tour through the control bridge (scripts/memtour.py: a save loaded, Zora's Domain, the Market, Hyrule Field and Death Mountain Trail, 40 s each with walking and camera swings). All 4 stops reached, a clean quit, and no device loss or Vulkan error in the log. Textures uploaded and drew through the tour. A failed upload or a later, smaller GPU write into a texture's storage did not come up; the checks cover them.Possibly related, for context:
c51e9d9558856036459cView command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.