WIP: fix(vk): an update that changes no shader no longer rebuilds every pipeline #1418
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1418
Loading…
Reference in a new issue
No description provided.
Delete branch "TrashPanda/Longshot:fix/no-pipeline-rebuild"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux, game-free only · Needs: any GPU, a start after an update, in the game
Why
state_list_preparedeletedlongshot_pipelines.bin.built, the marker that every noted pipeline is already in the pipeline cache, on every call. The preparation after every build calls it. So each update, even one that changed no shader, left the next start to build every noted pipeline again behind "Loading pipelines", after the preparation had already built them in its own process. Upstream that is 44,000 pipelines, minutes on a slower PC.Ported from Hookshot
0824d761(NotRexed):git.notrexed.dev/Hookshot/Hookshot@0824d76165The tracker was searched for
precompile_covers_notesand "Loading pipelines": no pull request or issue is this fix. #903, #900, #721 and #714 are other loading bugs.How to reproduce
Seen on: Hookshot's report. Not yet run on Longshot.
Settings and switches: defaults.
Steps: start the game once so the pipeline cache is built; update to a build that changes no shader (the preparation runs); start again.
Expected: the second start loads the pipelines at create, with no full "Loading pipelines".
Actual: on main, the second start builds every noted pipeline again.
What changed
The marker goes only when the preparation notes a pipeline that was not noted before. Those new notes are added without their pipelines, and
precompile_runbuilds them on the next start. A list that notes nothing new leaves the marker. The translator stamp still rebuilds everything when the shaders change (pcache_drop_stale).state_list_prepareingpu/vk/vk_draw.cppcounts whatpipeline_noteadded and removes the marker only then, with a log linevk: state list: N pipelines not noted before: ....tests/test_load_default.pychecks that the marker's removal instate_list_preparefollowsif (added) {and that the count comes frompipeline_note.No switch: an extra rebuild was the only thing the old removal bought, and
RX_VK_NO_STATE_LIST=1still turns the preparation off.Measurements
Verdict: the game-free check fails before the fix and passes here. Nothing in the game has been measured.
Hardware: Linux, no GPU run.
Scene: none.
Method:
python -I -B tests/test_load_default.pyon the test commit and on this head.How to reproduce: check out
db4c16d4(the test alone, on main544229a2) and run the command; then this head.544229a2plus the test @db4c16d49a234beftests/test_load_default.pyload_default: okpython -I -B tests/run_all.pyon this head (Linux, 16 at once): 602 passed, 38 failed, 81 skipped (no Vulkan or SPIR-V headers here). Each of the 38 also fails onmain544229a2run alone, excepttests/test_appimage_package_retry.py, which timed out under the full run's load (90 s) and passes alone on this head, twice.Flags and settings
None: no switch is added or changed.
Testing
Tested: Linux, game-free.
tests/test_load_default.pyfails on the test commit and passes on9a234bef.Not tested: the game, any GPU, Windows. This PC is not running the game. Opened with
--forcebecause Joey asked.Must be tested:
vk: state list: ... not noted beforeline. Then add a state the list did not have (or deletelongshot_pipelines.bin): the line appears and the next start builds the new ones.Risks
What could break: if
pipeline_notereported a note as old when its pipeline is not in the cache, that pipeline would not be prebuilt and would compile in play, a hitch the first time it draws. The marker is also still removed when the translator stamp changes and when the pipeline cache is dropped.Turning it off: none as a switch; reverting the commit restores the old removal.
Compatibility: nothing in saves or options.
MKWVK_API_VERSIONis unchanged.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). The 38 failures also fail onmainhere (this PC lacks the Vulkan headers and some tools); the one that did not,test_appimage_package_retry, is a timing failure under load and passes alone.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.@Beanstalk any GPU, two game starts in a row on this PR's build, from a new save, with the pipeline cache kept between them. Expected: the first start may build pipelines ("Loading pipelines"); the second start loads them at create with no full "Loading pipelines" and no log line
vk: state list: ... pipelines not noted before. Compare the second start's time to the menu with main's second start. Not run in the game yet; Joey asked for this pull request so the farm can take that run.@TrashPanda only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Possibly related, for context:
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.