vk: read a pipeline cache over 2 GB on Windows #1226

Merged

jarynclouatre merged 1 commit from fix/pipeline-cache-size into main

2026-11-09 18:22:40 +00:00

jarynclouatre commented

2026-11-09 18:14:09 +00:00

(Migrated from github.com)

A normal cache is well under the limit (500 to 700 MB here after a prepare and 20 minutes of play), so this matters for a cache that has grown over long play or for anything that adds to it. The fix uses _fseeki64 and _ftelli64, as the shader cache reader in vk_shader.cpp already does.

Testing

Windows 11, RTX 4070 Ti SUPER, on 8b2e1d3 with a local change that also keeps the GPL stage libraries in the cache (RX_VK_GPL=1), which took it to 2,151,265,735 bytes after --prepare-shaders. Without this fix the log said "pipeline cache ... (0 KB)" and the file was 882 MB after the run. With it the log said "(2100271 KB)" and the same 19-minute scripted route through eleven areas ran normally.

Build checks only for the rebased commit on current main: it compiles for Windows (llvm-mingw) and Linux.

On Windows, pcache_init took the size of mkw_vk_pipelines.bin from ftell, which returns a 32-bit long there. Past 2 GB the size came back wrong, the read was skipped and the game started with an empty pipeline cache. The next save then wrote that run's much smaller cache over the big one, so everything the prepare had put in it was gone. Linux was not affected. A normal cache is well under the limit (500 to 700 MB here after a prepare and 20 minutes of play), so this matters for a cache that has grown over long play or for anything that adds to it. The fix uses _fseeki64 and _ftelli64, as the shader cache reader in vk_shader.cpp already does. ## Testing Windows 11, RTX 4070 Ti SUPER, on 8b2e1d3 with a local change that also keeps the GPL stage libraries in the cache (RX_VK_GPL=1), which took it to 2,151,265,735 bytes after --prepare-shaders. Without this fix the log said "pipeline cache ... (0 KB)" and the file was 882 MB after the run. With it the log said "(2100271 KB)" and the same 19-minute scripted route through eleven areas ran normally. Build checks only for the rebased commit on current main: it compiles for Windows (llvm-mingw) and Linux.

hckrman101 (Migrated from github.com) approved these changes

2026-11-09 18:19:52 +00:00

hckrman101 (Migrated from github.com) left a comment

Thanks, this looks right to me.

On Win64 long is 32-bit, so ftell fails (returns -1) once the file is past 2 GB; sz > 32 was then false, the read was skipped, and the next pipeline_cache_save replaced the big file with the small in-memory cache. That matches what you saw.
_fseeki64/_ftelli64 are already used elsewhere in vk_draw.cpp (the pipe notes reader) and vk_shader.cpp, and on Linux they come from the host/posix/windows.h shim (fseeko/ftello), so nothing new is needed for the Linux build. Linux behaviour is unchanged since long was already 64-bit there.
A -1 from _ftelli64 still falls through sz > 32 and starts with an empty cache, same as before. ci.initialDataSize is size_t, so a >2 GB blob goes to the driver intact.
No overlap with #1046: that one only touches the save lambda further down the file, so the two should merge cleanly.

No test for this, but I don't think one is worth it for a three-line change like this.

Thanks, this looks right to me. - On Win64 `long` is 32-bit, so `ftell` fails (returns -1) once the file is past 2 GB; `sz > 32` was then false, the read was skipped, and the next `pipeline_cache_save` replaced the big file with the small in-memory cache. That matches what you saw. - `_fseeki64`/`_ftelli64` are already used elsewhere in `vk_draw.cpp` (the pipe notes reader) and `vk_shader.cpp`, and on Linux they come from the `host/posix/windows.h` shim (`fseeko`/`ftello`), so nothing new is needed for the Linux build. Linux behaviour is unchanged since `long` was already 64-bit there. - A -1 from `_ftelli64` still falls through `sz > 32` and starts with an empty cache, same as before. `ci.initialDataSize` is `size_t`, so a >2 GB blob goes to the driver intact. - No overlap with #1046: that one only touches the save lambda further down the file, so the two should merge cleanly. No test for this, but I don't think one is worth it for a three-line change like this.

Commenting is not possible because the repository is archived.

No reviewers

hckrman101