fix: hardening part 3, updates, shared files, guest buffers and texture sizes #1684
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!1684
Loading…
Reference in a new issue
No description provided.
Delete branch "hardening-part3"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Linux players on v2.0.12 and v2.0.13 cannot update: every download the builder makes fails before it connects, so no fix reaches them. This PR repairs that, and is the third hardening pass after #1398 and #1422. It covers what has landed since and the parts those passes did not reach: files players share, the updater, the OS layer and the renderer. Each commit is one topic.
Issues
Linux updates: every download failed (high: no update reaches players).
What:
linux_update._requestpassed the URL asurllib.request.Request's second argument, which is the POST body. Every request became a POST with astrbody, andurlopenraisedTypeErrorbefore connecting. The effects:latest_tag()returned nothing, so the builder acted as if it were offline;--updatefailed with "update-error";This came in with #1470 and shipped in v2.0.12.
Who: every AppImage player.
Fix: the request has no body.
Test:
test_linux_update.py test_a_download_is_a_get(it fails on main).Linux updates: a link chain could leave the install folder (medium, defence in depth).
_symlinkchecked a link's target as written, withabspath.x/../..looks like it stays inside the folder, but the system resolvesxfirst, and the same zip or a later part can makexa link afterwards. With such a chain,AppRun -> a/.../Land_finish_tree'scopyfilepluschmod 0755wrote an executable file anywhere in the home folder...only as its leading parts;test_a_link_chain_cannot_leave_the_staging_tree, with two chains through the realinstall(). Both fail on main, and the canary file is overwritten. Toolchain-style../lib/xlinks still unpack.Linux updates: the requested release is the one installed (low).
setup.candUpdater.cs.Linux updates: file modes from a part (low).
make_parts.pywrites.test_unpacked_files_are_not_group_or_world_writable.Windows setup: an unchecked zip could be unpacked (low).
<sha>.zip.part,DeleteFileW(zip); MoveFileW(part, zip)ignored both results. An old<sha>.zipthat could not be removed (locked by an antivirus, or read-only) was then unpacked in place of the checked one.MoveFileExW(..., MOVEFILE_REPLACE_EXISTING), and stop if it fails.hookshot_env.txtcould overwrite game files and press buttons (medium).What: #1398/#1422 refuse names that end in
_FILE/_DIRand values that are paths. Several test switches name a file or folder under other names, and a bare name resolves in the game folder:HOS_AUDIO_DUMP=image.bintruncated the game's image at the first sound, so the next start failed until a rebuild;RX_VK_HANG_DUMP=hookshot_options.inioverwrote the options file;RX_VK_SHOTS=userwith a largeRX_VK_SHOT_W/Hfilled the disk.HOS_INPUT_SCRIPTandRX_VK_MENU_SCRIPTpress game buttons and Insert menu keys on a timer, for example walking the file select to Erase.Who: anyone who hands a player a
hookshot_env.txt, which players are asked to share.Fix: these switches take no value from the player's file.
HOS_ENV_FILE's file still sets anything. The readme says so.Test:
test_env_file.py, with new lines for each case.The NV12 reader could copy any host memory to the game (high, for a game led astray).
InitializeDecoderImagePoolkeeps the pool when its buffers fail to allocate (#342 guarded only the decode path).ReadDecoderOutputImageNv12then readNULL + i*w*h*3/2 + y*w + x, which can be any host address, and copied it into guest memory. A fixture read a host string back this way.NULL.nn::fs reads and writes, and the libc copy hooks, stayed unbounded (medium).
read_atwrote up to 1 GB chunks atGP(buf + done), andWriteFileread fromGP(buf), for any 64-bitbufand size;memcpy,memmoveandmemsethooks passed guest addresses and lengths straight to the host's own calls;GetFileSizeon a folder's handle answered with 8 bytes of host stack.hos_guest_rangeon each file buffer (a write larger than a DWORD is refused, not cut);tests/test_guest_copies.py(new) buildsnn_libc.cover a small window with a watched page after it, and checks the fs and NV12 guards in the source.A texture's storage size wrapped in 32 bits (medium: multi-GB allocations, sizes past the GPU's limits).
What:
nvn_layout.cworked out the level sizes inuint32_t. A width of0x40000001RGBA8 texels came out as 512 bytes of storage, so the game allocated 512 bytes. The renderer then made a 4 GB-wideVkImage(beyond the device's limits, whichvkCreateImagedoes not define) and asked for multi-GB staging and scratch buffers (bad_alloc). A level count near 2^32 also looped for minutes, and shifts of 32 or more were undefined.Who: a texture whose size comes from game data (a mod, or a file the game reads).
Fix:
create()refuses a size, layer count or sample count the GPU cannot make (texture_size_ok).For ordinary sizes the layout is unchanged: 529,200 descriptions (formats, targets, sizes, levels, samples, MINIMAL_LAYOUT) give the same sizes, tiles and level offsets as before.
Test:
tests/test_texture_layout_sizes.py(new) builds the realnvn_layout.cand the cut-outtexture_size_ok. It fails on main.Cached descriptor counts and noted formats went to the driver unchecked (medium if a cache is shared, else low).
hookshot_programs.binrecord with two descriptor counts of 2^31 wrappedensure_layout'suint32_ttotal to 0. That made a push layout with billions of descriptors, and the bind loop ran that many times;VkFormatfor colour and depth targets. Linux has no crash guard, so the preparation crashed at every start until the file was deleted.formats_ok, besidetargets_ok).test_program_cache_reader.py test_descriptor_counts_are_boundedandtest_pipeline_state_counts.py(formats the device does not draw to).program_cache_cleanup_test.cpp's stand-in descriptors now have a count.Smaller checks (low):
prompts.cbind_font:so + 14 > lenwrapped for a subtable offset near 2^32, and the check then read 4 GB past the font (a SEGV in the test). It needs a changedprompts/<face>/External.bfotf.test_prompt_font_bind.pyhas a new case, which crashes on main;cmap4_bindnow checks the same span ascmap4_glyph.hookshot_options.iniasmicrophone_device=. A line break in it added lines to the file.mic_keyturns control characters into spaces (test_menu_utf8.py).rx_backend.cload_vulkan:strcpyofhookshot_vk.dlloverhookshot.exeoverranpath[MAX_PATH]for an exe path near 260 characters. It now refuses a name that does not fit.hos_sdl.cload_one: whenGetModuleFileNameAfailed, the bareSDL3.dllwent toLoadLibraryA(current folder, PATH). It now goes straight to the System32/default-dirs lookup.vk_dlss.cpp: when the game folder could not be found,game_dir()was empty and NGX searched and wrote a relativengxfolder in the current directory. NGX is not started then.discord_rpc.c: the pipe is opened withSECURITY_SQOS_PRESENT | SECURITY_IDENTIFICATION, so a program that made a pipe of Discord's name first cannot impersonate the game.Building from extracted folders followed links (low).
link_or_copy(os.link,copy2) andcopytreefollow symlinks. A link in--romfs/--exefsput the file it named, from anywhere on the PC, into the game folder.refuse_dump_linksstops the build with a message. The folders themselves may still be links.test_make_game_cache.py test_links_in_an_extracted_dump_are_refused.Not changed
Checked and found fine, or left for a decision:
CI:
pull_request_targetorworkflow_run;${{ }}of untrusted input in arun:;contents: readby default, withwriteonly in the release jobs (tags);persist-credentials: false;The release workflow's
npm installwithout a lock file was noted in #1398.pip installs (decision):
deps.shand the Windows Python part install pinned versions without--require-hashes(nx2/prompt_requirements.txttoo). A hijacked PyPI account can add a new, more specific wheel for a pinned version. Hash pins for every wheel and platform, plus--no-deps, would close it. That changes the release's dependency step, which I could not run here. The dav1d, Vulkan, SPIR-V and VMA archives without hashes are as #1422 noted.Source-checkout fallbacks on Windows:
C:\emsdkand the old../Eglantineand../mkw-recompCPM caches are used when nothing else is set. For a checkout atC:\Hookshot, those resolve to folders any local user can create. The builders setEMSDKandRX_CPMthemselves, so players are not affected. Dropping the fallbacks is a maintainer call (design.md names Eglantine's cache as a source).Lifetime races a game led astray could use (two threads on one object). Each needs its own change and testing:
CloseFile/CloseDirectoryagainst a read in progress;FinalizeagainstDecode;Finalizeagainst their readers;DestroyThread.Read-only libc hooks:
memcmp,strlenandstrcmpstill read guest addresses unbounded. They only read, and #880 covers the addressing.Program cache SPIR-V: a record's SPIR-V still goes to the driver unvalidated, and a cache from someone else is trusted once its stamp matches. An install key in the stamp remains open (#1422).
Renderer, game-code values (not file data):
texture_read_texelsuses a 32-bit byte counter (a hang with a 4 GB guest buffer);vkCmdBlitImage;maxUniformBufferRange.Eden's
UNREACHABLE()is__builtin_unreachable(). It is not reachable from game data today, since lowering never passes raw ISBERD bits. Throwing there would be safer, but it is third-party code.Linux updater:
_unpackreads each member into memory (a hash-pinned part only). The WindowsUpdater.csdoes not cap a download with noContent-Length.reuse_parttrusts the previous version'sparts.txt.Discord on Linux falls back to
$TMPDIRor/tmpfor its socket, as Discord itself does. A local user who made that socket learns the pid and version.Fine:
AddText), and its fixed buffers fit;RX_VK_MENU_SCRIPTparsing, rumble report sizes and handles, and Discord frame lengths are fine;hos_update.cpasses onlysafe_tagvalues to zenity and exec;ngx_sdk.pyand the hash-checked downloads indeps.share fine;How it was tested
python -I -B tests/run_all.py: 254 passed, 7 skipped (no GPU, game or .NET here).test_linux_update.py);hookshot_vk.so: a full build of the game andhookshot_vk.sopasses. The ABI check is clean, and the shader preparation at the end of the build ran through the changed cache and state-list readers.hookshot_vk.dllcross-built with llvm-mingw, with NVIDIA's NGX SDK, sovk_dlss.cppwas compiled. The changed host files andbuilder/setup/setup.cpass a syntax check with llvm-mingw's clang.nvn_layout.cover 529,200 texture descriptions. No size, tile or level offset differs.