Linux support, and an AppImage of the builder for x86_64 #66

Merged

SkippySteve merged 12 commits from linux-appimage into main

2026-11-07 16:49:01 +00:00

SkippySteve commented

2026-11-07 07:59:56 +00:00

(Migrated from github.com)

Linux support for Hookshot (x86_64), and a CI workflow that builds an AppImage of the builder.

What it does

host/posix/: the Win32 calls the runtime and the Vulkan back end make, implemented on POSIX. Its windows.h is on the include path of Linux builds only, so most Windows code is untouched. Where a shared file needed a change, it's behind #ifdef _WIN32 with the Windows branch as it was (vk_main.cpp, vk_memory.cpp, vk_draw.cpp, vk_replay.cpp, hos_main.c, hos_mem.c, hos_setjmp.c). The shared-file change that also affects Windows is a / fallback where the executable's folder is found (nn_applet.c, rx_backend.c). It covers SRW locks and condition variables (futex), threads, events, fibers (ucontext), files (\ paths made /), virtual memory (the guest window is one MAP_NORESERVE mapping, so committing is a no-op and decommitting is madvise), modules (dlopen, with x.dll → x.so, SDL2.dll → libSDL2-2.0.so.0, vulkan-1.dll → libvulkan.so.1), and vectored exception handlers on SIGSEGV (crash reports).
The window, keyboard and mouse on SDL2 (display_sdl.c, loaded at run time like pad_sdl.c loads it). SDL wants the window on the main thread, so on Linux the game runs on a thread of its own (64 MB stack) and the main thread owns the window. The Vulkan surface is made through SDL (X11 or Wayland). Sound: waveOut on SDL's audio. Insert menu text: the GDI calls vk_menu.cpp makes, rasterised with stb_truetype from the system's sans font.
Movies: FFmpeg with dav1d decodes OoT's AV1 movies (Media Foundation's job on Windows).
hos_setjmp.c: a System V version of recomp_sj_setjmp / recomp_sj_longjmp. It uses the same buffer layout and the same naked-function approach as the Win64 one, so setjmp still runs in the caller's frame.
nx2/build.py builds an ELF hookshot and hookshot_vk.so. make_game.py works on Linux: a compiler check instead of the Windows SDK check, the work folder in ~/.cache/hookshot-builder, and building without the play profile when the compiler is too old to read it.
AppImage (scripts/linux/deps.sh, scripts/linux/appimage.sh, .github/workflows/linux-appimage.yml): Hookshot Builder for Linux (a Tk window with the same steps as the WPF one, plus --cli for make_game.py), carrying LLVM 23 (clang, lld), a glibc 2.35 sysroot, Python with the tools' modules, static FFmpeg + dav1d, SDL2 and the prebuilt Vulkan back end. Players install nothing. It is built in Ubuntu 22.04, so the AppImage and the games it builds run on glibc 2.35 and newer. The workflow uploads it as an artifact, and on a tag also to the release. work/ (the hot lists and the play profile) is taken from the repository when it's there, else from the latest release's HookshotBuilder.zip.

Fixes found while running it

(Linux only) A Wayland compositor gives no swapchain image back while the window can't be seen (screen asleep, another workspace), and the infinite acquire stopped the game until the screen came back. Now an acquire that gets nothing in 100 ms marks the window unseen. From then on frames are dropped without waiting, so the game runs at its own speed, until an image comes back (vk_replay.cpp).
Controllers are polled on the window thread. A second SDL thread pumping the same queue corrupted the heap once a controller was attached. With no window they keep their own thread.
AV1 movies ask for dav1d, then libaom. Host objects are keyed on the FFmpeg headers they were compiled against, so a build does not reuse an object from another FFmpeg.
Saves go in $XDG_DATA_HOME/hookshot (~/.local/share/hookshot). A user folder beside the game from an earlier build is moved there once. --user still wins.
hookshot_env.txt beside the game sets the same switches as on Windows (RX_VK_NOOPT=1 and the rest), and it is read before the window so HOS_HEADLESS and HOS_WINDOW_SIZE apply.

The timer wait that used to be listed here is in main now (pull request 55). This branch is merged through current main, including the lighting, exposure, VRAM and builder fixes.

Tested

On one PC: Arch Linux (Hyprland, Wayland), RTX 2060 Super, NVIDIA 615.78, with the supported dump (build e0f3ccfa) from its .nup:

.nup extraction, the lift and the full build on Linux, then the game started from its folder.
The title screen, New Game, the whole opening movie (AV1), the Great Deku Tree scene, and Kokiri Forest into gameplay in Link's house, with the HUD. 60 fps in the Deku Tree scene. 15-minute and 4-minute runs with no crash.
Input from the project's own controller emulation (HOS_INPUT_LIVE) through to gameplay. A real controller (8BitDo Ultimate 2C) was found through SDL. The keyboard on XWayland: E started the game, and Insert opened the menu and changed and saved an option (the back end reads the executable's key state).
Saving: the game's autosave was written, and the next start showed it on the Load screen.
Sound opened through SDL. On Wayland, and on XWayland (SDL_VIDEODRIVER=x11).
After rebasing onto b306c4d: the first-start shader preparation (14,179 programs), the new Fullscreen row of the Insert menu (on and off), and loading the autosave by keyboard and walking around Link's house.
The AppImage built with scripts/linux/deps.sh + scripts/linux/appimage.sh in an Ubuntu 22.04 root filesystem, the same steps as the workflow. That AppImage (206 MB), run on the Arch machine, checked the dump's version, built the whole game from the romfs and exefs folders with its own compiler and sysroot in 28 minutes (3 jobs), and wrote the shader list. The game it built prepared its shaders at the first start (14,179 programs in about 55 s), played the opening, and took the scripted controller. That game needs only libc and libm from the system (glibc 2.35 or newer): FFmpeg, dav1d and the C++ runtime are inside it, and SDL2 sits beside it.

Not tested: AMD or Intel GPUs, other distributions, a long play session. The builder window was only seen to open (the screens were asleep during testing); the builds above ran through its --cli, which runs the same make_game.py. The workflow file itself hasn't run on GitHub's runners yet (my account can't run Actions at the moment), though its steps ran as above.

Relation to #32

#32 also adds Linux support and arrived while this was in progress. A few differences, for comparison:

#32's recomp_sj_setjmp wraps sigsetjmp in a static function. Lifted code calls recomp_sj_setjmp by name, so the game won't link, and a setjmp inside a function that has returned can't be longjmp'd back to (the reason for the naked assembly, as the file's comment explains). Here it's a System V naked version of the same.
This one links nothing from SDL at build time (it's loaded at run time, as on Windows), keeps the WebGPU renderer out on Linux with stubs, and adds the AppImage and CI.

Happy to fold anything useful from #32 in here, or to rebase onto it if you'd rather take that one first.

Linux support for Hookshot (x86_64), and a CI workflow that builds an AppImage of the builder. ## What it does - **`host/posix/`**: the Win32 calls the runtime and the Vulkan back end make, implemented on POSIX. Its `windows.h` is on the include path of Linux builds only, so most Windows code is untouched. Where a shared file needed a change, it's behind `#ifdef _WIN32` with the Windows branch as it was (`vk_main.cpp`, `vk_memory.cpp`, `vk_draw.cpp`, `vk_replay.cpp`, `hos_main.c`, `hos_mem.c`, `hos_setjmp.c`). The shared-file change that also affects Windows is a `/` fallback where the executable's folder is found (`nn_applet.c`, `rx_backend.c`). It covers SRW locks and condition variables (futex), threads, events, fibers (ucontext), files (`\` paths made `/`), virtual memory (the guest window is one `MAP_NORESERVE` mapping, so committing is a no-op and decommitting is `madvise`), modules (dlopen, with `x.dll` → `x.so`, `SDL2.dll` → `libSDL2-2.0.so.0`, `vulkan-1.dll` → `libvulkan.so.1`), and vectored exception handlers on SIGSEGV (crash reports). - **The window, keyboard and mouse on SDL2** (`display_sdl.c`, loaded at run time like `pad_sdl.c` loads it). SDL wants the window on the main thread, so on Linux the game runs on a thread of its own (64 MB stack) and the main thread owns the window. The Vulkan surface is made through SDL (X11 or Wayland). **Sound**: `waveOut` on SDL's audio. **Insert menu text**: the GDI calls `vk_menu.cpp` makes, rasterised with stb_truetype from the system's sans font. - **Movies**: FFmpeg with dav1d decodes OoT's AV1 movies (Media Foundation's job on Windows). - **`hos_setjmp.c`**: a System V version of `recomp_sj_setjmp` / `recomp_sj_longjmp`. It uses the same buffer layout and the same naked-function approach as the Win64 one, so setjmp still runs in the caller's frame. - **`nx2/build.py`** builds an ELF `hookshot` and `hookshot_vk.so`. **`make_game.py`** works on Linux: a compiler check instead of the Windows SDK check, the work folder in `~/.cache/hookshot-builder`, and building without the play profile when the compiler is too old to read it. - **AppImage** (`scripts/linux/deps.sh`, `scripts/linux/appimage.sh`, `.github/workflows/linux-appimage.yml`): Hookshot Builder for Linux (a Tk window with the same steps as the WPF one, plus `--cli` for `make_game.py`), carrying LLVM 23 (clang, lld), a glibc 2.35 sysroot, Python with the tools' modules, static FFmpeg + dav1d, SDL2 and the prebuilt Vulkan back end. Players install nothing. It is built in Ubuntu 22.04, so the AppImage and the games it builds run on glibc 2.35 and newer. The workflow uploads it as an artifact, and on a tag also to the release. `work/` (the hot lists and the play profile) is taken from the repository when it's there, else from the latest release's `HookshotBuilder.zip`. ## Fixes found while running it - (Linux only) A Wayland compositor gives no swapchain image back while the window can't be seen (screen asleep, another workspace), and the infinite acquire stopped the game until the screen came back. Now an acquire that gets nothing in 100 ms marks the window unseen. From then on frames are dropped without waiting, so the game runs at its own speed, until an image comes back (`vk_replay.cpp`). - Controllers are polled on the window thread. A second SDL thread pumping the same queue corrupted the heap once a controller was attached. With no window they keep their own thread. - AV1 movies ask for dav1d, then libaom. Host objects are keyed on the FFmpeg headers they were compiled against, so a build does not reuse an object from another FFmpeg. - Saves go in `$XDG_DATA_HOME/hookshot` (`~/.local/share/hookshot`). A `user` folder beside the game from an earlier build is moved there once. `--user` still wins. - `hookshot_env.txt` beside the game sets the same switches as on Windows (`RX_VK_NOOPT=1` and the rest), and it is read before the window so `HOS_HEADLESS` and `HOS_WINDOW_SIZE` apply. The timer wait that used to be listed here is in main now (pull request 55). This branch is merged through current main, including the lighting, exposure, VRAM and builder fixes. ## Tested On one PC: Arch Linux (Hyprland, Wayland), RTX 2060 Super, NVIDIA 615.78, with the supported dump (build `e0f3ccfa`) from its `.nup`: - `.nup` extraction, the lift and the full build on Linux, then the game started from its folder. - The title screen, New Game, the whole opening movie (AV1), the Great Deku Tree scene, and Kokiri Forest into gameplay in Link's house, with the HUD. 60 fps in the Deku Tree scene. 15-minute and 4-minute runs with no crash. - Input from the project's own controller emulation (`HOS_INPUT_LIVE`) through to gameplay. A real controller (8BitDo Ultimate 2C) was found through SDL. The keyboard on XWayland: E started the game, and Insert opened the menu and changed and saved an option (the back end reads the executable's key state). - Saving: the game's autosave was written, and the next start showed it on the Load screen. - Sound opened through SDL. On Wayland, and on XWayland (`SDL_VIDEODRIVER=x11`). - After rebasing onto `b306c4d`: the first-start shader preparation (14,179 programs), the new Fullscreen row of the Insert menu (on and off), and loading the autosave by keyboard and walking around Link's house. - The AppImage built with `scripts/linux/deps.sh` + `scripts/linux/appimage.sh` in an Ubuntu 22.04 root filesystem, the same steps as the workflow. That AppImage (206 MB), run on the Arch machine, checked the dump's version, built the whole game from the `romfs` and `exefs` folders with its own compiler and sysroot in 28 minutes (3 jobs), and wrote the shader list. The game it built prepared its shaders at the first start (14,179 programs in about 55 s), played the opening, and took the scripted controller. That game needs only libc and libm from the system (glibc 2.35 or newer): FFmpeg, dav1d and the C++ runtime are inside it, and SDL2 sits beside it. Not tested: AMD or Intel GPUs, other distributions, a long play session. The builder window was only seen to open (the screens were asleep during testing); the builds above ran through its `--cli`, which runs the same `make_game.py`. The workflow file itself hasn't run on GitHub's runners yet (my account can't run Actions at the moment), though its steps ran as above. ## Relation to #32 #32 also adds Linux support and arrived while this was in progress. A few differences, for comparison: - #32's `recomp_sj_setjmp` wraps `sigsetjmp` in a `static` function. Lifted code calls `recomp_sj_setjmp` by name, so the game won't link, and a setjmp inside a function that has returned can't be longjmp'd back to (the reason for the naked assembly, as the file's comment explains). Here it's a System V naked version of the same. - This one links nothing from SDL at build time (it's loaded at run time, as on Windows), keeps the WebGPU renderer out on Linux with stubs, and adds the AppImage and CI. Happy to fold anything useful from #32 in here, or to rebase onto it if you'd rather take that one first.

❤️ 1

shayderrr commented

2026-11-07 14:08:12 +00:00

(Migrated from github.com)