feat: pictures of the destinations on the Teleport page, from your own game #2371
Open
samaBR85 wants to merge 2 commits from
samaBR85/feat/teleport-pictures into main
pull from: samaBR85/feat/teleport-pictures
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2866
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2814
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2371
Loading…
Reference in a new issue
No description provided.
Delete branch "samaBR85/feat/teleport-pictures"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The Teleport list only shows map names, and many players don't remember where a name like "Sacred Forest Meadow" or "Zora's Fountain" leads. A picture of each place makes the choice easy at a glance.
How it works now. The pictures come from the player's own game. Nothing is shipped and nothing leaves the PC. The first time Link stays a few seconds in a place (no map loading, no cutscene), Hookshot copies that frame once and keeps it in
teleport_captures.binin the game folder. The page shows the picture for Link's current age, or the other age's when only that one exists. A place not visited yet shows a "No picture yet" frame instead.Retake picture. Debug › Travel gets a Retake picture row. The player chooses one key, mouse button or controller button, then presses it in play to take that place's picture again. A button the game or Hookshot already uses is refused, and the help says what uses it. Symbol keys show by the keyboard layout's own character with a name, for example "` (Backtick)". The Controls page uses the same names.
Tip: with Free camera on, a retake gets views Link can't reach.
What changed
host/hos_main.c:oot_capture_tick, run with the player's update, tells the menu when Link has settled in a map.gpu/vk/vk_replay.cpp:teleport_capturecopies the middle of the presented frame (480x270) to a host buffer when the menu asks, and hands it over a frame later.gpu/vk/vk_menu.cpp: the picture above the help, the "No picture yet" frame, the Retake picture row, the "Teleport picture saved" notice, and the key names. One texture shows the chosen picture, with its pixels sent again when the choice changes, since the UI's descriptor pool holds 16.gpu/vk/oot_teleport_pictures.handoot_teleport_capture.h: each destination's child and adult maps, and the 47 maps that can be taken.tests/test_teleport_pictures.py: the three lists (oot_destinations, the pictures, the maps that can be taken) agree. No Pillow needed.readme.md: a paragraph in the debug tools section.gpu/vk/menu_i18n.h: the new texts in the menu's 13 languages, the old Teleport help's entries replaced. Machine-made, as the tables' other texts; the reason a Retake picture button is refused is translated before the message is made of it. The "No picture yet" line breaks into two lines of about the same width when it is long, in any language (after a particle in Japanese and Chinese).tests/test_menu_keys.pyandtests/test_fps_overlay_order.py: their stubs know the new names. The notice is drawn by its owndraw_notice, somenu_recordstays simple to stub.How it was tested
tests/run_all.py: 394 pass, 7 fail. Two of those were mine and are fixed (test_menu_keys,test_fps_overlay_order). The other 5 (test_env_on,test_vram_budget_row,test_lift_cache,test_fpx_sticky,test_bake_portability) fail on main without this change too, on this PC.test_menu_i18n: the same problems as on main (texts from before this change), none from the new entries. The menu checked on screen in English, Portuguese, German, French, Japanese and Chinese.Not tested: Linux and the Steam Deck (the game-folder path now takes
/as well as\), and AMD or Intel cards.Screenshots
Debug › Travel: the Retake picture row and its help

The Teleport page with a picture

The Teleport page before a place's picture is taken

In game: a picture saved

Thanks for this. A picture for each destination is a good idea, and the single-texture approach for the descriptor pool is a nice touch. I don't think AI-generated images are the right source for them, though. Shipping generated art in the repo isn't ideal: it only suggests each place, it doesn't look like what the player actually sees, and it's 2.5 MB of assets the project would carry from now on.
The game already has a picture for every one of these places: its own map-screen art. It can be used without anything from the game leaving anyone's PC, the same way the button prompts are built.
Where the textures are. They're in the player's RomFS under
UI/CustomTex/, named almost exactly like your files:<Map>_<Child|Adult>_NN.bntx.zs, for exampleKokiriForest_Child_00,HyruleField_Adult_00andHountedWasteland_Child_00. That spelling is the game's own.Mini_<Map>_<Age>_NN, if a smaller one ever fits better.<Dungeon>_All(for exampleDodongoCavern_AllandBottomOfTheWell_All), next to the per-floor maps (ForestTemple_2F_Mainand so on).Matched against
oot_teleport_pictures.h, 45 of the 47 destinations have one under the same name. Gerudo Training Ground and Inside Ganon's Castle are the only two that don't by name, so they'd need the nearest map or no picture.How to build them. This could happen at build time, from the player's own dump, as
nx2/prompt_assets.pydoes withUI/Layout2Archive/Commonfor the button prompts:read_zs(with the zs dictionary) decompresses the.bntx.zsfiles.bntx_texturesanddecoderead and deswizzle the Switch 2 BNTX, usingtexture2ddecoder, which is already innx2/prompt_requirements.txtnext to the Pillow this PR uses.teleport_pictures.pythen scales them to the box and writesteleport_pictures.bininto the game folder, as it does now.Done that way, nothing made from the game is committed or released, which keeps it within CONTRIBUTING's rule that nothing made from the game leaves the PC.
assets/teleport_pictures/could go entirely. The player's ownteleport_pictures/override can stay as it is, and so can the rest of the PR (the menu drawing, the single reused texture, the test).A follow-up, and one correction to my comment above. I decoded the map textures from my own dump to compare them with the pictures in
assets/teleport_pictures. I'm not posting the images, since they're made from the game, but here's how to reproduce it locally.What they are
UI/CustomTex/.../<Map>_<Child|Adult>_00.bntx.zsis a 2000x1440 hand-drawn map of the area, in the map screen's parchment style. It's a top-down map, not a scenery view, so the Teleport page would show "where on the map" rather than "what it looks like".<Dungeon>_Allis not usable. It decodes as a dark silhouette of the layout, a mask. The finished picture is the per-floor<Dungeon>_<Floor>_Main(for exampleDodongoCavern_1F_MainorForestTemple_1F_Main). Some floors have one per state instead (ForestTemple_2F_State00_MaintoState03). The other layers next to them are_Back(a flat backdrop),_Alphaand_Grad.0x2f06) and the dungeon floors ASTC 6x6 (0x3106), all 2000x1440.How to reproduce it
nx2/prompt_assets.py'szs_dictionary,read_zs,bntx_texturesandgob_offsetas they are. Itsdecodeonly handles ASTC 4x4 and BC4 (what the prompts need), so it needs one change: take the ASTC block size from the format's top byte (0x2Dis 4x4,0x2F5x5,0x316x6 and so on) and pass it totexture2ddecoder.decode_astc.UI/CustomTex, so find each by name rather than building the path.zstandard, Pillow andtexture2ddecoder.The script I used (no game data in it)
The same calls would fit in
teleport_pictures.pyat build time: decode, scale to the box, and writeteleport_pictures.bin. Nothing made from the game would need to be committed. For the dungeons I'd use the entrance floor's_Main(_State00_Mainwhere a floor has states).Thanks for this. Pictures on the Teleport list are a good idea, and reusing one texture to stay inside the UI's 16-descriptor pool is the right call. There's one real bug, though, and a repo rule the build step breaks, so I'm asking for changes before merging.
blocking
gpu/vk/vk_menu.cpp:2280:read_teleport_picturesfinds the game folder withstrrchr(path, '\\'). On Linux,GetModuleFileNameA(host/posix/win32_posix.c) returns a/path, soslashis null and the function returns before it even logs. The Teleport page then shows no pictures on Linux and the Steam Deck, with nothing inhookshot_log.txtto say why.ini_path()in the same file already does this portably: take the folder withfind_last_of("\\/")(or loop back to the last\\or/), and log on that early return too. The Linux build isn't in the PR's testing, which is how this slipped through.blocking
nx2/teleport_pictures.py:19andnx2/make_game.py:1861: CONTRIBUTING accepts the button prompts' packages (Pillow among them) "fornx2/prompt_assets.pyandnx2/prompt_draw.pyalone, at build time", and asks reviewers to flag any other use. This PR adds a new build-time Pillow use in its own script. Either move the work intoprompt_assets.py(which is where nullifyshot's suggestion of decoding the player's own map textures would naturally live anyway), or get an explicit OK from the maintainer to widen that rule, and update CONTRIBUTING in the same PR.should fix The shipped pictures themselves. nullifyshot covered the alternative well (the game's own
UI/CustomTexmaps, decoded at build time from the player's dump, like the prompts), so I won't repeat it. Two things I'd add if the AI pictures stay:assets/teleport_pictures/orPROVENANCE.md).should fix
gpu/vk/oot_teleport_pictures.handvk_menu.cpp:2306: the page always shows the child map's picture, so 16 of the 17*_Adult.jpgfiles are never shown (onlyGerudoFortress_Adultis, andGanonCastlenever is). They still go intoteleport_pictures.bin: 47 × 480 × 270 × 4 bytes ≈ 24 MB in every game folder, all read into memory the first time the page opens. Either choose the picture by Link's current age (the menu already readslink_agefor the Kid/Adult row), or drop the adult pictures that are never shown and remove them from the header. Storing the pictures compressed in the.bin(or loading only the chosen one) would also cut that 24 MB a lot.minor
tests/test_teleport_pictures.py:30: without Pillow the check printsSKIPand exits 0, whichrun_all.pycounts as a pass. DeclareRUN_ALL_MODULES = ["PIL"], astests/test_prompt_guide_lines.pydoes, so the runner reports it as skipped instead.CI is green and the branch merges cleanly onto current main. I didn't check the in-game look beyond the description's screenshot.
Both earlier blockers still stand:
read_teleport_picturesusesstrrchr(path, '\\'), so it returns early on Linux and Steam Deck.nx2/teleport_pictures.pyadds a Pillow use outside the two files CONTRIBUTING allows.The 47 committed JPEGs (2.5 MB, AI-generated, no stated source or licence) are a maintainer call. Only 31 are shown, but all 47 expand to about 24 MB of RGBA in each game folder and load into memory when the page first opens. nullifyshot's suggestion of decoding the game's own
UI/CustomTexmap textures from the player's dump at build time, like the button prompts, would avoid shipping art.Thanks for this, the page is a nice idea and reusing one texture for the 16-descriptor pool is the right call. I read the code and the thread, and I agree with Joey-Magnus's two blockers, which are still open on this head:
read_teleport_picturesusesstrrchr(path, '\\')and so returns silently on Linux/Steam Deck (use thefind_last_of("\\/")approachini_path()has), andnx2/teleport_pictures.pyis a Pillow use outsideprompt_assets.py/prompt_draw.py, which CONTRIBUTING allows alone.On the pictures, which is the question you asked me to look at: they are AI-generated JPEGs committed to the repo, so no game files are in the PR itself, and that part of CONTRIBUTING is not broken on its face. What I can't tell from the PR is the input: if any image was made from a game screenshot or art (image-to-image), the result is "made from the game" and can't be committed. The description says only "AI-generated illustrations", with no tool, terms or inputs, and the GPL-3.0-or-later licensing needs those too (Joey asked for a provenance note). Beyond that, nullifyshot's suggestion (decode the player's own
UI/CustomTexmap textures at build time, like the prompts) avoids committing any art at all, and also solves the Pillow-scope point if the work moves intoprompt_assets.py. I'd go that way, but the shipped art, its 2.5 MB, and any widening of the Pillow rule are NotRexed's call.Smaller things:
[screenshot]and no image is attached. CONTRIBUTING asks for screenshots when a change shows on screen; please attach it.RUN_ALL_MODULES = ["PIL"]declaration for the test (Joey's points).teleport_picture, the lookup loop has nobreak, so the last match wins. Harmless, but abreaksays what it means.teleport_picturereturns null with no log line; fine for now since the builder makes them all one size, but a log would help if someone hand-edits the.bin.WantUpdateswhile the previous frame is still in flight. It worked in the Windows screenshot test you describe, but it is the part I'd want checked on AMD/Intel and Linux.CI: only the title check ran (pass); the branch is "unstable" per GitHub right now, so a rebase onto main and a full CI run are worth doing after the changes. This stays a COMMENT: the decision on the assets and the rule stays with NotRexed.
Thanks, all three of you. I reworked the PR along your suggestions. The pictures now come from the player's own game, so the AI pictures are gone.
assets/teleport_pictures,nx2/teleport_pictures.pyand themake_game.pystep are removed. Nothing is decoded at build time either. The page takes its own screenshot the first time Link stays in a place, and keeps it inteleport_captures.binin the game folder. That also answers the provenance and the 2.5 MB questions.find_last_of("\\/"), likeini_path(), and logged when it can't be found.link_age), else the other age's. Only places the player has visited are stored, at 480x270.[screenshot]placeholder is replaced by real screenshots, an unreadable picture or a size mismatch is logged, and the branch is rebased on current main.@nullifyshot, thanks for digging into
UI/CustomTex. Since those turned out to be top-down parchment maps rather than views of the place, screenshots from the player's own game fit the page better.Thanks for the rework. Taking the pictures from the player's own game is a much better fit, and both earlier blockers are fixed:
game_folder_filetakes the folder withfind_last_of("\\/")and logs when it can't, and there's no Pillow and no shipped art. Merged into current main (51da0464),tests/run_all.pypasses apart from a lift-cache check that fails on main here too. CI hasn't built this head yet (only the title check has run), so nothing has been compiled on Windows or Linux in CI.Two things about when the pictures are taken, which I'd like fixed before merging:
blocking
vk_menu.cpp:4072(menu_store_capture), called fromvk_replay.cpp:3386insidedo_present: each new picture rewrites the wholeteleport_captures.binon the replay thread, in the middle of a present. At 480x270 RGBA that's about 518 KB a map, so up to about 24 MB with all 47. The first capture also reads the whole file there (read_teleport_pictures). That's a hitch every time Link settles in a new map. Please hand the pixels to a worker (or the menu thread) for the file work, and consider writing them smaller: they're shown at a fraction of that size, and zstd is already linked.should fix
hos_main.c:1310:oot_capture_tickruns inoot_update_player_cheatsfor every player, Debug tools on or not. So everyone's game folder fills with screenshots, and "Teleport picture saved" shows over the game, for a page most players never open. Taking pictures only while the Debug tools are on, or from the first time the Teleport page is opened, would keep that to the people who use it.Smaller points:
present_native_pqis skipped. Please check that a picture taken with HDR Expanded doesn't come out washed out or clipped, sincesrcis then not plain sRGB.menu_take_capture_requestreturns -1 while it's open), so someone waiting with the menu up isn't confused.