Ultrawide: pause, item and map menus in the middle 16:9, sides blurred or black #1352
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!1352
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/ultrawide-menus"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
On a screen wider than 16:9, the pause/item and map menus were drawn stretched over the whole wide picture: their 16:9 layout was spread up to twice as wide, with parts of neighbouring menu pages at the sides. This keeps those menus in the middle 16:9, like the HUD. The sides get a blurred mirror of the middle (or black), so the item wheel, the menus and the map look clean at 21:9 and 32:9.
How
nx2/oot_wide.py: the HUD site's keep-whole-picture list (KEEP_VTABLES) is now a run-time mask (hos_wide_keep_mask, a bit per screen) instead of a fixed test. The menus (bits 0, 1, 3) are squeezed into the middle 16:9 like the HUD. The fades (bit 2) keep covering the whole picture.MkwVkHost.wide_menus(API version 10): the renderer sets that mask from the new option each present and gets the middle 16:9's margin back.HOS_WIDE_KEEP=<hex>overrides it for testing.1a7482c0). A pause, the map, the item wheel, a loading screen or a movie draws none, and then the window's scale pass (shaders/fg_scale.comp) fills the sides outside the middle 16:9 with a blurred mirror of the middle, or black, with a short soft edge. It allows 3 presents of grace for a frame the world skips.RX_VK_SHOT_FX=1:RX_VK_SHOTSscreenshots go through the window's scale pass, so the sides can be checked headless.tests/test_upscale_independent.py: covers the scale pass's new 20-byte push block, and the menu sides running the pass on their own.Tested
RTX 5080, Windows 11, 5120x1440 (32:9), headless runs of a save in Hyrule Field with
RX_VK_SHOT_FX=1:tests/run_all.py: the same result asmainhere (144 passed; the 13 failures are the same onesmainhas on this machine, missing tools such as clang++).Before (current main) and after screenshots attached.
The ultrawide picture itself still applies after a restart only, since the game sizes its pictures once at start-up.
BEFORE:

AFTER: (Blurred and Black Bars options)

Nice result, and the before/after makes the case well. I read it against main (
11232e8, after #1224). The geometry checks out:x0/x1are fractions of the cropped picture anduvspans the present rect, so the boundary lines up with the squeezed menu whether the picture is stretched to the window (main today) or fitted to its own aspect (#1339). At exactly 16:9 and on narrower screens (#1224),hos_wide_marginis 0, sowide_sidesreturnsside = 0and the pass is a no-op. Same with Ultrawide off:hos_wide_squeezeis 0 and the HUD patch never runs. 32:9, whether from the monitor or #1290's 5120x1440 output, gives margin 0.25 as expected. The feather sits entirely outside the middle, so menu pixels aren't touched. I don't see textual conflicts with #1339 or #1290.About the -316: 298 of those lines are the regenerated
fg_scale_comp_spv.h. I checked the sha256 on its first line against the newfg_scale.comp, and glslangValidator 1116.6.0 output matches the header byte for byte. The rest are edited lines: theKEEP_VTABLESfixed test became the mask lookup, the push block went from 8 to 20 bytes, the old "HUD is still stretched" hint is gone, and the tests were updated for both. Nothing functional was dropped.A few things I'd like to see addressed or answered before merge:
The "menu is open" signal is one hard-coded program key (
1a7482c0ec9991eanot drawn for 3 or more presents). If any gameplay scene, rendering option (DLSS, Resolution, the pipe-skip path when that pipeline variant isn't built yet), or later shader change stops that exact draw from reaching that line, live gameplay gets the outer half of a 32:9 screen blurred with nothing to show why. Have you checked a few different scene types (interiors, dungeons, Ganon's castle, fishing pond, title/file select) with DLSS on and off? A log line when the state flips would make field reports diagnosable. In-engine cutscenes and pre-rendered movies are also affected (movies now get blurred sides, see below), which the row's hint doesn't mention.The menus' background capture is squeezed too. The old
KEEP_VTABLEScomment says screens 0, 1 and 3 "redraw a capture of the whole picture". Clearing their keep bits scales every draw on those screens, including that capture quad, so the frozen world behind the item wheel and pause menus should now be horizontally compressed tosqueeze(50% at 32:9). The item-wheel panel in the after screenshot looks like it could be that. Is it, or does the capture take a different path?Blur cost and quality.
side_fillis a sparse 11x11 box (121 bilinear taps about 36 px apart at 1440p) for every side pixel. At 5120x1440 that's roughly 446M fetches per frame with poor cache locality, on top of the Lanczos/Catmullfiltered()result that then gets thrown away. It only runs in menus, but it will be noticeable on weaker GPUs. Because the taps are so far apart, fine detail (menu borders, text) also aliases into a visible plaid/grid pattern: see the top-left sides of the first after panel. A small downsampled copy (1/8 res) blurred with a few taps would be cheaper and smoother.Smaller points are inline. CI didn't report any checks on this branch.
121 bilinear taps per side pixel, spaced about 2.5% of the height apart (around 36 px at 1440p). That's about 446M fetches per frame at 5120x1440, and the gaps alias sharp detail into the grid pattern visible in the after screenshot's sides. Blurring a small downsampled copy would be cheaper and smoother. Also,
filtered(uv)(up to 16 Lanczos taps) runs for these pixels and is discarded whenoutside == 1.0; computing the side first and returning early would skip that.This is the only signal for "a menu is open", and the call sits after the pipe-skip and
RX_VK_SKIP_*early returns, so a frame where this pipeline isn't ready yet (a new area's first frames,targets_persistent) or where it's skipped counts as "no world". The 3-present grace covers short gaps, but it would help to log the transitions (vk: ultrawide sides on/off at present N) so a report of blurred gameplay can be diagnosed.With frame generation on,
framegen_scaleruns on the display thread (viapresent_image, underpresent_image_mu), whilewide_sides_tickwriteswide_nowon the replay thread without a lock. In practice onlysidechanges, so it's harmless, but it's a data race. Makingsideanstd::atomic<int>(x0/x1 are fixed once the margin is known) or writing it underpresent_image_muwould fix it. Also, made frames are about one frame behind, so the switch can lag by a frame there. That's fine, just noting it.Nit:
wide_menusdefaults to 1, so this branch (a sampled view plus alast_frame/last_valuebump) now runs on every present for every user, 16:9 included, even thoughframegen_scalethen returns early. Gating it on a nonzero margin (for example cachinghost->wide_menus's return) would keep 16:9 users on the old path.Nit (debug-only): in a windowed run, this resizes
scale.scaledto kShotW x kShotH, and the followingpresent_imageresizes it back to the window size. That's twodevice_wait_idles per shot frame. It also isn't underpresent_image_mu, so it can race the framegen display thread'sframegen_scale. Fine for headless use, but a short comment saying so would help.@ -19,2 +19,3 @@# (the screens whose UI keeps the whole picture: the item-wheel and pause / system menu backgrounds redraw a capture of# the whole picture; the event and loading fades cover it)# the whole picture; the event and loading fades cover it. Bit k of host/hos_main.c's hos_wide_keep_mask keeps screen k# whole; the Insert menu's Ultrawide menus row clears the menus' bits 1 and 3 (vk_framegen.cpp): their layout thenPer the line above, screens 0, 1 and 3 redraw a capture of the whole picture. With their bits cleared, that capture quad goes through the same
_row *= _qscaling as the menu widgets, so the world behind the item wheel and pause menu ends up squeezed to the middle 16:9 (half width at 32:9). Is that what the item-wheel panel in your after shot shows?Thanks for the careful read! Pushed fixes for all of it:
1. The "no world" signal. The
1a7482c0check now sits before the pipe-skip andRX_VK_SKIP_*returns, so a composite that is skipped or whose pipeline isn't built yet still counts as world. Every change is logged:vk: ultrawide menu sides on/off at present N (the world last drawn at present M). In a run with DLSS on (Market and Hyrule Field, title, loading, wheel, pause, map), the sides came on only for the loading screens and the pause and map menus, never in play. I haven't gone through interiors, dungeons, Ganon's castle or the fishing pond yet: my saves are early. The row's help now says that loading screens and pre-rendered movies get the same sides.2. The capture behind the menus. You were right that it changed, but it went the other way: KrysGG noticed that changing weapons made the background twice as wide. Testing each keep bit on its own showed that screen 0 is the item wheel's background (the last frame of play, blurred, HUD and all), and it draws only that. Bit 0 now stays whole, so the menus clear only bits 1 and 3. While the wheel's background program (
02ff9920) draws, we add no sides, so the wheel looks like it does on the Switch 2: the frozen frame across the whole screen, with the wheel in the middle. Pause and map still get the blurred or black sides.3. Blur cost and quality. A first small dispatch makes the picture at 1/16, each pixel the mean of its 16x16 block (16 bilinear taps). The side pixels then take 7x7 taps from that copy, about 6% of the height either way. That's about 49 fetches from a ~320x90 texture per side pixel, down from 121 spread over the whole picture. The plaid is gone, but it's still a little blocky, so a follow-up commit will smooth it further. Meanwhile, the black-bars setting is shown on the pause menu and the map (image). Side pixels wholly outside the middle return before
filtered()and the sharpen, so the Lanczos taps aren't spent and thrown away.Inline points:
wide_nowis now an 8-bytestd::atomic<WideSides>, holdingx0andside, withx1 = 1 - x0since the middle is centred. The display thread's read is no longer a race.vk_replay.cpp: the direct-view path is gated onwide_sides_possible()(Ultrawide with a margin and the row on), so 16:9 users keep the old path.RX_VK_SHOT_FXpath now has a comment saying it's for headless runs: two device waits a shot in a window, and not underpresent_image_mu.I also merged main again after #1398:
wide_menusnow comes after the newlog_flushinMkwVkHost, so the API version goes to 11.tests/test_device_lost_exit.pychecked thathos_log_flushcloses the host table, so it now checks that it followshos_sdl_gyro_statuswith members after it allowed.Tests updated: the scale-pass fixture covers the third binding, the 1/16 dispatch, and the atomic.
tests/run_all.pyshows the same results as before on this machine.Follow-up on point 3:
1badf67andb2ba650rework the blurred sides.The sides no longer mirror the middle. Each side pixel now blurs what the game draws there (a menu's neighbouring pages, for example), out of focus and dimmed to 75%. The blur is a Gaussian of 9x9 taps over a 1/8 copy, each pixel of which is the exact mean of its 8x8 block. That removes the streaks the mirror's clamp made at the middle's edge, and the blockiness of the 1/16 copy. Beside the map the game draws nothing, so those sides stay dark. Full width and black bars are unchanged.
Thanks for working through all of it. I re-read the PR at
b2ba650against405bbe9, leaving out the main merges.Everything from last time is addressed:
1a7482c0and02ff9920stores now sit right afterprogram_for, ahead ofskip_live,RX_VK_SKIP_PROG,cp->failedand the no-pipeline skip. So a composite that gets skipped, or whose pipeline isn't built yet, still counts as world. The on/off log line with the last world present is what I was after, and the row's help now mentions loading screens and movies.02ff9920draws makes sense, and theKEEP_VTABLEScomment now explains why.filtered()and the sharpen for side pixels.The inline points are done too:
wide_nowis an 8-byte atomic, the direct-view path is gated onwide_sides_possible(), and theRX_VK_SHOT_FXcomment is in. I regeneratedfg_scale_comp_spv.hfrom the newfg_scale.compwith glslangValidator 1116.6.0, and it matches byte for byte (the sha256 line matches too). The fixture covers the third binding, both side modes and the extra small-copy dispatch, andtest_device_lost_exit.pyis loosened the right way for the new table member. GitHub shows the branch as mergeable with main, and no CI checks run on it.One optional nit inline. Looks good to me.
@ -58,4 +79,4 @@void main() {const ivec2 g = ivec2(gl_GlobalInvocationID.xy);const ivec2 d = imageSize(dst);if (any(greaterThanEqual(g, d))) return;Nit, optional: 81 taps per side pixel is fine now that they come from a small, cache-friendly texture. If weaker GPUs ever show a menu-only cost, the Gaussian is separable, so a horizontal pass and a vertical pass over the 1/8 copy (or blurring the copy in place once and then sampling it with one bilinear tap) would bring this down to about 18 taps, or 1.
Fixed in
e552ab4a. The store now comes before every skip, and the transitions get logged. Thanks.@ -19,2 +19,3 @@# (the screens whose UI keeps the whole picture: the item-wheel and pause / system menu backgrounds redraw a capture of# the whole picture; the event and loading fades cover it)# the whole picture; the event and loading fades cover it. Bit k of host/hos_main.c's hos_wide_keep_mask keeps screen k# whole; the Insert menu's Ultrawide menus row clears the menus' bits 1 and 3 (vk_framegen.cpp): their layout thenAnswered: bit 0 (the wheel's background) now stays whole, and no sides are added while 02ff9920 draws. That makes sense.
Fixed: there's a 1/8 mean copy now, a 9x9 Gaussian from it, and the early return before filtered(). Looks good.
Fixed with the 8-byte std::atomic. Thanks.
Re-read at
0404628. Since my approval atb2ba650the only new commits are the two main merges (7f1d718,0404628). The PR's own files (fg_scale.comp, its SPIR-V header,vk_framegen.cpp/.h,oot_wide.py, the scale-pass fixture) are byte-identical to what I approved, and theMkwVkHostresolution in7f1d718(prompt_face, then wide_menus, version 12) was right for main at that point. Nothing new to raise in the code.It conflicts with main again though (main is at
0d8e755now, after the imgui menu and a few others). I ran a three-way merge of the overlapping files.vk_draw.cpp,vk_menu.cpp,vk_menu.handhos_main.cmerge cleanly. Three files conflict:gpu/vk/vk_api.h: main addedconst char *(*romfs_dir)(void);afterprompt_faceand is already atMKWVK_API_VERSION 12. Keep both, withwide_menusafterromfs_dir, and bump to 13 so a mismatched back end/executable pair still gets rejected.host/rx_backend.c: thevk_hostinitializer. Same order:..., hos_prompts_face, host_romfs_dir, host_wide_menus,.test_device_lost_exit.pyonly checkshos_sdl_gyro_status, hos_log_flush,, so it's fine as is.gpu/vk/vk_replay.cpp, theRX_VK_SHOT_FXshot block: main replacedmenu_record(r.cmd, shot_image, (int)(presents % 3), ...)withmenu_record_image(r.cmd, shot_image, shot_view, kShotW, kShotH). Keep your fx/blit lines and take main'smenu_record_imageline.No CI checks run on this branch. With main merged in, this looks good to me.
Merged main again (now at
e4ea245), resolved as you described:romfs_dir, thenwide_menus, withMKWVK_API_VERSION13, and main'smenu_record_imageline kept under theRX_VK_SHOT_FXblock. One small follow-up from #1448 is in450e77a: the blur's 1/8 copy is now sized from the checkedsourcerectangle instead of the raw crop, the same rectangle the crop blit above it uses. Rebuilt and re-checked at 5120x1440 on the new menu: the item wheel keeps the full-width background, and pause and map sit in the middle 16:9 with their sides.