feat(vk): add live colour vision correction #1832
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1832
Loading…
Reference in a new issue
No description provided.
Delete branch "futondestroyer/Longshot:codex/colour-vision"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Adds a live, saved Colour vision correction row under Graphics → Accessibility. Off is the default. Protan and Deutan address red-green distinctions, and Tritan addresses blue-yellow distinctions. The row and help are translated into all supported menu languages.
The correction uses the lost-channel estimates from the linear Rec.709 model by Machado, Oliveira and Fernandes. It shifts colour differences onto an opponent axis while preserving luminance and neutral colours, then compresses chroma at the display gamut boundary.
The earlier Colour filter implementation has Linux RADV measurements and a headless capture path. The current PR adds correction in the final display path, including Native PQ HDR. Its independent Windows measurements and Linux follow-up below are for this implementation. Related PR #139
Start at
present_imageingpu/vk/vk_replay.cppand the sharedcolour_vision.glsl. SDR correction draws directly into the existing swapchain image in place of the blit. HDR correction runs in the existing HDR10 encoder after grading, in linear Rec.2020 light. Both game-rendered and generated frames use the same final route. The Insert menu's colour palette follows the selected mode. Off retains the existing uncorrected GPU paths.Testing
Rebase verification (November 20): Candidate
3a237573is on89c3d21e. All three commit patches were preserved, including dyst's fixture fixes. The exact Windows pack and runtime built and the runtime module hashes matched the pack. On the RTX 4070, the disposable Kokiri SDR run applied Protan, Deutan and Tritan live with built-in 2× frame generation. The renderer logged correction on both game and generated frames. The saved setting returned to Off and the game exited with code 0.Eleven focused source checks passed, including capture dimensions and INI key coverage. All seven offscreen Vulkan geometry cases passed. The checked-in present-source crop fixture failed on exact main, the former head and the rebase because its
hud_sourcestub has eight parameters while production passes nine. Both sources passed the crop behavior check with only that stub corrected in a disposable fixture copy. The rebase carries no stub edit. The full-suite comparison, HDR runs and timings below retain their original pins.Tested: Windows 11, NVIDIA GeForce RTX 4070, NVIDIA 621.01. The production build
eea12a98loaded a disposable Kokiri checkpoint. All three modes updated the live SDR output, saved, and switched back to Off. Built-in 2× frame generation showed both game-rendered and generated frames through the SDR, Expanded HDR10 and Native PQ HDR10 routes. All runs exited normally. The subsequent docs and fixture changes leave the game and renderer source identical.The live SDR captures matched an independent reference within four sRGB levels on 6,522/6,646 stable coloured samples in Protan, 6,610/6,646 in Deutan, and 6,598/6,646 in Tritan. The colour fixture, both shader builds and regeneration checks, HDR10 grading, locale, settings, scaler, frame routing and overlay-order fixtures passed. Exact-source review used the local Graphify maps for the base and candidate.
The Windows game-free checks were compared with exact
mainin shards. All changed fixtures passed. The control-client suite passed on rerun with two skips, and the intermittent stop-receipt case passed separately on both trees. The queue self-test exited without a report on both snapshots, with identical test and queue code.56 fixtures that failed or timed out on both snapshots
tests/test_appimage_player_build.pytests/test_auto_handheld.pytests/test_bug_report_defaults.pytests/test_build_long_paths.pytests/test_capture_read_once.pytests/test_crash_dialog.pytests/test_crt_options.pytests/test_damage_debug_menu.pytests/test_dev_ab.pytests/test_dev_build.pytests/test_dev_migrate.pytests/test_dev_packs.pytests/test_discover.pytests/test_display_startup_failure.pytests/test_dlss_feature_size.pytests/test_failed_compile_keeps_objects.pytests/test_fastfp_fma.pytests/test_fma_soft.pytests/test_fpx_sticky.pytests/test_frame_overlap.pytests/test_game_menu_mouse_buttons.pytests/test_guest_imported.pytests/test_high_fps_fixes.pytests/test_host_guest_writes.pytests/test_hud_layout_row.pytests/test_libc_hooks.pytests/test_lift_cache.pytests/test_lift_lookups.pytests/test_linux_preflight.pytests/test_linux_pressure.pytests/test_make_game_cache.pytests/test_make_game_ngx.pytests/test_mise_build.pytests/test_missing_nup.pytests/test_mods_list.pytests/test_ngx_sdk.pytests/test_pads_after_menu.pytests/test_pending_guest_import.pytests/test_pipeline_cache_save.pytests/test_pipeline_play_cache.pytests/test_prepare_cache_save.pytests/test_present_thread.pytests/test_profiling_mode.pytests/test_psychov35_fsr3_fg.pytests/test_readback_timeout_source.pytests/test_sdiv_overflow.pytests/test_seed_reader.pytests/test_slow_frames.pytests/test_song_note_cue.pytests/test_spawn_emit.pytests/test_upscaler_rows.pytests/test_vbytes.pytests/test_vcore_escape.pytests/test_vfpx_elem.pytests/test_walk.pytests/test_walk_key.pyStill to test:
Measurements
Verdict: With Deutan, the candidate ran at 110.0 FPS against 113.9 FPS with correction Off (-3.4%). Game-frame p99 was 10.95 ms against 11.41 ms. The process-level GPU busy values varied between repeats, with overlapping ranges.
Windows 11, RTX 4070 and NVIDIA 621.01. Disposable Kokiri checkpoint with a verified 1920×1080 game frame and 1600×900 window, SDR, V-Sync off and frame generation off. Two interleaved 25-second windows per arm after warm-up, in the order main, Off, Deutan, main, Deutan, Off. Each value is the median of the two run statistics. FPS and frame p99 come from the game trace. GPU busy is the process-level PresentMon measurement. Foreground changes affected the displayed-interval samples.
068fb8ebfdda302e(Off)fdda302e(Deutan)Game-frame p99 across the two repeats: main 11.03–15.14 ms, Off 11.02–11.80 ms, Deutan 10.95–10.95 ms. Recorded pipeline-wait frames across all six windows: 0.
Risks
The correction changes the display palette, and the fixed presets can help different scenes and players by different amounts. Choose Off to restore the uncorrected picture. The bridge's upstream captures retain the source colours. SDR shader initialisation failure leaves the existing presentation route available. There is no extra image allocation or readback in the correction path.
Screenshots
Same view from the disposable Kokiri checkpoint, cropped to the game client. Both runs used SDR with frame generation off.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.mainand here, for a change that shows on screen.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Tested in game: Windows 11, RTX 4070, NVIDIA 621.01. The disposable Kokiri checkpoint switched Protan, Deutan and Tritan live, saved the selected settings and returned to Off. Both game and generated frames were verified in SDR, Expanded HDR10 and Native PQ HDR10. All runs exited normally. The two cropped SDR views confirmed the display output.
The same view from the disposable Kokiri checkpoint, cropped to the game client. Both runs used SDR with frame generation off.
Possibly related, for context:
Completed the six matched, warmed SDR windows. On the RTX 4070, candidate Off and Deutan measured 113.9 and 110.0 FPS, with 11.41 and 10.95 ms game-frame p99. The description includes process GPU busy, the same-spot screenshots, the source-check comparison and the Linux and guide follow-up.
WIP: feat(vk): add live colour vision correctionto feat(vk): add live colour vision correctionVerdict: Request changes. The feature is well built and Off leaves every present path as it is. But the new
present_imageandframegen_hdrarguments break three tests that pass onmain: two in the source suite (test_capture_dimensions.py,test_present_source_crop.py) and the Vulkan fixturetest_present_aspect_vulkan.py. The fix is test-only and is below as a patch. Tier 1 AMD, Linux and Steam Deck runs are still owed, and the sRGB-only swapchain branch they would exercise has never run.For players with red-green or blue-yellow colour vision deficiency, this adds a saved Accessibility row that recolours the final picture and the Insert menu, live, in SDR and both HDR modes, game and generated frames alike. It makes the code better: the correction lives in one shared GLSL function. In SDR it replaces the final blit, so it adds no images and no readback, and in HDR it is part of the existing encoder pass. The row is a menu module, and nothing changes while it is Off.
Reviewed:
fdda302eagainst068fb8eb(its merge base, still the PR's base):colour_vision.h,colour_vision.glsl,colour_vision.frag,hdr_encode.comp,vk_colour_vision.cpp/.h,menu/colour_vision.cpp, and the changed parts ofvk_replay.cpp(all ofpresent_imageand thedo_presentpart),vk_framegen.cpp/.h(framegen_scale,framegen_hdr,pass),vk_imgui.cpp/.h,vk_menu.cpp/.h(set_row,menu_setting_set, the options load),vk_main.cpp's swapchain usage, every changed test, and every test that cutspresent_imageorframegen_hdrout of the source. Only skimmed: the two SPIR-V headers (their provenance lines) andmenu_i18n.h(6 strings × 13 languages, spot-read). Nothing run (a code review).Fixed in this review: nothing pushed. The patch below is committed on my side but couldn't be pushed to
codex/colour-visionfrom here, so it's yours to apply (or say so and I'll push it).Checked and fine:
do_presentandpresent_imagetake the same branches as onmain:allow_scaleisdirect_scale, which is exactly the old condition for makingdirect_view. The HDR fast path for Native PQ codes still applies whenever correction is off.present_imagecovers the game's writes.framegen_scale'spassends in a full barrier, and its outputs are SAMPLED and in GENERAL. The swapchain image goes TRANSFER_DST → COLOR_ATTACHMENT → TRANSFER_DST around the draw, and its older content is kept, so the letterbox clear stays outsiderenderArea(LOAD_OP_DONT_CAREapplies only inside it).dev.swap_view_format, the format ofdev.swap_views. Both sRGB cases resolve correctly: an sRGB-only chain drawn through sRGB views gets linear output, and a mutable sRGB chain drawn through UNORM views gets encoded output. Every surface allows COLOR_ATTACHMENT, and Vulkan 1.3 dynamic rendering and sync2 are already required at device creation.dot(axis, Y709) == 0for all three axes, so luminance holds in SDR. In HDR,shift - dot(shift, Y2020)keeps Rec.2020 luminance. The gamut scale can't divide by zero. The CPU copy incolour_vision.hgives the same result as the GLSL in SDR.present_image_muserialises the pipeline map and the statics. Each row change, including the bridge'smenu_setting_set, goes throughset_rowand on toon_change, andmenu_initseeds the mode after the options load, so the atomic can't fall behindopts.colour_vision.preset_exempt, no restart and its key inFROZEN. The docs row and section are there, and the menu-section tests are updated.test_present_geometry_tracefails ifallow_scale &&or!scaled &&is undone.test_colour_visionpins the three reference colours, greys, luminance and gamut.Blocking (to change before merge)
tests/test_capture_dimensions.py:37asserts the old textpresent_image(src, sw, sh, crop, direct_view);, whichvk_replay.cpp:5579no longer has. It stops at that assert on this head and passes onmain. Its fixture'spresent_imagestub also needs the two new parameters.tests/test_present_source_crop.pystitches the definition offramegen_hdr, whose newint colour_visionhas no default (the default is only invk_framegen.h). The template's call atpresent_source_crop_test.cpp:150passes 8 arguments, so the fixture no longer compiles.tests/test_present_aspect_vulkan.pypastes everything inpresent_imagefromconst PresentAspect aspect{up to the frame log. That span now usesallow_scale,source_srgb,index,dev.swap_views,menu_colour_vision_mode,colour_vision::enabled,colour_vision_sdrand a 9-argumentframegen_hdr, andpresent_aspect_vulkan.cppdefines none of them. The source suite skips this test (it needs a device), butrun_vulkan_fixtures.pyand the lavapipe workflow run it.None of the three is on the Testing section's "fails on both" list, so they probably didn't run on your PC (the first two need
--headers). The patch makes each accept both the old and new source, as your other fixture updates do, and runs it with correction Off. It also movescolour_visionto its alphabetical place inFROZENwith the table's comment back on its own line, and makesframegen_hdr's default0(it wasfalse) with a doc line. I wrote it by reading and haven't run it, so please run those three tests (andtest_menu_ini_keys.py) once it's in.The patch (on
fdda302e)Suggestions (the author may take or leave)
vk_replay.cpp:6008: the mode is read twice per frame, once indo_present(to decide whether to makedirect_view) and once inpresent_image. If the row flips between those two reads,present_imagefinds correction on with no view, logs "unavailable for this presentation route" once and never again, so a real failure later goes unreported. Passing the modedo_presentdecided on (assource_srgbis passed) would close that gap.test_colour_vision.pypins the CPU copy only. The GLSL has its own copy of the coefficients and axes, and nothing checks that the two agree. A test that reads both sets of constants would catch a one-sided edit, which would leave the menu and the picture corrected differently.main, e.g. on #1852).Questions
mainat 101.66 fps against this PR's Off at 113.90 fps is a 12% gap between two code paths that are the same when Off. Wasmainbuilt the same way (the same pack over the same base, and the same runtime folder and options file)? It doesn't change the verdict, since Off can't cost anything here by construction. But it means the 3.4% for Deutan sits within the noise of these runs. A Deutan cost of about 0.3 ms per frame for one full-screen pass at 1600×900 is plausible, and it's opt-in.Testing: Windows 11, RTX 4070, NVIDIA 621.01, by the author: all three modes live in SDR, Expanded HDR10 and Native PQ, game and generated frames, saved and back to Off, with before/after shots. Missing, and it holds the merge (Tier 1, a present path): AMD RX 6000-9000 on Windows and Linux/RADV, and the Steam Deck, with Off and one mode. A Linux surface with no UNORM format takes the
target_srgbbranch ofcolour_vision.frag, which NVIDIA on Windows never does. CI on this head: the Python source checks and SPIR-V fixtures were skipped, so they couldn't catch the three tests above. The guide check fails, as onmain.Overlaps and merge order: #139 is the same feature, built into
fg_scale.comp. It doesn't merge cleanly withmain, and its row is invk_menu.cpp's table, which the menu-module rule no longer allows. Only one should merge. On the code, I'd take this one: it's on currentmain, it's a module, it covers Native HDR, and it corrects the menu with the picture. Simon's Linux RADV runs and farm setup could cover this PR's AMD gap. The two authors and a BigBean should settle it on that PR.@ -63,3 +63,3 @@/// else the SDR picture, expandedVkImage framegen_hdr(VkCommandBuffer cmd, VkImageView src, uint32_t sw, uint32_t sh, const int crop[4], uint32_t dw, uint32_t dh,bool pq = false);bool pq, int colour_vision = false);= 0rather than= falsefor an int. This default is only here, not on the definition, andtest_present_source_crop.pycompiles the definition and calls it with 8 arguments (present_source_crop_test.cpp:150), so that fixture no longer compiles (blocking 2; the patch in the summary).@ -5568,3 +5577,3 @@if (dev.offscreen) frame_log_present(); // (no window: the copy into the offscreen image above is its present)if (dev.offscreen || interpolated) return;present_image(src, sw, sh, crop, direct_view);present_image(src, sw, sh, crop, direct_view, source_srgb, direct_scale);tests/test_capture_dimensions.py:37asserts the old text of this call (present_image(src, sw, sh, crop, direct_view);), so it stops at that assert on this head, and its fixture'spresent_imagestub has 5 parameters. The patch in the summary takes both forms (blocking 1).@ -5997,2 +6006,3 @@const VkImage scaled = src && view ? framegen_scale(f.cmd, view, src, sw, sh, crop, rect.w, rect.h, native_pq) : VK_NULL_HANDLE;const VkImage scaled = allow_scale && src && view ? framegen_scale(f.cmd, view, src, sw, sh, crop, rect.w, rect.h, native_pq) : VK_NULL_HANDLE;// (longshot) the Insert menu's HDR: the picture (scaled, or the game's own with its crop) to HDR10, copied in as the SDR one isconst int colour = menu_colour_vision_mode();Suggestion:
do_presentreads the mode separately to decide ondirect_view. A flip between the two reads gives correction with no view, which logs the one-time "unavailable" line here, and a real failure later then goes unreported. Passing the mode along withsource_srgbwould make the two agree.@ -6002,2 +6012,3 @@: framegen_hdr(f.cmd, view, sw, sh, crop, rect.w, rect.h, native_pq, colour)): VK_NULL_HANDLE;if (hdr_image) {const bool corrected = correct_sdr && index < dev.swap_views.size() && colour_vision_sdr(test_present_aspect_vulkan.pypastes this span (fromconst PresentAspect aspect{to the frame log) intopresent_aspect_vulkan.cpp, which defines none ofallow_scale,source_srgb,index,dev.swap_views,colour_vision_sdr,menu_colour_vision_mode,colour_vision::enabled, or a 9-argumentframegen_hdr(blocking 3; the patch in the summary).@ -21,2 +21,3 @@# (the keys of 2026-11-18, main c507b98c, as players' files hold them: odd ones included, "autosave:" and "amd_anti-lag_2")FROZEN = { # (a key: the value its row saves, as players' files hold them)FROZEN = {"colour_vision": "&opts.colour_vision", # (a key: the value its row saves, as players' files hold them)Nit: the table is in key order and this line took the table's own comment. In the patch,
colour_visiongoes aftercamera_speedand the comment goes back onFROZEN = {.Verdict: Comment. The three broken tests from my review are fixed in
1c499df4(pushed tocodex/colour-vision, on top offdda302e). What still holds the merge is the Tier 1 hardware run and the choice between this PR and #139, not the code.Reviewed:
1c499df4againstfdda302e: only my own commit, which is the patch from the review above with nothing added. Nothing run.Fixed in this review:
1c499df4.test_capture_dimensions.py,test_present_source_crop.pyandtest_present_aspect_vulkan.pynow follow the newpresent_imageandframegen_hdrarguments (each accepts the old source too, and runs with correction Off).colour_visionis in its alphabetical place inFROZEN, andframegen_hdr's default is0, with a doc line.To do before merge
test_capture_dimensions.py,test_present_source_crop.py,test_menu_ini_keys.pyand, on a GPU,run_vulkan_fixtures.py'stest_present_aspect_vulkan.pyon this head. I wrote the commit by reading. If any of them fails, it's mine to fix.colour_vision.frag'starget_srgbbranch, which hasn't run yet.The suggestions and the measurement question from the review above still stand. None of them blocks.
1c499df4a73a2375733fRebased onto
89c3d21ewith all three patches preserved, including the fixture follow-up from dyst. Verified3a237573in the Windows pack and in Kokiri SDR with built-in 2x FG. All three modes applied live, both game and generated frames used correction, the saved setting returned to Off, and the game exited cleanly. Eleven focused checks and seven offscreen Vulkan cases passed. The crop fixture stub error reproduces on exact main and the former head. See Testing for the detailed scope and remaining checks.