fix(vk): the descriptor array census builds on Windows and cuts source with srccut #1729
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1729
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:fix/census-windows-near-macro"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux, RTX 4090 (two starts in the game of the exact head, the llvm-mingw cross compile for Windows and the game-free checks) · Needs: Windows, any GPU
TL;DR
Since the descriptor array census (#475) merged, main's Windows build does not compile: one of the census's counters is named near, and the Windows headers turn that word into nothing. This PR renames the counter, and also makes the census's check cut source with the shared helper that new checks are asked to use (#475 merged before its last review was answered). Compiled for Windows on Linux, the census fails on main with five errors and compiles here. On Linux the game starts from a Kokiri save on this head and the census logs its numbers; nothing was run on Windows.
Breakdown
Why
Main's Windows build stopped at the census when #475 merged: a Windows build of main fails until it is fixed. The cause is one word: Windows' headers define near and far (a leftover of 16-bit pointers) as empty macros, so a member called near is no longer a name there. The Linux checks could not see it, because nothing in them defined the Windows macros.
The last review of #475 also asked that the census's check use the shared source cutter (
tests/srccut.py, as AGENTS.md asks of a new check) instead of a brace counter of its own, which raised a bare error when a marker was missing and took the first of two matches without a word. #475 merged before that was done, so it is in this PR as a separate commit.How to reproduce
Seen on: Linux x86_64, llvm-mingw 20261104 (clang 23.1.5, target x86_64-w64-mingw32); no GPU is used.
Settings and switches: defaults.
Steps: on main
868f6be871, compilegpu/vk/vk_programs.cppfor Windows with the flags nx2/build.py gives the renderer's own files (vflags:-std=c++20 -fms-extensions -DVK_NO_PROTOTYPES -DMKWVK_BUILDING -DMKWVK_INCLUDES=2and the include folders of gpu/vk, the CPM Vulkan, SPIR-V and VMA headers). Without a Windows toolchain: run this PR'stests/test_descriptor_array_census.py --source <a checkout of main>.Expected: the file compiles.
Actual: five errors, all in gpu/vk/descriptor_array_census.h: line 48 "expected member name or ';' after declaration specifiers", and lines 107, 132, 133 and 136 "expected expression". The check run against main's header stops with the same five.
What changed
The census's counter near is near_count, which is all the Windows build needs. The census's check now compiles its test program the way vk_programs.cpp sees the header (after windows.h), and cuts source with the shared helper.
ArrayCensus::nearisnear_countingpu/vk/descriptor_array_census.h(its declaration, the increment innote(), and the two tests and the print argument insummary()); nothing else named it.FIXTUREintests/test_descriptor_array_census.pydefines near, far, IN, OUT, small and interface as the Windows headers define them before it includes the census header, and its docstring says so.tests/test_descriptor_array_census.pyimportsblockfromtests/srccut.pyin place of its own brace counter; itsblock(...)calls are as they were, andSRCCUT_CHECK_NAIVE=1shows it cuts the same text. With a marker renamed in a scratch copy it stops withsrccut: '...' is not in gpu/vk/vk_programs.cpp (cut by tests/test_descriptor_array_census.py) (renamed or reshaped? update the check with the source), and with the marker twice, with... more than once (lines 747 and 748), where the old counter passed.Measurements
Nothing to measure: a counter renamed with the same output, and a change to a check; no frame, memory or start-up cost changes.
Flags and settings
None.
Testing
Tested: on Linux (no game run, no GPU used), through the workbench's idle gate:
gpu/vk/vk_programs.cppwith the renderer's flags: main868f6be871fails with the five errors above; this head compiles. Everygpu/vk/*.cppunit (41, withoutvk_ngx_msvc_rt.cpp, which the build leaves out) passes a syntax check for Windows on this head, 40 of 41 on main.tests/test_descriptor_array_census.py: stops with the same five errors when run with--sourceon main's header, and passes here, also withSRCCUT_CHECK_NAIVE=1. Its fixture also passes the Windows-target syntax check of the fixture helper (1 captured unit; Windows execution not tested).tests/test_srccut.py,tests/test_doc_anchors.py, the program cache checks that cutvk_progcache.cpp, and the byte-compile pass.python -I -B tests/run_all.py --jobs 6 --strict, through the idle gate, with a fresh HOME) on this head and then on main868f6be871: 791 passed, 43 failed, 17 skipped on each, and the same 43 fail on both, so none comes from this PR: test_bug_report_defaults, test_capture_read_once, test_capture_readback_visibility, test_crt_options, test_damage_debug_menu, test_dev_packs, test_display_startup_failure, test_dlss_feature_size, test_draw_stream_visibility, test_failed_compile_keeps_objects, test_fsr_image_retirement, test_fsr_uniform_ownership, test_guest_imported, test_high_fps_fixes, test_host_guest_writes, test_hud_layout_row, test_libc_hooks, test_linux_preflight, test_linux_pressure, test_linux_update, test_log_open_failure, test_missing_nup, test_mods_size_table, test_nonuniform_image_arrays, test_parallel_init_order, test_pending_guest_import, test_pipeline_play_cache, test_prepare_cache_save, test_present_source_crop, test_present_thread, test_profiling_mode, test_psychov35_fsr3_fg, test_readback_timeout_source, test_replay_write_order, test_slow_frames, test_song_note_cue, test_staging_burst, test_texel_buffer_ownership, test_upscaler_rows, test_vbytes, test_vcore_escape, test_vk_occlusion_queries and test_walk_key.44ab7f749b(built from it with the project's builder, seeded from a finished work folder whose lift inputs differ only innx2/oot_patches.py; the game logslongshot v3.0.0-rc.8-50-g44ab7f749b (source 38b427da)and both binaries carry that stamp), on Linux with an RTX 4090, windowless, on the side lane beside another game withRX_VK_VRAM_MB=9000, from a Kokiri save, with warm program and pipeline caches from a build with the same translator inputs. Both starts reach Kokiri Forest with Link in control after about a minute of start-up, and no frame rate is claimed:RX_VK_PRECOMPILE=1, the other switches as they ship): 117.6 s, 10,559 frames,fatal=0 crash=0 devlost=0 nocore=0 kernel=0 nvrm=0; a picture after control shows Kokiri Forest with the hearts, the sign, Link and a Kokiri, as on any build. The census logs only when the window closes or after a preparation, and this start was ended by a SIGTERM, so it has no census line.RX_VK_LOAD=start, so the programs are read at the start, and a quit through the control bridge, which is when the census logs its close line): exit 0, 133.8 s,fatal=0 crash=0 devlost=0 kernel=0 nvrm=0, andnocore=2(two "ran without a core" lines, for the audio thread and the actor-compose thread, which each waited 5.3 s during the opening movie, before the title; start 1 had none, and no census code runs there; main was not started under this switch, so the two lines are not compared with it). The census ran and logged:vk: indexed descriptor arrays (close): 2610 in 2610 of 52174 program stages, 0 of them near a limitandvk: indexed descriptor arrays (close), fragment stages: 2610 texture (17-64: 2610); the longest 64.868f6be871plus an unrelated change, preparing every shipped program on the same GPU) loggedvk: indexed descriptor arrays (the preparation): 4580 in 4578 of 93403 program stages, 7 of them near a limit (named above),fragment stages: 4579 texture (17-64: 4573, 257-995: 4, 1024 and more: 2); the longest 1024andcompute stages: 1 texture (1024 and more: 1). Same lines, same kind of numbers: about 5 % of the stages have an array in both (4.9 % there, 5.0 % here), and here the sample is the 52,174 stages the start read, not all 93,403, so the counts are not compared one for one.Not testable by hand: a Windows link and run: this PC is Linux with no Windows installed here. The cross compile of the file and the fixture compiled with the Windows macros stand in for it.
Not tested: a link of longshot_vk.dll and a start on Windows; a start of main under start 2's switches (to compare its two
ran without a corelines).Must be tested:
vk_programs.cppandlongshot_vk.dllto link, and a start to logvk: indexed descriptor arrays (the preparation)(by hand: a person with a Windows PC)Risks
What could break: a Windows build that this PC could not link: only the one header and the check changed, and the cross compile covers every renderer file.
Turning it off: revert the commit.
Compatibility: Nothing changes. The census is not in the translator's stamp (the check still verifies that), so no cache is rebuilt, and no setting, save or file format changes.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.New on this forge; not opened on GitHub.
WIP: fix(vk): the descriptor array census builds on Windows and cuts source with srccutto fix(vk): the descriptor array census builds on Windows and cuts source with srccut@Beanstalk
Beanstalk test farm
Commit
44ab7f7· tier reviewed · approved by mooki · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On Windows, this plan builds both revisions and runs from the Kokiri checkpoint into Hyrule Field, checking that the descriptor-array census logs a summary and that the scene matches main. It takes about 10–15 minutes of game runs per tester, excluding builds. A PR build failure, crash, missing census summary, or substantial image difference fails the test. Main is reported not to compile on Windows, which may block the comparison until the baseline builds.
kokiri-start, then teleport to Hyrule Field, take the picture field to compare with main; 1 run on each side.Done (started 09:39 UTC, last report 09:47 UTC). Comment
@Beanstalkto run it again.NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 3070, 621.24.0Measurements · windows
Verdict: could not be judged on windows (2/1 report).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 3070, 621.24.0AMD Ryzen 5 5600X3D 6-Core Processor, 32 GB RAMScene: windows-census, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); windows-census: 1 game start a side; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Tested on Windows: the Must be tested item passes on the exact head
44ab7f74.PC: Windows 11 Pro 10.0.26300, RTX 4070, NVIDIA 616.56, Ryzen 9 7900X. Built with tools.lock's llvm-mingw through
scripts/dev_build.pyand the job queue.dev_build.py vkon main868f6be8stops atvk_programs.cppwith the five errors in the description:descriptor_array_census.hline 48 "expected member name or ';'", and lines 107, 132, 133 and 136 "expected expression".dev_build.py vkcompiles every renderer file (0 failed) and linkslongshot_vk.dllwith its symbols checked. A whole game pack (dev_build.py pack, the game and the renderer, 4 min 42 s) linkslongshot.exeandlongshot_vk.dll.--prepare-shaders --compile-shaders mostlogged:vk: indexed descriptor arrays (the preparation): 4580 in 4578 of 93403 program stages, 7 of them near a limit (named above), thenfragment stages: 4579 texture (17-64: 4573, 257-995: 4, 1024 and more: 2); the longest 1024andcompute stages: 1 texture (1024 and more: 1); the longest 1024. These are the same numbers as the main run quoted in the description, and the near-a-limit count (7) now comes fromnear_count.longshot_ctl.py launch --profile test --repo-save kokiri-start, pressed A until play, waited 30 s, took a picture, thenstop. It reachedKokiriForest_Childwith Link in control, and the picture shows Link and Navi by Link's house with the hearts and the prompts. The quit was clean:quit, exit 0, not forced. The log has no fatal, crash or device-lost lines, and the close line isvk: indexed descriptor arrays (close): 2661 in 2661 of 52366 program stages, 0 of them near a limit.tests/test_descriptor_array_census.pypasses, and so doestests/test_srccut.py(10 tests). I did not run the whole suite here.The runtime is the pack over an older local base (
310e7c9d, from before #475), because a base of current main can't build on Windows until this merges.