WIP: docs(build): the lifted-chunk -O2 budget: costs, coverage and a LONGSHOT_CHUNK_OPT patch #1180
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1180
Loading…
Reference in a new issue
No description provided.
Delete branch "Marcuss2/Longshot:perf/build-opt-budget"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: docs · Risk: low · Tested on: Linux, build-time only (no game run — nothing here changes the game) · Needs: an in-game A/B decision by the Lead (commands in Testing)
Why
The hot-giant split (task-1) made a build that carries a hot list cost the same as a no-list build (7:13 -> 3:17). Task-2 asked the follow-up with numbers: what does it cost to compile more of the lift at -O2, and is any of it worth recommending? This PR answers with from-scratch builds and proposes the switch that would ship such an experiment — as a patch in the report, not applied:
nx2/build.pyis untouched here (task-1 is landing lines in it).What changed
Nothing in code. Two hand-written measurement files under the git-ignored build folder, force-added on purpose so the numbers travel with the branch:
build-linux/opt-budget.md— the report: the table, the budget answer, the recommendation, the patch, reproduction commands.build-linux/chunk_opt.patch— the proposedLONGSHOT_CHUNK_OPTenv switch next toOPTinnx2/build.py(default = today's behaviour;allor a file ofcode_*.c/f_names sends those chunks to -O2 with the light flags off). It is also embedded in the report.Measurements
Verdict: more -O2 is not worth recommending by default — the report's honest answer. All chunks at -O2 costs 23:25 wall / 1,345 s compile (9.6x chunk CPU) vs 3:29 / 139.5 s for the baseline list build, and the exe shrinks 19.1 MB (546,891,360 -> 527,756,744 B), so there is no size trade either. The sample mass outside the hot list sits in the 25 MB chunks (code_000.c holds 927 of 5,218 innermost samples); each costs +610..745 s CPU at -O2 — the split's ~240 s buys none of them whole. The only sane chunk-level A/B is the efficient-3 selection (code_002_1.c, code_002.c, code_014_1.c: +201 s CPU, ~+20-35 s wall, +6.8% sample coverage). Cheaper still: extend the hot list (+25 functions ≈ +18.7% coverage ≈ +10-40 s wall) — the split made hot pieces cost ~4-25 s each.
71b1edd8+ split line, hot listcc757427,LONGSHOT_CHUNK_OPT=allHardware: Linux, Ryzen ZNVER5 32-thread, 62 GB. Method:
rm -rf build-linuxthenpython3 -I scripts/dev_build.py buildthroughscripts/job_queue.py(mode build), one build at a time,LONGSHOT_OBJ_STORE=0 LONGSHOT_LIFT_STORE=0, no shader preparation; per-chunk times frombuild_times.json; samples counted asscripts/hotlist.pycounts (innermostf_frame) over the Market profile. The report also keeps the first all-O2 attempt as the artifact it was — a scoping bug in the patch (local vs globalCHUNK_OPT_O2) left the -O2 flag off the chunks: same 62-light-chunks line, byte-identical exe; the rerun shows 0 light chunks, 4-9x per-chunk times, a different exe.Flags and settings
One switch is proposed (not shipped here):
LONGSHOT_CHUNK_OPT— seebuild-linux/chunk_opt.patch. Default (unset) is exactly today's behaviour;alllifts every lifted chunk to -O2 (light flags off); a file ofcode_*.cnames orf_names lifts the named chunks / the chunks defining the named functions. Verified in the rerun: the light-chunk line 62 -> 0, per-chunk times 4-9x (hand-checked by the Lead: code_022.c -O1 light 55.0 s / -O1 all passes 73.4 s / -O2 223.6 s), exe bytes change. Recommendation: keep it unset by default; use it for targeted experiments (the efficient-3 set is the one worth an A/B).LONGSHOT_CHUNK_OPT(proposed)allcosts 9.6x chunk CPUTesting
Tested:
python -I -B tests/run_all.pyon this branch (report-only change): no new failures; the 17 known environmental failures also fail on main71b1edd8(test_audio_output_devices, test_bug_report, test_bug_report_defaults, test_crt_options, test_dlss_feature_size, test_failed_compile_keeps_objects, test_guest_imported, test_hardware_report, test_linux_preflight, test_linux_update, test_make_game_cache, test_missing_nup, test_mods_size_table, test_pending_guest_import, test_run_all_changed, test_vk_occlusion_queries, test_walk_key). Both measured builds compiled the whole game with 0 failed compiles. The patch's flag path is verified by the builds themselves (the light-chunk line, per-chunk times, exe bytes).Not tested: no game run — this PR changes no code the game runs, and task-2 says the Lead does the fps A/B.
Must be tested:
printf 'code_002_1.c\ncode_002.c\ncode_014_1.c\n' > /tmp/chunk_opt3.txt, applybuild-linux/chunk_opt.patch, then a from-scratch list build withLONGSHOT_CHUNK_OPT=/tmp/chunk_opt3.txt(expected ~3:40-4:00), and the in-game comparison against the current config in the Market area. Expected result: only build time changes unless fps moves; the report says what to conclude either way.Risks
What could break: nothing here — no code change lands. If the patch were landed later, its default (env unset) compiles exactly as today, and a wrong
LONGSHOT_CHUNK_OPTvalue only changes which chunks compile at -O2 (slower builds, no behaviour change; object keys name the flags).Turning it off: unset the env var; or revert the commit.
Compatibility: nothing changes for a player; no shader or pipeline cache effects (no game code change).
Issues
See the team board task-2 (opt budget); no tracker issue.
Checklist
Everyone
docs(build): ...).main(branch is71b1edd8= origin/main, no other commits).python -I -B tests/run_all.pypasses (the 17 known environmental failures are named; they fail on main too).build-linux/opt-budget.md).third_party/untouched.When it applies
Assisted-by:trailer (the commit).docs/development.mdgets the row.docs/tested/— N/A: these are build-time measurements, not game runs; the report underbuild-linux/is the record the task asked for.MKWVK_API_VERSION— N/A (renderer untouched).tests/updated — N/A (no source changed).Possibly related, for context:
Note: This is marked WIP as this shows what is possible. Compiling everything at -O2 is something worth considering for major releases.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.