WIP: perf(build): the runtime's own files built with a play profile of their own (RX_PGO_RT, off unless set) #1986
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1986
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/build-pgo-rt"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The hot lifted pieces (
RX_PGO) and the renderer (RX_VK_PGO, the shipped profile) are built with a play profile; theruntime's own files (
host/*.c: the OS layer under every game thread, the input, the sound, the threads' waits) areplain
-O2with none. This addsRX_PGO_RT, off unless set:RX_PGO_RT=gen:host/*.c(and on Linux the executable's fourhost/posixfiles; the renderer's own copy ofwin32_posix.ckeeps its flags) compile counting their branches with atomic adds (-fprofile-update=atomic: everygame thread runs them), and
host/pgo_rt.cis in the executable, so a run withHOS_PGOset to a file name writestheir counters through that name's
.resetand.dumpfiles as anRX_PGO=genrun does (with the hot pieces' toowhen both are set; their adds stay plain, as on
main).RX_PGO_RTset to a merged profile file: they are built with that profile. It is read once and every compile readsa copy of its bytes,
build/rt_profile/<hash>.profdata; the hash sits beside the flag(
-DLONGSHOT_RT_PROFILE_SHA1), so the object key follows the profile's contents, not its path or a later merge. Afterthe compile the build says how many runtime files found their counts and names those clang noted (out of date,
missing, none); it stops when none did.
RX_VK_ONLY) and the release'sthird-party objects never open it.
What changes for the existing counting builds:
host/pgo_rt.c, the writerRX_PGO=gen(and the renderer's copy forRX_VK_PGO=gen) already use, now clears and copies the counters word by word with atomic stores and loads, writes<file>.tmp, checks every write, moves it into place and only then deletes<file>.dump(a failure is said once andtried again while the marker stays); its marker names have no length limit; a failed allocation or thread start is said
and cleaned up. A counting build (
RX_PGO=gentoo) stops on a clang older than 19, whosellvm-profdatareads neitherraw format the writer writes. Nothing else changes: no object of a build without these switches, not the lifted code,
not the renderer's objects. It is compile-only (left out of the lift key in
make_game.pyand dev.py), anddev.py packs packwith it set builds the game.docs/dev/developing.mdgets "Record the runtime's play profile" (count,runtime, record, merge, build; PowerShell, the queue, files under
work/,RX_PGO=genleft unset since its dump holdsthe game's counters),
docs/dev/reference/build.mdits row, andHOS_PGO's row ingpu/vk/switches.defnames it.Left for a separate change, already on
mainforRX_PGO=gen: on Linux,host/posix/win32_posix.creads the processaffinity in a constructor that the writer's thread, started from another constructor, can race.
From the local branch
exp/x86-asm-sync(RX_PGO_RTof 4eed88f7, with 8c5ecbe9's fix of the profile-key test foldedin: the test admitting the RT define; on main that fix has nothing of its own to fix, so it is not a separate PR). Not ported: the branch also gave the renderer this profile and rewrote
host/pgo_rt.c(both modules onHOS_PGOwith a.vkfile, a write at exit, clang's own profile runtime); that wouldchange main's
RX_VK_PGO=genandRX_PGO=genrecordings, andRX_VK_PGO_FILEalready records the renderer.Read first:
rt_profile_flags,rt_profile_reportand the lines that usert_fl,host_builtandcountinginnx2/build.py;host/pgo_rt.c;tests/test_rt_profile_build.py(new).Testing
Tested: Windows 11, i7-9700, RTX 4060, llvm-mingw clang 23.1.5, Python 3.14.7 (the Builder's); game-free checks
only. No
RX_PGO_RTbuild has run in the game on this PC yet and no runtime profile has been recorded (Still to test).Each check ran in a hidden console of its own, through the PC's job queue, on head
348857bb(this branch onmain@6e1521a7;mainhas since gainedf1f2dc5b, #1870, which changes another row ofgpu/vk/switches.defand mergescleanly, so these runs are of
348857bbas pushed):python -I -B tests/test_rt_profile_build.py(its compiler: LONGSHOT_MINGW's clang): "Ran 13 tests", OK. The jobplan from
build.mainfor unset, gen, gen withRX_PGO=gen, a profile (compiled from its copy, the copy unchangedwhen the profile is merged again), a missing profile (stops the game build), the renderer alone and the third-party
objects with a missing profile (never opened);
rt_profile_report; the clang check; the routes. The Windows roundtrip: 1,000,000 calls on four threads after a reset counted exactly, a blocked write that keeps
.dumpand says soand then completes, a UTF-8 folder with the game's manifest, the merged profile used with every count found, and
clang's out-of-date and unprofiled notes through
-Wno-everything.1ca31bbd(before the fix), in the same job: 5 failures and 6 errors. The rendereralone stops on the missing profile ("the profile cannot be read"), the counting flags have no atomic adds, the
profile is compiled from its own path, and the round trip counted 324,706 of the 1,000,000 calls (an earlier run:
887,320). The errors are the functions the fix adds (
rt_profile_report,pgo_writer_supported).python -I -B tests/test_profile_path_key.py: "Ran 2 tests", OK.tests/run_all.py --strictover the other checks that read the build's compile path orswitches.def(
test_a64_helper_build_flags,test_vk_pgo,test_build_long_paths,test_compile_stops_on_exception,test_switches_removed,test_cpu_target,test_image_size_guard,test_build_cleanup_scope,test_failed_compile_keeps_objects): 8 passed, 1 failed (test_failed_compile_keeps_objects.py), and the same 8and 1 on
main@6e1521a7in the same job. The full--changedrun was not repeated on this head. The renderer checks that the opener's earlier run could not compile (no Vulkan headers:RX_CPMunset), run again with the headers at348857bband onmain@6e1521a7:test_pipeline_states.pypasses on both;test_recorder_sync.py,test_present_queue_default.py,test_target_reuse_memory.pyandtest_vk_occlusion_queries.pyfail onmaintoo with the same errors.Still to test:
RX_PGO_RT=genpack, a run withHOS_PGOset in a heavy place (its.resetfile once loaded, its.dumpfile after a few minutes),llvm-profdata merge, then a pack withRX_PGO_RTset to the merged profile measured against
main(CPU ms a frame, fps), expecting the merge to succeed and theprofile build to start and play (by hand)
RX_PGO_RT=genbuild links and writes its counters (host/pgo_rt.c's ELF path, as forRX_PGO=gen),and the game-free round trip with the POSIX shim added to
tests/test_rt_profile_build.py(a later round; by hand)head with a recorded profile (a later round; by hand)
Measurements
Verdict: not measured in play: the switch ships off, and a profile has to be recorded before a build with one can
be compared (Still to test). Only the game-free checks in Testing have run for this PR.
Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5; the checks in Testing, through the PC's job queue, at the test's
own commit and at the fix, in one job
6e1521a7with66df7649and the new tests (1ca31bbd)348857bbtests/test_rt_profile_build.pytests/test_profile_path_key.pyRisks
Off (unset), every flag and object key is as on
main:rt_flis empty andcountingisRX_PGO=genas before. Set,it changes only developers' builds of
host/*.c: a counting build is slower (never measure it); a profile build is abuild nobody has measured yet.
RX_PGO=genandRX_VK_PGO=genrecordings go through the changed writer (atomicclear and copy, the dump written whole before its marker goes), and an
RX_PGO=genbuild now refuses a clang olderthan 19. Players notice nothing: no build they get sets these. A revert is enough.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
Recommendation: Request changes. A stale runtime profile can stop a renderer-only build, and the new recording path can lose counts when game threads update them concurrently. Fix those paths and verify that a real game build consumes a usable profile before judging the performance gain.
Adds a developer-only build option to record and apply branch profiles for the executable’s host runtime. It reuses the existing writer and object cache, which keeps the build integration small. The cost is a new recording workflow and a recompile of the host jobs whenever the selected profile changes. The default build keeps its existing optimization flags.
@fixwill do (16)nx2/build.py:1816: The builder opens RX_PGO_RT for any standalone invocation, before it filters out jobs the command will never build. (more on the line)nx2/build.py:1300: The generated counters use Clang’s default non-atomic updates. (more on the line)nx2/build.py:1316: The cache key hashes one read of the mutable profile, but parallel Clang jobs reopen the original path later. (more on the line)host/pgo_rt.c:118: The watcher deletes .dump regardless of whether it wrote a complete profile. Renaming the output directory or filling the drive makes the documented completion signal disappear while the profile is absent or truncated. The unchecked allocation at line 84 can also crash the recording process. Check allocation and every write and close, publish the completed file atomically, and acknowledge the marker only after success. Report failures so a developer can retry.host/pgo_rt.c:108: The watcher silently truncates the marker paths. A valid Windows output path of 255 bytes already overflows the 260-byte buffer once .reset and its terminator are added. The requested marker can be missed, or a different truncated path can be acted on. Construct both paths without truncation and report OS path errors. Add a round trip using a long UTF-8 path and the game’s Windows manifest.host/pgo_rt.c:54: The new recording mode inherits a writer that emits raw format 10 for every Clang through 22. For example, LLVM 17 uses raw format 8, so its own llvm-profdata cannot merge a dump from this writer. Use the selected compiler’s profiling runtime where available, or reject unsupported recording toolchains at build time and document the supported pair.host/pgo_rt.c:130: The reused watcher starts from a constructor and ignores allocation and thread-creation failures. A failed strdup can pass NULL to the worker, and a failed CreateThread leaks the path and leaves no watcher. Successful startup also leaves the handle open. On Linux the worker can reach apply_process_affinity before the separate affinity constructor has finished writing the plain mask. Start after affinity setup or make that setup a thread-safe once operation, and handle path and thread ownership explicitly.nx2/build.py:1931: The host jobs inherit -Wno-everything, so a readable renderer-only or wholly stale profile can produce a successful runtime build while applying no useful counts. (more on the line)docs/dev/developing.md:502: The recipe uses POSIX environment syntax on a feature first tested on Windows, and jumps from making a pack to running it. (more on the line)docs/dev/developing.md:510: The blanket description of the dump is false with the supported combined counting mode. (more on the line)tests/test_rt_profile_build.py:122: The build coverage asserts source strings, while the round trip supplies its own compile and link commands. (more on the line)tests/test_rt_profile_build.py:162: The only record/merge/use test skips Linux, leaving ELF section retention and the POSIX watcher untested. (more on the line)tests/test_rt_profile_build.py:148: On Windows, long is 32-bit and the sum over 100,000 calls is about 10.8 billion, so the fixture itself has signed overflow. (more on the line)For the author (2) (a run, a measurement or a decision)
Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request),Website / Website on Cloudflare (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The author reports 7 runtime-profile checks and 2 profile-key checks passing on Windows at
66df7649. The earlier comparison used tests copied onto main at51f5f9da. There are no in-game results, clean build time or size figures, or paired frame-time measurements against6e1521a7.Other systems: Lifted code, A64 helpers, third-party audio libraries and renderer-module jobs keep their own flags. The Linux executable also profiles its POSIX window and Win32 shim jobs, whose renderer copies keep their original flags. An enabled profile can alter host-side input, audio, waits, pacing and presentation on every GPU and on Steam Deck. Those paths have no play coverage here. Save formats, shader cache formats and the renderer API are untouched. Renderer-only and prebuilt-object commands can currently fail before the job lists are filtered.
Switches: RX_PGO_RT is build-only and defaults to unset or empty. Opt-in is appropriate for a developer experiment with no shipped profile. The build reference has its row. HOS_PGO still defaults to unset and now also drives the executable’s writer in RX_PGO_RT=gen builds. Its runtime switch-table row is updated, and setting it still disables the crash window.
Reviewed:
66df764957against6e1521a70a: Read the full nine-file diff, the new runtime-profile test and the changed profile-key test in full, and the affected sections and callers in build.py, make_game.py, devtools/build.py and devtools/packs.py. Read host/pgo_rt.c, the relevant POSIX shims, toolchain and test-runner paths, AGENTS.md, CONTRIBUTING.md, the Forgejo template and the developer standards. Skimmed adjacent profiling docs, CI setup and the run summaries supplied in the description. The rest of the large build modules was outside the affected paths. No local review checklist or sibling Tooling checkout was present.; nothing run (a code review).Checked and fine:
Testing: Static review only. The reported Windows checks pass at HEAD. The broader changed-source run was on the earlier rebase and the author reports its 23 failures on that base as well. No comments or reviews contain additional runs. The supplied CI checks are pending.
Overlaps and merge order: None found in the checkout or supplied review data. #1948 already landed in the merge base.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -497,0 +499,4 @@The runtime's own files (`host/*.c`) build at `-O2` with no profile unless `RX_PGO_RT` names one (an experiment: noprofile is shipped, and no gain is measured yet):1. Build a counting game: `RX_PGO_RT=gen python -I scripts/dev.py packs pack <base> rt-count` (`host/pgo_rt.c` writes theL10 (minor): The recipe uses POSIX environment syntax on a feature first tested on Windows, and jumps from making a pack to running it. PowerShell cannot run the assignment as written, and starting the base executable records nothing from the pack. Show the PowerShell form, how to assemble the runtime from the pack, and the queued build and run commands. Use quoted absolute paths in a local ignored directory, since pack builds can change checkout and the profile files are not ignored by extension.
@ -497,0 +507,4 @@4. Build with it: `RX_PGO_RT=<rt.profdata> python -I scripts/dev.py packs pack <base> rt-prof`, and measure that packagainst `main`. A function changed since the recording compiles without its counts.Like the renderer's, it holds function names, shape hashes and branch counts of Longshot's own code, nothing of the game.L11 (minor): The blanket description of the dump is false with the supported combined counting mode. With RX_PGO=gen and RX_PGO_RT=gen, HOS_PGO writes lifted-game function records as well as host records. State that the runtime-only recipe must leave RX_PGO=gen off, and say explicitly to keep combined captures local.
@ -1292,0 +1297,4 @@# run's counters); its contents' hash is in the flags as the other two profiles' are, so a new profile compiles these files# again and the same one does not. Off unless set: every other object, and these without it, as before. The renderer keeps# its own profile (RX_VK_PGO); host/pgo_rt.c itself counts nothingRT_GEN = ["-fprofile-instr-generate", "-mllvm", "-disable-vp=true"]L2: The generated counters use Clang’s default non-atomic updates. Two game threads entering the same host wait or timing routine can lose increments. The watcher also clears live counters with memset and reads them through fwrite while game threads are updating them. Use atomic updates and atomic reset/snapshot operations, or quiesce the counting threads for those operations. Cover the combined RX_PGO=gen mode too, since it shares the same counter section.
@ -1292,0 +1313,4 @@with open(prof, "rb") as f:# (the profile's contents beside its path, as the hot pieces' and the renderer's have them: the object key follows# the contents, tests/test_profile_path_key.py)flags = ["-fprofile-instr-use=" + prof,L3: The cache key hashes one read of the mutable profile, but parallel Clang jobs reopen the original path later. If a developer replaces the profile with a fresh merge during a build, objects compiled against the replacement can be stored under the old digest and reused by later packs. Snapshot the selected bytes to a stable file and compile every job against that snapshot.
@ -1780,1 +1813,4 @@[os.path.join(rt64, "a64_rt.c")]# (RX_PGO_RT: the runtime's own files' profile flags; a counting build of them or of the hot pieces has host/pgo_rt.c, the# counters' writer, in the executable)rt_fl = rt_profile_flags() if a.standalone else []L1: The builder opens RX_PGO_RT for any standalone invocation, before it filters out jobs the command will never build. For example, an exported profile path that has been deleted makes dev.py vk or a standalone --third-party-objects build exit. Resolve the profile only when the game host jobs will actually be compiled.
@ -1891,6 +1928,7 @@ def main():DEPFILE_JOBS[obj] = (host_blob, host_roots, extras)jobs.append((cc, s, obj,host_fl + ["-MD", "-I" + os.path.join(OPUS, "include")] +(rt_fl if os.path.basename(s) != "pgo_rt.c" else []) + # (RX_PGO_RT; the counters' writer counts none)L8: The host jobs inherit -Wno-everything, so a readable renderer-only or wholly stale profile can produce a successful runtime build while applying no useful counts. The build still prints that it is using the profile. Surface missing and stale profile diagnostics for this mode and require evidence that runtime functions matched, while allowing individual functions to fall back after edits.
@ -0,0 +119,4 @@def test_the_build_gives_it_to_the_runtimes_files_alone(self):# (main()'s lines, as they are: the flags only on host/*.c's jobs and never on the counters' writer; the writer in# the executable, and its link without clang's profile library, whenever something counts)self.assertIn('rt_fl = rt_profile_flags() if a.standalone else []', BUILD)L12: The build coverage asserts source strings, while the round trip supplies its own compile and link commands. Both can pass if main drops the writer from the final object list or adds renderer jobs with the wrong flags. Capture the actual jobs and executable link command from build.main and assert the flags and inputs for unset, gen, use, combined counting and renderer-only builds.
@ -0,0 +145,4 @@int main(void) {const char *f = getenv("HOS_PGO");char dump[MAX_PATH];long s = 0;L14: On Windows, long is 32-bit and the sum over 100,000 calls is about 10.8 billion, so the fixture itself has signed overflow. It also accepts a profile with every count zero, since it checks function names and matching hashes only. Use defined 64-bit arithmetic, run known branch distributions before and after an acknowledged reset, and assert the post-reset counts in the merged profile.
@ -0,0 +159,4 @@"""@unittest.skipUnless(os.name == "nt", "host/pgo_rt.c's Windows writer: the round trip runs on Windows")L13: The only record/merge/use test skips Linux, leaving ELF section retention and the POSIX watcher untested. The suite also forces --cc clang over CC, and a missing llvm-profdata becomes a successful unittest skip even in the strict suite. Add the ELF round trip with the real POSIX shim and link flags, honor the selected compiler, and make missing required tools visible to strict CI.
Recommendation: Request changes. The new profile path must be scoped to the jobs that consume it, and the recorder must produce reliable counts before any speed result can be interpreted.
I independently confirm the unrelated-build failure and multithreaded-counter blockers reported at this head.
rt_profile_flags()opens the specified profile for standalone renderer-only operations before host jobs are filtered. The test fixture compiles its own commands, so it cannot catch this route. Add an actual job-plan case with a missing runtime profile for a renderer-only build. Atomic Clang increments alone would still leave the watcher clearing counters with a rawmemsetand reading them withfwritewhile game threads run. Define synchronized reset and snapshot behavior as well, then validate the actual writer on Windows and Linux. Ensure dump acknowledgement follows a confirmed complete write. The cache key hashes bytes from a mutable path that parallel compiler jobs subsequently reopen, so give the jobs the same stable snapshot the key hashes.Does it help: Unproven. The author reported seven runtime-profile and two profile-key checks on Windows. The local compiler/profile round trip is single-threaded, skips Linux, and does not prove that a game build consumes a useful profile. No paired gameplay or clean build time and size result exists, and CI is pending. Keep the developer switch off by default. A normal gameplay farm plan cannot generate, merge, and consume an exact-head runtime profile and then compare the requested build-time switch. Use an explicitly supported multi-stage flow with matching compiler and private inputs once correctness is fixed.
Reviewed:
66df764957against6e1521a70a. Read the full patch and the compile, pack-routing, linker and runtime profile-writer paths. No code executed.Pushed
348857bbonmain@6e1521a7, for the reviews of66df7649.1ca31bbdadds the failing tests: build.main's real job plan, and a Windows round trip with four threads, a reset and a blocked write.348857bbis the fix. The profile is now read only where the runtime's files are compiled, and every compile reads a stable copy named by its hash. Counting uses atomic adds, and host/pgo_rt.c clears and copies the counters atomically and deletes .dump only after the profile is in place whole. Counting builds need clang 19 or newer, a profile build reports which runtime files matched, and the recipe is reworked. Summary, Testing, Measurements and Risks describe this head. The Linux writer run, the in-game measurement, and build times and sizes are still in Still to test.Re #1986 (comment): done in
348857bb, with the failing tests first in1ca31bbd. The profile is read only when host/*.c are compiled, so RX_VK_ONLY and the third-party objects never open it; tests/test_rt_profile_build.py now runs build.main's real job plan, including a renderer-only build with a missing RX_PGO_RT. RX_PGO_RT=gen counts with atomic adds (-fprofile-update=atomic), and host/pgo_rt.c clears and copies the counters word by word with atomic stores and loads and writes the copy. It writes .tmp, checks every write and the close, moves the file into place, and only then deletes .dump; a failure is reported and the marker kept. Every compile reads a copy of the profile named by the hash the key holds (build/rt_profile/.profdata). On Windows the round trip counts exactly 1,000,000 calls made on four threads after a reset, and checks a blocked write. The Linux run of the writer is not covered by this push: it needs a Linux machine and is listed under Still to test. The switch stays off by default, and the measurements are pending.Re #1986 (comment) (L1): fixed in
348857bb. rt_profile_flags runs only when the build compiles the runtime's files (standalone, not RX_VK_ONLY, not --third-party-objects). Plan.test_the_renderer_alone_never_opens_it and test_the_third_party_objects_never_open_it run build.main with a missing profile and check that it plans without opening it; both fail at1ca31bbd.Re #1986 (comment) (L2): fixed in
348857bb. RT_GEN adds -fprofile-update=atomic, and host/pgo_rt.c's reset and dump now clear and copy each counter word with atomic stores and loads, and write the copy, not the live section. The Windows round trip runs 1,000,000 calls on four threads after a reset and requires exactly that count in the merged profile. Combined mode (RX_PGO=gen too) is in the job-plan test; the hot pieces' own adds stay plain as on main, and the description says so.Re #1986 (comment) (L3): fixed in
348857bb. The profile is read once, and its bytes are copied to build/rt_profile/.profdata, which every runtime compile reads. A new merge at the same path cannot reach objects stored under the old hash. Plan.test_profile_is_read_once_and_compiled_from_a_copy rewrites the profile after planning and checks that the copy is unchanged and the flags name it.Re #1986 (comment) (L8): fixed in
348857bb. With a profile, the runtime jobs add -Wprofile-instr-unprofiled/-out-of-date/-missing after their -Wno-everything (the round trip checks that both notes come through). After the compile the build says how many runtime files found their counts and names the files clang noted. If none found counts, it stops and drops those objects, so a renderer-only or wholly stale profile fails instead of building quietly. Single functions changed since the recording still fall back.Re #1986 (comment) (L10): fixed in
348857bb. The recipe now gives the PowerShell form, makes a runtime from the pack (dev.py packs rt) and runs that runtime's executable, and runs the build and the recording through the queue. It also keeps the files under the ignored work/ by absolute path.Re #1986 (comment) (L11): fixed in
348857bb. The recipe says to leave RX_PGO=gen unset, because with it the dump also holds the lifted game's counters, which stay on the developer's PC. The closing sentence now describes only a profile recorded with RX_PGO_RT=gen alone.Re #1986 (comment) (L12): fixed in
1ca31bbd. The source-string checks are gone; the test captures build.main's real job plan for unset, gen, a profile, combined RX_PGO=gen, renderer-only and third-party-object builds, and asserts each job's flags and whether the writer is in the build. The executable's link command is not captured: it is built after the compile, and its profile options follow the samecountingvalue that decides whether pgo_rt.c is in the plan. Capturing the link is left for a later round.Re #1986 (comment) (L13): partly fixed in
1ca31bbd. The test no longer forces --cc clang: it takes --cc, then CC, then LONGSHOT_MINGW's clang, then clang on PATH. A clang without llvm-profdata beside it now fails instead of skipping. The ELF round trip with the real POSIX shim is not in this push, because it needs a Linux machine to verify; it is in Still to test and comes in a later round.Re #1986 (comment) (L14): fixed in
1ca31bbd. The fixture uses int64_t, runs 100,000 calls before an acknowledged reset and 1,000,000 on four threads after it, and requires exactly 1,000,000 as pick's function count in the merged profile, so an all-zero or pre-reset profile fails.Re #1986 (comment) (L4): fixed in
348857bb. pgo_write checks its allocations, every fwrite and the fclose, writes .tmp, and moves it over . Only then is .dump deleted. A failure is printed to stderr once, the marker stays, and the write is retried every 500 ms. The round trip makes a folder so the move fails, checks that .dump stays and the message is printed, then removes the folder and checks that the profile is written.Re #1986 (comment) (L5): fixed in
348857bb. The marker names are built with malloc at any length, and a lookup failure other than "not found" is reported once. The round trip links the game's manifest and runs under a folder with a UTF-8 name. A path over MAX_PATH is not tested: neither the manifest nor this PC opts into Windows long paths, so such a path fails at the OS, which the writer now reports instead of truncating.Re #1986 (comment) (L6): fixed in
348857bb. A counting build (RX_PGO_RT=gen, and RX_PGO=gen, which uses the same writer) now stops on a clang older than 19 (build.pgo_writer_supported, from -dumpversion). Raw format 10 came with LLVM 19, and older llvm-profdata reads neither format the writer emits. The build reference says so, and CountingClang checks 17/18/4.2.1 against 19/22/23.Re #1986 (comment) (L7): partly fixed in
348857bb. The constructor checks every allocation and CreateThread, reports a failure and frees what it took, closes the handle on success, and treats an empty HOS_PGO as unset. The Linux affinity race (init_process_affinity's constructor against apply_process_affinity on a thread started by another constructor) is in host/posix/win32_posix.c and is already present on main for RX_PGO=gen. I can't verify a change there on this PC, so it is left for a later round, and the description says so.Re #1986 (comment) (L9): fixed in
348857bb. The build reference, the recipe and the code comment now name the executable's host/posix files on Linux, and the recipe says the renderer's own copy of win32_posix.c keeps its flags.Re #1986 (comment) (L15): the part an edit can do is in
348857bb. After the compile, a profile build reports how many runtime files found their counts and names the files clang noted as out of date or missing; it stops if none matched. The in-game comparison (counting and profile builds on Windows and Linux, profile head against its merge base, interleaved, average fps, 1% low, median and p99) needs a recorded profile and is pending with the in-game tester; it stays in Still to test.Re #1986 (comment) (L16): pending. Clean build and link times and executable sizes for the merge base, the head unset and the head with a recorded profile need full game builds and a recorded profile. They are in Still to test for a later round, and this push claims no figures.
Tested corrected: an earlier run on this PC had no Vulkan headers for the renderer checks (
RX_CPMunset), so the ones it listed as also failing onmainhad compiled nothing on either side. They ran again with the headers at348857bband onmain(queue job fb2-vk-1); the description now gives those results. No check fails at this head alone; nothing in the branch changed.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.