feat(dev): LONGSHOT_TRACY=1 builds the game and the renderer with the Tracy profiler's zones, localhost only #1543
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1543
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:feat/tracy"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Linux, RTX 3090 (NVIDIA 620.70) · Needs: Windows, any GPU (a build with the option on)
TL;DR
Finding where a frame's time goes took a sampler, wait records and a per-frame CSV put together by hand, and that misled us several times. This PR adds a developer's build option,
LONGSHOT_TRACY=1, that builds the game and the renderer with the Tracy profiler: every thread, wait, lock, NVN call and renderer stage on one timeline. A player's build is unchanged: with the option off nothing of Tracy is compiled. It ran in two played sessions (89 s, 29.7 million zones), which showed the main thread of a Market frame spending about 10 of 18.6 ms in one job-loop wait.Breakdown
LONGSHOT_TRACY=1carries Tracy's client (127.0.0.1 only, nothing announced on the network) and zones in the host and the renderer.__LINE__constants).main), an object-by-object comparison with the option off.tracy-captureandtracy-csvexport).#ifdef TRACY_ENABLE.Why
The frame rate in Castle Town's market is limited on the CPU side and there are half-second freezes in play. The tools we had (the profiling mode's sampler,
HOS_CHAIN's wait and wake records, the frame trace) each show one part, and putting them together by hand gave wrong answers more than once. Tracy shows all of it on one timeline, live or from a file. No issue in the tracker asks for this (searched the open issues for "tracy" and "profil").What changed
A build option, a vendored client, one wrapper header and zones at the places that matter; nothing else in the build moves.
third_party/tracyis Tracy 0.15.0'spublic/folder and LICENSE, unmodified, with itsPROVENANCE.md(tag v0.15.0, commit f2a4f25, archive and tree hashes). 0.15.0 is the version of the AUR'stracyviewer: the protocol is locked to the version.nx2/build.py):LONGSHOT_TRACY=1compilespublic/TracyClient.cppinto the game's executable alone, which exports Tracy's C API (___tracy_*);longshot_vk.socalls that one client, so the process has exactly one. Defines:TRACY_ONLY_LOCALHOST,TRACY_NO_BROADCAST,TRACY_ON_DEMAND,TRACY_NO_CRASH_HANDLER,TRACY_NO_CODE_TRANSFER,TRACY_MANUAL_LIFETIME(the game'smainstarts the client, so the crash window's helper process has none),TRACY_FIBERS. The lifted game code is not compiled again. On Windows the executable's import library is what the renderer's DLL links: written, never built.host/hos_tracy.h: the only header a call site includes (C and C++): scoped zones, waits in their own colour, zone text, frame marks, plots, thread names, fibers, and "lock wait" zones for every SRW lock and critical section of a file that has to wait. Every use in the sources is inside#ifdef TRACY_ENABLE, because many checks compile one function cut from a source file.nvnQueuePresentTexture; guest waits and signals inhost/nn_os_sync.c(events, semaphores, queues, WaitAny, condition variables, mutexes, with the object's guest address and the guest caller); sleeps, yields, virtual-core waits, kernel waits, joins and fiber switches inhost/hos_thread.c; the NVN calls that work or wait on a game thread inhost/nvn_native.c; thread names where the host already names threads.draw,build_descriptors,bind_descriptors,program_for;RX_VK_TRACY_DRAWS=0leaves them out), memory sync and captures, texture create and upload, pipeline builds, the workers, the recorder's idle time, the present; plots of the frame time and, with the frame trace on, its main columns.nx2/tracy_funcs.py,nx2/tracy_funcs.txt,host/hos_tracy_game.c; Linux): listed lifted functions become zones at the link, by rewriting the symbol table of a copy of the chunk's object (no translation, no compile), and the indirect calls of the two job loops become zones by compiling one chunk again (about 20 s). Recorded only withHOS_TRACY_GAME=1and not working yet: see Testing.docs/development/tracy.md(build, capture, what each zone is, the privacy rule), linked fromdocs/development.md.tests/test_tracy_funcs.py(the list, the object patch on two small C files, the calls header, and that no zone macro sits outside#ifdef TRACY_ENABLE);tests/test_shader_stamp_cache.pygets the new build globals.A capture holds the game's function names and timings: it is made from the game and never leaves the PC. The doc says so, and the client cannot be reached from another machine.
Measurements
Verdict: the option off costs nothing (same objects). The option on, while a capture is taken, costs frame rate; these are first numbers, not a controlled A/B, which is still to do.
Hardware: Linux, RTX 3090, NVIDIA 620.70, Ryzen 9 7900X.
Scene: played sessions at 2560x1440 with DLSS Quality; headless smoke runs at 2048x1152 in Kokiri Forest.
Method: the played sessions' frame counts and file sizes from
tracy-capture; the headless runs'vk: ... fpslog lines and the captures' frame counts.RX_VK_TRACY_DRAWS=0RX_VK_TRACY_DRAWS=0Option off, object by object (the base build's own compile commands,
main@310e7c9dagainst this branch):hos_thread.c,nn_os_sync.c,nvn_native.c,hos_svc.c,hos_mem.c,posix/win32_posix.c,vk_replay.cpp,vk_memory.cpp,vk_texture.cpp,vk_jobs.cpp,vk_framegen.cpp: the same object, byte for byte.vk_draw.cpp: the same size and code; four__LINE__constants of itsSKIP_HEREmacro differ, because lines moved.Build time with the option on, as a pack over a base: 27 s to 1 min 21 s (the host's and the renderer's own files, Tracy's client, one lifted chunk for the job loops' calls, the links).
Flags and settings
Three: a build option and two run-time switches that exist only in a build made with it.
LONGSHOT_TRACY=1nx2/build.py)RX_VK_TRACY_DRAWS=0leaves the four per-draw zones out of a capture=0for long capturesHOS_TRACY_GAME=1records the lifted game code's zones (funcs,calls: one kind)LONGSHOT_TRACY_FUNCS=<file>Testing
Tested: Linux, RTX 3090, NVIDIA 620.70, Ryzen 9 7900X. Two played sessions on the first version of this branch with
tracy-captureon 127.0.0.1:8086 (numbers above): the capture connected at once,tracy-csvexportread both files, no error line in either log. Headless smoke runs of the current code with the game's zones off: a 20 s capture with 61 zone names from the host and the renderer, every game and renderer thread by its name (the job workers, the actor executors, the render, recorder and presenter threads), a fiber's row, and Tracy's sampler's samples as an ordinary user (perf_event_paranoid2). Game-free checks:tests/run_all.py --changed382 pass, 36 fail, and the same 36 fail on a clean checkout ofmain@310e7c9don this PC (I ran them there:test_switch_table.pyand 35 others);tests/test_tracy_funcs.pyandscripts/check_third_party.pypass.Not tested: the last small change (the game's zones off by default, the order check) was built in the step before it, not after; a capture with the viewer GUI (not installed here); a capture longer than 90 s; whether the sampler's symbols of the lifted functions resolve (the 20 s smoke capture ended before any symbol was resolved); AMD and Intel GPUs.
Not working: the game-code zones (
HOS_TRACY_GAME=1). With the listed functions and the job loops' calls both on, Tracy stops the capture a few milliseconds in ("Invalid order of zone begin and end events"): a 0.0 s capture. With the functions alone a 20 s capture is fine; with the calls alone it records andtracy-capturecrashes while saving. Fibers were ruled out (no fiber switch happens inside such a zone). A check that names the zone ending out of turn is in; the fix follows on this branch.Must be tested:
LONGSHOT_TRACY=1(llvm-mingw), expecting the game andlongshot_vk.dllto link (the DLL against the executable's import librarylongshot_tracy.lib) and the game to start; thentracy-capture -o run.tracy, expecting zones from both modules (Test Farm: @Beanstalk windows, a build with LONGSHOT_TRACY=1 and a start of the game).maindoes (Test Farm: @Beanstalk the usual comparison against main).Risks
What could break: nothing in a build without the option, as far as the object comparison shows. In a Tracy build: a capture slows the game; a zone that ends out of turn stops the capture (the game goes on).
Turning it off: do not set
LONGSHOT_TRACY; in a Tracy build, do not connect a viewer.Compatibility: Nothing changes for a player: no save, option, cache or
MKWVK_API_VERSIONchange.vk_shader.cppis untouched, so the shader caches' stamp is the same.Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (only--changedwas run: 36 failures, the same 36 onmainon this PC; not yet all named here)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Tested in game: RTX 3090, NVIDIA 620.70, Ryzen 9 7900X, Linux: two played sessions (Castle Town's market among the places) on the first version of this branch with tracy-capture on 127.0.0.1:8086, 89 s and 90 s, 29.7 and 16.8 million zones, no error line in either log; then headless smoke runs of the current code in Kokiri Forest with a 20 s capture: 61 zone names from the host and the renderer, the threads by name, the sampler's samples. The game-code zones (HOS_TRACY_GAME=1, off by default) do not work yet: Tracy stops the capture with 'Invalid order of zone begin and end events'.
WIP: feat(dev): LONGSHOT_TRACY=1 builds the game and the renderer with the Tracy profiler's zones, localhost onlyto feat(dev): LONGSHOT_TRACY=1 builds the game and the renderer with the Tracy profiler's zones, localhost only