1 Performance: what to fix next
HumanBean edited this page 2026-11-18 03:53:03 +00:00

Performance: what to fix next

Written 2026-11-17, from one day of measurements on one PC: Ryzen 9 7900X, RTX 3090 (NVIDIA 620.70), Linux, 2560x1440, Castle Town Market unless said otherwise. It follows the Renderer rework plan. Numbers marked estimate were not measured. Nothing here has run on Windows, AMD or Intel.

The short version

The Market runs at about 70 fps here, and two limits sit almost on top of each other at about 13 ms a frame:

  • the game's own loop (about 12.4 ms), most of it the game's code running on its job threads;
  • the render chain (render thread, then the GPU, then the present; about 13 to 14 ms), which the game waits for once a frame because its frame limiter keeps it one frame ahead of the GPU.

Fixing either one alone changes nothing. That is why most single changes measured as "no change" this week. The first item below is the one that makes the others count.

Where a Market frame goes

From a Tracy capture of a play session (frame 18.6 ms under Tracy and six cores; about 13 to 14 ms without). The frame's critical chain, followed backwards wait by wait over 631 frames:

On the chain ms a frame Share
Executor threads running game code (the actor batch) 7.86 43%
Worker threads running game code 3.91 21%
Main thread running 1.24 7%
Memory copy for the renderer at the frame's first submit (capture_make, on the main thread) 1.64 9%
Hand-offs between threads, 54 a frame 1.24 7%
The game's 20 us polling sleeps 1.22 7%
Yields, virtual-core waits, host locks 0.79 4%
  • The main thread waits once a frame for about 8.4 ms: the actor batch. The executors are running 89.5% of that time.
  • On average 3.8 of the 6 executors are busy in the batch (all six 38% of the time, exactly one 23%). The count and the job graph are the game's.
  • Instant hand-offs would save at most 0.6 to 0.8 ms of that wait.

What the actor batch computes (grouped by job body; names inferred from what the code does and calls):

Group ms a frame on the chain Share of the batch Shape
Actor updates (movement, IK, model matrices, NPC behaviour) about 2.5 30% spread over hundreds of functions
Cloth simulation (middleware) about 1.7 21% one job body of 1.2 ms with a hot core; 17% of it in the host's float helpers
Rigid-body physics (middleware) about 1.5 19% a few hot cores; 2 to 26% in the float helpers
Not attributed 0.9 11%

The grouping comes from a profile standing still mapped onto a walking capture; 11 to 18% of the samples did not map.

What to fix, in order

# What Evidence Expected State
1 Let the game run two frames ahead of the GPU, and find why shadows flickered when #916 did it All game code at -O3 shortens the game's loop from 12.4 to 10.6 ms (two pairs) and fps does not move (70 to 72 against 67 to 70): the saved time becomes a wait for the GPU's previous frame (0 to 0.7-2.2 ms). The same with the frame gate lifted. #916 measured 11 to 15% on Windows before it was closed for the flicker The ceiling moves from the render chain to the game's loop: about 90 fps with faster game code (estimate) #916's commit is rebased onto main in a local workspace with its switch (HOS_FRAME_AHEAD=0); built, not measured. The flicker's cause is unknown
2 Loading threads must not hold a virtual core through a whole decompression While the player walks and the game streams, 57 of 637 frames (9%) had a frame thread waiting over 1 ms for a core held by a loader inside libzstd; those frames are 26.0 ms against 17.3 ms The +4.5 ms on about one frame in eleven while streaming A first fix (give the core up for the whole decode) removed the long waits (longest 97 ms to 2.8 ms, one noisy pair) but was slower standing still in three of three pairs: not to merge. Build it as: keep the core, decode in 64 to 128 KB pieces, check for a request between pieces. Measure walking, with Tracy, not standing
3 The other threads that hold a core through a host call (vibration, audio, actor creation), then lending a lock waiter's priority to the lock's owner Over a thousand long core waits behind the vibration thread in one run; audio 0.13 ms a frame. The loan only helps once holders can see a request, so it follows item 2 Unknown; this is the pattern behind the long freezes Not looked at; the priority loan is written, unbuilt
4 Cloth and physics cores on the hot list at -O2, with the float helpers inline They are about 40% of the batch's chain and each has a hot core; all code at -O3 gave 15% on the loop but costs 746 s of compile against about 298 s About 0.5 ms a frame for a few dozen functions and little build time (estimate) Not built. Shows as fps only after item 1
5 Handle lookup without the shared table (keep the host object's pointer in the game's own structure) obj_lookup is 0.55 ms a frame on the chain and 1.0 ms of CPU a frame over all threads, from cache misses on a table of 131,000 objects; 19 to 31% of render-pass command building About 0.5 ms a frame (estimate) A larger per-thread cache was tried: no gain in three pairs, dropped
6 The memory copy on the main thread at submit 1.5 to 1.65 ms a frame on the critical chain; the longest single copy was 30.7 ms. A fix that copies only pages inside the game's pools cut pages copied by 40 to 45% in an earlier version Up to 1 ms a frame, and fewer long frames (estimate) The pools fix is written, never run in the game
7 The render chain itself Under Tracy the render thread's submit jobs are 7.3 ms a frame, against about 2 ms measured earlier with the renderer's own counters: one of the two is wrong. #1121 (jobs replayed as they come) gave +1% in the Market and +4.6% in Kokiri Forest over four pairs here, against 6 to 8% on its author's Intel Arc Unknown until the two figures are reconciled Open
8 The half-second freezes a player hit at 1440p (220 to 575 ms every 5 to 10 s in the Market, no shader or texture work in them) Seen in one session; two Tracy sessions since did not catch one - Needs a longer captured play session; items 2 and 3 are the likely family

Tried and ruled out for the Market's frame time

Do not repeat these without a new reason:

  • short sleeps turned into yields; a 1 us timer slack (a 20 us sleep already takes 27 us);
  • virtual cores switched off (one pair, no change);
  • read-backs forced late;
  • RX_VK_PARALLEL (10 to 30% slower: its segments write back only at the submission's end);
  • #1469 (lifter register locals): no gain over four pairs here, build 23% longer;
  • a larger per-thread object cache;
  • all game code at -O3 on its own: no fps, and a much longer build.

Counters that misled: any figure that counts a wait as running (the render thread "98% busy", "GPU time" under parallel replay); a thread with no wait zones is not thereby computing.

Tools that now exist

  • Tracy (#1543, merged): LONGSHOT_TRACY=1 builds the game and the renderer with zones for every wait, lock, hand-off and renderer stage, localhost only; docs/development/tracy.md. A capture is made from the game and never leaves the PC. #1546 (open) fixes a setting that made a Tracy build of main capture at 10 to 19 fps instead of 36, and makes the game-code zones (about 2,000 job calls a frame, named where the game's objects carry a name) give a valid capture. Still open: sampled game code shows as bare addresses; a Windows build with the option on has never been made.
  • The frame trace (RX_VK_FRAME_TRACE=1): per frame, the game's loop (game_ms) and its wait for the GPU (wait_gpu_ms). Judge a change by these two, not by fps alone.

How to measure a change here

  • Interleaved pairs, at least three, each alone on the PC (the job queue's timing mode). Sessions of one build differ by 5 to 10 fps on a PC short of memory.
  • A change to the game's loop shows in game_ms; whether it reaches the player shows in the frame time, and only once item 1 is in.
  • Hitches need a walking session, not standing still.