Game: the performance statistics sender deletes summaries on a 404 or a redirect, can send twice, and sends more than the service takes at once #1265

Open
opened 2026-11-16 23:27:15 +00:00 by mooki · 1 comment
Member

Longshot version

main at 17394b7111 (the merge of #983). nx2/perf_report.py is unchanged on main 2547a8f3dd.

Channel

Built from source

What kind of problem

Something else

What happened

nx2/perf_report.py, which the launcher starts at its start to send what the game left in perf_pending/, has three ways to lose or double data, all silent (its output goes to the null device, line 415):

  1. A summary is deleted on a response that is not the service saying no. answer() (
    def answer(code, retry_after=None):
    ) calls every status except 2xx, 429 and 5xx "refused", and send_these then deletes the file (line 367). A 404 from the ingress while the service is redeployed or its route is missing, a redirect (http.client does not follow one), 408 or 425 would drop up to 12 summaries a start. Only 400, 413, 415 and 422 mean "this summary will never be taken".
  2. Two senders can overlap. start_sending (line 409) starts a detached --send at each launcher start, with no lock. If the first is still waiting on its 10 s timeout or a slow link, a second reads the same perf_pending/ and can post the same summaries, which the service counts twice (a summary has no ID, so it cannot dedupe).
  3. The default sends more than the ingress allows. The sender posts up to 12 summaries a start (LONGSHOT_PERF_MAX_SENDS, line 304). The perf route's ingress rate limit is 6 a minute with a burst of 6 per address (deploy/telemetry/40-ingress.yaml in the deploy repo, lines 37 and 39). A player with more than 6 waiting (more than half an hour of play since the launcher last ran) gets a 429 on the 7th, which the sender reads as "busy" and answers with a wait of at least an hour, doubling to a day (BUSY_FIRST, BUSY_MOST, line 52). Nothing is lost, but the rest arrives a day late. I did not test whether the 429 carries a Retry-After.

Also worth knowing, not a bug: the sender only runs when the launcher starts, so a game started from Steam or a shortcut never sends and up to 48 summaries wait (older ones are dropped past that, line 310).

Where in the game

Not in the game: the launcher's start, and python -I nx2/perf_report.py <game folder> --send.

How often

1 and 3: every time the condition holds (a missing route during a deploy; more than 6 summaries waiting). 2: rarely.

Graphics card and driver version

Not applicable.

CPU, RAM and operating system

Not specific; the sender is the same Python on Windows and Linux.

Changes from the defaults

None.

longshot_log.txt

Not attached: read from the code, the game-free tests/test_perf_report.py (passes on main), and the deploy config.

Screenshots or video

Not applicable.

What would fix it

  • Treat 3xx, 404, 408 and 425 as "later" (keep the file, note the backoff or not); keep "refused" for 400, 413, 415 and 422. Add a case for each to tests/test_perf_report.py (it already has a local relay).
  • A lock for --send: create perf_pending/.sending with O_EXCL, treat it as stale after a few minutes, and exit when it is held.
  • Make the defaults agree: at most 5 sends a start, or a short pause between sends, or raise the ingress burst; whichever the team prefers, but the sender's number and the ingress's should be written next to each other.

Before you send it

  • The problem is still there on the newest release or nightly (main).
  • I searched the open issues for it.
  • I'm not attaching game files, keys, the game's executable, its .bin caches or saves.
### Longshot version main at 17394b71111f8b7d9d6ecc65f430175145394743 (the merge of https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/983). `nx2/perf_report.py` is unchanged on main 2547a8f3dd3267be6612bbc1edd79196093a10c4. ### Channel Built from source ### What kind of problem Something else ### What happened `nx2/perf_report.py`, which the launcher starts at its start to send what the game left in `perf_pending/`, has three ways to lose or double data, all silent (its output goes to the null device, line 415): 1. **A summary is deleted on a response that is not the service saying no.** `answer()` (https://git.magicbean.dev/MagicBeanDev/Longshot/src/commit/17394b71111f8b7d9d6ecc65f430175145394743/nx2/perf_report.py#L196) calls every status except 2xx, 429 and 5xx "refused", and `send_these` then deletes the file (line 367). A 404 from the ingress while the service is redeployed or its route is missing, a redirect (http.client does not follow one), 408 or 425 would drop up to 12 summaries a start. Only 400, 413, 415 and 422 mean "this summary will never be taken". 2. **Two senders can overlap.** `start_sending` (line 409) starts a detached `--send` at each launcher start, with no lock. If the first is still waiting on its 10 s timeout or a slow link, a second reads the same `perf_pending/` and can post the same summaries, which the service counts twice (a summary has no ID, so it cannot dedupe). 3. **The default sends more than the ingress allows.** The sender posts up to 12 summaries a start (`LONGSHOT_PERF_MAX_SENDS`, line 304). The perf route's ingress rate limit is 6 a minute with a burst of 6 per address (`deploy/telemetry/40-ingress.yaml` in the deploy repo, lines 37 and 39). A player with more than 6 waiting (more than half an hour of play since the launcher last ran) gets a 429 on the 7th, which the sender reads as "busy" and answers with a wait of at least an hour, doubling to a day (`BUSY_FIRST`, `BUSY_MOST`, line 52). Nothing is lost, but the rest arrives a day late. I did not test whether the 429 carries a Retry-After. Also worth knowing, not a bug: the sender only runs when the launcher starts, so a game started from Steam or a shortcut never sends and up to 48 summaries wait (older ones are dropped past that, line 310). ### Where in the game Not in the game: the launcher's start, and `python -I nx2/perf_report.py <game folder> --send`. ### How often 1 and 3: every time the condition holds (a missing route during a deploy; more than 6 summaries waiting). 2: rarely. ### Graphics card and driver version Not applicable. ### CPU, RAM and operating system Not specific; the sender is the same Python on Windows and Linux. ### Changes from the defaults None. ### longshot_log.txt Not attached: read from the code, the game-free `tests/test_perf_report.py` (passes on main), and the deploy config. ### Screenshots or video Not applicable. ### What would fix it - Treat 3xx, 404, 408 and 425 as "later" (keep the file, note the backoff or not); keep "refused" for 400, 413, 415 and 422. Add a case for each to `tests/test_perf_report.py` (it already has a local relay). - A lock for `--send`: create `perf_pending/.sending` with `O_EXCL`, treat it as stale after a few minutes, and exit when it is held. - Make the defaults agree: at most 5 sends a start, or a short pause between sends, or raise the ingress burst; whichever the team prefers, but the sender's number and the ingress's should be written next to each other. ### Before you send it - [x] The problem is still there on the newest release or nightly (main). - [x] I searched the open issues for it. - [x] I'm not attaching game files, keys, the game's executable, its .bin caches or saves.
Contributor

Claiming this for the fix: answer() reclassified so 3xx/404/408/425 keep the summary (later) and only 400/413/415/422 delete it (refused); an O_EXCL lock on perf_pending/.sending with stale takeover for the detached sender; and LONGSHOT_PERF_MAX_SENDS default lowered to 5 with the ingress limit (6 a minute, burst 6) written next to both numbers. Tests per case in tests/test_perf_report.py's local relay.

Claiming this for the fix: answer() reclassified so 3xx/404/408/425 keep the summary (later) and only 400/413/415/422 delete it (refused); an O_EXCL lock on perf_pending/.sending with stale takeover for the detached sender; and LONGSHOT_PERF_MAX_SENDS default lowered to 5 with the ingress limit (6 a minute, burst 6) written next to both numbers. Tests per case in tests/test_perf_report.py's local relay.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1265
No description provided.