MagicBeanDev/Longshot
MagicBeanDev/Longshot
Watch
54
67
Fork
You've already forked Longshot
 71
Code Issues 124 Pull requests 196 Projects Releases 8 Packages Wiki Activity Actions 442

WIP: ci: run checks, builds, releases and the website on CircleCI #417

Draft
drcrow wants to merge 6 commits from drcrow/Longshot:ci/circleci into main
pull from: drcrow/Longshot:ci/circleci
merge into: MagicBeanDev:main
Conversation 3 Commits 6 Files changed 22 +2095 -10
drcrow commented 2026-11-14 11:49:53 +00:00
First-time contributor

Why

Pull requests get only the title, description and third-party checks today. The game-free checks and the builders run by hand or at release, so a break is found after it merges. This moves CI and the release pipeline to CircleCI, which runs the checks and builders each pull request needs and nothing more.

What changed

  • tests: tests/run_all.py gains --list (the checks a run would start, running none), --junit FILE, and an opt-in --exact (names must match exactly; nothing matching fails closed) that CircleCI's test splitter uses. --shard, --write-times and substring names are unchanged. Tests: tests/test_run_all_junit.py.
  • ci: .circleci/config.yml has one small setup job per event (pull request, push to main, v* tag, schedule, manual API run). Each one looks at which areas changed (.circleci/mapping.conf: tests, gpu, linux, windows, website, third-party, docs) and continues with .circleci/shared.yml plus that event's file in .circleci/events/. A docs-only PR runs lint, third-party and gate; a gpu/ change adds the Vulkan builds and SPIR-V fixtures.
  • ci: the shared setup, fork handling, release sign/publish steps and gate come from one CircleCI orb, magicbeandev/ci, used by all three repos (source in a new MagicBeanDev/orb repo). This repo keeps only its own jobs.
  • ci: the game-free checks run with CircleCI Smarter Testing (.circleci/test-suites.yml), split across 4 containers by timing, with JUnit results.
  • ci: every push to main runs everything, then releases a nightly, as release.yml does: the release jobs start only after every check and build passes, so a red main ships nothing. v* tags rerun the core checks, then release. Release jobs run main's copy of scripts/release.py. Signing (release-sign, signing key only) and publishing (release-publish, Forgejo token only, one release at a time) are separate jobs. release_dry_run runs it all with a throwaway key and stops before publishing.
  • ci: the Wine checks run on an Ubuntu 24.04 VM, like the Forgejo runner. The other jobs use CircleCI's Arm (Gen2) containers where the tools exist (docs/ci.md lists each job and why).
  • ci: website: a pull request's built site is saved as a CircleCI artifact and linked in one PR comment. Main deploys it with the production token. PR code never runs next to a Cloudflare token.
  • ci: fork pull requests: CircleCI doesn't build them from Forgejo, so a second CircleCI project on this repo runs .circleci/forks.yml, started by a Forgejo webhook. A maintainer adds the ci: approved label (Forgejo only lets people with write access do that), and it runs that exact commit with no secrets, then posts ci/circleci: gate on the PR. The label stays on, so new pushes keep running. After 14 quiet days it's removed.
  • ci: caches are saved from main only. The orb's conftest policies keep secrets off any job that runs PR code. Commit messages are checked against the Conventional Commit rule. PR title and description checks stay on Forgejo.
  • build(dev): mise.toml and mise.lock (pinned tools and the deps, lint, test, ci:smoke, validate tasks), pre-commit hooks (commit-msg included), a devcontainer, and a Chunk sidecar config.
  • docs: docs/ci.md with flow diagrams; CONTRIBUTING.md's CI lines.
  • The Forgejo workflows stay until CircleCI has been green for a while. A later PR removes them.

Flags and settings

None. No game switch, menu row or option, and no new environment variable. CircleCI pipeline parameters (release_dry_run, vulkan_dlss, full) are in docs/ci.md.

Measurements

main: f87557ae (2026-11-14)
this PR: ci/circleci @ 2c00a9a6
Hardware: devbox, x86_64 Linux, Docker; the jobs' own CircleCI images (no game, no GPU)
Scene: none: CI only
Method: run_all.py --strict in one Ubuntu 24.04 image with test-linux's packages, main and this PR interleaved (main, PR, main, PR), 2 runs each, the median. These numbers are from 2c00a9a6. run_all.py has been reworked since to keep --shard and substring names; a new interleaved measurement against current main is running, and this section will be updated with it.
How to reproduce: the same image and python3 -I -B tests/run_all.py --strict on each checkout; mise run validate for the CI config

main @ f87557ae this PR @ 2c00a9a6
CI on a pull request title, description, third-party, actionlint lint, meta, third-party, plus the affected checks and builds, then gate
run_all.py --strict (Linux, 24.04 image): passed / failed / skipped 531 / 4 / 15 538 / 4 / 15 (the 7 new checks pass)
run_all.py --strict wall time (16 at once) 210 s 215 s 
runs: main 531/4/15 213 s, PR 538/4/15 208 s, main 531/4/15 209 s, PR 538/4/15 223 s
failing on both sides: test_pad_layout.py (main: hid.c can't find HoldToggle since #137),
  test_oot_saves.py, test_pipeline_share.py, test_linux_preflight.py (these three only in the measuring container,
  which runs as a user without a home folder; in the test-linux job's own environment only test_pad_layout fails)

Testing

Tested: circleci config validate on every event's merged config with the orb inlined, and the orb's conftest policies (28 pass). run_all.py --strict as measured above. mise run lint and mise run test on a fresh Chunk sidecar (Ubuntu 22.04).

Not tested: a live CircleCI run (it starts once this is on main and the orb is published), the Arm containers, the Wine VM job, publishing (it needs the signing key and bot token in CircleCI), Cloudflare deploys, and the fork flow end to end (it needs the org webhook).

Needs other testers: None

Screenshots

Nothing on screen

Risk and rollback

A wrong path rule could skip a job a change needed. Main pushes run everything, and full: true on a manual run forces it all. The Forgejo workflows stay on, so reverting this commit is enough.

Reviewer notes

Start with docs/ci.md, then .circleci/config.yml and .circleci/events/pr.yml. test_pad_layout.py fails on main too (after #137, its compile of hid.c doesn't find host/hold_toggle.h); that is for a separate fix. The shared CI logic lives in the magicbeandev/ci orb (MagicBeanDev/orb), and #353's pull request builds can sit on top of this pipeline. Feedback very welcome, especially on the path areas in .circleci/mapping.conf (what should run for what) and on the fork-PR approval flow.

Issues

See #283

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main.
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
  • Tested in the game, or what wasn't is named under Testing.
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
## Why Pull requests get only the title, description and third-party checks today. The game-free checks and the builders run by hand or at release, so a break is found after it merges. This moves CI and the release pipeline to CircleCI, which runs the checks and builders each pull request needs and nothing more. ## What changed - tests: `tests/run_all.py` gains `--list` (the checks a run would start, running none), `--junit FILE`, and an opt-in `--exact` (names must match exactly; nothing matching fails closed) that CircleCI's test splitter uses. `--shard`, `--write-times` and substring names are unchanged. Tests: `tests/test_run_all_junit.py`. - ci: `.circleci/config.yml` has one small setup job per event (pull request, push to main, `v*` tag, schedule, manual API run). Each one looks at which areas changed (`.circleci/mapping.conf`: tests, gpu, linux, windows, website, third-party, docs) and continues with `.circleci/shared.yml` plus that event's file in `.circleci/events/`. A docs-only PR runs lint, third-party and `gate`; a `gpu/` change adds the Vulkan builds and SPIR-V fixtures. - ci: the shared setup, fork handling, release sign/publish steps and gate come from one CircleCI orb, `magicbeandev/ci`, used by all three repos (source in a new `MagicBeanDev/orb` repo). This repo keeps only its own jobs. - ci: the game-free checks run with CircleCI Smarter Testing (`.circleci/test-suites.yml`), split across 4 containers by timing, with JUnit results. - ci: every push to main runs everything, then releases a nightly, as `release.yml` does: the release jobs start only after every check and build passes, so a red main ships nothing. `v*` tags rerun the core checks, then release. Release jobs run main's copy of `scripts/release.py`. Signing (`release-sign`, signing key only) and publishing (`release-publish`, Forgejo token only, one release at a time) are separate jobs. `release_dry_run` runs it all with a throwaway key and stops before publishing. - ci: the Wine checks run on an Ubuntu 24.04 VM, like the Forgejo runner. The other jobs use CircleCI's Arm (Gen2) containers where the tools exist (`docs/ci.md` lists each job and why). - ci: website: a pull request's built site is saved as a CircleCI artifact and linked in one PR comment. Main deploys it with the production token. PR code never runs next to a Cloudflare token. - ci: fork pull requests: CircleCI doesn't build them from Forgejo, so a second CircleCI project on this repo runs `.circleci/forks.yml`, started by a Forgejo webhook. A maintainer adds the `ci: approved` label (Forgejo only lets people with write access do that), and it runs that exact commit with no secrets, then posts `ci/circleci: gate` on the PR. The label stays on, so new pushes keep running. After 14 quiet days it's removed. - ci: caches are saved from main only. The orb's conftest policies keep secrets off any job that runs PR code. Commit messages are checked against the Conventional Commit rule. PR title and description checks stay on Forgejo. - build(dev): `mise.toml` and `mise.lock` (pinned tools and the `deps`, `lint`, `test`, `ci:smoke`, `validate` tasks), pre-commit hooks (commit-msg included), a devcontainer, and a Chunk sidecar config. - docs: `docs/ci.md` with flow diagrams; CONTRIBUTING.md's CI lines. - The Forgejo workflows stay until CircleCI has been green for a while. A later PR removes them. ## Flags and settings None. No game switch, menu row or option, and no new environment variable. CircleCI pipeline parameters (`release_dry_run`, `vulkan_dlss`, `full`) are in docs/ci.md. ## Measurements main: f87557ae (2026-11-14) this PR: ci/circleci @ 2c00a9a6 Hardware: devbox, x86_64 Linux, Docker; the jobs' own CircleCI images (no game, no GPU) Scene: none: CI only Method: `run_all.py --strict` in one Ubuntu 24.04 image with test-linux's packages, main and this PR interleaved (main, PR, main, PR), 2 runs each, the median. These numbers are from 2c00a9a6. `run_all.py` has been reworked since to keep `--shard` and substring names; a new interleaved measurement against current main is running, and this section will be updated with it. How to reproduce: the same image and `python3 -I -B tests/run_all.py --strict` on each checkout; `mise run validate` for the CI config | | main @ f87557ae | this PR @ 2c00a9a6 | |---|---|---| | CI on a pull request | title, description, third-party, actionlint | lint, meta, third-party, plus the affected checks and builds, then `gate` | | `run_all.py --strict` (Linux, 24.04 image): passed / failed / skipped | 531 / 4 / 15 | 538 / 4 / 15 (the 7 new checks pass) | | `run_all.py --strict` wall time (16 at once) | 210 s | 215 s | ``` runs: main 531/4/15 213 s, PR 538/4/15 208 s, main 531/4/15 209 s, PR 538/4/15 223 s failing on both sides: test_pad_layout.py (main: hid.c can't find HoldToggle since #137), test_oot_saves.py, test_pipeline_share.py, test_linux_preflight.py (these three only in the measuring container, which runs as a user without a home folder; in the test-linux job's own environment only test_pad_layout fails) ``` ## Testing **Tested:** `circleci config validate` on every event's merged config with the orb inlined, and the orb's conftest policies (28 pass). `run_all.py --strict` as measured above. `mise run lint` and `mise run test` on a fresh Chunk sidecar (Ubuntu 22.04). **Not tested:** a live CircleCI run (it starts once this is on main and the orb is published), the Arm containers, the Wine VM job, publishing (it needs the signing key and bot token in CircleCI), Cloudflare deploys, and the fork flow end to end (it needs the org webhook). **Needs other testers:** None ## Screenshots Nothing on screen ## Risk and rollback A wrong path rule could skip a job a change needed. Main pushes run everything, and `full: true` on a manual run forces it all. The Forgejo workflows stay on, so reverting this commit is enough. ## Reviewer notes Start with docs/ci.md, then `.circleci/config.yml` and `.circleci/events/pr.yml`. `test_pad_layout.py` fails on main too (after #137, its compile of hid.c doesn't find `host/hold_toggle.h`); that is for a separate fix. The shared CI logic lives in the `magicbeandev/ci` orb (MagicBeanDev/orb), and #353's pull request builds can sit on top of this pipeline. Feedback very welcome, especially on the path areas in `.circleci/mapping.conf` (what should run for what) and on the fork-PR approval flow. ## Issues See #283 ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated.
docs: describe the CircleCI pipeline and how to contribute
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 6s
Details
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
CI / Conventional Commit title (pull_request) Has been cancelled
Details
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
Details
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Details
Third-party parts / As their notes say (pull_request) Has been cancelled
Details
Third-party parts / The same as upstream (pull_request) Has been cancelled
Details
6a8e365958
mooki reviewed 2026-11-14 15:04:29 +00:00
mooki left a comment

Looked over what runs when, at 6a8e3659. Read from the files; where I ran something it says so. Nothing here was seen in a live CircleCI run.

Stops every run as written

1. The fork condition is always true, and it guards the checkout. .circleci/shared.yml:80-105 (get-source) and .circleci/events/fork.yml:4 use greater: [<< pipeline.parameters.fork_pr >>, 0]. greater is not one of CircleCI's logic statements, and the compiler does not reject it: an unknown operator counts as true. I compiled a miniature of get-source through CircleCI's config compiler: with fork_pr at 0 the job gets the "Fork PR checkout" step and no checkout; at 7 the same. So circleci config validate passes, and in every pull request, main and tag pipeline each job that uses get-source (lint, meta, the tests, the builders, the website jobs, the release builds) fetches pull/0/head and fails. Forms that compiled correctly: at step level not: {equal: [0, << pipeline.parameters.fork_pr >>]}; at workflow level when: pipeline.parameters.fork_pr > 0 (inside a step's condition: that expression string also counted as true).

2. meta cannot find the description template. shared.yml:380-384 copies main's checker to /tmp/main-scripts/check_pr_description.py and unpacks the template under /tmp/main-scripts/.forgejo/. The checker looks one folder above itself (scripts/check_pr_description.py:41), that is in /tmp/.forgejo/.

3. website checks a site it has not built. shared.yml:803-804 runs npm ci and npm run check; website/scripts/check-site.ts starts by reading dist. Forgejo's website.yml builds first.

4. Lint installs from a lock file with no Linux Arm entries. The small executor is arm.medium.gen2, the install runs with MISE_LOCKED=1 (shared.yml:255-261), and mise.lock lists linux-x64 and macos-arm64 only. As I read mise's locked mode it refuses a platform the lock does not have; not run.

5. tests/run_all.py loses --shard, which the Forgejo checks still pass. On this head python -I -B tests/run_all.py --shard 1/2 --list ends with error: unrecognized arguments: --shard, and .forgejo/workflows/ci.yml:163 runs tests/run_all.py --strict --shard ... (so does the new scripts/ci/check.py:159-160). The Forgejo source checks would fail from the merge on, while they are meant to keep running alongside.

Differs from the description

6. The nightly does not wait for the checks. .circleci/events/main.yml has two workflows: ci (lines 3-43) and nightly-release (45-62). Signing requires the release builds and notes only, so a red ci does not stop a signed nightly. Each workflow also has its own gate.

7. Main does not run everything. In ci, linux-appimage, windows-builder, windows-vulkan and the website jobs keep their path filters (main.yml:14-30); docs/ci.md:20 says "All areas".

8. Fork approval. .circleci/approvers.txt has no login, so gatekeeper.py:220-224 refuses every label. The gatekeeper also takes the label as a one-time approval that must come after the latest push and removes it (:225-242), where the team summary and the launcher and site descriptions say the label stays and new pushes keep running. And it reads the first 50 open pull requests only (:94); there are over 170 open.

9. full. The description lists a full parameter; .circleci/config.yml:6-27 does not declare it and events/manual.yml has no route for it.

10. Paths. A .circleci/ change is "full" but the website jobs filter on website alone (events/pr.yml:29-41). Forgejo's website.yml also triggers on docs/tested-configurations.md and docs/tested/**; mapping.conf:33 has website/ only. mise.lock, ci/linux/ and .forgejo/ have no rule.

Release chain

11. Main's release.py is run from .ci/main-release/ (shared.yml:223-226), so its root is <checkout>/.ci (scripts/release.py:64). Loaded from there it finds 0 trusted keys and no cliff.toml; from scripts/ it finds 1 key and the file. The verify at shared.yml:1120 then has no key to trust.

12. A nightly's builder version is the word "nightly". shared.yml:1028-1036 turns nightly-<n>-<hash> into VERSION=nightly for dotnet publish -p:Version. Forgejo's build-windows.yml:103-104 takes a numeric version or 0.0.0.

13. Dry-run signing. shared.yml:1107-1108 uses /tmp/dry-sign.pem as the key file, but keygen makes a folder of that name holding release.pem and release.pub.pem (scripts/release.py:715-717), and verify gets no public key for it.

14. released-as ... || true (shared.yml:914) turns a failed check of an earlier release into "none found".

15. The release Windows build restores a compiler cache. release-windows uses the llvm-mingw command, which restores v1-llvm-mingw-* after the hash-checked install (shared.yml:196-207, :1017); docs/ci.md:21 says release builds restore no caches.

16. Two publishers. .forgejo/workflows/release.yml still publishes on main and on v* tags. Which one is the only writer while both run?

Smaller

  • shared.yml:878-886: the comment job, which holds the comment token, falls back to the pull request's copy of four helper scripts when main lacks them (it does today). Failing instead is safer.
  • .circleci/config.yml:79 and :95-97 put fork_sha, the branch and the tag into shell text; checking fork_sha as 40 hex digits and passing the others through the environment avoids surprises.
  • On main the lint range is HEAD..HEAD (shared.yml:322-323).
  • The game-free checks run on Arm (test-linux, arm.large.gen2); checks that skip off x86-64, such as tests/test_fpx_sticky.py:189, then do not run anywhere. Intended?
  • events/release.yml: release-version and gate have no tag filter while the jobs between them do. CircleCI's rule for tags asks for one on every job in the chain (https://circleci.com/docs/guides/orchestrate/workflows/#executing-workflows-for-a-git-tag).

Checked and fine

The three examples hold: a docs change adds nothing to lint, meta and third-party; gpu/ adds the Linux checks, SPIR-V fixtures and Windows Vulkan build; the builder adds its CLI checks and the Windows builder. With fork_pr set, the checkout compares the fetched commit with fork_sha before using it. Signing and publishing ask for separate contexts; the manual dry run has no publish job. run_all.py --list still finds every existing check, and the new JUnit test passes.

Questions

What turns Forgejo's events into the pull_request, push and api events the setup expects, and what posts a failed ci/circleci: gate when a required job fails and the gate job never starts? Which CircleCI settings limit the signing, bot and Cloudflare contexts to main and tags, given that a branch in this repository runs its own configuration?

Looked over what runs when, at `6a8e3659`. Read from the files; where I ran something it says so. Nothing here was seen in a live CircleCI run. ### Stops every run as written **1. The fork condition is always true, and it guards the checkout.** `.circleci/shared.yml:80-105` (`get-source`) and `.circleci/events/fork.yml:4` use `greater: [<< pipeline.parameters.fork_pr >>, 0]`. `greater` is not one of CircleCI's logic statements, and the compiler does not reject it: an unknown operator counts as true. I compiled a miniature of `get-source` through CircleCI's config compiler: with `fork_pr` at 0 the job gets the "Fork PR checkout" step and no `checkout`; at 7 the same. So `circleci config validate` passes, and in every pull request, main and tag pipeline each job that uses `get-source` (lint, meta, the tests, the builders, the website jobs, the release builds) fetches `pull/0/head` and fails. Forms that compiled correctly: at step level `not: {equal: [0, << pipeline.parameters.fork_pr >>]}`; at workflow level `when: pipeline.parameters.fork_pr > 0` (inside a step's `condition:` that expression string also counted as true). **2. `meta` cannot find the description template.** `shared.yml:380-384` copies main's checker to `/tmp/main-scripts/check_pr_description.py` and unpacks the template under `/tmp/main-scripts/.forgejo/`. The checker looks one folder above itself (`scripts/check_pr_description.py:41`), that is in `/tmp/.forgejo/`. **3. `website` checks a site it has not built.** `shared.yml:803-804` runs `npm ci` and `npm run check`; `website/scripts/check-site.ts` starts by reading `dist`. Forgejo's `website.yml` builds first. **4. Lint installs from a lock file with no Linux Arm entries.** The `small` executor is `arm.medium.gen2`, the install runs with `MISE_LOCKED=1` (`shared.yml:255-261`), and `mise.lock` lists `linux-x64` and `macos-arm64` only. As I read mise's locked mode it refuses a platform the lock does not have; not run. **5. `tests/run_all.py` loses `--shard`, which the Forgejo checks still pass.** On this head `python -I -B tests/run_all.py --shard 1/2 --list` ends with `error: unrecognized arguments: --shard`, and `.forgejo/workflows/ci.yml:163` runs `tests/run_all.py --strict --shard ...` (so does the new `scripts/ci/check.py:159-160`). The Forgejo source checks would fail from the merge on, while they are meant to keep running alongside. ### Differs from the description **6. The nightly does not wait for the checks.** `.circleci/events/main.yml` has two workflows: `ci` (lines 3-43) and `nightly-release` (45-62). Signing requires the release builds and notes only, so a red `ci` does not stop a signed nightly. Each workflow also has its own `gate`. **7. Main does not run everything.** In `ci`, `linux-appimage`, `windows-builder`, `windows-vulkan` and the website jobs keep their path filters (`main.yml:14-30`); `docs/ci.md:20` says "All areas". **8. Fork approval.** `.circleci/approvers.txt` has no login, so `gatekeeper.py:220-224` refuses every label. The gatekeeper also takes the label as a one-time approval that must come after the latest push and removes it (`:225-242`), where the team summary and the launcher and site descriptions say the label stays and new pushes keep running. And it reads the first 50 open pull requests only (`:94`); there are over 170 open. **9. `full`.** The description lists a `full` parameter; `.circleci/config.yml:6-27` does not declare it and `events/manual.yml` has no route for it. **10. Paths.** A `.circleci/` change is "full" but the website jobs filter on `website` alone (`events/pr.yml:29-41`). Forgejo's `website.yml` also triggers on `docs/tested-configurations.md` and `docs/tested/**`; `mapping.conf:33` has `website/` only. `mise.lock`, `ci/linux/` and `.forgejo/` have no rule. ### Release chain **11. Main's `release.py` is run from `.ci/main-release/`** (`shared.yml:223-226`), so its root is `<checkout>/.ci` (`scripts/release.py:64`). Loaded from there it finds 0 trusted keys and no `cliff.toml`; from `scripts/` it finds 1 key and the file. The `verify` at `shared.yml:1120` then has no key to trust. **12. A nightly's builder version is the word "nightly".** `shared.yml:1028-1036` turns `nightly-<n>-<hash>` into `VERSION=nightly` for `dotnet publish -p:Version`. Forgejo's `build-windows.yml:103-104` takes a numeric version or `0.0.0`. **13. Dry-run signing.** `shared.yml:1107-1108` uses `/tmp/dry-sign.pem` as the key file, but `keygen` makes a folder of that name holding `release.pem` and `release.pub.pem` (`scripts/release.py:715-717`), and `verify` gets no public key for it. **14. `released-as ... || true`** (`shared.yml:914`) turns a failed check of an earlier release into "none found". **15. The release Windows build restores a compiler cache.** `release-windows` uses the `llvm-mingw` command, which restores `v1-llvm-mingw-*` after the hash-checked install (`shared.yml:196-207`, `:1017`); `docs/ci.md:21` says release builds restore no caches. **16. Two publishers.** `.forgejo/workflows/release.yml` still publishes on main and on `v*` tags. Which one is the only writer while both run? ### Smaller - `shared.yml:878-886`: the comment job, which holds the comment token, falls back to the pull request's copy of four helper scripts when main lacks them (it does today). Failing instead is safer. - `.circleci/config.yml:79` and `:95-97` put `fork_sha`, the branch and the tag into shell text; checking `fork_sha` as 40 hex digits and passing the others through the environment avoids surprises. - On main the lint range is `HEAD..HEAD` (`shared.yml:322-323`). - The game-free checks run on Arm (`test-linux`, `arm.large.gen2`); checks that skip off x86-64, such as `tests/test_fpx_sticky.py:189`, then do not run anywhere. Intended? - `events/release.yml`: `release-version` and `gate` have no tag filter while the jobs between them do. CircleCI's rule for tags asks for one on every job in the chain (https://circleci.com/docs/guides/orchestrate/workflows/#executing-workflows-for-a-git-tag). ### Checked and fine The three examples hold: a docs change adds nothing to lint, meta and third-party; `gpu/` adds the Linux checks, SPIR-V fixtures and Windows Vulkan build; the builder adds its CLI checks and the Windows builder. With `fork_pr` set, the checkout compares the fetched commit with `fork_sha` before using it. Signing and publishing ask for separate contexts; the manual dry run has no publish job. `run_all.py --list` still finds every existing check, and the new JUnit test passes. ### Questions What turns Forgejo's events into the `pull_request`, `push` and `api` events the setup expects, and what posts a failed `ci/circleci: gate` when a required job fails and the gate job never starts? Which CircleCI settings limit the signing, bot and Cloudflare contexts to main and tags, given that a branch in this repository runs its own configuration?
madebyjordan commented 2026-11-14 15:11:44 +00:00
Contributor

Thanks for this. It's the first ask in #394 (CI that gates merges), so I'm keen to see it land. mooki's review covers the config itself; these are the process points around the cutover. They apply to MagicBeanDev/LongshotLauncher#65 and MagicBeanDev/Site#10 too.

1. Make ci/circleci: gate a required check on main as part of the cutover. Today main's protection requires no status checks in Longshot or LongshotLauncher, and Site's main isn't protected at all, so a red gate wouldn't stop a merge. Until the gate is required it's advice only. It's also what Forgejo's "merge when checks pass" needs if we use that in place of a merge bot: with no required check, it merges straight away.

2. Get main green first. Once the gate is required and the nightly waits for it (mooki's point 6), a red main blocks every merge and every nightly. test_pad_layout.py (failing since #137) needs fixing before the switch. After that, when main goes red, reverting the PR that broke it should be the first move, so nightlies don't stall.

3. The fork label becomes a step before every merge. Most contributors work from forks. With the gate required, a fork PR can't merge until a maintainer adds ci: approved, and in this PR the label is needed again after every push. With approvers.txt empty, that's one maintainer's waking hours again, so fork PRs would wait overnight for a label instead of a merge. Two asks before the cutover:

  • Here, the behaviour the launcher and Site descriptions give: the label stays on until the PR has been idle for 14 days. The jobs that run fork code get no secrets either way.
  • People in approvers.txt spread across timezones.

4. Where the release key lives. The signing key moves from a Forgejo secret to a CircleCI context. Keeping it to the sign job is good, but it does mean an outside service holds the key that signs every build the launcher installs. That may well be the right call; I'd just like the team to make it on purpose, and note it in RELEASING.md, rather than have it arrive with the CI move. This goes with mooki's question about how the contexts are kept to main and tags.

Thanks for this. It's the first ask in #394 (CI that gates merges), so I'm keen to see it land. mooki's review covers the config itself; these are the process points around the cutover. They apply to MagicBeanDev/LongshotLauncher#65 and MagicBeanDev/Site#10 too. **1. Make `ci/circleci: gate` a required check on main as part of the cutover.** Today main's protection requires no status checks in Longshot or LongshotLauncher, and Site's main isn't protected at all, so a red gate wouldn't stop a merge. Until the gate is required it's advice only. It's also what Forgejo's "merge when checks pass" needs if we use that in place of a merge bot: with no required check, it merges straight away. **2. Get main green first.** Once the gate is required and the nightly waits for it (mooki's point 6), a red main blocks every merge and every nightly. `test_pad_layout.py` (failing since #137) needs fixing before the switch. After that, when main goes red, reverting the PR that broke it should be the first move, so nightlies don't stall. **3. The fork label becomes a step before every merge.** Most contributors work from forks. With the gate required, a fork PR can't merge until a maintainer adds `ci: approved`, and in this PR the label is needed again after every push. With `approvers.txt` empty, that's one maintainer's waking hours again, so fork PRs would wait overnight for a label instead of a merge. Two asks before the cutover: - Here, the behaviour the launcher and Site descriptions give: the label stays on until the PR has been idle for 14 days. The jobs that run fork code get no secrets either way. - People in `approvers.txt` spread across timezones. **4. Where the release key lives.** The signing key moves from a Forgejo secret to a CircleCI context. Keeping it to the sign job is good, but it does mean an outside service holds the key that signs every build the launcher installs. That may well be the right call; I'd just like the team to make it on purpose, and note it in RELEASING.md, rather than have it arrive with the CI move. This goes with mooki's question about how the contexts are kept to main and tags.
drcrow force-pushed ci/circleci from 6a8e365958
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 6s
Details
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
CI / Conventional Commit title (pull_request) Has been cancelled
Details
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
Details
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Details
Third-party parts / As their notes say (pull_request) Has been cancelled
Details
Third-party parts / The same as upstream (pull_request) Has been cancelled
Details
to b866a018e3
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
Details
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Details
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Details
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Details
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Details
PR description / Description follows the template (pull_request_target) Successful in 3s
Details
2026-11-14 16:47:33 +00:00 Compare
drcrow commented 2026-11-14 16:47:42 +00:00
Author
First-time contributor

Thanks @mooki and @madebyjordan, both reviews were really useful. Since 6a8e3659 the branch has been reworked: the shared setup, fork handling, signing and publishing moved into one orb (magicbeandev/ci), and the branch is rebuilt on current main. So some points are fixed by the rework and some by specific changes. Point by point:

mooki

  1. greater: is always true: confirmed with circleci config process. The fork workflow now uses when: pipeline.parameters.fork_pr > 0, which is dropped at 0 and kept at 7. The old get-source step condition is gone: fork checkout is a separate orb command, used only in the fork workflow.
  2. meta template path: gone. PR title and description checks stay on Forgejo, so CircleCI no longer runs check_pr_description.py.
  3. website checks an unbuilt site: it now runs npm run build before npm run check.
  4. Arm lint with a linux-x64-only lock: the lock now has linux-arm64 entries.
  5. --shard: you were right, and it was worse than that: our change also made name selection exact-only, which broke run_all.py ub_. Redone as purely additive. --shard, --write-times and substring names are as on main; new are --list, --junit FILE, and an opt-in --exact (fail closed) that only CircleCI's splitter uses. tests/test_run_all_junit.py checks that --shard 1/2 and 2/2 cover every check exactly once.
  6. The nightly doesn't wait for the checks: it's now one workflow. release-version requires lint, third-party, test-linux, builder-cli, spirv-fixtures and the builds, so a red main signs nothing. A v* tag reruns the core checks before it builds anything.
  7. Main doesn't run everything: main is force-full now.
  8. Fork approval: approvers.txt and the gatekeeper are gone. The label is the approval (Forgejo only lets people with write access add labels), it's sticky, so new pushes keep running, and it's removed after 14 quiet days. The new doorbell is webhook-driven, so there's no 50-PR scan.
  9. full: declared in config.yml and used by main, tags and API runs.
  10. Paths: .circleci/ is full and includes the website jobs. The website jobs also run for docs/tested-configurations.md and docs/tested/**. mise.lock is full; .forgejo/ is lint only.
  11. release.py root: main's copy now goes to .mb-main/release.py in the checkout, so its root is the checkout: it finds the trusted key and cliff.toml. The orb's sign and publish jobs had the same bug and are fixed the same way.
  12. Nightly builder version: now numeric, or 0.0.0, as in build-windows.yml.
  13. Dry-run signing: the orb's release-sign uses keygen's release.pem and release.pub.pem and verifies against the public key.
  14. released-as ... || true: removed.
  15. Release Windows build cache: llvm-mingw takes a cache parameter, and release builds pass false.
  16. Two publishers: until cutover, CircleCI runs the main and tag chains and the website deploy as dry runs, so Forgejo's release.yml and website.yml stay the only writers. The cutover PR turns CircleCI live and removes those workflows in the same commit.

Smaller ones:

  • Comment-job fallback to the PR's scripts: gone; the preview comment is an orb command now.
  • fork_sha, branch and tag in shell text: they go through environment: now, and the orb's checkout refuses a fork_sha that isn't 40 hex digits.
  • Lint range on main: no longer HEAD..HEAD; it compares against HEAD~1.
  • Arm and x86-only checks: test-linux runs on x86 (large), so test_fpx_sticky and the other x86-only checks still run.
  • Tag filters: every job in events/release.yml has one now, and so do the tag setup and noop jobs.

Questions:

  • Events: CircleCI's Forgejo integration turns pushes and same-repo PRs into push and pull_request pipelines; api is a manual run. Fork PRs come in through a second CircleCI project on this repo, started by a Forgejo webhook.
  • A failed job and gate: each job posts its own ci/circleci: <job> status, so the cause shows red on the commit; gate never appears. Once gate is required, that blocks the merge. I'd rather not give gate an API token on PR pipelines just to turn it red.
  • How contexts are kept to main and tags: each context has an expression restriction, e.g. release-signing is pipeline.git.branch == "main" or pipeline.git.tag starts-with "v" and is limited to the Longshot and Launcher projects. Those values come from the event, not from the config, so a branch's own config can't change them.

Jordan

  1. Required gate: agreed. It's on the cutover checklist, along with protecting Site's main and v* tags, before "merge when checks pass" is turned on.
  2. Main green first: agreed. test_pad_layout.py needs fixing before the gate is required, and reverting first when main goes red makes sense.
  3. The fork label: sticky now, as you asked. With no approvers list, anyone with write access can approve, which spreads it across time zones.
  4. The signing key: fair, it should be a deliberate team decision. It's on the agenda with the owner, and RELEASING.md will say where the key lives and who can rotate it before CircleCI signs anything real. Until then, signing on CircleCI only ever uses a throwaway key.

The orb's source will be in MagicBeanDev/orb. The owner is creating it, and I'll link the first PR here as soon as it's up.

Thanks @mooki and @madebyjordan, both reviews were really useful. Since `6a8e3659` the branch has been reworked: the shared setup, fork handling, signing and publishing moved into one orb (`magicbeandev/ci`), and the branch is rebuilt on current main. So some points are fixed by the rework and some by specific changes. Point by point: **mooki** 1. **`greater:` is always true:** confirmed with `circleci config process`. The fork workflow now uses `when: pipeline.parameters.fork_pr > 0`, which is dropped at 0 and kept at 7. The old `get-source` step condition is gone: fork checkout is a separate orb command, used only in the fork workflow. 2. **`meta` template path:** gone. PR title and description checks stay on Forgejo, so CircleCI no longer runs `check_pr_description.py`. 3. **`website` checks an unbuilt site:** it now runs `npm run build` before `npm run check`. 4. **Arm lint with a linux-x64-only lock:** the lock now has `linux-arm64` entries. 5. **`--shard`:** you were right, and it was worse than that: our change also made name selection exact-only, which broke `run_all.py ub_`. Redone as purely additive. `--shard`, `--write-times` and substring names are as on main; new are `--list`, `--junit FILE`, and an opt-in `--exact` (fail closed) that only CircleCI's splitter uses. `tests/test_run_all_junit.py` checks that `--shard 1/2` and `2/2` cover every check exactly once. 6. **The nightly doesn't wait for the checks:** it's now one workflow. `release-version` requires lint, third-party, test-linux, builder-cli, spirv-fixtures and the builds, so a red main signs nothing. A `v*` tag reruns the core checks before it builds anything. 7. **Main doesn't run everything:** main is `force-full` now. 8. **Fork approval:** `approvers.txt` and the gatekeeper are gone. The label is the approval (Forgejo only lets people with write access add labels), it's sticky, so new pushes keep running, and it's removed after 14 quiet days. The new doorbell is webhook-driven, so there's no 50-PR scan. 9. **`full`:** declared in `config.yml` and used by main, tags and API runs. 10. **Paths:** `.circleci/` is full and includes the website jobs. The website jobs also run for `docs/tested-configurations.md` and `docs/tested/**`. `mise.lock` is full; `.forgejo/` is lint only. 11. **`release.py` root:** main's copy now goes to `.mb-main/release.py` in the checkout, so its root is the checkout: it finds the trusted key and `cliff.toml`. The orb's sign and publish jobs had the same bug and are fixed the same way. 12. **Nightly builder version:** now numeric, or `0.0.0`, as in `build-windows.yml`. 13. **Dry-run signing:** the orb's `release-sign` uses `keygen`'s `release.pem` and `release.pub.pem` and verifies against the public key. 14. **`released-as ... || true`:** removed. 15. **Release Windows build cache:** `llvm-mingw` takes a `cache` parameter, and release builds pass `false`. 16. **Two publishers:** until cutover, CircleCI runs the main and tag chains and the website deploy as dry runs, so Forgejo's `release.yml` and `website.yml` stay the only writers. The cutover PR turns CircleCI live and removes those workflows in the same commit. Smaller ones: - **Comment-job fallback to the PR's scripts:** gone; the preview comment is an orb command now. - **`fork_sha`, branch and tag in shell text:** they go through `environment:` now, and the orb's checkout refuses a `fork_sha` that isn't 40 hex digits. - **Lint range on main:** no longer `HEAD..HEAD`; it compares against `HEAD~1`. - **Arm and x86-only checks:** `test-linux` runs on x86 (`large`), so `test_fpx_sticky` and the other x86-only checks still run. - **Tag filters:** every job in `events/release.yml` has one now, and so do the tag setup and noop jobs. Questions: - **Events:** CircleCI's Forgejo integration turns pushes and same-repo PRs into `push` and `pull_request` pipelines; `api` is a manual run. Fork PRs come in through a second CircleCI project on this repo, started by a Forgejo webhook. - **A failed job and `gate`:** each job posts its own `ci/circleci: <job>` status, so the cause shows red on the commit; `gate` never appears. Once `gate` is required, that blocks the merge. I'd rather not give `gate` an API token on PR pipelines just to turn it red. - **How contexts are kept to main and tags:** each context has an expression restriction, e.g. `release-signing` is `pipeline.git.branch == "main" or pipeline.git.tag starts-with "v"` and is limited to the Longshot and Launcher projects. Those values come from the event, not from the config, so a branch's own config can't change them. **Jordan** 1. **Required gate:** agreed. It's on the cutover checklist, along with protecting Site's main and `v*` tags, before "merge when checks pass" is turned on. 2. **Main green first:** agreed. `test_pad_layout.py` needs fixing before the gate is required, and reverting first when main goes red makes sense. 3. **The fork label:** sticky now, as you asked. With no approvers list, anyone with write access can approve, which spreads it across time zones. 4. **The signing key:** fair, it should be a deliberate team decision. It's on the agenda with the owner, and RELEASING.md will say where the key lives and who can rotate it before CircleCI signs anything real. Until then, signing on CircleCI only ever uses a throwaway key. The orb's source will be in MagicBeanDev/orb. The owner is creating it, and I'll link the first PR here as soon as it's up.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
Details
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Details
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Details
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Details
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Details
PR description / Description follows the template (pull_request_target) Successful in 3s
Details
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'ci/circleci:drcrow-ci/circleci'
git switch 'drcrow-ci/circleci'
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
area/accessibility
Barriers for people with disabilities

  
area/audio
Sound and music

  
area/input
Controllers, keyboard, mouse

  
area/install
Installing, updating, building from your game

  
area/menu
In-game menu and settings

  
area/renderer
Vulkan renderer, shaders, effects

  
area/saves
Save files

  
area/upscaling
DLSS, FSR, frame generation

  
good first issue
A good start for a new contributor

  
gpu/amd
AMD GPUs

  
gpu/igpu
Integrated GPUs

  
gpu/intel
Intel GPUs

  
gpu/nvidia
NVIDIA GPUs

  
help wanted
Extra hands or knowledge wanted

  
kind
bug
 Something doesn't work as it should

  
kind
chore
 Build, CI, dependencies, tidying

  
kind
docs
 Documentation

  
kind
feature
 A new feature or an improvement

  
kind
performance
 Frame rate, frame time, loading, memory

  
kind
question
 A question rather than a change

  
needs measurements
Waiting for before/after measurements

  
platform/linux
Linux only

  
platform/steam-deck
Steam Deck

  
platform/windows
Windows only

  
preview
Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  
priority
critical
 Crashes, data loss, or blocks most players

  
priority
high
 Important: next in line

  
priority
low
 Nice to have

  
status
blocked
 Waits on something else first

  
status
confirmed
 Reproduced or agreed on

  
status
duplicate
 Already reported or proposed

  
status
invalid
 Not a problem in this project

  
status
needs-info
 Waiting for more information from the reporter

  
status
wontfix
 Will not be worked on

  
testing wanted
Wanted: a test on other hardware (posts a test request in #test-requests)

No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!417
No description provided.