feat(dev): dev.py update keeps a game folder on main's newest, picks merged PRs, merges open ones on top, gathers logs (--gui) #1932
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1932
Loading…
Reference in a new issue
No description provided.
Delete branch "GummiNaomi/Longshot:feat/dev-update"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Testing what landed on
mainan hour ago means waiting for Nightly or rebuilding a game folder by hand. This addsdev.py update: it keeps a game folder you play on the newest commit ofmain(or another branch), or on its owncommit plus only the merged pull requests you pick, with open pull requests merged on top if you want to try them
before they land. It also gathers the folder's logs into one zip for a bug report, and does all of it in a small window
with
--gui.An update:
update/treein the development folder,detached) on the newest commit, so your checkouts and branches are not touched;
image.bin,longshot_env.txt,longshot_options.ini, the texts...) intoupdate/backups/, the last three kept;dev.pywhen it has this command (so its owntools.lockand builder), in the player's build environment (build.environment), through the job queue, sharingthe development folder's stores: when the translation didn't change, only what changed compiles. A folder that keeps
its game files (
romfs/,.longshot_exefs/) is built again from them (make_game.py --update); a new one from thefiles
dev.pyfinds (the romfs hard-linked, asmake_game.pydoes);dev.py updatecommand, once its process is gone. On success it writes the commit into the folder(
.longshot_dev_update) and what came in intoupdate/history.md.Picking (
--pick, or unticking merged pull requests in the window): the worktree goes on the folder's commit andthe picked pull requests' commits are cherry-picked onto it in their order. One that needs a pull request left out is
refused, naming it and the files it conflicts in. Every one picked is the newest commit itself. The picks are written
beside the commit (
.longshot_dev_update_picks), and the next check marks them "in your build".Open pull requests (
--with, or ticking them in the window): each one's head (refs/pull/N/head, which Forgejo,Gitea and GitHub keep) is fetched and merged onto the build in number order, after the main commit or the picks. One
that conflicts is refused before the folder is touched, named with its files. The open ones in a build are written
beside its commit (
.longshot_dev_update_open) and marked by the next check ("updated since" when the pull requestmoved); an update without
--withleaves them out and says so. The list is read from the remote's forge withoutsigning in (Forgejo or Gitea's
/api/v1, or GitHub's API, from the remote's URL). The forge list'smergeablefieldis not shown: it was stale here for pull requests that merge cleanly, so the real merge decides.
Logs (
--logs, the window's Gather logs):logs/(the log, its older copies, the frame log; neverprofile-*folders),
bug-reports/butbug-reports/local-only/(minidumps: game and private data, never shared), thesettings files, the last build log and the history, and a summary of the PC and the build, zipped into
update/logs/.Logs can name the user's folder: the command says to read them before sharing.
The window (
--gui): on the left, the pull requests merged since the folder's build (colour-coded by kind, tickedto pick) and the open ones (ticked to merge on top; drafts and WIPs greyed), with a search box over both (a number
matches the pull requests whose number holds it, other text the titles and authors; Ctrl+F); on the right, the build
log in colour (errors, warnings, stages, done). Check, Update, Roll back, Stop, Gather logs. Each action runs
dev.py update ..., so the window and the command line behave the same.It refuses while the game in that folder is running. Saves, mods,
longshot_options.iniandlongshot_env.txtarekept by the builder, as in a launcher update.
Read first:
scripts/devtools/update.py(cmd_update,run_build,merge_open,recover), then the new sectionKeep a game on main in
docs/dev/developing.md.Testing
Tested: Windows 11 Pro 26200, RTX 4070 SUPER (NVIDIA 617.42), Ryzen 9 9950X3D, Python 3.13.
dev.py update --game <new folder>on a PC with no development folder yet:tools.lock's Windows tools installed, main @87c0d7b4built from the game filesdev_build.jsonnames (romfshard-linked), the shaders prepared: 20.5 min in all (the build 14:45). The folder's commit, the history and a
backup written. The game started from it, ran 75 s with no window and sat at the title with no
CRASHor[error]line in its log.
--forceon that folder: 0.5 min, every object and the translation reused; thegame, renderer and
image.binbyte for byte the same.dev.py updatecommand said "the update to87c0d7b4did not finish: the game folder is put back as it was"; thegame, renderer and
image.binwere byte for byte as before, no update left pending, no build process left.the summary (system, game commit, GPU and driver).
89c3d21e: 36 merged pull requests listed, colour-coded; untickingtwo shows "34 of 36 merged"; 292 open pull requests listed from the forge without signing in; searching "18"
narrows both sections (15 merged, 29 open shown); ticking an open one shows "1 open on top".
tests/test_dev_update.py(new, game-free: a temporary repository stands for upstream, withrefs/pull/N/headrefsfor open pull requests, and a stand-in for the build writes the folder), 25 cases: the new commits listed; an update
that builds, records the commit and the history and keeps three backups; an up-to-date folder not built (and built
with
--force); a failed, an interrupted and a stopped build put back; an update still running not put back;--rollback; the settings remembered; make_game.py's command for a folder with and without its game files; thebuild handed to a newer tree's own
dev.py; a dirty worktree and a running game refused;--pick(only the pickedones built, a pick that needs one left out refused and named, every one picked = the newest);
--with(open headsmerged on top, with
--picktoo, one that conflicts refused and named, an unknown number refused, an updatewithout it leaving them out); the search; the forge API from https, scp-style, ssh and GitHub remote URLs; the
window's JSON list;
--logs(the minidumps and profiles left out). All pass.test_dev_update,test_doc_anchors,byte_compile_trackedpass.test_job_queue,test_dev_ab,test_dev_packs,test_dev_migrateandtest_dev_buildfail on this Windows PC with exactly the same failing caseson
main@66dc0a51(Linux paths and file behaviour); ruff isn't installed here, so CI's run is the lint.build.environment()prints "no NVIDIA DLSS in this build" before it appliesdev_build.json'senv, so withLONGSHOT_NGX_SDKset there it warns although the build has DLSS (its log says"with NVIDIA DLSS").
Still to test:
--withfor real (an open pull request merged on top and built), and--pickfor real once main has moved(by hand, me)
--with, and--gui(by hand: someone with Linux)Measurements
Nothing to measure: a development tool that runs the existing builder; the game is unchanged. Its own times are in
Tested (a fresh folder 20.5 min, an update with nothing changed 0.5 min).
Risks
Developers only: nothing reaches players. It writes only into the folder you name (after backing it up) and into the
development folder's
update/(a worktree, a work folder and up to three backups of about 0.7 GB each). It reads theforge's public API for the open pull requests and fetches their heads; nothing is sent. A revert is enough to remove
it;
update/can be deleted.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). (The dev tools' checks and the docs'ones; the five Windows failures are main's own, named in Testing.)
main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Possibly related, for context:
Recommendation: Request changes. Rollback and recovery can leave a game folder with mismatched build files or restore the wrong folder, and the running-game and concurrent-update guards are incomplete. The feature also has not yet completed a real build on either supported OS.
This adds a command and window to follow a branch, build a game folder, and restore a backup. The intended convenience comes with data-integrity risks in rollback and recovery, and the claimed reuse of the target commit’s build environment does not hold for commits predating this command.
Does it help: Unproven. The reported Windows tests use a stand-in builder; Check and opening the window were exercised, but no real update, incremental rebuild, or Stop was completed. This is not a game performance change.
Other systems: Windows launcher-built folders can store the game and libraries in bin/, which the backup omits. Linux process detection can miss games started with relative paths. Both OSes share the recovery and concurrent-update risks. The real builder also writes files and metadata beyond the backup list; its installed-file manifest can then disagree with restored files. Linux and Windows real builds and GUI Stop remain untested.
Switches: Adds --game, --remote, --branch, --check, --rollback, --gui, --force and internal --build-here/--commit options. Remote defaults to origin, branch to main, and force is off; the chosen game and branch settings are remembered. It sets LONGSHOT_KEEP_BUILD_FILES=1 for builds; this is appropriate for incremental updates, but their behavior is not yet demonstrated with a real build.
Reviewed:
10b32ae4fdagainst3df5c13ed2: Read AGENTS.md, CONTRIBUTING.md, the PR template, standards, the diff, scripts/devtools/update.py, scripts/dev.py, scripts/devtools/common.py and tests/test_dev_update.py. Read relevant builder, process and queue paths; skimmed the changed documentation and test references. docs/review-checklist.md and ../Tooling/AGENTS.md were not present. No recorded runs, translations or generated files were changed. No earlier comments or reviews were supplied; overlapping PRs could not be checked with the network closed.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Suggestions (the author may take or leave)
Testing: Review only; no commands from the project or tests were run. The author reports 15 stand-in tests passing on Windows, plus Check and window opening; the real builds and Linux run remain outstanding. Full test and lint results and CI on the head are not established.
Overlaps and merge order: Not checked: network access is closed.
Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.
@ -0,0 +125,4 @@want = os.path.normcase(os.path.abspath(game))for pid, exe in procs.exes().items():if exe.lower() != GAME_EXE:continueA Linux game launched as ./longshot is resolved against the updater's cwd here, and a launcher-built game's executable can be in bin/. Both can evade this check. Please resolve the actual executable path and recognize the folder's bin/ game.
@ -0,0 +158,4 @@return ddef put_back(d):The builder can replace bin/longshot.exe and libraries in launcher-built folders, as well as other outputs and its installed-file manifest. Restoring only this list can leave a mixed build or stale manifest. Please restore the complete builder-managed state, including files that were previously absent, and test a partial finish.
@ -0,0 +185,4 @@p = json.load(f)except (OSError, ValueError):returnstart = procs.proc_start(p.get("pid", 0))The updater PID can exit while its Linux child queue runner and builder remain alive. A subsequent command can then restore the backup while that builder is still writing. Please ensure the entire build tree has stopped before recovery.
@ -0,0 +228,4 @@log = part("build-%s.log" % commit[:8])if os.path.isfile(os.path.join(tree, "scripts", "devtools", "update.py")) and \os.path.normcase(os.path.realpath(tree)) != os.path.normcase(os.path.realpath(ROOT)):return subprocess.call([sys.executable, "-I", os.path.join(tree, "scripts", "dev.py"), "update", "--build-here",For a target commit without update.py, this fallback uses the current checkout's build environment and tools.lock. That includes current main before this PR lands and can select incompatible tools. Please construct the environment from the target tree in this case too.
@ -0,0 +240,4 @@if a.gui:return gui()if a.build_here: # (the newer tree's own build step: run_build)return _build_here(a)Two update processes can both pass recover() before either writes pending.json, then race on the shared worktree and pending record. Please lock the complete update and recovery transaction across processes.
@ -0,0 +252,4 @@die("there is no backup to put back")if running_in(game):die("the game in %s is running: close it first" % game)put_back(b[0])Backups are global, but the running check uses the selected game. If A was updated and the user selects B, this restores A without checking whether A is running. Please select a backup for the requested folder and check its actual destination.
WIP: feat(dev): dev.py update keeps a game folder you play on main's newest commit, with a window (--gui)to WIP: feat(dev): dev.py update keeps a game folder you play on main's newest commit, picks its merged PRs, gathers logs (--gui)d14f4fe388ee08cc5635WIP: feat(dev): dev.py update keeps a game folder you play on main's newest commit, picks its merged PRs, gathers logs (--gui)to WIP: feat(dev): dev.py update keeps a game folder on main's newest, picks merged PRs, merges open ones on top, gathers logs (--gui)WIP: feat(dev): dev.py update keeps a game folder on main's newest, picks merged PRs, merges open ones on top, gathers logs (--gui)to feat(dev): dev.py update keeps a game folder on main's newest, picks merged PRs, merges open ones on top, gathers logs (--gui)