feat(queue): an interface for tools: status and history as JSON, move to reorder, settings read again while a job waits #1852
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1852
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:feat/queue-interface"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The PC's job queue (
dev.py queue) had no interface a tool could use. A dashboard, a script or an agent wanting to show what runs and waits, and why, could only scrapestatus's text: #1802 leftstatus --jsonandhistory --jsonbehind withjob_queue.py. And nobody could change the order of waiting jobs. A waiting job also readjob_queue.jsononce, when it was queued, so a slot changed there reached only the jobs queued after the change.This PR adds three things:
dev.py queue status --jsonandhistory --json [-n N]: one JSON object ("schema": 1) with whatstatusprints and more:job_queue.jsonor default) and the settings file's path.dev.py queue move JOB --first | --last | --to N | --before JOB | --after JOB, andmove --clear:--cleargives the order back to the queue.job_queue.jsonagain every 5 s. A slot opened there starts the jobs already waiting. The queue's folders are kept: a job never changes queue.Builds on #1816: this branch's first four commits are #1816's, needed for the memory in
status --jsonon Windows and formemory_wait()'s reason in it. Review the last commit,feat(queue): an interface for tools....Read first:
scripts/devtools/jobqueue.py:snapshot(),rank()/ahead()/placed(),move(), the settings re-read inrun().How the order is kept:
<queue>/.order, JSON, written under the queue's.lock, as runner pid and start time, so a pid used again by a later job is not the placed one.*.jsonfile because every version of the queue reads those in that folder as tickets, and drops one with no live runner. A first try named itorder.json: the runners deleted it at once.Prior work: searched the tracker for a queue order or reorder, queue priority,
status --json,history --json, queue status, move, cancel and settings: nothing but #1523, which broughtstatus --json's Windows fields that #1802 then dropped. No claims in the chat.Testing
Tested: Windows 11, 64 GB, on a private queue and read-only on this PC's own.
tests/test_job_queue.py:Order: placed jobs go first and before a high priority; a reused pid is not the placed job; a timed run's group follows the order.Move: every way to place a job, by name and by id; a job queued after a move goes behind it;--clear; a running job, an unknown name and a doubled name are refused;.ordersurvives the runners' cleanup of dead tickets;status's snapshot gives each job's place, its reason and whether it was placed.StatusJson: throughdev.py, the schema, the slots, and each setting's source (environment over file over default).RealOrder, real jobs on any OS: a job moved first starts first; a waiting job with no light slot starts oncejob_queue.jsonopens one.*.json, and the order ignored for a timed group.dev.py queue status,status --json,history -n 2 --jsonandmove --helpon this PC's queue;status --jsonreadLONGSHOT_QUEUE_STATEas coming from the settings file, and the file's path.tests/run_all.py(mise run test's checks), with the Builder's llvm-mingw first onPATH: 692 passed, 56 failed, 123 skipped. The failing checks, rerun here and on a cleanmainwith the same compiler, fail the same on both, none here only (main's known Windows failures).Still to test:
python -I -B tests/test_job_queue.py Order Move StatusJson RealOrder, expecting all to pass. Nothing in the order or the JSON is system-specific (by hand).Measurements
Verdict: no cost.
statustakes as long as onmain, and the settings re-read costs 236 µs every 5 s while a job waits.Setup: Windows 11, 64 GB. No game run: a development script. Measured
dev.py queue statuswall time, 5 runs each, interleaved in two rounds, the median of the second (the first had a cold start onmain), on this PC's queue with one job running; andsettings()timed over 5 × 1000 calls, the median.3f2c9b3c13df2323dev.py queue statusdev.py queue status --jsonRisks
statusmarks it(placed).move --clear.status's reason for a waiting job is now exact, where it saidstarting (or waiting for memory).move, ordev.py queue move --clear; revert the commit for the rest. Nothing changes until someone moves a job or editsjob_queue.json.LONGSHOT_QUEUE_DIR,LONGSHOT_QUEUE_STATEandLONGSHOT_QUEUE_QUIETshow in new lines only as the queue's existing settings, read as before (the re-read keeps the first two as the job started with), at their defaults.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.