fix(privacy): one shared rule keeps the user's name out of the launcher's log and every report #164
No reviewers
Labels
No labels
area/build
area/install
area/saves
area/tooling
area/ui
area/updates
ci: approved
good first issue
gpu/amd
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
platform/linux
platform/steam-deck
platform/windows
priority
critical
priority
high
priority
low
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/LongshotLauncher!164
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/LongshotLauncher:fix/build-report-scrub"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The launcher's log, its bug reports and the build summaries must never carry the OS user's name, and #162's summaries could: its filter replaced whole paths, missed some shapes, and still let folder names through. This PR makes one shared rule that takes the user's name out of every path, and leaves the rest of the path, which is what tells a helper where a build failed.
The shared helper:
Privacy.Scrub(Core/Privacy.cs)BugReport.Scrub(home folder as~) moves into its own class and is extended. With no arguments it reads this PC's home folder and user itself; tests pass their own. Its placeholders are the game's own (Longshot'sgpu/vk/vk_bug_report.cppandhos_crash_redact_lineinhost/hos_crash_dialog.h), so a launcher's report and a game's read alike.%USERPROFILE%on Windows and~on Linux, wherever a path starts with it: in any case on Windows, either separator, doubled as JSON writes it,%20in afile://link.C:\Users\Name2andC:\Users\Name Smith\xstay as they are.<user>, wherever it is a whole folder's name in a path: another drive (D:\Users\Ana\x), a backup or OneDrive folder (E:\Backup\Ana\saves,D:\OneDrive\Ana\x),/media/<name>,/run/media/<name>,/var/home/<name>(Steam Deck and other image-based distros), and on Windows in any case and as its 8.3 short form (C:\Users\JANEAN~1).%USERPROFILE%and<user>are never replaced again (checked, also for users calleduserandUSERPROFILE).Environment.UserName, which can differ from it (a renamed account, a Microsoft account's 5-letter folder).Anain a sentence,Ana2andAna.oldstay; a name with spaces, an apostrophe or non-ASCII is taken out whole (D:\Users\Jane Ann Doe\x,D:\Users\Seán O'Brien\x).Where it runs:
LauncherLog.Addscrubs every line before it keeps it, so no caller has to remember. Whatever reads the log gets no user name either. #171 (the log inlogs/launcher.log) gets this for free if its file writer uses the linesAddkeeps.BugReport.MakeandBuildReports.Jsonuse the same helper (on each field, then on the whole JSON).The build summary's diagnostic (Core/BuildReports.cs)
[path]patterns andFailure.Known(the player's own texts, matched as substrings) are gone.[game file](CONTRIBUTING's no-game-files rule):.nup,.nx(fromGameFiles.Extensions, kept in step by a check), other dump and key formats, the game's own files; a quoted one whole, spaces and all. A call such asd.keys()orstruct.pack()stays.?token=) and a fragment never go, for magicbean.dev too.Other fixes, as before
whyis required, and the launcher's own SIGTERM sendsBuildReports.LauncherStopped. Its summary now sends no earlier error line, so it counts under one signature.why.[GeneratedRegex], as inProblems.cs.Not done
perf_sent:perf_sent/is the game's sender's folder, and a failed first build has no game folder yet. It needs a place of its own and a Settings wording decision: its own PR.ProblemKindplus the stage and exit code: with paths kept, a free-text signature now differs per PC folder layout, so this matters more. It's a design choice for whoever owns Longshot-api's dashboards; this PR keeps the free text and makes it safe.C:\Games\The Legend Dump v2.nupgoes asC:\Games\The Legend Dump [game file].e.Messageon two of the build step's failures is the OS's own text, in its language on Windows: not changed.Testing
python -I tests/test_launcher_core.py: 1584 checks passed. New:PrivacyChecks(the moved home-folder cases; names with several spaces, an apostrophe, non-ASCII; another drive, OneDrive, a backup folder, an 8.3 name;/media,/run/media,/var/home;Environment.UserNamebeside the folder's name; prose, longer names and relative paths kept; this PC's own home and user by default; a line throughLauncherLog.Add), the summary's JSON end to end on Windows and Linux paths (no user name anywhere in what is sent, the rest of the path kept,%USERPROFILE%and<user>written as they are in a summary's and Send's JSON, never\u003C), the diagnostic's table, and a 100 KB word.python -I tests/test_release.py: 37 tests OK.dotnet build App -c Release: 0 warnings, 0 errors.mise run ci:smoke(no mise here), a build through the app, Linux.Request changes: automatic failure reports can still disclose parts of private paths containing multiple spaces.
Reviewed head:
46018097c1cd46c0684991e3ff6c0733c8f7c4b4, including the stacked #162 changes. The inline finding is reproduced throughBuildReports.Jsonwith an automatic failed report, using authored strings only; no telemetry was sent.Findings
Failure.Knowncan retain middle words. See the inline reproduction onCore/BuildReports.cs:91. Please cover these cases through the final JSON serializer as well asDiagnostic.Validation
python -I tests/test_launcher_core.py: 1,470 checks passed outside the sandbox. Windows symlink cases were skipped because the required privilege is unavailable.python -I tests/test_release.py: 37 tests passed, with Git's OpenSSL on PATH and access to the suite's local fake HTTP server.dotnet build App -c Release -warnaserror: 0 warnings, 0 errors.Review and tests performed with OpenAI Codex. No repository source changes were retained.
@ -65,0 +88,4 @@/// An absolute path: a drive (C:\, C:/, not https:/), ~\ or ~/, \\server, or a / that starts a word; through a space when the/// word after it holds a separator too (D:\Jane Doe\x), and through the words that end a game file's name when its last name/// has no extension (C:\Games\The Dump v2.nup)[GeneratedRegex("""(?:(?<![A-Za-z0-9])[A-Za-z]:[\\/]|(?<![^\s'"(\[=,])(?:~[\\/]|\\\\|/(?=[^\s/'"<>|])))[^\s'"<>|]*(?:[ \t]+(?=[^\s'"<>|]*[\\/])[^\s'"<>|]*)*(?:(?<=[\\/][^\s'"<>|.\\/]*)(?:[ \t]+[^\s'"<>|\\/]+){1,6}?\.(?i:""" + GameExtensions + """)\b)?""")][P1] Redact the whole path when a directory has multiple spaces
The continuation only crosses a space when the immediately following token contains a slash. Consequently
cannot open D:\Jane Ann Doe\tools\make_game.pybecomescannot open [path] Ann [path], and/media/alex/My Very Private Drive/tools/make_game.pybecomes[path] Very Private [path]. Both outputs are also present in the finalBuildReports.Jsonpayload for an automatic failed report when these paths are outsideFailure.Known(for example, a tool/dependency path). This silently transmits parts of private directory names despite the diagnostic's redaction guarantee. Please redact the complete path, including intermediate words, and add end-to-end JSON regression cases for Windows and Linux paths with multiple spaces.I read this against #162's head (
f8a0786) and ran its checks. The shape is good, but the "known texts" part, the main fix here, does not work yet for the most common layout (a dump or build folder under the home folder), and the tests would not notice. Changes needed before it merges.Run (Linux only; no Windows run, no game, no real failed build):
python -I tests/test_launcher_core.pyon this head: 1491 checks passed (the body says 1477); on #162's head: 1465.tests/test_release.py: 37 OK. About 280 inputs of my own throughBuildReports.Json, this head next to #162's, and I broke each fix once in a scratch copy to see if a test notices. Not run:dotnet build App(can't restore the apphost pack offline here),mise run ci:smoke.Blocking
~first, so a knownC:\Users\Ana\Downloads\The Game v2.nupno longer matches, and the bare name then becomes[game file](it holds a space) inside~\Downloads\....can't read C:\Users\Ana\Downloads\The Game v2.nup: bad headergoes out ascan't read [path] file]: bad header; with the file onD:\it is[path], so one failure lands in two signatures. A build or work folder under home that ends in a spaced name leaks its last word (C:\Users\Ana\Longshot Workbecomes[path] Work). The check usesD:\My Dumps, outside home, so it passes. Fix: scrub the known texts with the same home folder first, and replace them in one regex pass.Known()adds: BuildStepViewModel.cs L544-L547). A dump calledgame.nupturnsnot a game executableintonot a [game file] executable; a folder calleddumprewritestry another dump of the game;mainturnsdomain remain maintainintodo[game file] re[game file] [game file]tain. What the player named the file then picks the signature (Zelda.nup,dump.nup,game.nup: three hashes for one failure). Fix: whole words only, and no bare name that is a single plain word./data/Longshotknown,/data/Longshot/My Secret Folder/xgoes out as[path]/My Secret Folder/x.Should fix
whyis fixed, butError = lastErroris still sent (L534, L596) and Longshot-api hasheswhy | error. Three different last stderr lines gave three signatures; the fixture sets an error line and asserts onlywhy.Failsending the translated why,Known()never passed to theFailure,Known()without the build and work folders, and the English choice dropped on the Win32 path and in oneExitedbranch.Known()is in the App project, which the core tests don't compile, and the regex at BuildReportChecks.cs L253 matches no line of the code. Moving the list building into Core would let a check cover it.D:\Games\The Legend of Zelda\x.nupbecomes[path] Legend of [path], a spaced last folder (/media/jane/My Passportbecomes[path] Passport), andC:\Program Files (x86)\...leavesFiles. make_game prints its paths unquoted, so this is the usual shape of its error lines, and the Settings text says "No paths, names or saves". Either the rule goes further or the text says less.?token=query and a game file name in the path all go out. Keep scheme, host and path only.e.Message(L194, L226) is the OS's own text on Windows. I could not test that here.Nits:
struct.pack,d.keys()andnx2.bntxnow become[game file]; a path plus up to five words plusx.zsis swallowed as one[path]; one 100 KB word with many=takes 8 s in the regexes (3 s on #162 already; no timeout, and it runs on the window's thread); the picker filter and Finder keep their own*.nupand*.nxlists instead ofGameFiles.Extensions; an apostrophe in a folder name ends the path; the body's 1477 is stale.Held up: every home-folder spelling I tried (spaces, case,
/or\,\\?\, 8.3 names,/var/home, WSL), UNC paths, non-ASCII names,3/10,N/A,and/or, version strings, and the look-alike hostsevilmagicbean.devandmagicbean.dev.evil.com.On the open question in the body: the problem kind plus the stage as the signature looks better to me. It needs no scrubbing for the key, though the free text would still have to be safe.
WIP: fix(build): a summary's diagnostic loses paths with spaces and every dump's name, and says why in Englishto fix(build): a summary's diagnostic loses paths with spaces and every dump's name, and says why in English46018097c12a75d314aefix(build): a summary's diagnostic loses paths with spaces and every dump's name, and says why in Englishto fix(privacy): one shared rule keeps the user's name out of the launcher's log and every reportReworked (
2a75d31, rebased onto main now that #162 is in). Thanks both: the reviews showed whole-path scrubbing couldn't be made tight. The requirement is narrower: only the OS user's name must not leave the PC; the rest of a path is fine and useful. So there is now one shared rule,Privacy.Scrubin Core/Privacy.cs, whichLauncherLog.Add, bug reports and build reports all use. The description has the details.futondestroyer, [P1] multiple spaces: paths are now kept on purpose, and the user's name in them goes whole, whatever its spaces.
cannot open D:\Users\Jane Ann Doe\tools\make_game.pyis sent ascannot open D:\Users\[user]\tools\make_game.py, and/media/alex/My Very Private Drive/tools/make_game.pyas/media/[user]/My Very Private Drive/tools/make_game.py. New end-to-end checks go throughBuildReports.Jsonfor Windows and Linux, and assert no user name anywhere in the JSON.mooki:
Failure.Knownand the substring replacement are gone. The rest of a path staying is now the intent.why. The check asserts both.Known(): gone with it. The English-why source checks stay as they were.The Legend of Zelda,My Passport,Program Files (x86)): kept by design now. The consent text said "No paths, names or saves" and now says "No user name, game files or saves", in all 12 catalogs.e.Messagebeing the OS's text: not changed, noted in the description.d.keys()andstruct.pack()stay; a path plus words swallowed whole is gone; a 100 KB word is cut before the patterns run (checked); an apostrophe no longer matters.nx2.bntx-style module names still become[game file], and the picker's and Finder's own*.nup/*.nxlists are unchanged (out of scope). The count is updated: 1579 checks.ProblemKindplus the stage: agreed, and with paths kept it matters more. It's listed as the open question; it's a call for whoever owns the dashboards.Screenshots: before and after
Drawn with
tools/screenshots(headless, the same scene and window on both sides): before frommain@321a9e1, after from this branch.Settings: the performance statistics text
The consent line now says what is actually left out, since paths are kept and only the user name goes.
Before
After
The placeholders now match the game's (
49ba742). Longshot's gpu/vk/vk_bug_report.cpp and hos_crash_redact_line write the home folder as%USERPROFILE%on Windows and~on Linux, and the user's name elsewhere in a path as<user>; the launcher's log, bug reports and build reports now write the same ([user]and a Windows~are gone).\u003C. Its encoder already leaves<and>alone, and it's scrubbed again after it is written. A check reads a summary's and Send's final JSON.userandUSERPROFILE.futondestroyer, mooki: could you have another look when you have a moment? The rework from your reviews is in
2a75d31, and this commit only changes the placeholders.