fix(host): the saved logs leave out the player's user name #1459
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1459
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:fix/log-redact"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: medium · Tested on: Windows 11 (game-free checks only) · Needs: Windows and Linux, any GPU, an in-game run
Why
Players send
longshot_log.txtaround all the time: in issues, on Discord, in chats. Until now only the copies the Bug reportand the crash window make left the home folder and the user's name out. The log itself kept
C:\Users\<name>\...and/home/<name>/...wherever a line had a path, and so did the builder's--logfile. Now every log the game and thebuilder save leaves the user name out as it is saved. The rest of each path stays, so a helper still sees where
things are.
For #1093 (Windows user names in generated files; this covers the logs). See #1264 (the crash report's own redaction).
What changed
The saved logs use the rule the Bug report already used, with one implementation instead of two: the home folder, as a
whole, becomes
~(%USERPROFILE%on Windows), and the user's name becomes<user>wherever it is a folder in a path.The game's own lines are redacted as they are written. Lines other code prints straight to stderr are redacted in the
file once the run has ended. The builder's
--logfile gets the same rule in Python.gpu/vk/compat/longshot_redact.hlongshot_redact_line, a C header the C++ renderer also includes.hos_crash_redact_line(host/hos_crash_dialog.h) and the Bug report'sredact()(gpu/vk/vk_bug_report.cpp) now callit. Their output is unchanged: the new test checks it against
main'sredact()on every case.hos_vlogandhos_fatal(host/hos_thread.c) redact each line before the control bridge, thelog queue or stderr get it, through
hos_log_redact(host/log_redact.c). It takes out this PC's home folder and username, read once from the environment. On Windows it also takes out:
C:\Users\JOHNSM~1, which TEMP paths often use);USERNAME(john.PC).It allocates nothing and takes no lock after the first call. The renderer's
logfgoes through the same path.fprintfs):hos_log_redact_filerewrites the log in place once its run has ended. It locks the file (
LockFileEx/flock) so two rewrites neveroverlap, and leaves the file alone when it's already clean. It runs in two places:
host/hos_crash_dialog.chelper): right after the game is gone, before it reads the log. It onlytouches its own run's log, checked by the
run_id:line, which may have been rotated to.1by then;host/hos_main.c): on the log it has just rotated tolongshot_log.1.txt. This covers runs that hadno crash watch (
--prepare-shaders,HOS_HEADLESS,HOS_CRASH_WATCH=0).make_game.py --log FILEwrites throughnx2/log_redact.pyRedactedLines, the same rule in Python. Itwrites whole lines only, so a path split across two writes is still caught. The console, which a launcher reads, stays
as it was.
docs/crash-report-window.md.What the game and the builder save, and what this PR does with each
logs/longshot_log.txtand.1–.4.txt__GL_SHADER_DISK_CACHE_PATH, the Vulkan loader's layer manifestsmake_game.py --logfilelogs/longshot_frames.csv, the frame log (RX_VK_FRAME_LOG)logs/profile-*/(HOS_PROFILE=1)process.jsoncould; the rest is numbers, thread names and module+offsetprocess.jsonalready writes<game folder>/or<a path>bug-reports/pending/)hos_report_privacy)bug-reports/<date>/cache/longshot_driver_faults.txtlogs/longshot_startup.logMeasurements
No measurement yet, because the game hasn't been built or run on this PC. The cost per line is a scan of a line of at most
2,200 bytes for up to four short strings, done on the thread that logs, against 30–90 lines a second in play. Frame time
must still be compared in game (Must be tested).
Flags and settings
None.
Testing
Tested: game-free, on Windows 11 with GCC 16.1 (WinLibs).
tests/test_log_redact.py(new) passes. It covers:another drive's
\Users\<name>; OneDrive under the home folder;/home,/media,/run/mediaand/var/home; aname that is also a word of the line (kept); relative paths (kept);
redact()gives the same output.main'sredact()gives the same output for every case too;hos_log_redactfrom the environment: a profile folder named apart fromUSERNAME, and a real 8.3 short name onthis drive;
hos_log_redact_file: another run's log is left alone; this run's is rewritten, both shorter and longer; analready-clean file isn't written; a missing file;
RedactedLineswith a line split across writes, and the source checks on every caller.tests/test_build_log.pypasses: it now compares the log with the console's text as the file holds it, and checksthe console keeps the path while the file doesn't.
test_log_levels.py(a line throughhos_vlogcomes out redacted),test_crash_record.py(theorder in
hos_vlogandhos_fatal),test_bug_report.py,test_log_batch.pyandtest_report_privacy.py.test_crash_dialog.py's behaviour build linkshost/log_redact.c.python -I -B tests/run_all.py --changed origin/main: 105 passed, 16 failed, 14 skipped. The 16 also fail onmain@
4dc2a6a5on this PC, run the same way:test_bug_report_defaults,test_crash_dialog(Windows code-page cases, see#1012),
test_display_startup_failure,test_failed_compile_keeps_objects,test_hud_layout_row,test_invalidate_vcore,test_job_queue,test_missing_nup,test_oot_state_tap,test_perf_report,test_prepare_stall,test_profiling_mode,test_slow_frames,test_tested_configurations,test_vcore_escapeandtest_walk_key.test_build_logfaileduntil its update.
main@c14641a6, the checks named above ran again with the same results.Not tested:
host/hos_thread.candhos_main.cneed the lift'sgenerated
a64_suyu.h) and hasn't been run.host/log_redact.c(flock,pread/pwrite). They haven't been compiled: there's no Linux here.host/hos_crash_dialog.ccompiles on Windows (-fsyntax-only -Wall -Wextra, clean). Its Linux behaviour testwasn't run.
Must be tested:
logs/longshot_log.txtmust show%USERPROFILE%where the game folder or saves are under your profile, and no
C:\Users\<you>. Search the file for your user name.@Beanstalk linux, start to the title and quit, then check logs/longshot_log.txt for the user name): the same check, expecting~and<user>and no/home/<you>.@Beanstalk steamdeck): a game folder on the SD card (/run/media/<user>/...) shows/run/media/<user>/in the log.HOS_TEST_CRASH, or kill the game from Task Manager or withkill -SEGV):longshot_log.txtafterwards has no user name, third-party lines included;longshot_log.1.txtthe same.longshot_log.txtstarts fresh, and.1is the previous run'slog, clean.
mainin one scene (the FPS counter's Dev level, orRX_VK_FRAME_TRACE=1), 2 runs each. Expect no difference.make_game.py --nup ... --out ... --log build.loggives abuild.logwith~or%USERPROFILE%for thework and game folders, while the console shows the real paths.
Risks
What could break:
left cut short.
window. The rewrite is locked against the other rewrite, not against readers.
D:\x\al\ybecomesD:\x\<user>\y),as the Bug report already did.
Turning it off: revert the commits.
Compatibility:
~/%USERPROFILE%and<user>instead of the real paths.nx2/pipeline_share.pyreads the log for the GPU and theversion, not paths, so it's unaffected.
Issues
See #1093
See #1264
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.WIP: fix(host): the saved logs leave out the player's user nameto fix(host): the saved logs leave out the player's user name