test: the fixtures that crash on purpose leave no core dump on Linux #2293

Merged

mooki000 merged 1 commit from test/no-core-dumps into main

2026-11-11 20:31:40 +00:00

mooki000 commented

2026-11-11 13:49:06 +00:00

(Migrated from github.com)

Every run of tests/run_all.py on a Linux desktop leaves three core dumps, and with a crash notifier three "a program
crashed" messages: three fixtures crash on purpose, and the kernel hands each crash to the core-dump handler
(systemd-coredump here).

Part of #2334.

Fixture	Case	Signal
`tests/test_linux_driver_fault_guard.py`'s `guard`	`fault-outside` (a fault outside the guarded call must still kill the process)	SIGSEGV
`tests/test_recorder_sync.py`'s `recorder_sync_test`	broken variant "thread: no backpressure (the ring overwritten)"	SIGSEGV
the same	broken variant "thread: no join at teardown"	SIGABRT

On one machine that ran the checks many times in one night, that made 107 crash notifications.

What changed

Both fixtures call prctl(PR_SET_DUMPABLE, 0) at the start of main, on Linux only (#ifdef __linux__). The
kernel then makes no core dump for the process and starts no handler, whatever ulimit -c and core_pattern say;
the process still ends by its signal, so the checks see exactly what they saw before.
In tests/recorder_sync_test.cpp it sits beside the existing SetErrorMode(SEM_NOGPFAULTERRORBOX | SEM_FAILCRITICALERRORS), which does the same job on Windows (no crash dialog).
Why in the fixtures and not a core-size limit set by the Python tests: for a piped core_pattern the kernel does not
apply RLIMIT_CORE (core(5)), except the special value 1, so a limit of 0 still starts the handler; and the dumpable
flag cannot be set from the parent, since execve resets it.

Tests only; nothing in the game or the builder changes.

How it was tested

Arch Linux, ulimit -c unlimited, core_pattern = |/usr/lib/systemd/systemd-coredump ...; both tests run directly
(not through run_all.py), coredumpctl list before and after:

Before this change, this machine's journal holds one guard (SIGSEGV), one broken12/recorder_sync_test.exe
(SIGSEGV) and one broken13/recorder_sync_test.exe (SIGABRT) core dump per run of the checks (40, 32 and 32 over
two days).
With it: tests/test_linux_driver_fault_guard.py with Clang 23.1.3 and GCC 16.2.1, and tests/test_recorder_sync.py
(the two variants still end with exit codes -11 and -6, and are caught): coredumpctl list shows no new entry. The
kernel still logs its usual segfault at ... line for each fault.
The same results as main: main's tests/test_recorder_sync.py (49f75dc8), run with core dumps blocked from outside
(a core-size limit of 1), prints the same 22 broken-variant lines, exit codes included, and the same PASS line as this
branch's run (the two outputs compared with diff); main's guard test passes the same way.
python -I -B tests/run_all.py on main 49f75dc8 with this change (2026-11-11, load about 70 to 135): 362 passed, 2
failed, 10 skipped; both changed tests pass. The two failures are this machine's and fail on main as well, each with a
pull request of its own: tests/test_wayland_libdecor_hint.py (Clang 23, #2290) and
tests/test_updater_asset_digest.py (only a .NET 10 SDK here).

Not tested

Windows and macOS (nothing changes there: the line is Linux only).
Other core-dump handlers (apport, abrt): the kernel never starts one for a process that is not dumpable, so the same
is expected; not tried.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 [email protected]

Every run of `tests/run_all.py` on a Linux desktop leaves three core dumps, and with a crash notifier three "a program crashed" messages: three fixtures crash on purpose, and the kernel hands each crash to the core-dump handler (systemd-coredump here). Part of #2334. | Fixture | Case | Signal | |---|---|---| | `tests/test_linux_driver_fault_guard.py`'s `guard` | `fault-outside` (a fault outside the guarded call must still kill the process) | SIGSEGV | | `tests/test_recorder_sync.py`'s `recorder_sync_test` | broken variant "thread: no backpressure (the ring overwritten)" | SIGSEGV | | the same | broken variant "thread: no join at teardown" | SIGABRT | On one machine that ran the checks many times in one night, that made 107 crash notifications. ## What changed - Both fixtures call `prctl(PR_SET_DUMPABLE, 0)` at the start of `main`, on Linux only (`#ifdef __linux__`). The kernel then makes no core dump for the process and starts no handler, whatever `ulimit -c` and `core_pattern` say; the process still ends by its signal, so the checks see exactly what they saw before. - In `tests/recorder_sync_test.cpp` it sits beside the existing `SetErrorMode(SEM_NOGPFAULTERRORBOX | SEM_FAILCRITICALERRORS)`, which does the same job on Windows (no crash dialog). - Why in the fixtures and not a core-size limit set by the Python tests: for a piped `core_pattern` the kernel does not apply `RLIMIT_CORE` (core(5)), except the special value 1, so a limit of 0 still starts the handler; and the dumpable flag cannot be set from the parent, since `execve` resets it. Tests only; nothing in the game or the builder changes. ## How it was tested Arch Linux, `ulimit -c` unlimited, `core_pattern` = `|/usr/lib/systemd/systemd-coredump ...`; both tests run directly (not through `run_all.py`), `coredumpctl list` before and after: - Before this change, this machine's journal holds one `guard` (SIGSEGV), one `broken12/recorder_sync_test.exe` (SIGSEGV) and one `broken13/recorder_sync_test.exe` (SIGABRT) core dump per run of the checks (40, 32 and 32 over two days). - With it: `tests/test_linux_driver_fault_guard.py` with Clang 23.1.3 and GCC 16.2.1, and `tests/test_recorder_sync.py` (the two variants still end with exit codes -11 and -6, and are caught): `coredumpctl list` shows no new entry. The kernel still logs its usual `segfault at ...` line for each fault. - The same results as main: main's `tests/test_recorder_sync.py` (49f75dc8), run with core dumps blocked from outside (a core-size limit of 1), prints the same 22 broken-variant lines, exit codes included, and the same PASS line as this branch's run (the two outputs compared with `diff`); main's guard test passes the same way. - `python -I -B tests/run_all.py` on main 49f75dc8 with this change (2026-11-11, load about 70 to 135): 362 passed, 2 failed, 10 skipped; both changed tests pass. The two failures are this machine's and fail on main as well, each with a pull request of its own: `tests/test_wayland_libdecor_hint.py` (Clang 23, #2290) and `tests/test_updater_asset_digest.py` (only a .NET 10 SDK here). ## Not tested - Windows and macOS (nothing changes there: the line is Linux only). - Other core-dump handlers (apport, abrt): the kernel never starts one for a process that is not dumpable, so the same is expected; not tried. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]>

mooki000 commented

2026-11-11 17:08:17 +00:00

(Migrated from github.com)

Rebased onto main b29967bf so the workflows run here: they had waited for an approval on the old head, and main's game-free tests are green again since #2380. The patch is unchanged (git range-diff: the commit is equal), and its test passes on the new base.

Rebased onto main `b29967bf` so the workflows run here: they had waited for an approval on the old head, and main's game-free tests are green again since #2380. The patch is unchanged (`git range-diff`: the commit is equal), and its test passes on the new base.

Joey-Magnus (Migrated from github.com) approved these changes

2026-11-11 20:07:53 +00:00

Joey-Magnus (Migrated from github.com) left a comment

Looks good. prctl(PR_SET_DUMPABLE, 0) at the top of main is the right place for this: as the description says, execve resets the flag, so the parent can't set it, and RLIMIT_CORE doesn't help with a piped core_pattern. Both changes are behind #ifdef __linux__, so Windows is untouched, and the fixtures still end by their signal. I ran tests/test_linux_driver_fault_guard.py and tests/test_recorder_sync.py at this head on Linux and both pass. I also checked that neither fixture reads /proc/self or uses ptrace, which a non-dumpable process would lose access to. No other test in tests/ or nx2/ expects a signal exit, so these really are the only crashing fixtures.

The red CI checks don't come from this PR. On Windows the failure is tests/test_native_astc_texture_count.py (scanf deprecated under -Werror), which main fixed in #2397. The appimage job hit Launchpad connection timeouts. A rebase onto current main should clear the Windows one.

Looks good. `prctl(PR_SET_DUMPABLE, 0)` at the top of `main` is the right place for this: as the description says, `execve` resets the flag, so the parent can't set it, and `RLIMIT_CORE` doesn't help with a piped `core_pattern`. Both changes are behind `#ifdef __linux__`, so Windows is untouched, and the fixtures still end by their signal. I ran `tests/test_linux_driver_fault_guard.py` and `tests/test_recorder_sync.py` at this head on Linux and both pass. I also checked that neither fixture reads `/proc/self` or uses ptrace, which a non-dumpable process would lose access to. No other test in `tests/` or `nx2/` expects a signal exit, so these really are the only crashing fixtures. The red CI checks don't come from this PR. On Windows the failure is `tests/test_native_astc_texture_count.py` (`scanf` deprecated under `-Werror`), which main fixed in #2397. The `appimage` job hit Launchpad connection timeouts. A rebase onto current main should clear the Windows one.

mooki000 commented

2026-11-11 21:26:09 +00:00

(Migrated from github.com)