Retired synchronization objects are lost when multi-wait links outlive their first collection pass #2804

Open

opened 2026-11-12 14:41:14 +00:00 by github-sync · 0 comments

github-sync commented

2026-11-12 14:41:14 +00:00

Owner

@Arkex-ai opened on GitHub · 2026-11-12 14:26 UTC · original

Problem

bury_locked in host/nn_os_sync.c at 51da0464b4cf0c7f56b81ebd6bb309325835e175 removes an aged object from the retirement queue even when its multi-wait links still prevent recycling. When teardown later unlinks that object, the collector has lost its only retirement record. The object and its payload buffers cannot return to the existing spare-object pool.

This is different from the intentional spare-object cache: eligible objects should retain their node for cached pointers but release their payloads and become reusable. Here an ineligible object disappears from the collection path entirely.

Reproduction

A game-free Windows/Clang regression uses the actual object table and LinkMultiWaitHolder, UnlinkMultiWaitHolder and UnlinkAllMultiWaitHolder implementations with a controlled monotonic clock:

Link a holder and multi-wait, finalize one, and let a colliding key replace its table slot.
Advance past the existing 30-second grace. The object must remain pending while links still use it.
Use the actual unlink path, then trigger another collection pass.

Current source loses the retirement record; the formerly linked node is not reusable, and a retired multi-wait retains its link buffer. Retaining the record for retry fixes both paths. The same regression covers ring growth/wrapping with 1,100 deferred objects, 100 retry passes, an eligible object behind a blocked oldest entry, and the HOS_KEEP_DEAD_OBJECTS opt-out. All five candidate cases pass; four base cases fail behaviorally and the opt-out base control passes.

Scope

The retirement-queue fix is published as draft #2858. It retains the existing grace period and eligibility checks, never recycles a linked object, and does not change wait/signal semantics, the per-thread cache policy, or lock order. Private investigation preceded the delivered team-room claim. Current hunks of #2708, #2709, #2765, #1543 and #2430 affect other code in this file, not retirement collection.

No game run has established the frequency or total retained bytes. This is not attributed to multi-GiB runtime pressure; the PR will remain draft until runtime validation.

Priority and affected-player reach are not scored without an in-game record/byte count. The actual table/teardown path is reproduced; in-game scale remains unknown. The worklist row was removed and its unrelated encoding change restored.

Review followup 9bba8f35 also addresses collection racing Unlink: acquire reads of eligibility pair with final release publication after the last object/array access, without reversing the existing lock order. Seven native-thread/table scenarios (40 assertions) pass in three Windows runs and repeated GCC/Clang/sanitizer Linux runs; the previous published head fails both new interleavings. Negative reclamation is quarantined rather than executed as an invalid memory access. No general race-freedom or gameplay claim is made.

> [@Arkex-ai](https://github.com/Arkex-ai) opened on GitHub · 2026-11-12 14:26 UTC · [original](https://github.com/NotRexed/Hookshot/issues/2804) ## Problem `bury_locked` in `host/nn_os_sync.c` at `51da0464b4cf0c7f56b81ebd6bb309325835e175` removes an aged object from the retirement queue even when its multi-wait links still prevent recycling. When teardown later unlinks that object, the collector has lost its only retirement record. The object and its payload buffers cannot return to the existing spare-object pool. This is different from the intentional spare-object cache: eligible objects should retain their node for cached pointers but release their payloads and become reusable. Here an ineligible object disappears from the collection path entirely. ## Reproduction A game-free Windows/Clang regression uses the actual object table and `LinkMultiWaitHolder`, `UnlinkMultiWaitHolder` and `UnlinkAllMultiWaitHolder` implementations with a controlled monotonic clock: - Link a holder and multi-wait, finalize one, and let a colliding key replace its table slot. - Advance past the existing 30-second grace. The object must remain pending while links still use it. - Use the actual unlink path, then trigger another collection pass. Current source loses the retirement record; the formerly linked node is not reusable, and a retired multi-wait retains its link buffer. Retaining the record for retry fixes both paths. The same regression covers ring growth/wrapping with 1,100 deferred objects, 100 retry passes, an eligible object behind a blocked oldest entry, and the `HOS_KEEP_DEAD_OBJECTS` opt-out. All five candidate cases pass; four base cases fail behaviorally and the opt-out base control passes. ## Scope The retirement-queue fix is published as draft #2858. It retains the existing grace period and eligibility checks, never recycles a linked object, and does not change wait/signal semantics, the per-thread cache policy, or lock order. Private investigation preceded the delivered team-room claim. Current hunks of #2708, #2709, #2765, #1543 and #2430 affect other code in this file, not retirement collection. No game run has established the frequency or total retained bytes. This is not attributed to multi-GiB runtime pressure; the PR will remain draft until runtime validation. Priority and affected-player reach are not scored without an in-game record/byte count. The actual table/teardown path is reproduced; in-game scale remains unknown. The worklist row was removed and its unrelated encoding change restored. Review followup 9bba8f35 also addresses collection racing Unlink: acquire reads of eligibility pair with final release publication after the last object/array access, without reversing the existing lock order. Seven native-thread/table scenarios (40 assertions) pass in three Windows runs and repeated GCC/Clang/sanitizer Linux runs; the previous published head fails both new interleavings. Negative reclamation is quarantined rather than executed as an invalid memory access. No general race-freedom or gameplay claim is made.

github-sync referenced this issue from a pull request that will close it,

2026-11-12 17:57:19 +00:00

fix(host): retain linked synchronization objects for later reclamation #2858

github-sync referenced this issue from a commit

2026-11-12 17:57:55 +00:00

fix(host): retain linked synchronization objects for later reclamation

github-sync referenced this issue

2026-11-12 18:00:06 +00:00

fix(host): retain linked synchronization objects for later reclamation #2858

Commenting is not possible because the repository is archived.