build(release): adult areas and dungeons added to jarynclouatre-most-of-the-game, 1406 pipelines #2064

Merged

jarynclouatre merged 2 commits from build/pipeline-list-adult into main

2026-11-11 22:01:36 +00:00

jarynclouatre commented

2026-11-11 01:39:11 +00:00

(Migrated from github.com)

More play added to my contributed list from #2029, mostly adult areas and dungeons I had not reached before. The data, plus one test change.

The list file was re-made with the tool from #2040:

python -I nx2/pipeline_states.py add jarynclouatre-most-of-the-game <my lists> --build ... --settings ... --route ...

It keeps only pipelines no other contributed list has. Of 8,021 pipelines in my recordings, 1,406 are new, so my file goes from 7,251 to 8,657 pipelines and the merged list from 12,800 to 14,206. The .md note lists the new areas: Graveyard, Hyrule Field, Kokiri Forest, Lost Woods, Sacred Forest Meadow, Forest Temple, Death Mountain Trail and Crater, Goron City, Fire Temple and its boss, Zora's River, Domain and Fountain, and the Ice Cavern, all as adult.

Recorded on Windows 11, RTX 4070 Ti SUPER (driver 621.14), no mods, 2560x1440 with dynamic resolution, on main with #1961 (some runs also with #2020 and #2050, which change no shaders). As before it holds only hashes, formats and draw state, no game code.

Checked:

nx2/pipeline_states.py check passes.
tests/test_pipeline_states.py, tests/test_ship_state_list.py and tests/test_pipeline_state_sources.py pass.
Based on main at 6cecdd1; it merges cleanly with the tour list (#2065).

test_split_list_lost_nothing pinned each migrated list's exact size, so growing a list failed it. It now takes each list's first n pipelines, since add keeps a list's own pipelines first, and still checks their count, that they don't overlap, and their hash. The same file grown by add passes, and main's form of the test fails on it.

I have not timed the first start's shader preparation with this list yet.

More play added to my contributed list from #2029, mostly adult areas and dungeons I had not reached before. The data, plus one test change. The list file was re-made with the tool from #2040: ``` python -I nx2/pipeline_states.py add jarynclouatre-most-of-the-game <my lists> --build ... --settings ... --route ... ``` It keeps only pipelines no other contributed list has. Of 8,021 pipelines in my recordings, 1,406 are new, so my file goes from 7,251 to 8,657 pipelines and the merged list from 12,800 to 14,206. The `.md` note lists the new areas: Graveyard, Hyrule Field, Kokiri Forest, Lost Woods, Sacred Forest Meadow, Forest Temple, Death Mountain Trail and Crater, Goron City, Fire Temple and its boss, Zora's River, Domain and Fountain, and the Ice Cavern, all as adult. Recorded on Windows 11, RTX 4070 Ti SUPER (driver 621.14), no mods, 2560x1440 with dynamic resolution, on main with #1961 (some runs also with #2020 and #2050, which change no shaders). As before it holds only hashes, formats and draw state, no game code. **Checked:** - `nx2/pipeline_states.py check` passes. - `tests/test_pipeline_states.py`, `tests/test_ship_state_list.py` and `tests/test_pipeline_state_sources.py` pass. - Based on main at 6cecdd1; it merges cleanly with the tour list (#2065). `test_split_list_lost_nothing` pinned each migrated list's exact size, so growing a list failed it. It now takes each list's first n pipelines, since `add` keeps a list's own pipelines first, and still checks their count, that they don't overlap, and their hash. The same file grown by `add` passes, and main's form of the test fails on it. I have not timed the first start's shader preparation with this list yet.

jarynclouatre commented

2026-11-11 01:48:13 +00:00

(Migrated from github.com)

Checked against main with the tour list (#2065): check passes, and the merged list goes from 19,038 to 20,095 pipelines with this, so 1,057 of the new ones are not in the tour either. The branch merges cleanly as it is.

Checked against main with the tour list (#2065): `check` passes, and the merged list goes from 19,038 to 20,095 pipelines with this, so 1,057 of the new ones are not in the tour either. The branch merges cleanly as it is.

hckrman101 (Migrated from github.com) reviewed

2026-11-11 02:19:27 +00:00

hckrman101 (Migrated from github.com) left a comment

Game-files check (CONTRIBUTING.md): clean. The only files are the contributed list and its note. I decompressed the .bin.xz: it is HKSTATE1, record size 528, exactly 8,657 fixed-size records (12-byte header, no trailing bytes), which matches the Pipelines: 8657 in the note. So it holds only the hashes, formats and draw state that the format describes, nothing else, and the change is data only. The .md edit is consistent with the PR description.

One thing to fix: CI is red on tests/test_pipeline_state_sources.py (test_split_list_lost_nothing): AssertionError: 8657 != 7251 : jarynclouatre-most-of-the-game. That test pins MIGRATED["jarynclouatre-most-of-the-game"] = 7251 and a sha256 of the three lists joined (MIGRATED_SHA256), as a one-time check that the split of the old assets/pipeline_states.bin.xz lost nothing. Growing this list breaks both the count and the hash, so the PR needs to touch the test as well (it is not data only, and the description says the tests pass, which was not true of this one). It can't just bump 7251 to 8657, because the hash would still fail. Options: check that the original 7251 records are all still in the file (keep them in the test as a hash of the sorted original records, not of the whole list), or assert >= the original counts. Please don't weaken it beyond that, per AGENTS.md. Note the same will apply to anything else that grows these lists, so a form that tolerates growth is better than another number to bump.

Python source checks (ubuntu-24.04) also fails tests/test_make_game_ngx.py ('SimpleNamespace' object has no attribute 'switching'). That is in a builder DLSS test this PR does not touch, so I'm not blaming the PR for it, but the windows jobs are red too and I did not check whether they are the same test or this one.

On the content: the author already checked the overlap with the tour list in #2065 (1,057 of these are not in the tour either), so the dedupe looks fine. Still open from the description: first-start shader preparation time with this list is not measured. With 14k+ pipelines it is worth a number before this ships, not a blocker for the data itself. The decision to merge stays with @NotRexed.

Game-files check (CONTRIBUTING.md): clean. The only files are the contributed list and its note. I decompressed the `.bin.xz`: it is `HKSTATE1`, record size 528, exactly 8,657 fixed-size records (12-byte header, no trailing bytes), which matches the `Pipelines: 8657` in the note. So it holds only the hashes, formats and draw state that the format describes, nothing else, and the change is data only. The `.md` edit is consistent with the PR description. One thing to fix: CI is red on `tests/test_pipeline_state_sources.py` (`test_split_list_lost_nothing`): `AssertionError: 8657 != 7251 : jarynclouatre-most-of-the-game`. That test pins `MIGRATED["jarynclouatre-most-of-the-game"] = 7251` and a sha256 of the three lists joined (`MIGRATED_SHA256`), as a one-time check that the split of the old `assets/pipeline_states.bin.xz` lost nothing. Growing this list breaks both the count and the hash, so the PR needs to touch the test as well (it is not data only, and the description says the tests pass, which was not true of this one). It can't just bump 7251 to 8657, because the hash would still fail. Options: check that the original 7251 records are all still in the file (keep them in the test as a hash of the sorted original records, not of the whole list), or assert `>=` the original counts. Please don't weaken it beyond that, per AGENTS.md. Note the same will apply to anything else that grows these lists, so a form that tolerates growth is better than another number to bump. `Python source checks (ubuntu-24.04)` also fails `tests/test_make_game_ngx.py` (`'SimpleNamespace' object has no attribute 'switching'`). That is in a builder DLSS test this PR does not touch, so I'm not blaming the PR for it, but the windows jobs are red too and I did not check whether they are the same test or this one. On the content: the author already checked the overlap with the tour list in #2065 (1,057 of these are not in the tour either), so the dedupe looks fine. Still open from the description: first-start shader preparation time with this list is not measured. With 14k+ pipelines it is worth a number before this ships, not a blocker for the data itself. The decision to merge stays with @NotRexed.

jarynclouatre commented

2026-11-11 02:33:55 +00:00

(Migrated from github.com)

Thanks, and sorry, I missed that test. e1720ae takes each migrated list's first n pipelines (add keeps a list's own pipelines first, "after the ones it holds already"), asserts >= on the count, and keeps the overlap and hash checks on those n, so a grown list passes and a changed or reordered original part still fails. Main's form of the test fails on this branch and the new one passes; the description is corrected. I'll post the preparation time once I have it.

Thanks, and sorry, I missed that test. e1720ae takes each migrated list's first n pipelines (`add` keeps a list's own pipelines first, "after the ones it holds already"), asserts `>=` on the count, and keeps the overlap and hash checks on those n, so a grown list passes and a changed or reordered original part still fails. Main's form of the test fails on this branch and the new one passes; the description is corrected. I'll post the preparation time once I have it.

jarynclouatre commented

2026-11-11 13:27:35 +00:00

(Migrated from github.com)

Preparation time, on the same PC (Windows 11, RTX 4070 Ti SUPER, driver 621.14), prepare-only runs on main f915af9's merged list, without and with this file.

With the pipeline cache and the driver's shader cache emptied first, as on a first start: 15,323 graphics pipelines precompiled in 108.1 s, and 15,763 in 110.9 s. So the 440 this adds on top of current main cost about 3 s at a first start. With the caches an earlier session left, 115.1 s and 115.9 s.

Preparation time, on the same PC (Windows 11, RTX 4070 Ti SUPER, driver 621.14), prepare-only runs on main f915af9's merged list, without and with this file. With the pipeline cache and the driver's shader cache emptied first, as on a first start: 15,323 graphics pipelines precompiled in 108.1 s, and 15,763 in 110.9 s. So the 440 this adds on top of current main cost about 3 s at a first start. With the caches an earlier session left, 115.1 s and 115.9 s.

Joey-Magnus (Migrated from github.com) approved these changes

2026-11-11 21:46:41 +00:00

Joey-Magnus (Migrated from github.com) left a comment

I merged this into today's main, which has gained the #2075, #2082, #2429, #2445 and #2133 lists since you rebased. nx2/pipeline_states.py check passes. Your file goes from 7,251 to 8,657 pipelines, and the merged list from 28,902 to 29,688, so 786 of the new ones are still not in any other source. test_pipeline_state_sources, test_pipeline_states and test_ship_state_list pass, and so does the full run_all (376 passed, 0 failed, Linux).

The new test_split_list_lost_nothing is the growth-tolerant form that was asked for. It checks the first n records of each migrated list against the original hash and count, so a grown list passes and a reordered or changed original part still fails. The preparation-time numbers (about 3 s more on a first start) answer the last open point.

The red checks are from the run on e1720ae's base. actionlint's SC2012 and test_make_game_ngx come from main at the time, not from this diff. A re-run on current main should clear them.

I merged this into today's main, which has gained the #2075, #2082, #2429, #2445 and #2133 lists since you rebased. `nx2/pipeline_states.py check` passes. Your file goes from 7,251 to 8,657 pipelines, and the merged list from 28,902 to 29,688, so 786 of the new ones are still not in any other source. `test_pipeline_state_sources`, `test_pipeline_states` and `test_ship_state_list` pass, and so does the full run_all (376 passed, 0 failed, Linux). The new `test_split_list_lost_nothing` is the growth-tolerant form that was asked for. It checks the first n records of each migrated list against the original hash and count, so a grown list passes and a reordered or changed original part still fails. The preparation-time numbers (about 3 s more on a first start) answer the last open point. The red checks are from the run on e1720ae's base. actionlint's SC2012 and `test_make_game_ngx` come from main at the time, not from this diff. A re-run on current main should clear them.