Skip to content

Core: add list/dict merging feature to triggers#2793

Merged
black-sliver merged 24 commits intoArchipelagoMW:mainfrom
Silvris:yaml_merge
Mar 12, 2024
Merged

Core: add list/dict merging feature to triggers#2793
black-sliver merged 24 commits intoArchipelagoMW:mainfrom
Silvris:yaml_merge

Conversation

@Silvris
Copy link
Collaborator

@Silvris Silvris commented Feb 4, 2024

What is this fixing or adding?

A common pain point with triggers is the inability to append to a list or dict of already defined values with new ones based on the trigger result, instead needing to redefine all items in the new dict/list. This adds that functionality in the form of appending + to the start of the option's name. This indicates that the list/dict should be merged into the main yaml's option rather than replace it.

How was this tested?

Manually, by applying the tag to various scalars/lists/dicts. Ran unittests and a all-games gen to confirm no regressions.

If this makes graphical changes, please attach screenshots.

Use of the new tag

Super Metroid:
  start_location: 
    landing_site: 50
    aqueduct: 50
  start_hints:
    - Morph Ball
triggers:
  - option_category: Super Metroid
    option_name: start_location
    option_result: aqueduct
    options:
      Super Metroid:
        +start_hints:
          - Gravity Suit

@ScipioWright ScipioWright added is: enhancement Issues requesting new features or pull requests implementing new features. affects: core Issues/PRs that touch core and may need additional validation. labels Feb 5, 2024
@PoryGone PoryGone added the waiting-on: peer-review Issue/PR has not been reviewed by enough people yet. label Feb 10, 2024
Copy link
Collaborator

@agilbert1412 agilbert1412 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big fan of the @merge syntax, as it feels out of place in the yaml. Are there precedents for a similar "@x" syntax being used as a modifier in a yaml file?

@Silvris
Copy link
Collaborator Author

Silvris commented Feb 13, 2024

I'm not a big fan of the @merge syntax, as it feels out of place in the yaml. Are there precedents for a similar "@x" syntax being used as a modifier in a yaml file?

There's no real precedent for this type of feature. I'm open to other ideas for syntax, but the yaml spec doesn't give us too many options. Utilizing the tag system introduces too many caveats than I'm willing to add, and yaml doesn't allow for any additional info to be placed onto an object.

@ScipioWright ScipioWright added waiting-on: core-review Issue/PR has been peer-reviewed and is ready to be merged or needs input from a core maintainer. and removed waiting-on: peer-review Issue/PR has not been reviewed by enough people yet. labels Feb 14, 2024
@TheT113
Copy link
Contributor

TheT113 commented Mar 2, 2024

I feel like a more syntax appropriate option would be just add another option, like "merge: yes".

Anyway, hugely in favor of this being added regardless of the syntax.

@Berserker66
Copy link
Member

We could use "+start_hints" or "|start_hints" similar to python's list and set combine, it's not yaml, but at least it's the next closest thing we have. This would also be able to be processed faster, since just checking the first letter is less instructions.

@Silvris
Copy link
Collaborator Author

Silvris commented Mar 4, 2024

Went ahead and swapped to the + prefix, since I think that's a pretty clear indicator in of itself for what it should do (adds two lists/dicts together).

Co-authored-by: Fabian Dill <[email protected]>
Copy link
Member

@black-sliver black-sliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

| has special meaning in yaml, so I think + is the better choice. It is still somewhat odd-looking, but I think it's the best suggestion so far, because it allows more flexibility than merge: <bool>.
I think I found a bug, see below, and also I have a suggestion to make it what I think is cleaner. You can take that or leave it.

@black-sliver
Copy link
Member

I'm wondering if it's somewhat easy to add unit tests to this. I'm not sure how valuable it is, but def update_weights() could be tested in a vacuum. Do you think that would be possible?

Copy link
Member

@black-sliver black-sliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure where to put the test exactly, but test/programs/ isn't really correct. The idea of the tests in that folder is to test EXEs (or the scripts that get built into EXEs).

Maybe test/general/test_triggers.py ? Or test_player_options.py? And create a new class that inherits from unittest.TestCase.

Copy link
Member

@black-sliver black-sliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't want to be nit-picky, but rather show what can be done in case either you or someone else takes this as reference for future tests.

Copy link
Member

@black-sliver black-sliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it does what it should.

Thanks!

@black-sliver black-sliver merged commit fb9ef19 into ArchipelagoMW:main Mar 12, 2024
EmilyV99 pushed a commit to EmilyV99/Archipelago that referenced this pull request Apr 15, 2024
* proof of concept

* add dict support, block top/game level merge

* prevent key error when option being merged is new

* update triggers guide

* Add documentation about add/remove/replace

* move to trailing name instead of proper tag

* update docs

* confirm types

* Update Utils.py

* Update Generate.py

* pep8

* move to + syntax

* forgot to support sets

* specify received type of type error

* Update Generate.py

Co-authored-by: Fabian Dill <[email protected]>

* Apply suggestion from review

* add test for update weights

* move test to new test case

* Apply suggestions from code review

Co-authored-by: black-sliver <[email protected]>

---------

Co-authored-by: Fabian Dill <[email protected]>
Co-authored-by: black-sliver <[email protected]>
EmilyV99 pushed a commit to EmilyV99/Archipelago that referenced this pull request Apr 15, 2024
* proof of concept

* add dict support, block top/game level merge

* prevent key error when option being merged is new

* update triggers guide

* Add documentation about add/remove/replace

* move to trailing name instead of proper tag

* update docs

* confirm types

* Update Utils.py

* Update Generate.py

* pep8

* move to + syntax

* forgot to support sets

* specify received type of type error

* Update Generate.py

Co-authored-by: Fabian Dill <[email protected]>

* Apply suggestion from review

* add test for update weights

* move test to new test case

* Apply suggestions from code review

Co-authored-by: black-sliver <[email protected]>

---------

Co-authored-by: Fabian Dill <[email protected]>
Co-authored-by: black-sliver <[email protected]>
FlySniper added a commit to FlySniper/Archipelago that referenced this pull request Aug 3, 2024
…pelagoMW#2922)

* remove component checking

* fix missing deathlink messages

* move reads under deathlink check
Core: Fix OptionList and OptionSet to allow Iterable of Iterable (ArchipelagoMW#2911)

* fix, maybe

* typegard for iterable of any

* wow I'm so tired I just changed the method name without changing what it actually does...

* also exclude bytes in is_iterable_but_str

* apply pr comments

* Update Utils.py

Co-authored-by: Doug Hoskisson <[email protected]>

* Revert "also exclude bytes in is_iterable_but_str"

This reverts commit cf087d2.

---------

Co-authored-by: Doug Hoskisson <[email protected]>
Docs: Added snes9x-nwa as recommended emulator to the setup guides for SNES games (ArchipelagoMW#1778)

* Added snes9x-nwa as recommended emulator to the setup guides

* Removed snes9x-nwa from the setup guides of DKC3 and SMW

* Update worlds/alttp/docs/multiworld_en.md

Co-authored-by: Aaron Wagener <[email protected]>

* Removed duplicate text
Minor grammar and spelling fixes

* Unified required software for SM, SMZ3 and SoE with ALTTP

* Added instructions for usage of BSNES-Plus for ALTTP, SM and SMZ3

---------

Co-authored-by: Aaron Wagener <[email protected]>
KH2: Update all instances of multiworld.option_name to option.option_name (ArchipelagoMW#2634)

* update the multiworld to options

* Update worlds/kh2/Rules.py

Co-authored-by: Exempt-Medic <[email protected]>

* does this work

* namine sketches

* wrong branch :)

---------

Co-authored-by: Exempt-Medic <[email protected]>
The Messenger: fix items accessibility reachability bug due to new rules (ArchipelagoMW#2937)

CI: Don't auto-remove content based labels (ArchipelagoMW#2941)

Docs: improve AutoWorld method docstrings (ArchipelagoMW#2509)

* clarify some autoworld docstrings

* revert accidental change
The Witness: Don't unnecessarily break people's 0.4.4 yamls (ArchipelagoMW#2940)

kvui: allow sorting hints in the hint tab (ArchipelagoMW#2684)

MultiServer: send new read_hints datastore values on change (ArchipelagoMW#2558)

The Witness: Obelisk Keys (ArchipelagoMW#2805)

Core: String comparison with FreeText class (ArchipelagoMW#2942)

CommonClient: use rich text for /received (ArchipelagoMW#2715)

CommonClient: Fix item link group name when member slot name contains brackets (ArchipelagoMW#2794)

SMW: v2.0 Content Update (ArchipelagoMW#2762)

Changelog:

Features:
- New optional Location Checks
  - 3-Up Moons
  - Hidden 1-Ups
  - Bonus Blocks
  - Blocksanity
    - All blocks that contain coins or items are included, with the exception of:
      - Blocks in Top Secret Area & Front Door/Bowser Castle
      - Blocks that are unreachable without glitches/unreasonable movement
- New Items
  - Special Zone Clear
  - New Filler Items
    - 1 Coin
    - 5 Coins
    - 10 Coins
    - 50 Coins
  - New Trap Items
    - Reverse Trap
    - Thwimp Trap
- SFX Shuffle
- Palette Shuffle Overhaul
  - New Curated Palette can now be used for the Overworld and Level Palette Shuffle options
  - Foreground and Background Shuffle options have been merged into a single setting
- Max possible Yoshi Egg value is 255
  - UI in-game is updated to handle 3-digits
  - New `Display Received Item Popups` option: `progression_minus_yoshi_eggs`

Quality of Life:
- In-Game Indicators are now displayed on the map screen for location checks and received items
- In-level sprites are displayed upon receiving certain items
- The Camera Scroll unlocking is now only enabled on levels where it needs to be
- SMW can now handle receiving more than 255 items
- Significant World Code cleanup
  - New Options API
  - Removal of `world: MultiWorld` across the world
- The PopTracker pack now has tabs for every level/sublevel, and can automatically swap tabs while playing if connected to the server

Bug Fixes:
- Several logic tweaks/fixes

"Major credit to @TheLX5 for being the driving force for almost all of this update. We've been collaborating on design and polish of the features for the last few months, but all of the heavy lifting was all @TheLX5."
Core: typing for `Option.default` and a few other ClassVars (ArchipelagoMW#2899)

* Core: typing for `Option.default` and a few other `Option` class variables

This is a replacement for ArchipelagoMW#2173

You can read discussion there for issues we found for why we can't have more specific typing on `default`

instead of setting a default in `Option` (where we don't know the type), we check in the metaclass to make sure they have a default.

* NumericOption doesn't need the type annotation that brings out the mypy bug

* SoE default ClassVar
Core: add list/dict merging feature to triggers (ArchipelagoMW#2793)

* proof of concept

* add dict support, block top/game level merge

* prevent key error when option being merged is new

* update triggers guide

* Add documentation about add/remove/replace

* move to trailing name instead of proper tag

* update docs

* confirm types

* Update Utils.py

* Update Generate.py

* pep8

* move to + syntax

* forgot to support sets

* specify received type of type error

* Update Generate.py

Co-authored-by: Fabian Dill <[email protected]>

* Apply suggestion from review

* add test for update weights

* move test to new test case

* Apply suggestions from code review

Co-authored-by: black-sliver <[email protected]>

---------

Co-authored-by: Fabian Dill <[email protected]>
Co-authored-by: black-sliver <[email protected]>
CI: build: create setup (ArchipelagoMW#2936)

* CI: build: create setup

also add /DNO_SIGNTOOL to inno_setup.iss

* CI: build: trigger when changing setup-related files
Shivers: Renaming for clarity and consistency (ArchipelagoMW#2869)

* Moves plaque location to front for better tracker referencing.

* Tiki should be Shaman.

* Hanging should be Gallows.

* Merrick spelling.

* Clarity change.
FFMQ: Update Map Shuffle Seed description (ArchipelagoMW#2658)

* Update Map Shuffle Seed description

* Update worlds/ffmq/Options.py

Co-authored-by: Exempt-Medic <[email protected]>

---------

Co-authored-by: Exempt-Medic <[email protected]>
Core: fix incorrect ordering on the always_allow static method (ArchipelagoMW#2938)

CI: update actions (ArchipelagoMW#2943)

HK: Removes Vanilla Items from ItemPool and Uses Grimmchild1 when relevant (ArchipelagoMW#2898)

KDL3: Ensure all abilities accessible on non-minimal (ArchipelagoMW#2929)

Pokemon Emerald: v2 Update (ArchipelagoMW#2918)

Core: add layer for patches that don't use `Patch.py` (ArchipelagoMW#2889)

* Core: add layer for patches that don't use `Patch.py`

* bump container version

* APAutoPatchInterface name

* mystic quest change

* OoT and Adventure changes

* missed name in docstring

* container version compatibility
Lingo: Pre-compile datafile to improve loading time (ArchipelagoMW#2829)

CommonClient: Don't retry connection when connection details are invalid (ArchipelagoMW#2831)

Stardew Valley: 5.x.x - The Allsanity Update (ArchipelagoMW#2764)

Major Content update for Stardew Valley, including the following features

- Major performance improvements all across the Stardew Valley apworld, including a significant reduction in the test time
- Randomized Farm Type
- Bundles rework (Remixed Bundles and Missing Bundle!)
- New Settings:
  * Shipsanity - Shipping individual items
  * Monstersanity - Slaying monsters
  * Cooksanity - Cooking individual recipes
  * Chefsanity - Learning individual recipes
  * Craftsanity - Crafting individual items
- New Goals:
  * Protector of the Valley - Complete every monster slayer goal
  * Full Shipment - Ship every item
  * Craftmaster - Craft every item
  * Gourmet Chef - Cook every recipe
  * Legend - Earn 10 000 000g
  * Mystery of the Stardrops - Find every stardrop (Maguffin Hunt)
  * Allsanity - Complete every check in your slot
- Building Shuffle: Cheaper options
- Tool Shuffle: Cheaper options
- Money rework
- New traps
- New isolated checks and items, including the farm cave, the movie theater, etc
- Mod Support: SVE [Albrekka]
- Mod Support: Distant Lands [Albrekka]
- Mod Support: Hat Mouse Lacey [Albrekka]
- Mod Support: Boarding House [Albrekka]

Co-authored-by: Witchybun <[email protected]>
Co-authored-by: Witchybun <[email protected]>
Co-authored-by: Jouramie <[email protected]>
Co-authored-by: Alchav <[email protected]>
Launcher: make scrollbar more prominent (ArchipelagoMW#2955)

TUNIC: Updated display name for a few options (ArchipelagoMW#2953)

SMW: Add CHANGELOG.md (ArchipelagoMW#2947)

Celeste 64: Add CHANGELOG.md (ArchipelagoMW#2948)

DKC3: Add CHANGELOG.md (ArchipelagoMW#2946)

Core: increment version (ArchipelagoMW#2958)

SA2B: Add CHANGELOG.md (ArchipelagoMW#2945)

The Witness: Add newly submitted junk hints (ArchipelagoMW#2949)

OoT: Entrance Spoiler Fixes (ArchipelagoMW#2500)

Stardew Valley: Added a Great Combat requirement to an entrance that could block its own key (ArchipelagoMW#2959)

SC2: Multi-campaign (ArchipelagoMW#2954)

Adds HotS, LotV and NCO campaigns to SC2 game.
The world's name has changed to reflect that (it's not only Wings of Liberty now)
The client was patched in a way that can still join to games generated prior this change
---------

Co-authored-by: Magnemania <[email protected]>
Co-authored-by: EnvyDragon <[email protected]>
Co-authored-by: Matthew <[email protected]>
Co-authored-by: hopop201 <[email protected]>
Co-authored-by: Salzkorn <[email protected]>
Co-authored-by: genderdruid <[email protected]>
Co-authored-by: MadiMadsen <[email protected]>
Co-authored-by: neocerber <[email protected]>
Co-authored-by: Exempt-Medic <[email protected]>
Co-authored-by: Fabian Dill <[email protected]>
Zork Grand Inquisitor: Implement New Game (ArchipelagoMW#2539)

Adds Archipelago support for Zork Grand Inquisitor, the 1997 point-and-click PC adventure game.

The client (based on `CommonClient`), on top of its regular Archipelago duties, fully handles the randomization of the game and the monitoring / modification of the game state. No game modding needed at all; the player is ready to play an Archipelago seed if they can play the vanilla game through ScummVM.

The "reverse engineering" (there's likely a better term for this...) of the game is my own original work and I included an MIT license at the root of my world directory.

A PopTracker pack was also created to help people learn the game: https://github.com/SerpentAI/ZorkGrandInquisitorAPTracker
TUNIC: Implement support for connection plando (ArchipelagoMW#2864)

The Witness: Add junk hint for Zork: Grand Inquisitor (ArchipelagoMW#2961)

SMW: Increment Required Client Version (ArchipelagoMW#2962)

Core: implement APProcedurePatch and APTokenMixin (ArchipelagoMW#2536)

* initial work on procedure patch

* more flexibility

load default procedure for version 5 patches
add args for procedure
add default extension for tokens and bsdiff
allow specifying additional required extensions for generation

* pushing current changes to go fix tloz bug

* move tokens into a separate inheritable class

* forgot the commit to remove token from ProcedurePatch

* further cleaning from bad commit

* start on docstrings

* further work on docstrings and typing

* improve docstrings

* fix incorrect docstring

* cleanup

* clean defaults and docstring

* define interface that has only the bare minimum required
for `Patch.create_rom_file`

* change to dictionary.get

* remove unnecessary if statement

* update to explicitly check for procedure, restore compatible version and manual override

* Update Files.py

* remove struct uses

* ensure returning bytes, add token type checking

* Apply suggestions from code review

Co-authored-by: Doug Hoskisson <[email protected]>

* pep8

---------

Co-authored-by: beauxq <[email protected]>
Co-authored-by: Doug Hoskisson <[email protected]>
Pokemon Emerald: Bump required client version (ArchipelagoMW#2963)
@Silvris Silvris deleted the yaml_merge branch May 29, 2025 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

affects: core Issues/PRs that touch core and may need additional validation. is: enhancement Issues requesting new features or pull requests implementing new features. waiting-on: core-review Issue/PR has been peer-reviewed and is ready to be merged or needs input from a core maintainer.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants