Core: Option.default typing and definition checking#2173
Core: Option.default typing and definition checking#2173beauxq wants to merge 10 commits intoArchipelagoMW:mainfrom
Option.default typing and definition checking#2173Conversation
|
Getting a lot of mypy errors with this. The reason seems to be that, e.g., I think a solution to this problem might be to specify explicit typing for
(I assume a bunch of subclasses in the worlds would have to be changed, too, to avoid the same error, but I did only check the classes defined in core.) |
|
Can I see what errors you're getting? and how? |
|
The errors happen when you subclass the options, not in the core Options.py |
|
Well, most of the errors are in world-specific subclasses, but some are actually also in core, e.g.: |
|
I can't reproduce that. Do you have some hidden mypy settings? or what version of mypy? |
|
nevermind, I found how to reproduce it |
As long as they inherit from one of these with a concrete type, they shouldn't need it any further down the subclass chain. |
|
Now anything that inherits directly from Anything further down the inheritance chain will take that concrete annotation. |
I don't think this is true. As soon a class anywhere in the chain sets an explicit value for default without also specifying an appropriate type hint, all lower subclasses will have mypy errors if they want to set a different For example, The Messenger's |
|
Ok, I think this is a bug in mypy that has nothing to do with the |
|
Yeah, here's an already existing issue: python/mypy#10506 |
|
is it worth merging this while there is still an upstream bug surrounding this? |
|
We definitely need to make FreeText a more proper class at some point. Not sure if this is the right place for it, since I think no matter how you approach it with the goal of this PR it'll still be a bit of a mess. |
|
I found a few more issues when applying el-u's suggestion: pyright correctly tells us that it's not type-safe to narrow the type of
|
|
Sorry, I didn't find the time to look at this again after your last changes. We didn't break this in the meantime and should be able to merge, right? |
I just looked over it again, and can't think of any new problems from it. |
(and `default` isn't needed in `FlagsProtocol`)
|
I found something in SoE affected by this: beauxq#2 |
|
I think |
|
Merged the other PR. So we get proper mypy output when updating this branch. |
soe: change `FlagProtocol` to match `default` typing
|
Hm, I still see a couple of errors in Options.py and soe/options.py that we can maybe partially fix, I'll give it a look when I have the time - maybe tonight. You said something about ReadOnly not being available to use yet. When does this become available and what would it fix? I'd maybe leave a comment since we will definitely drop 3.8 at latest when it becomes EoL in a year and at that point there is also no reason for 3.9 anymore. |
PEP 705 doesn't give us what we need to fix this. It has this line:
So there isn't even a PEP written yet to give us what might fix this.
I'm not confident that it would fix everything here. This issue python/typing#1486 is separate from the mypy bug. And that's the issue that If a protocol has a property (like I just thought there's a chance that |
I'm not sure what you're thinking of. I'm aware there are lots more issues in There are different levels of complexity with other things that could be fixed. |
|
Mainly the prog balancing override |
|
Oh I see. That's the same problem. We'd have to put What do we do? Wait for mypy to fix it? Is pyright any better? (Never used it, but I guess we could switch if it's as good or better) |
|
I feel like a protocol with |
|
I think pyright is generally a little better. I use both mypy and pyright because there are a few things pyright won't detect that mypy will. |
|
Waiting on Author for merge conflicts. |
|
The lack of a ReadOnly specifier to avoid invariance, and other issues, I think make this not worth it. |
…ables This is a replacement for ArchipelagoMW#2173 You can read discussion there for issues we found for why we can't have more specific typing on `default` instead of setting a default in `Option` (where we don't know the type), we check in the metaclass to make sure they have a default.
* Core: typing for `Option.default` and a few other `Option` class variables This is a replacement for #2173 You can read discussion there for issues we found for why we can't have more specific typing on `default` instead of setting a default in `Option` (where we don't know the type), we check in the metaclass to make sure they have a default. * NumericOption doesn't need the type annotation that brings out the mypy bug * SoE default ClassVar
…agoMW#2899) * Core: typing for `Option.default` and a few other `Option` class variables This is a replacement for ArchipelagoMW#2173 You can read discussion there for issues we found for why we can't have more specific typing on `default` instead of setting a default in `Option` (where we don't know the type), we check in the metaclass to make sure they have a default. * NumericOption doesn't need the type annotation that brings out the mypy bug * SoE default ClassVar
…agoMW#2899) * Core: typing for `Option.default` and a few other `Option` class variables This is a replacement for ArchipelagoMW#2173 You can read discussion there for issues we found for why we can't have more specific typing on `default` instead of setting a default in `Option` (where we don't know the type), we check in the metaclass to make sure they have a default. * NumericOption doesn't need the type annotation that brings out the mypy bug * SoE default ClassVar
…pelagoMW#2922) * remove component checking * fix missing deathlink messages * move reads under deathlink check Core: Fix OptionList and OptionSet to allow Iterable of Iterable (ArchipelagoMW#2911) * fix, maybe * typegard for iterable of any * wow I'm so tired I just changed the method name without changing what it actually does... * also exclude bytes in is_iterable_but_str * apply pr comments * Update Utils.py Co-authored-by: Doug Hoskisson <[email protected]> * Revert "also exclude bytes in is_iterable_but_str" This reverts commit cf087d2. --------- Co-authored-by: Doug Hoskisson <[email protected]> Docs: Added snes9x-nwa as recommended emulator to the setup guides for SNES games (ArchipelagoMW#1778) * Added snes9x-nwa as recommended emulator to the setup guides * Removed snes9x-nwa from the setup guides of DKC3 and SMW * Update worlds/alttp/docs/multiworld_en.md Co-authored-by: Aaron Wagener <[email protected]> * Removed duplicate text Minor grammar and spelling fixes * Unified required software for SM, SMZ3 and SoE with ALTTP * Added instructions for usage of BSNES-Plus for ALTTP, SM and SMZ3 --------- Co-authored-by: Aaron Wagener <[email protected]> KH2: Update all instances of multiworld.option_name to option.option_name (ArchipelagoMW#2634) * update the multiworld to options * Update worlds/kh2/Rules.py Co-authored-by: Exempt-Medic <[email protected]> * does this work * namine sketches * wrong branch :) --------- Co-authored-by: Exempt-Medic <[email protected]> The Messenger: fix items accessibility reachability bug due to new rules (ArchipelagoMW#2937) CI: Don't auto-remove content based labels (ArchipelagoMW#2941) Docs: improve AutoWorld method docstrings (ArchipelagoMW#2509) * clarify some autoworld docstrings * revert accidental change The Witness: Don't unnecessarily break people's 0.4.4 yamls (ArchipelagoMW#2940) kvui: allow sorting hints in the hint tab (ArchipelagoMW#2684) MultiServer: send new read_hints datastore values on change (ArchipelagoMW#2558) The Witness: Obelisk Keys (ArchipelagoMW#2805) Core: String comparison with FreeText class (ArchipelagoMW#2942) CommonClient: use rich text for /received (ArchipelagoMW#2715) CommonClient: Fix item link group name when member slot name contains brackets (ArchipelagoMW#2794) SMW: v2.0 Content Update (ArchipelagoMW#2762) Changelog: Features: - New optional Location Checks - 3-Up Moons - Hidden 1-Ups - Bonus Blocks - Blocksanity - All blocks that contain coins or items are included, with the exception of: - Blocks in Top Secret Area & Front Door/Bowser Castle - Blocks that are unreachable without glitches/unreasonable movement - New Items - Special Zone Clear - New Filler Items - 1 Coin - 5 Coins - 10 Coins - 50 Coins - New Trap Items - Reverse Trap - Thwimp Trap - SFX Shuffle - Palette Shuffle Overhaul - New Curated Palette can now be used for the Overworld and Level Palette Shuffle options - Foreground and Background Shuffle options have been merged into a single setting - Max possible Yoshi Egg value is 255 - UI in-game is updated to handle 3-digits - New `Display Received Item Popups` option: `progression_minus_yoshi_eggs` Quality of Life: - In-Game Indicators are now displayed on the map screen for location checks and received items - In-level sprites are displayed upon receiving certain items - The Camera Scroll unlocking is now only enabled on levels where it needs to be - SMW can now handle receiving more than 255 items - Significant World Code cleanup - New Options API - Removal of `world: MultiWorld` across the world - The PopTracker pack now has tabs for every level/sublevel, and can automatically swap tabs while playing if connected to the server Bug Fixes: - Several logic tweaks/fixes "Major credit to @TheLX5 for being the driving force for almost all of this update. We've been collaborating on design and polish of the features for the last few months, but all of the heavy lifting was all @TheLX5." Core: typing for `Option.default` and a few other ClassVars (ArchipelagoMW#2899) * Core: typing for `Option.default` and a few other `Option` class variables This is a replacement for ArchipelagoMW#2173 You can read discussion there for issues we found for why we can't have more specific typing on `default` instead of setting a default in `Option` (where we don't know the type), we check in the metaclass to make sure they have a default. * NumericOption doesn't need the type annotation that brings out the mypy bug * SoE default ClassVar Core: add list/dict merging feature to triggers (ArchipelagoMW#2793) * proof of concept * add dict support, block top/game level merge * prevent key error when option being merged is new * update triggers guide * Add documentation about add/remove/replace * move to trailing name instead of proper tag * update docs * confirm types * Update Utils.py * Update Generate.py * pep8 * move to + syntax * forgot to support sets * specify received type of type error * Update Generate.py Co-authored-by: Fabian Dill <[email protected]> * Apply suggestion from review * add test for update weights * move test to new test case * Apply suggestions from code review Co-authored-by: black-sliver <[email protected]> --------- Co-authored-by: Fabian Dill <[email protected]> Co-authored-by: black-sliver <[email protected]> CI: build: create setup (ArchipelagoMW#2936) * CI: build: create setup also add /DNO_SIGNTOOL to inno_setup.iss * CI: build: trigger when changing setup-related files Shivers: Renaming for clarity and consistency (ArchipelagoMW#2869) * Moves plaque location to front for better tracker referencing. * Tiki should be Shaman. * Hanging should be Gallows. * Merrick spelling. * Clarity change. FFMQ: Update Map Shuffle Seed description (ArchipelagoMW#2658) * Update Map Shuffle Seed description * Update worlds/ffmq/Options.py Co-authored-by: Exempt-Medic <[email protected]> --------- Co-authored-by: Exempt-Medic <[email protected]> Core: fix incorrect ordering on the always_allow static method (ArchipelagoMW#2938) CI: update actions (ArchipelagoMW#2943) HK: Removes Vanilla Items from ItemPool and Uses Grimmchild1 when relevant (ArchipelagoMW#2898) KDL3: Ensure all abilities accessible on non-minimal (ArchipelagoMW#2929) Pokemon Emerald: v2 Update (ArchipelagoMW#2918) Core: add layer for patches that don't use `Patch.py` (ArchipelagoMW#2889) * Core: add layer for patches that don't use `Patch.py` * bump container version * APAutoPatchInterface name * mystic quest change * OoT and Adventure changes * missed name in docstring * container version compatibility Lingo: Pre-compile datafile to improve loading time (ArchipelagoMW#2829) CommonClient: Don't retry connection when connection details are invalid (ArchipelagoMW#2831) Stardew Valley: 5.x.x - The Allsanity Update (ArchipelagoMW#2764) Major Content update for Stardew Valley, including the following features - Major performance improvements all across the Stardew Valley apworld, including a significant reduction in the test time - Randomized Farm Type - Bundles rework (Remixed Bundles and Missing Bundle!) - New Settings: * Shipsanity - Shipping individual items * Monstersanity - Slaying monsters * Cooksanity - Cooking individual recipes * Chefsanity - Learning individual recipes * Craftsanity - Crafting individual items - New Goals: * Protector of the Valley - Complete every monster slayer goal * Full Shipment - Ship every item * Craftmaster - Craft every item * Gourmet Chef - Cook every recipe * Legend - Earn 10 000 000g * Mystery of the Stardrops - Find every stardrop (Maguffin Hunt) * Allsanity - Complete every check in your slot - Building Shuffle: Cheaper options - Tool Shuffle: Cheaper options - Money rework - New traps - New isolated checks and items, including the farm cave, the movie theater, etc - Mod Support: SVE [Albrekka] - Mod Support: Distant Lands [Albrekka] - Mod Support: Hat Mouse Lacey [Albrekka] - Mod Support: Boarding House [Albrekka] Co-authored-by: Witchybun <[email protected]> Co-authored-by: Witchybun <[email protected]> Co-authored-by: Jouramie <[email protected]> Co-authored-by: Alchav <[email protected]> Launcher: make scrollbar more prominent (ArchipelagoMW#2955) TUNIC: Updated display name for a few options (ArchipelagoMW#2953) SMW: Add CHANGELOG.md (ArchipelagoMW#2947) Celeste 64: Add CHANGELOG.md (ArchipelagoMW#2948) DKC3: Add CHANGELOG.md (ArchipelagoMW#2946) Core: increment version (ArchipelagoMW#2958) SA2B: Add CHANGELOG.md (ArchipelagoMW#2945) The Witness: Add newly submitted junk hints (ArchipelagoMW#2949) OoT: Entrance Spoiler Fixes (ArchipelagoMW#2500) Stardew Valley: Added a Great Combat requirement to an entrance that could block its own key (ArchipelagoMW#2959) SC2: Multi-campaign (ArchipelagoMW#2954) Adds HotS, LotV and NCO campaigns to SC2 game. The world's name has changed to reflect that (it's not only Wings of Liberty now) The client was patched in a way that can still join to games generated prior this change --------- Co-authored-by: Magnemania <[email protected]> Co-authored-by: EnvyDragon <[email protected]> Co-authored-by: Matthew <[email protected]> Co-authored-by: hopop201 <[email protected]> Co-authored-by: Salzkorn <[email protected]> Co-authored-by: genderdruid <[email protected]> Co-authored-by: MadiMadsen <[email protected]> Co-authored-by: neocerber <[email protected]> Co-authored-by: Exempt-Medic <[email protected]> Co-authored-by: Fabian Dill <[email protected]> Zork Grand Inquisitor: Implement New Game (ArchipelagoMW#2539) Adds Archipelago support for Zork Grand Inquisitor, the 1997 point-and-click PC adventure game. The client (based on `CommonClient`), on top of its regular Archipelago duties, fully handles the randomization of the game and the monitoring / modification of the game state. No game modding needed at all; the player is ready to play an Archipelago seed if they can play the vanilla game through ScummVM. The "reverse engineering" (there's likely a better term for this...) of the game is my own original work and I included an MIT license at the root of my world directory. A PopTracker pack was also created to help people learn the game: https://github.com/SerpentAI/ZorkGrandInquisitorAPTracker TUNIC: Implement support for connection plando (ArchipelagoMW#2864) The Witness: Add junk hint for Zork: Grand Inquisitor (ArchipelagoMW#2961) SMW: Increment Required Client Version (ArchipelagoMW#2962) Core: implement APProcedurePatch and APTokenMixin (ArchipelagoMW#2536) * initial work on procedure patch * more flexibility load default procedure for version 5 patches add args for procedure add default extension for tokens and bsdiff allow specifying additional required extensions for generation * pushing current changes to go fix tloz bug * move tokens into a separate inheritable class * forgot the commit to remove token from ProcedurePatch * further cleaning from bad commit * start on docstrings * further work on docstrings and typing * improve docstrings * fix incorrect docstring * cleanup * clean defaults and docstring * define interface that has only the bare minimum required for `Patch.create_rom_file` * change to dictionary.get * remove unnecessary if statement * update to explicitly check for procedure, restore compatible version and manual override * Update Files.py * remove struct uses * ensure returning bytes, add token type checking * Apply suggestions from code review Co-authored-by: Doug Hoskisson <[email protected]> * pep8 --------- Co-authored-by: beauxq <[email protected]> Co-authored-by: Doug Hoskisson <[email protected]> Pokemon Emerald: Bump required client version (ArchipelagoMW#2963)
What is this fixing or adding?
The typing of
Option.defaulthas been making trouble for type-checkers,and it doesn't make sense to set a default (
0) that is the wrong type for some subclasses (FreeText)I consulted the python typing discussion board about this
python/typing#1460
and it seems there is no perfect solution.
This is the best solution offered there.
Also, instead of setting a default in
Option(where we don't know the type), we check in the metaclass to make sure they have a default.How was this tested?
unit tests, and breaking some options to make sure the assertion in the metaclass would fail unit tests if an option doesn't set a default