Networks statistics (i)plot refactor and new sanitize functions#1401
Networks statistics (i)plot refactor and new sanitize functions#1401
Conversation
- Extract carrier names from MultiIndex in stochastic networks - Automatically replicate new carriers across all scenarios - Preserve MultiIndex structure when concatenating carriers - Add comprehensive unit tests for stochastic network support - Update stochastic_network() example to set carriers and call add_missing_carriers()
Detect stochastic scenarios and adjust plotting behavior to ensure seaborn automatically aggregates scenarios with error bars. Collection indices are used for faceting instead of color when scenarios are present. This maintains error bar functionality while supporting NetworkCollection plotting.
Stochastic networks now display error bars in plotly interactive bar plots. Scenarios are automatically aggregated (mean) and standard deviations are shown as error bars. Detection ensures only true stochastic networks are aggregated, not collections with indices named 'scenario'. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Fix 15/20 failing tests by accounting for AC carrier from buses - Add NetworkCollection.has_scenarios property for consistent API - Simplify stochastic carrier wrapping logic in add_missing_carriers() - Remove duplicated scenario checking code, use native has_scenarios - Add release notes for new features - Clean up .gitignore duplicates and add development files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Use self.n_save.components for safer component iteration - Change logger level from info to debug for no missing carriers - Move generate_colors to common.py as standalone function - Update tests to use common.generate_colors - Simplify _get_investment_periods to use n.periods directly - Use n._index_names directly (available on both Networks and Collections) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
|
Strong support for automatic carrier management! |
- Add Network.unique_carriers property to collect all carriers from components - Create Carriers.assign_colors() method for flexible color assignment - Refactor add_missing_carriers() to use new assign_colors() method - Simplify API: replace assign_colors/color_palette params with single palette param - Update all tests to match new API (palette instead of assign_colors/color_palette) Addresses review comments #1 and #3: - Issue #1: Create separate assign_colors() method instead of parameters - Issue #3: Add c.unique_carriers property similar to c.ports 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Addresses review comment #10 (lkstrp at plotter.py:759) about reducing boilerplate in statistics plotting code. This change consolidates ~110 lines of duplicated logic between StatisticPlotter and StatisticInteractivePlotter. Changes: - Created _StatisticPlotterBase base class with shared _prepare_chart_data() method - Both StatisticPlotter and StatisticInteractivePlotter now inherit from this base - Moved common logic for parameter validation, schema application, and data preparation into the base class method - Fixed _get_investment_periods() to handle NetworkCollection gracefully by catching NotImplementedError when accessing the periods attribute Benefits: - Reduces code duplication and maintenance burden - Ensures consistency between static and interactive plotters - Makes future changes easier to implement in one place - Improves code readability 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Add unique_carriers as a property on the Components base class, following the established pattern for component-related properties like ports. Changes: - Added unique_carriers property to pypsa.components.components.Components - API: `n.c.<component>.unique_carriers` (e.g., `n.c.generators.unique_carriers`) - Updated carriers.py to iterate over all components and collect unique carriers - Updated release notes to document the new API location Benefits: - Follows consistent API pattern similar to ports property - Each component can report its own unique carriers - More modular and intuitive API design 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Refactor get_carrier_colors and get_carrier_labels to reduce code duplication by extracting common logic into a helper method _get_carrier_attribute. Changes: - Added _get_carrier_attribute() helper method to extract carrier attributes with optional nice name mapping - Simplified get_carrier_colors() to use the helper method - Simplified get_carrier_labels() with clearer logic - Added comprehensive docstrings to all methods - Added explanatory comments for stochastic network handling Benefits: - Reduces code duplication (~15 lines) - Makes the code more maintainable and easier to extend - Provides a pattern that could be generalized for other component attributes - Clearer separation of concerns The helper method could potentially be extended in the future to work with other component types beyond carriers, making it a foundation for more general component attribute access patterns. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Add a `unique` property to the Carriers class that handles extracting unique carrier names from both regular and stochastic networks. This eliminates repeated conditional logic throughout the codebase. Changes: - Added `unique` property to pypsa.components._types.carriers.Carriers - Returns unique carrier names, handling MultiIndex for stochastic networks - Updated assign_colors() to use self.unique (2 usages simplified) - Updated add_missing_carriers() to use self.unique (1 usage simplified) Benefits: - Eliminates 3 instances of if/else blocks checking for scenarios - Makes code more readable and maintainable - Provides a single source of truth for accessing unique carrier names - Reduces coupling to internal index structure Example usage: >>> n.c.carriers.unique Index(['AC', 'gas', 'solar', 'wind'], dtype='object', name='name') 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Add timestep-to-snapshot mapping for multi-period networks in area/line plots - Fix artificial trace generation for stacked area plots with proper array dimensions - Extend filtered columns to include scenario and period for proper grouping - Add auto-faceting for multi-period, stochastic, and collection networks
- Add missing `import pypsa` statements to doctest examples - Mark illustrative examples with `# doctest: +SKIP` where appropriate - Add `# doctest: +NORMALIZE_WHITESPACE` to handle pandas DataFrame formatting - Update expected output in add_missing_carriers examples to include 'AC' carrier 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Add tests for previously untested functionality to improve code coverage: - Test assign_colors method with various scenarios: - Basic color assignment - Specific carrier selection - Different color palettes - Overwrite behavior - Stochastic networks - Test unique_carriers property: - Empty components - Components with and without carrier attribute - Filtering empty strings and NaN values - Test has_scenarios property for NetworkCollection: - Collections without scenarios - Collections with mixed networks - Collections with only stochastic networks - Test error handling for conflicting arguments in add_missing_carriers All 32 tests pass successfully. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Add tests to achieve 100% coverage on pypsa/components/_types/carriers.py: - Test assign_colors with single string carrier name (covers line 127) - Test add_missing_carriers on stochastic network without palette (covers branch 295->300 with palette=None case) All 33 carrier tests pass. Carriers module now has 100% line and branch coverage. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
|
This is amazing, thanks @FabianHofmann and @lkstrp, I tested this PR in my notebooks, it fixes the errors I also experienced when using n.statistics() on stochastic networks. What's still missing to merge this into master? :) Happy holidays! |
Great! Just some clean up and tests. I haven't been rushing this since it goes in |
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
|
@FabianHofmann I fixed some bugs which I found via the pypsa-app and changed a couple of things. Specially the API: Also @fneum Could you have a look, specially on the newly introduced API? |
|
@lkstrp wonderful, looked through your changes and like the cleanup. |
fneum
left a comment
There was a problem hiding this comment.
Looks great! Only optional stuff, mainly the idea to rename assign_colors to add_missing_colors to align with the other sanitize functions.





Enhances interactive and non-interactive plotting functionality for network collections and stochastic networks, with automatic carrier management.
Key Features:
add_missing_carriers()method: Automatically discovers and adds carriers used in components with optional color palette assignment (defaults to matplotlib's tab10)How to check
Checklist
docs.docs/release-notes.mdof the upcoming release is included.