Very elegant solution given the current capital_cost attribute, and covers what i had in mind. However...

However, I would prefer instead to have investment costs and capital cost side by side and only allow one of them to be defined by the user.

I totally agree.

The way I see it, this PR tries to achieve three things:

1. A way to disentangle FOM from capital_cost attribute.
2. A way to access both annualized and overnight investment costs.
3. More convenience for calculating annualized investment costs (common task) from known
   • overnight cost,
   • lifetime,
   • discount rate, and
   • number of years in snapshots.

(1.) and (2.) are relatively straightforward to implement with backwards compatibility and minimal invasiveness to the current use of the capital_cost attribute; (3.) is a challenge but also an add-on and could be left for later.

A solution for (1.) and (2.) could look like this (recycled and adapted from #371 (comment)):

1. Split capital_cost into fom_cost and capital_cost. The new optional input attribute fom_cost would default to 0. In the objective, both attributes are simply summed up and associated with the investment variable. This is compatible with old networks where FOM is included in capital_cost.
2. Add a new optional input attribute discount_rate (defaults to 0). Together with the existing optional input attribute lifetime, the annuity factor can be calculated with pypsa.common.annuity.
3. Add a new output attribute overnight_cost that is a dependent attribute (similar to x_pu_eff) and a function of capital_cost, lifetime, and discount_rate (excluding fom_cost. Use annuity_factor() function to calculate it in n.calculate_dependent_values(). The status quo remains that the annualized investment costs are entered by the user, the overnight cost are derived.
4. Add a new statistics function for calculating overnight costs (upfront investments) based on overnight_cost, called n.statistics.overnight_cost().
5. The statistics function n.statistics.total_cost() must be amended by fom_cost since they are no longer included in n.statistics.capex().

Overall, I think this offers a good compromise between backward compatibility, invasiveness to the current use of capital_cost, and the ability to distinguish investment costs (as annualized and overnight value) and FOM.

The suggestion does not offer more convenience for calculating annualized capital_costs (3.), but that's a fairly simple calculation with the utility function pypsa.common.annuity at hand. The dilemma here is the clear distinction of inputs and outputs in PyPSA by design. Ideally, we'd want the user to specify either annualized or overnight costs, and the respective other attribute is filled based on derived values. But that option requires handling if both attributes are filled with values. Maybe a simple consistency check and routine in calculating dependent values will already do the trick?

The dilemma here is the clear distinction of inputs and outputs in PyPSA by design. Ideally, we'd want the user to specify either annualized or overnight costs, and the respective other attribute is filled based on derived values.

Isn't that very simple to achieve with a wrapper, that throws an error when both are there?

Basically n.add would start by checking its arguments for consistency, and derive either capital_cost or overnight_cost, and explain what's wrong otherwise. Or maybe I am missing something?

Basically n.add would start by checking its arguments for consistency, and derive either capital_cost or overnight_cost, and explain what's wrong otherwise. Or maybe I am missing something?

Doesn't sound too bad, actually. Just needs to be extended to all other cases of IO and network building / manipulation with a strong consistency check.

Basically n.add would start by checking its arguments for consistency, and derive either capital_cost or overnight_cost, and explain what's wrong otherwise. Or maybe I am missing something?

That just starts mixing input and output variables, which we don't currently do by design. But I am not opposed to the idea in general. There is just more to it. In n.add, this can also be easily checked with good user feedback. But n.add is not our only interface, we also need to check any dataframe manipulation and IO. And then the UX get's more tricky.

Doesn't sound too bad, actually. Just needs to be extended to all other cases of IO and network building / manipulation with a strong consistency check.

We can't extend the same check, since in n.add I only pass one of both. Via IO or in the dataframe, I have both values calculated already, without knowing which one was passed. The consistency check would basically need to recalculate all values to check if they still match, and not just check "are both given?". And if they don't match you say "add one of them again, to overwrite the other"? If you changed discount_rate/ life_time, do you take capital or overnight cost to calculate the other? Also n.add would only solve this with overwrite=True, which is not default. It is also currently not deriving any attributes. Compute wise that would probably work, but the additional compute as well as UX isn't very clean.

I am mainly wondering how many other use cases we would have, where an attribute could be both output and input. If there are not many, I would rather keep the existing input vs output structure, but assign them dynamically which we could do here with a hidden helper attribute, which stores the info which one was the input. And keep to (or start to properly) enforce only input can be changed, which would be best for any interface (n.add, DataFrame, Excel, App etc.). But as @FabianHofmann said, it's not much fun to implement without proper validation. I should probably just start resurrecting #1128

That just starts mixing input and output variables, which we don't currently do by design.

Yes, ideally that does not change.

The consistency check would basically need to recalculate all values to check if they still match, and not just check "are both given?"

Yes, that's what I thought. Not sure if that's elegant. Probably not.

And if they don't match you say "add one of them again, to overwrite the other"?

Hmm, difficult. The user has to fix it by aligning either capital_cost or overnight_cost with the rest? Not great.

Thanks Lukas for elaborating a bit on the complexity of the I/O. Regarding the consistency_check:

And if they don't match you say "add one of them again, to overwrite the other"?

Hmm, difficult. The user has to fix it by aligning either capital_cost or overnight_cost with the rest? Not great.

I would still suggest to do exactly that and I don't think it's too bad. If the user specifies both values for some odd reason, that's most likely due to a mistake in their model building pipeline. So in the best case throwing a hard error in the consistency check would only improve their model by forcing the elimination of ambiguities.

thanks all for the comments. I am wondering if there is good way to make the stretch between @lindnemi requirements for convenience which are valid and touch quite some use cases and @fneum's request to keep a clear data model and avoid ambiguity which I also strongly second, why I came up with the overloading approach.

Proposal 1:

A boolean attribute capital_as_overnight_cost (default false) decides the kind of cost input and how to process. (It's essentially what is now hidden in the discount rate which is bad API I have to admit).

Like this we:

• allow user to set all costs with
  • overnight cost, lifetime, discount rate → new use case
  • capital cost (old use case)
• don't change inputs on the fly and set ambiguous attributes potentially seen as inputs
• can derive all important cost parameters internally and expose them via component attributes:
  • n.c.Generator.overnight_cost
  • n.c.Generator.capital_cost
  • n.c.Generator.total_cost
  • n.c.Generator.annuity
• new component attributes feed into statistics functions

Note: FOM is independent and simply adds to period capital costs in optimization.

# Old behavior (default)
n.add("Generator", "gas", capital_cost=105.8)

# New behavior with overnight costs
n.add("Generator", "wind",
    capital_cost=1000,  # interpreted as overnight
    capital_as_overnight_cost=True,
    discount_rate=0.07,
    lifetime=25,
    fom_cost=20,
)
# → n.c.generators.capital_cost["wind"] ≈ 85.8 (derived)
# → n.c.generators.total_cost["wind"] ≈ 105.8

Proposal 2:

Introduce an overnight_cost attribute (default NaN) which takes precedence over capital_cost when set. Don't calculate one from the other—full information is exposed through component attributes (n.c.Generator....). Raise consistency_check warning if both are given, skip checks in n.add and other IO until we have proper data validation.

# Old behavior
n.add("Generator", "gas", capital_cost=105.8)

# New behavior
n.add("Generator", "wind",
    overnight_cost=1000,
    discount_rate=0.07,  # 0% is valid here!
    lifetime=25,
    fom_cost=20,
)
# → optimization uses: 1000 * annuity(0.07, 25) + 20 ≈ 105.8

Proposal 2 might be cleaner—no boolean flag, separate attributes with clear semantics, and discount_rate=0 works correctly. Proposal 1 reuses capital_cost for two meanings, same flexibility, mitigates confusion on the user side.

Out of proposals 1 and 2, proposal 2 is cleaner.

@lindnemi needs to chime in, since he already uses overnight_cost in PyPSA-DE networks where both are stored.

The attribute total_cost should be called periodic_cost, since (e.g. in statistics), total cost includes also OPEX.

Not sure I understood the n.c.Generator.capital_cost. Wouldn't it be an issue if n.generators.capital_cost would not return simply the values of the column in n.generators but a (potentially) computed value? That I would find confusing.

@lindnemi needs to chime in, since he already uses overnight_cost in PyPSA-DE networks where both are stored.

For me both is fine. I like Proposal 2 better, even though it will mean a bit more refactoring work on the PyPSA-DE side because there every asset already has capital_cost as well as overnight_cost defined, which will then lead to warnings.

What's most important for me, is that capital_cost, overnight_cost, FOM and lifetime are separately accesible in the solved networks (and ideally via n.statistics)

@lkstrp @fneum thanks for the reviews. I addressed all of them. And yes, I still have to learn to better handle iterative changes with CC

@fneum I guess two things were not clear:

1. fom_cost is expected to be periodized by the user, just as capital_costs. they are added wo periodization. I have updated the docs, they were a bit vague
2. back-calculation in n.c.<component>.overnight_cost is automatically done internally. this will raise an error now if lifetime and discount_rate are not given.