Change LCOX to use same optimisation as NPV

[alexdewar]

Description

This PR changes the approach to LCOX appraisal, which means the mini-optimisation performed is now the same as for NPV. See #1199 for background and discussion.

I've held off on carrying out more substantial refactoring of the appraisal and investment code -- though this is sorely needed -- because we have other branches based on this (e.g. #1357). After all this work has settled down a bit, we can take another pass at it and tidy things up a bit.

The circularity_npv model unfortunately no longer works with these changes (#1368). I haven't had a chance to look into why this is, but disabling the test temporarily in order to get this branch merged and unblock other work seemed like the best approach. We should try to figure out what's going wrong there, though.

The first three commits are changes I made for the sake of debugging, but I think they're worth merging:

• Better error messages for a couple of macros in fixture.rs
• regenerate_test_data.sh: Carry on if model fails to run
• If investment appraisal fails, print the number of non-feasible options which weren't considered

Closes #1199.

Type of change

• Bug fix (non-breaking change to fix an issue)
• New feature (non-breaking change to add functionality)
• Refactoring (non-breaking, non-functional change to improve maintainability)
• Optimization (non-breaking change to speed up the code)
• Breaking change (whatever its nature)
• Documentation (improve or add documentation)

Key checklist

• All tests pass: $ cargo test
• The documentation builds and looks OK: $ cargo doc
• Update release notes for the latest release if this PR adds a new feature or fixes a bug
  present in the previous release

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works

[Copilot]

Pull request overview

This PR refactors the LCOX investment appraisal so that it uses the same optimisation formulation as the NPV appraisal (maximise activity surplus with fixed capacity equal to the supplied max_capacity), instead of the previous LCOX-specific formulation that minimised annualised cost with capacity as a decision variable and a VoLL-priced unmet-demand term. LCOX-specific per-time-slice costs are now computed separately and used only for the final LCOX metric calculation. This addresses issue #1199 where the old LCOX approach overinvested capacity and relied on VoLL to clear demand.

Changes:

• Removed the capacity decision variable, the unmet-demand objective penalty, and the LCOX-specific Sense::Minimise path from the appraisal optimisation; both LCOX and NPV now call a single shared perform_optimisation that maximises activity surplus.
• Restructured ObjectiveCoefficients to hold a shared activity_coefficients map plus an optional lcox_costs map; calculate_coefficients_for_assets now always builds the activity coefficients via a single helper and fills lcox_costs only when the objective is LCOX.
• Propagated removal of capacity/unmet-demand coefficients through callers (appraisal.rs, output.rs, fixture.rs), so AppraisalOutput.capacity is now simply the supplied max_capacity and the LCOX metric uses annual_fixed_cost(asset) directly.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/simulation/investment/appraisal/optimisation.rs	Drops capacity variable, hard-codes Sense::Maximise, and remaps solution columns to activity + unmet-demand only.
src/simulation/investment/appraisal/constraints.rs	Removes add_capacity_constraint and candidate-specific activity constraints; activity bounds are now derived from max_capacity.total_capacity().
src/simulation/investment/appraisal/coefficients.rs	Unifies coefficient calculation; adds lcox_costs and removes capacity_coefficient/unmet_demand_coefficient.
src/simulation/investment/appraisal.rs	calculate_lcox/calculate_npv now share the same optimisation call and use max_capacity directly for metrics.
src/output.rs	Removes capacity_coefficient column from appraisal debug output.
src/fixture.rs	Updates test fixture to the new ObjectiveCoefficients shape.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tsmbland]

There's a few problems here:

1. Assets are frequently choosing not to serve at all. This may be fair enough and something we anticipated. However, we don't have a solution for if all assets choose not to serve, so currently it fails. This happens for circularity, simple_marginal and simple_marginal_average

2. We're calculating demand limiting capacity incorrectly for processes with seasonal/annual activity constraints. See Fix get_demand_limiting_capacity to account for seasonal/annual activity limits #1326

3. Even with the above fix it still doesn't work. Hard to explain, but the issue seems to be that the DLC calculation inherently assumes that activity is fully fungible across capacity, which isn't the case in the way that we do tranching.

   Taking the example of windturbine in the muse1_default model: DLC is calculated based on 40% annual utilisation (-> ~13.2 -> tranch size of ~0.132). Meeting the full annual demands at a total capacity of the DLC means running at the full 40% annual activity. Since no individual tranche asset can exceed 40%, this also means that each individual tranche asset would have to run at 40% annual activity. Because each timeslice is 16.67% of the year, full utilisation requires activity in at least three timeslices. For the first few tranches it runs with activity in three timeslices, then switching to different timeslices in later tranches when these are full (note, there is a tendency to settle on "corner solutions", rather than spreading activity across timeslices, which links back to Potential for different dispatch results on different platforms #1174). Eventually, in the last couple of tranches, all but two timeslices are full, so it can only run with activity of up to 33.33% (2 x 16.67%). Thus, by the time all 10 tranches are appraised, the DLC has been fully invested, but since some of the tranche assets were not active at 40%, there's still some unmet demand remaining in two of the timeslices. At that point the algorithm believes the demand to be unreachable and exits the simulation.

   So, there's a mismatch between the DLC that we're calculating and the actual demand that can be realised via the tranching approach. That said, since we ultimately pool the tranche assets for dispatch, and the combined asset can allocate demand however it wants across capacity (within the overall constraints for the total capacity), DLC capacity actually is sufficient and correct.

   We could try and address this by changing the DLC calculation, or allowing more than DLC capacity investment, but ultimately that would allow too much capacity to be installed which would end up under-utilised in the dispatch. Doesn't seem realistic to me, but maybe it is in the real world? (I have to say, the premise of a wind turbine that can be active for 40% of the year with no specifying what 40% is a bit ridiculous, and maybe wasn't the intention anyway for this model, so the question of "realistic" is a bit out the window anyway. Still, this raises a general point).

   Alternatively, we could try to reframe the way that we do tranching. Rather than looking at an entirely new asset with capacity = capacity_limit_factor * DLC, maybe we need to take into account the capacity that's been selected so far so that the new capacity can allocate activity in a way that's compatible with the overall asset rather than the new capacity in isolation. I imagine possibly that this could be different for divisible assets, which are new physical entities, vs non-divisible assets, in which tranches represent growth of a single entity.

Going forward:

• Figure out why assets are choosing not to be active. Presumably because it's not profitable to do so, in which case that asset probably shouldn't be selected (which currently it wouldn't be)
• Figure out what to do if all candidates are inactive. I think part of the problem is that choice of whether it's worth serving demands is based on previous years prices. In reality, prices would go up to the point where it becomes profitable to serve (and/or demand would drop), but there's no mechanism for this
• I think there's a genuine problem with the way that we're doing investments by breaking assets into independent sub-assets. We need to address this

[tsmbland]

I've figured out the problem with the marginal and marginal_average models. These were actually using shadow pricing for heat, because the patch wasn't configured properly (fixed in #1324). The problem with this is that if heat uses shadow pricing and electricity etc use cost-based pricing, then heat ends up underpriced compared to the other commodities in the system (shadow prices are set without any context of the cost-based prices we later calculate).

When it comes to investment appraisal, assets that were active in the previous year dispatch choose not to be active in the appraisal because they're not profitable with the price of heat that's given to them.

I've rebased on top of #1324 and these two models now work.

The circularity model still fails, possibly because it's still using shadow prices for some commodities, although hopefully that will be fixed up with #1322.*

Overall, I think this reveals an inherent issue with mixing shadow pricing and cost-based pricing. I think we have to either use shadow pricing throughout or cost-based pricing throughout, but not mix and match (although it would probably be fine if all the shadow-priced commodities were upstream of the cost-based commodities in the price calculation order).

*I still have doubts about this though. I think what this shows is the importance of commodity prices being consistent with one another, and in the current approach of iterating just once, rather than aiming for an equilibrium, it's possible that prices won't be consistent with one another.

[tsmbland]

I've tried to come up with a simpler/more generic example to demonstrate the point above about tranching.

---

Consider the following scenario:
3 timeslices (TS1, TS2, TS3) each of which is 1/3 of the year
We're considering a process with an output coefficient of 1, and an annual activity limit of 2/3

Together, these imply:

• Max timeslice output for 1 unit of capacity = 1/3
• Max annual output for 1 unit of capacity = 2/3

If demand = 100 in each timeslice (total annual demand = 300), we can calculate the demand limiting capacity (the capacity required to meet the full demands) as the max of:

• TS1 DLC: 100 / (1/3) = 300
• TS2 DLC: 100 / (1/3) = 300
• TS3 DLC: 100 / (1/3) = 300
• Annual DLC: 300 / (2/3) = 450
  -> 450

In other words, we can calculate in advance that 450 capacity is sufficient to meet the full demand profile, based on the activity limits of the process.

Now, suppose we split this into three tranche assets of 150 capacity. Treating each as an independent asset, it would have a production limit of 50 in each timeslice (based on each timeslice being 1/3 of the year) and an annual limit of 100 (based on 2/3 utilisation)

Tranche 1 discovers that TS1 and TS2 are most profitable (e.g. if the output commodity price is highest in these timeslices) so it runs at full utilisation there, giving remaining demand of:

• TS1: 50
• TS2: 50
• TS3: 100

Tranche 2 does the same, giving remaining demand of:

• TS1: 0
• TS2: 0
• TS3: 100

By the time of tranche 3, there's 100 units remaining demand in TS3, but this asset can only serve 50 of this because of its own timeslice-level limits.

Therefore, after 3 tranches and 450 capacity investment, according to the investment algorithm there's still unmet demand, despite the fact that we earlier calculated 450 capacity to be sufficient (at this point it would currently give up and exit the simulation). We could overcome this by allowing it to consider a 4th tranche, but in the end that would allow it to invest in more capacity than it actually needs.

[alexdewar]

Thanks for looking into this! There's a lot to think about here... Would probs be easiest to have a chat about it, but I'll dump some thoughts in here before I forget.

There's a few problems here:

1. Assets are frequently choosing not to serve at all. This may be fair enough and something we anticipated. However, we don't have a solution for if all assets choose not to serve, so currently it fails. This happens for circularity, simple_marginal and simple_marginal_average

Ok, so you figured this out for the second two (nice work!).

Out of curiosity, I tried merging #1322 into this branch to see if the circularity example works and it doesn't:

No feasible investment options left for commodity 'H2YPRD', region 'GBR', year '2040', agent 'A0_ELC' after appraisal.

There was a candidate asset but it wasn't dispatched, therefore not considered.

2. We're calculating demand limiting capacity incorrectly for processes with seasonal/annual activity constraints. See Fix get_demand_limiting_capacity to account for seasonal/annual activity limits #1326
3. Even with the above fix it still doesn't work. Hard to explain, but the issue seems to be that the DLC calculation inherently assumes that activity is fully fungible across capacity, which isn't the case in the way that we do tranching.
   [snip]

Do you think it's worth trying to get #1326 merged? If it fixes a problem with the current approach, it might be worth doing even if we expect we'll go down a different route soon anyway.

Going forward:

• Figure out why assets are choosing not to be active. Presumably because it's not profitable to do so, in which case that asset probably shouldn't be selected (which currently it wouldn't be)

Presumably! Would be worth checking...

• Figure out what to do if all candidates are inactive. I think part of the problem is that choice of whether it's worth serving demands is based on previous years prices. In reality, prices would go up to the point where it becomes profitable to serve (and/or demand would drop), but there's no mechanism for this

It feels a bit wrong that assets which could meet some (all?) of the demand aren't even being considered. Sure, it would be better to have a profitable asset than an unprofitable one, but better to have an unprofitable one than not meet demand at all. That's like choosing to starve to death because you don't fancy the hamburger in front of you.

Here are my half-baked ideas about how to handle the "only remaining options are non-feasible" case:

1. Change the mini-optimisation so that we still get a number out for the options that are currently non-feasible (could we do a second optimisation that ranks the non-feasible options somehow so we can choose the least bad one, if we have to?)
2. Use some even more arbitrary heuristics to decide between the non-feasible options (e.g. we could prioritise existing assets over candidates, among other things). We ideally don't want to be using silly criteria to decide between them, but note that choosing something is still better than choosing nothing! Note that if there's only one option and it's non-feasible there's not even a decision to be made: we should just try it.

• I think there's a genuine problem with the way that we're doing investments by breaking assets into independent sub-assets. We need to address this

I think your idea of including capacity for the same asset that was selected in a previous tranche is an interesting one. I wonder if there will be subtle consequences to doing this though. Let's discuss!

Overall, I think this reveals an inherent issue with mixing shadow pricing and cost-based pricing. I think we have to either use shadow pricing throughout or cost-based pricing throughout, but not mix and match (although it would probably be fine if all the shadow-priced commodities were upstream of the cost-based commodities in the price calculation order).

If Adam is happy with this, then shall we just require that users use only shadow pricing or only cost-based? We don't want to let users do silly things. The other option sounds intruiging but a bit fiddly, so let's not make our lives hard until we have a clear use-case.

[alexdewar]

Rebasing now that #1322 is merged

[tsmbland]

Do you think it's worth trying to get #1326 merged? If it fixes a problem with the current approach, it might be worth doing even if we expect we'll go down a different route soon anyway.

I think I tried it with #1326 and it worked, although I may be misremembering! (Edit: Actually don't think I've tried this) I think #1326 is mostly fine, I'm just not confident that it's correctly dealing with zero-availabilities (it's just getting really complicated). It would be a lot easier if the demand map was keyed by TimeSliceSelection (i.e. #1329), but we'd need to discuss before going down that route

It feels a bit wrong that assets which could meet some (all?) of the demand aren't even being considered. Sure, it would be better to have a profitable asset than an unprofitable one, but better to have an unprofitable one than not meet demand at all. That's like choosing to starve to death because you don't fancy the hamburger in front of you.

Or rather, choosing not to make a hamburger for a starving person because it looses you money. In reality, the starving person would offer to pay more, or buy a salad instead, but in the MUSE world you'd have to wait until the next milestone year for either of these to happen.

Here are my half-baked ideas about how to handle the "only remaining options are non-feasible case":

1. Change the mini-optimisation so that we still get a number out for the options that are currently non-feasible (could we do a second optimisation that ranks the non-feasible options somehow so we can choose the least bad one, if we have to?)
2. Use some even more arbitrary heuristics to decide between the non-feasible options (e.g. we could prioritise existing assets over candidates, among other things). We ideally don't want to be using silly criteria to decide between them, but note that choosing something is still better than choosing nothing! Note that if there's only one option and it's non-feasible there's not even a decision to be made: we should just try it.

Sounds reasonable. Should discuss with Adam

• I think there's a genuine problem with the way that we're doing investments by breaking assets into independent sub-assets. We need to address this

I think your idea of including capacity for the same asset that was selected in a previous tranche is an interesting one. I wonder if there will be subtle consequences to doing this though. Let's discuss!

👍

Overall, I think this reveals an inherent issue with mixing shadow pricing and cost-based pricing. I think we have to either use shadow pricing throughout or cost-based pricing throughout, but not mix and match (although it would probably be fine if all the shadow-priced commodities were upstream of the cost-based commodities in the price calculation order).

If Adam is happy with this, then shall we just require that users use only shadow pricing or only cost-based? We don't want to let users do silly things. The other option sounds intruiging but a bit fiddly, so let's not make our lives hard until we have a clear use-case.

Yeah, I think this is sensible

[alexdewar]

I've rebased this now. @tsmbland: I poached your bits for working out unmet demand from #1329, because this doesn't really make sense without it.

We now have four failures:

failures:
    regression::missing_commodity
    regression::muse1_default
    regression::two_outputs
    regression::two_regions

In every case, there is one non-feasible option not considered (see #1347). I'm going to try forcing the model to select them anyway and see if we can at least get the models to run to completion.

[tsmbland]

I've rebased this now. @tsmbland: I poached your bits for working out unmet demand from #1329, because this doesn't really make sense without it.

We now have four failures:

failures:
    regression::missing_commodity
    regression::muse1_default
    regression::two_outputs
    regression::two_regions

In every case, there is one non-feasible option not considered (see #1347). I'm going to try forcing the model to select them anyway and see if we can at least get the models to run to completion.

Unless anything major has changed, muse1_default and two_regions is probably the thing I described above about tranching.

missing_commodity and two_outputs is a new one (could be the same issue though)

[alexdewar]

So now all the models that were previously failing are passing 🥳.

Unfortunately, I didn't notice before that circularity_npv now fails:

Caused by:
    The solver has indicated that the problem is infeasible, probably because the supplied assets could not meet the required demand. Demand was not met for the following markets: [(CommodityID("TPASKM"), RegionID("GBR"))]

I don't think we had that model when I first opened this PR.

Any thoughts @tsmbland? I don't have lots of time to dig into this in detail today.

I suppose we do have the option of temporarily disabling this one test until we figure out why it's not working, if that turns out to be difficult. It may be that #1201 magically fixes things (or makes them worse...).

[tsmbland]

Let's just disable it for now, in the interests of getting this merged

[codecov]

Codecov Report

❌ Patch coverage is 97.77778% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.68%. Comparing base (76db0a2) to head (20b6823).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/simulation/investment/appraisal.rs	84.61%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1319      +/-   ##
==========================================
- Coverage   89.80%   89.68%   -0.12%     
==========================================
  Files          58       58              
  Lines        8524     8406     -118     
  Branches     8524     8406     -118     
==========================================
- Hits         7655     7539     -116     
+ Misses        556      550       -6     
- Partials      313      317       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Copilot reviewed 64 out of 66 changed files in this pull request and generated 4 comments.

[alexdewar]

At least for LCOX, if activity is zero, then lcox() will return None and so the value of the metric will also be None.

[alexdewar]

PS -- I tried running circularity_npv with a hack to force it to select the one non-feasible option in 2040 and it still fails.

[tsmbland]

Looks good!