Skip to content

About

Powering Compute: what load flexibility, financial cover and price forecasts are worth to an AI data centre (MQFS working paper)

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

mqfs-compute-energy

Powering Compute: what load flexibility, financial cover and price forecasts are worth to an AI data centre.

A Mediterranean Quantitative Finance Society (MQFS) working paper.

Author: Nicolò Angileri (ORCID 0009-0004-3971-1114) · MIT licence · tests

An AI training data centre buys electricity at an hourly price with a heavy, persistent tail, and part of its load does not care exactly when it runs. This repository measures, on eleven years of archived market data from five European bidding zones, what that deferrability is worth, how it combines with forward and cap cover, and what a price forecast adds when work may wait longer than the day-ahead auction has published. Sicily, the dearest zone of the five, is the primary case; northern Italy, France, southern Sweden and south-western Norway are there so that no conclusion rests on one series.

The working paper: paper/paper.pdf. Every number in it is written by paper/build.py from results/*.json before it compiles. The three interactive pages run in the browser at nicoloangileri.github.io/mqfs-compute-energy.

Findings

The operational saving in each of the fifteen zone-periods, with its 95% interval

The saving of the schedule on realised prices, unhedged, in each zone and period, with a 95% moving-block bootstrap interval over days. Italy 2025 covers October to December and Norway 2025 covers 142 days, because of gaps in the source feed; 2026 runs to 5 August.

question answer paper
Operational saving of the schedule on realised prices, unhedged, 15 zone-periods 2024-2026 4.1% to 15.6% of the electricity bill, mean 8.3%; lowest day-level 95% bound 3.7% Section 9
Value of flexibility at the CVaR-optimal hedge, Sicily, simulation EUR 85,772 per MW-year, 6.25% of the optimal tail Section 8
What a forecast of the unpublished hours adds at a 48-hour window, EUR per MW-year over published prices only, 95% interval linear expert -1,167 [-4,952, +3,401]; neural network -1,387 [-6,969, +3,183]; perfect foresight +1,391 [-1,119, +3,790] Section 10
Test-block CRPS, frozen network against linear expert (lower is better) 0.11124 against 0.10890 Section 10
Tail reduction of the optimal hedge, Sicily 52.6% of the 95% CVaR, mostly by construction of the modelled forward curve Section 8
Price cap 0.0% cover at the base loading; dominated by the forward Section 8
Quantum amplitude estimation no practical advantage; see the resource estimate Appendix B

The claim the paper stands on is the operational one in the first rows: it is measured on the unhedged facility against prices that actually occurred, with the schedule restricted to what the auction had published, and its intervals are over real days. The financial result is reported as context, because the forward curve for Sicily had to be modelled from the same simulation it is measured on, which makes most of the tail reduction an identity; the paper says so where the number appears and sweeps the premium to size it. Three results are negative and are reported as such: a neural network does not beat a linear autoregression as a forecaster, a calibrated two-factor model generates better years than a neural SDE, and quantum amplitude estimation, verified exact on this problem, would need two-qubit error rates three orders of magnitude beyond today's hardware.

Reproducing

python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
python experiments/download_data.py --what all   # network, about 25 minutes
python cpp/build.py                              # optional C++ core
python experiments/run_all.py                    # every result, figure and page
python paper/build.py                            # numbers.tex, MODEL_CARD.md, PDF
pytest -q

requirements.txt pins the exact versions the published results were produced with. run_all.py runs the experiments in dependency order; --from STEP resumes. The download script is the only code that touches the network, and every file it writes is checked against data/manifest.json (endpoint, parameters, timestamp, size, SHA-256). The full pipeline takes a few hours on a laptop CPU; no accelerator is used.

What is in the repository

src/compute_energy/data/     ingestion, provenance manifest, cluster trace
src/compute_energy/stats/    seasonality, jumps, two-factor model, EVT, copulas
src/compute_energy/models/   quantile forecaster, neural SDE, baselines, scoring
src/compute_energy/mc/       Monte Carlo engine, receding-horizon scheduler,
                             realised-price runs with day-level intervals
src/compute_energy/hedge/    CVaR linear programme and efficient frontier
src/compute_energy/quantum/  amplitude loading, iterative QAE, resource counts
cpp/                         C++17 core through pybind11, exact twin of the engine
experiments/                 one script per result, all seeded; run_all.py
results/                     JSON results, figures, frozen models, three pages
site/                        landing page for the pages on GitHub Pages
paper/                       LaTeX sections, build.py, generated numbers
tests/                       unit and equivalence tests, run in CI

The scheduler decides once a day at the gate closure, sees the thirty-six hours of published prices, re-plans every unit of deferrable work not yet run, commits twenty-four hours, and moves work only when the saving beats the latency penalty. The NumPy reference and the C++ core agree exactly on the energy moved; tests/test_mc.py checks it, together with windows, capacity, re-planning and the forecast extension.

Three interactive pages in results/ carry real fitted objects and recompute something in the browser, reporting how far they land from the offline reference: the forecaster with its weights (forecaster_brain.html), the cost simulation with its scheduler (monte_carlo_3d.html), and the amplified quantum state (quantum_3d.html). GitHub Pages serves them from this repository with a landing page from site/ (.github/workflows/pages.yml); each is a single file that also works offline.

Data and licences

what source in this repository
day-ahead prices, five zones, 2016-2026 energy-charts (Fraunhofer ISE, ENTSO-E) France, Sweden, Norway (CC BY 4.0, Bundesnetzagentur / SMARD.de); Italy restored by the download script
load and generation by source energy-charts restored by the download script
weather at six sites Open-Meteo, ERA5 (CC BY 4.0, Copernicus) yes
GPU cluster trace, two months Alibaba PAI v2020 (research use; cite Weng et al., NSDI 2022) restored by the download script

The MIT licence covers the code. The Italian price series are marked by their provider "for private and internal use only" and the generation and load series carry no licence statement, so they are not redistributed. The Italian feed misses 1 January to 29 September 2025; the gap is recorded, never interpolated, and every realised-price analysis cuts at it.

Pre-registration and corrections

PREREGISTRATION.md fixed the sample split (training 2016-2022, validation 2023, test from 2024), the selection rules and four falsifiable hypotheses before any model was fitted. Every departure is recorded there with its date, including the corrections of v1.0 (see CHANGELOG.md): a jump family simulated with the wrong variance, CVaR programmes that had not reached optimality, a scheduler that could make losing moves, an out-of-sample metric that mixed the schedule with the hedge, and three leaks in the forecaster's data pipeline. MODEL_CARD.md documents both networks and is generated from the result files.

Limitations and what would change the answer

  • No quoted forward curve for Sicily. The curve is the simulated expected spot plus a stated premium, which makes the hedge's tail reduction largely mechanical. Real forward quotes would replace it.
  • The facility takes prices as given. Every value is per megawatt, on prices its own load does not move. A campus large against its zone would move them both ways: shifting work narrows the spreads the schedule captures, and leaving the dear hours lowers their price on the load that remains. The saving cannot simply be multiplied by the size of the facility, and the paper does not estimate the zonal supply curve that scaling it would need.
  • Seven training years calibrate the dispersion of an annual average, two of them from the 2021-2022 crisis; the simulated tail of the annual bill is an order of magnitude, not a measurement.
  • The Italian 2025 gap. An ENTSO-E token (ENTSOE_TOKEN) would let src/compute_energy/data/entsoe.py fill it from the primary source and return the test block to 2.6 years.
  • The deferrable share is a parameter, set by a resource-request proxy on a two-month trace from one operator, and swept in the paper.

Notes for developers

  • pyarrow must not be installed alongside qiskit-aer: after Aer is imported, pandas operations that go through Arrow strings fail with an opaque ArrowException. Pandas works without it.
  • The paper compiles with tectonic; paper/build.py --no-compile writes the numbers without compiling.
  • The energy-charts API truncates over-long requests silently; the client splits requests adaptively and audits every stored window.

Use of AI tools

This project was developed with extensive use of Anthropic's Claude models (in claude.ai and Claude Code), which wrote most of the code and drafted most of the text under the author's direction. The author set the scope, the data sources and the rules the work is bound by (every number from downloaded data and executed code, negative results reported, every citation checked against its source) and is responsible for the content. The commit history records how the work was produced.

Citation

See CITATION.cff.

About

Powering Compute: what load flexibility, financial cover and price forecasts are worth to an AI data centre (MQFS working paper)

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages