Skip to content

feat: fixed-view-point forecasting pipeline, available as CLI cmd flexmeasures add forecasts#1546

Merged
Flix6x merged 471 commits intomainfrom
feat/move-forecasting-to-flexmeasures
Nov 3, 2025
Merged

feat: fixed-view-point forecasting pipeline, available as CLI cmd flexmeasures add forecasts#1546
Flix6x merged 471 commits intomainfrom
feat/move-forecasting-to-flexmeasures

Conversation

@BelhsanHmida
Copy link
Copy Markdown
Contributor

@BelhsanHmida BelhsanHmida commented Jun 23, 2025

Description

This PR introduces a new forecasting pipelines into FlexMeasures. The pipelines train a model on historical sensor data and then generates forecasts over a specified prediction period. It supports both:

  • Regressor-based forecasting. ( we can use forecasts in regressors if available)
  • Autoregressive forecasting .

The pipeline works in forecasting cycles:

  • In each cycle, the model is trained on a moving training window (train-period).
  • Then, it forecasts the target sensor values for the next predict-period (up to max-forecast-horizon).
  • If the configured end-date hasn't been reached yet, the training window is extended and another cycle is run.
  • This repeats until the end-date is covered.

This design follows the fixed view point forecasting paradigm: each cycle re-trains the model from scratch with a slightly extended training window.

The forecasting jobs can be run immediately or scheduled in the forecasting queue using the --as-job option.

Key components include:

  • train_pipeline.py, predict_pipeline.py, and train_predict_pipeline.py for running forecasting cycles.
  • A custom_models/ module where ML models (e.g., LGBM) can be plugged in.
  • Supporting utilities for logging, error handling, and file management.
  • A CLI command to run forecasts.

  • Moved residential forecasting module from smart-buildings/flexmeasures_smartbuildings/residential to flexmeasures/data/models/forecasting
  • Ensured all relevant code and functionality are preserved in the new location
  • Updated import paths and dependencies accordingly
  • Forecasting can be done on forecasting queue
  • Added changelog item in documentation/changelog.rst
  • Added changelog item in documentation/cli/change_log.rst

Look and Feel

  • flexmeasures add --help:
  Commands:
  account            Create an account for a tenant in the FlexMeasures...
  account-role       Create an account role.
  annotation         Add annotation to accounts, assets and/or sensors.
  asset              Add an asset.
  asset-type         Add an asset type.
  beliefs            Add sensor data from a CSV or Excel file.
  forecasts          Generate forecasts for a target sensor.
  holidays           Add holiday annotations to accounts and/or assets.
  initial-structure  Initialize useful structural data.
  report             Create a new report using the Reporter class and...
  schedule           (Deprecated) Create a new schedule for a given power...
  sensor             Add a sensor.
  source
  sources            Create data sources for the data generators found...
  toy-account        Create a toy account, for tutorials and trying things.
  user               Create a FlexMeasures user.
  • flexmeasures add forecasts --help:
Usage: flexmeasures add forecasts [OPTIONS]

  Generate forecasts for a target sensor.

  Example
    flexmeasures add forecasts --sensor 2092 --regressors 2093
      --start-date 2025-01-01T00:00:00+01:00 --to-date 2025-10-15T00:00:00+01:00

  Workflow
    - Training window: defaults from --start-date until the CLI execution time.
    - Prediction window: defaults from CLI execution time until --to-date.
    - max-forecast-horizon: defaults to the length of the prediction window.
    - Forecasts are computed immediately; use --as-job to enqueue them.
    - Sensor 2093 is used as a regressor in this example.

  Notes:
  - Use --from-date to explicitly set when the forecasts will start.
  - Use --train-period to set the training window, which will grow each cycle
      until the specified --to-date is reached.
  - Use --predict-period to set the prediction window. It rolls forward by the
      forecast period each cycle, similar to the training window, but its size
      does not grow.

Options:
  --sensor TEXT                   Create forecasts for this sensor. Follow up
                                  with the sensor's ID. This argument can be
                                  given multiple times.  [required]
  --regressors TEXT               Comma-separated list of sensor IDs to be
                                  used as regressors. This is the full set of
                                  regressors and can include both past
                                  (realizations) and future (forecasts).
                                  Default is 'autoregressive', which uses the
                                  target sensor's own history.
  --future-regressors TEXT        Comma-separated list of sensor IDs to be
                                  treated as future regressors. Future
                                  regressors are assumed to have forecasts
                                  only.
  --past-regressors TEXT          Comma-separated list of sensor IDs to be
                                  treated as past regressors. Past regressors
                                  are assumed to have realizations only.
  --train-start, --start-date TEXT
                                  Start date for running the pipeline, i.e.
                                  when training begins (format: YYYY-MM-
                                  DDTHH:MM:SS±HH:MM).  [required]
  --to-date, --end-date TEXT      End date for running the pipeline (YYYY-MM-
                                  DDTHH:MM:SS+HH:MM)  [required]
  --train-period INTEGER RANGE    Duration of the initial training period in
                                  days (minimum of 2). After each forecast
                                  period, each next cycle will increase the
                                  training period by the forecast period. If
                                  not set, derives a training period from
                                  --start-predict-date instead. If that is
                                  also not set, defaults to 2 days.  [x>=2]
  --predict-period TEXT           Number of days to predict into the future in
                                  each cycle. After each cycle, the prediction
                                  window will move forward by this number of
                                  days.
  --from-date TEXT                Start date for predictions (YYYY-MM-
                                  DDTHH:MM:SS+HH:MM)
  --max-forecast-horizon TEXT     Maximum forecast horizon in hours
  --forecast-frequency INTEGER    Forecast frequency in hours, i.e. how often
                                  to recompute forecasts.
  --model-save-dir TEXT           Directory to save the trained model
  --output-path TEXT              Directory to save prediction outputs
  --probabilistic                 Enable probabilistic predictions
  --sensor-to-save TEXT           Sensor ID to save forecasts into a specific
                                  sensor. By default, forecasts are saved to
                                  the target sensor.
  --as-job                        Whether to queue a forecasting job instead
                                  of computing directly. To process the job,
                                  run a worker (on any computer, but
                                  configured to the same databases) to process
                                  the 'forecasting' queue. Defaults to False.
  --help                          Show this message and exit.

Further Improvements

follow-up PR's:

  • Streamline CLI options for forecasting commands and set defaults
  • Remove old forecasting code.
  • Add tests for the new forecasting CLI command

Related issue


  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

@BelhsanHmida BelhsanHmida self-assigned this Jun 23, 2025
Copy link
Copy Markdown
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just taking an early quick look :)

  • In the PR description, describe what the new feature is (a CLI command) and any other additions. People do not care too much that this code lived somewhere else before.
  • Do we need to use the word "residential" anymore? It is still in package names and the save path, but I don't see a reason to use it.
  • We don't have a src package convention in FlexMeasures, so IMO that can also go.
  • We usually write tests for code in FlexMeasures, at least outside of the CLI package. Is there a plan how we can test the new code that defines forecasting pipelines? Maybe we can later re-use the current test code for forecasting jobs?

@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community bot commented Jul 5, 2025

Documentation build overview

📚 flexmeasures | 🛠️ Build #30175466 | 📁 Comparing 6eee775 against latest (e7aaa76)


🔍 Preview build

Show files changed (25 files in total): 📝 15 modified | ➕ 10 added | ➖ 0 deleted
File Status
changelog.html 📝 modified
genindex.html 📝 modified
py-modindex.html 📝 modified
_autosummary/flexmeasures.cli.utils.html 📝 modified
_autosummary/flexmeasures.data.models.data_sources.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.custom_models.base_model.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.custom_models.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.custom_models.lgbm_model.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.exceptions.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.pipelines.base.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.pipelines.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.pipelines.predict.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.pipelines.train.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.pipelines.train_predict.html ➕ added
_autosummary/flexmeasures.data.models.forecasting.utils.html 📝 modified
_autosummary/flexmeasures.data.models.reporting.html 📝 modified
_autosummary/flexmeasures.data.schemas.forecasting.html ➕ added
_autosummary/flexmeasures.data.schemas.forecasting.pipeline.html ➕ added
_autosummary/flexmeasures.data.schemas.html 📝 modified
_autosummary/flexmeasures.data.schemas.times.html 📝 modified
api/v3_0.html 📝 modified
cli/change_log.html 📝 modified
features/forecasting.html 📝 modified
tut/forecasting_scheduling.html 📝 modified

@BelhsanHmida BelhsanHmida requested a review from Copilot July 6, 2025 20:38

This comment was marked as outdated.

@BelhsanHmida BelhsanHmida linked an issue Jul 6, 2025 that may be closed by this pull request
@nhoening nhoening added this to the 0.28.0 milestone Jul 20, 2025
@nhoening nhoening changed the title feat: move residential package to flexmeasures feat: new forecasting pipeline Jul 20, 2025
@nhoening nhoening added the CLI label Jul 20, 2025
@nhoening nhoening changed the title feat: new forecasting pipeline feat: fixed-view-point forecasting pipeline, available as CLI cmd flexmeasures add forecasts Jul 20, 2025
@BelhsanHmida BelhsanHmida requested a review from Copilot July 28, 2025 14:01
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new forecasting pipeline system into FlexMeasures, implementing "fixed-view-point forecasting" where models are retrained from scratch with progressively extended training windows. The pipeline supports both autoregressive and regressor-based forecasting modes, with the ability to run forecasting jobs immediately or schedule them in a forecasting queue.

Key changes include:

  • New forecasting pipeline architecture with modular components for training, prediction, and combined train-predict cycles
  • CLI integration replacing the deprecated flexmeasures add forecasts command with new functionality
  • U8darts library integration for advanced time series forecasting capabilities

Reviewed Changes

Copilot reviewed 20 out of 23 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
requirements/app.in Adds u8darts[notorch]>=0.29.0 dependency for forecasting capabilities
flexmeasures/cli/data_add.py Replaces deprecated forecasting command with new train_predict_pipeline functionality
flexmeasures/data/schemas/forecasting/pipeline.py Implements comprehensive validation schema for forecasting pipeline parameters
flexmeasures/data/models/forecasting/pipelines/ New modular pipeline architecture with base, train, predict, and train_predict components
flexmeasures/data/models/forecasting/custom_models/ Custom model implementations including LGBM model for multi-horizon forecasting
Comments suppressed due to low confidence (4)

requirements/app.in:68

  • The version 0.29.0 for u8darts appears to be non-existent. The latest available version of u8darts is 0.28.0. Consider using u8darts[notorch]>=0.28.0 instead.
u8darts[notorch]>=0.29.0

flexmeasures/data/models/forecasting/exceptions.py:16

  • The parameter name error_detail is misleading as it expects a sys module, not error details. Consider renaming it to sys_module or error_sys for clarity.
    _, _, exc_tb = error_detail.exc_info()

flexmeasures/data/models/forecasting/pipelines/base.py:376

  • This function appears to be deprecated based on the TODO comment. Consider removing it if the ForecastingPipelineSchema has replaced its functionality, or update the documentation to clarify its current status.
                )

@BelhsanHmida BelhsanHmida requested a review from nhoening July 30, 2025 06:10
Copy link
Copy Markdown
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A review from the train, without ability to install dependencies and run by myself.

Mostly doc&naming but also one part where I believe we might be able to save ourselves some time.

@nhoening
Copy link
Copy Markdown
Contributor

nhoening commented Aug 7, 2025

Two more questions:

  • Can you look at data/services/forecasting.py? There is a code stub for making fixed-view forecasts (as well as the code for rolling forecasts). However, this PR is not touching that file at all. I don't think we can ignore it.
    • Should we not use that stub?
    • Did we not plan to remove the rolling forecasts code or will we maintain it?
  • There is a util function called check_data_availability() (in data/models/forecasting/utils.py) which checks if we have the data we'd need to make forecasts (being called by make_rolling_viewpoint_forecasts()) - is that useful for use as well? We seem to fill in missing dates, but I guess more explicit checks could be useful beforehand.

BelhsanHmida and others added 22 commits October 16, 2025 15:03
Signed-off-by: Mohamed Belhsan Hmida <[email protected]>
… PR.

This reverts commit f1c9ca7.

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>
…e-forecasting-to-flexmeasures

# Conflicts:
#	flexmeasures/cli/data_add.py
#	flexmeasures/cli/utils.py
#	requirements/app.in
* feat: add as_job field to ForecasterParametersSchema for job configuration

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* fix: add missing comma dict return

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* test: add test case for as_job field in TrainPredictPipeline

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* fix: merge forecasting  job_kwargs into cycle params

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* fix: as_job should not become a DataSource attribute

Signed-off-by: F.N. Claessen <[email protected]>

---------

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>
Signed-off-by: F.N. Claessen <[email protected]>
Co-authored-by: F.N. Claessen <[email protected]>
* feat: add max-training-period option to CLI command

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* feat: add max_training_period field to ForecasterParametersSchema and enforce its limit in validation

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* feat: add logging for train-period exceeding max-training-period in ForecasterParametersSchema validation

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* test: add test to verify logging and capping of train_period exceeding max_training_period

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

* style: run pre-commit

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>

---------

Signed-off-by: Mohamed Belhsan Hmida <[email protected]>
Signed-off-by: Mohamed Belhsan Hmida <[email protected]>
Signed-off-by: F.N. Claessen <[email protected]>
Signed-off-by: F.N. Claessen <[email protected]>
Signed-off-by: F.N. Claessen <[email protected]>
Signed-off-by: F.N. Claessen <[email protected]>
@Flix6x Flix6x added this to the 0.30.0 milestone Nov 3, 2025
Signed-off-by: F.N. Claessen <[email protected]>
Signed-off-by: F.N. Claessen <[email protected]>
@Flix6x Flix6x merged commit 7fdccc4 into main Nov 3, 2025
7 of 8 checks passed
@Flix6x Flix6x deleted the feat/move-forecasting-to-flexmeasures branch November 3, 2025 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

4 participants