Skip to content

Comments

Refactoring Stacking, Multiplexer, Ensembler and TransformedTarget Forecasters#977

Merged
fkiraly merged 16 commits intosktime:mainfrom
thayeylolu:refactor-stmultiplex
Jun 24, 2021
Merged

Refactoring Stacking, Multiplexer, Ensembler and TransformedTarget Forecasters#977
fkiraly merged 16 commits intosktime:mainfrom
thayeylolu:refactor-stmultiplex

Conversation

@thayeylolu
Copy link
Collaborator

@thayeylolu thayeylolu commented Jun 18, 2021

Reference : #955

-- This is a copy of 965 without changes to reducer.py (a time when there was no bug)
We need to slowly re-factor all forecasters to be compliant with the new interface (see PR #912). i.e., no overrides of fit etc and descendant classes only implementing core logic, while the BaseForecaster implements check etc logic.

Currently this is running due to various downwards compatibility workarounds, which should be removed once all forecasters are ready (in "no-override" state)

The forecasters that still need a downstream refactor are:

  • EnsembleForecaster
  • MultiplexForecaster
  • TransformedTargetForecaster
  • StackingForecaster

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 18, 2021

looks like checks are all fine, so do you want to convert to "proper" PR and request reviews?

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 18, 2021

and you may want to close #955 of which this is now a duplicate

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks for your work!

Although the tests pass, I have some questions about the right tags, and the potential to dynamically attach tags based on components, to @mloning, @aiwalter, @koralturkk, see above.

Further, I think the pre-refactor mixins for MultiplexForecaster and EnsembleForecaster were incorrect, fh could be required in fit if the components do require it. Why did the tests not catch this?

This is not an issue with your refactor, @thayeylolu, but I just noticed it in looking at the code. We should perhaps fix these before at this opportunity.

@mloning
Copy link
Contributor

mloning commented Jun 19, 2021

I'm not really sure what the best solution is. A few thoughts:

  • I see tags as being defined on the class (not object) level, conceptually I think of them as describing algorithm properties, so I would prefer not to update them dynamically
  • In the cases of the above mentioned (composition) algorithms, for the implementation (i.e. input validation of the forecasting horizon), the tag should be set to False, the composite doesn't require the forecasting horizon during fitting, it merely passes it on to the components, which have their own tag and raise an error accordingly if they require the forecasting horizon during fitting
  • For documentation, when users may use tags for finding relevant models, setting it to False may be misleading, but conceptually still makes sense, as it refers to the composition algorithm, rather than a particular instance of the composite

Either way, I think this is a separate issue. I created one here: #981.

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 19, 2021

ok, agreed - let's split off the larger tags discussion, and just do a "safe" refactoring here.
Outstanding changes:

  • the _SktimeForecaster removal is not tags related and should be done
  • multiplexer tag should be changed to requires fh in fit (see above)

Copy link
Collaborator Author

@thayeylolu thayeylolu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the necessary changes and I would I to convert the draft for review

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 21, 2021

the test examples are now failing - they ran before.
Can you please check what's happening?

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 21, 2021

I think it's the multiplexer - the tutorial notebook passes the fh too late

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 21, 2021

I'd suggest:

What do you think, @mloning?

@thayeylolu
Copy link
Collaborator Author

thayeylolu commented Jun 21, 2021

@fkiraly Setting requires-fh-in-fit: to True is the cause for the failure of the multiplexer

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 21, 2021

@fkiraly Setting requires-fh-in-fit: to True is the cause for the failure of the multiplexer

You mean, to False, no? We want to set it back to True according to my suggestion - please confirm?

@fkiraly fkiraly requested a review from mloning June 22, 2021 11:42
@fkiraly
Copy link
Collaborator

fkiraly commented Jun 22, 2021

pinging @ninamiolane

Copy link
Contributor

@mloning mloning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @thayeylolu, you need to check the update methods again, I think some of the code can still be removed there. Otherwise looks good to me!

@thayeylolu thayeylolu marked this pull request as ready for review June 23, 2021 20:39
@thayeylolu thayeylolu requested review from fkiraly and mloning June 24, 2021 06:00
@thayeylolu
Copy link
Collaborator Author

@mloning this I ready for review :)

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me - the earlier request is partly related to tags and will be addressed separately.

Updated from main and let's wait for tests to pass.

@fkiraly fkiraly merged commit 22fb2a4 into sktime:main Jun 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants