-
Notifications
You must be signed in to change notification settings - Fork 294
[DEPRECATIONS] Support mlflow version 3.x #9102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…n new version of mlflow
theSaarco
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our general approach should be to not install the mlflow package unless specifically asked for. This means that we want to remove this package from the complete extra, and install it only if the user asks for the mlflow extra. This will reduce our exposure to the package and its possible vulns and dependencies.
I've added some comments explaining the exact changes that are needed to facilitate this.
@liranbg, @guy1992l - FYI.
dependencies.py
Outdated
| ], | ||
| "redis": ["redis~=4.3"], | ||
| "mlflow": ["mlflow~=2.22"], | ||
| "mlflow": ["mlflow~=3.0.0"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having discussed this in the release-tracking meeting, we want to remove the mlflow requirement from the complete extra. This means that this line should be removed from here, and added below (see L84 and further) so that when users ask for mlrun["mlflow"] they get just mlflow, and to install complete + mlflow they'll need to do install mlrun["complete, mlflow"].
extras-requirements.txt
Outdated
| redis~=4.3 | ||
| graphviz~=0.20.0 | ||
| mlflow~=2.22 | ||
| mlflow~=3.0.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As mentioned above, this also needs to be removed so we don't install this package by default in our images.
@omermaim - please mark this as a breaking change :) Also in Jira |
dependencies.py
Outdated
| { | ||
| "dev-postgres": ["pytest-mock-resources[postgres]~=2.12"], | ||
| "kfp18": ["mlrun_pipelines_kfp_v1_8[kfp]~=0.6.0"], | ||
| "mlflow": ["mlflow~=3.0.0"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we went all the way up to 3, why ~=3.0.0 and not 3.8.0? Does something break for us between 3.0 and 3.8?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in the ticket it said to move to version 3.0.0. Anyway this change should support later versions as well.
theSaarco
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
guy1992l
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VERY GOOD JOB :)
dependencies.py
Outdated
| { | ||
| "dev-postgres": ["pytest-mock-resources[postgres]~=2.12"], | ||
| "kfp18": ["mlrun_pipelines_kfp_v1_8[kfp]~=0.6.0"], | ||
| "mlflow": ["mlflow~=3.0.0"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in the ticket it said to move to version 3.0.0. Anyway this change should support later versions as well.
dependencies.py
Outdated
| { | ||
| "dev-postgres": ["pytest-mock-resources[postgres]~=2.12"], | ||
| "kfp18": ["mlrun_pipelines_kfp_v1_8[kfp]~=0.6.0"], | ||
| "mlflow": ["mlflow~=3.0.0"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mlflow~=3.0.0 is very strict. Question is should we strict it as security violations do not exist for it or should we move to mlflow~=3.0. The changes in their SDK and API should be fine.
dockerfiles/test/requirements.txt
Outdated
| @@ -1 +1,2 @@ | |||
| mlrun-pipelines-kfp-v1-8[kfp]~=0.6.0 | |||
| mlflow~=3.0.0 | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here...
tests/test_requirements.py
Outdated
| "aioresponses": {"~=0.7"}, | ||
| "testcontainers[k3s]": {"~=4.10.0"}, | ||
| "scikit-learn": {"~=1.5.2"}, | ||
| "mlflow": {"~=3.0.0"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here
| local_model_path = mlflow.artifacts.download_artifacts( | ||
| artifact_uri=str(model_uri) | ||
| ) | ||
| model_path = pathlib.Path(local_model_path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please keep the headline comments.
| local_model_path = mlflow.artifacts.download_artifacts( | |
| artifact_uri=str(model_uri) | |
| ) | |
| model_path = pathlib.Path(local_model_path) | |
| # Download the model and set the path to local path: | |
| local_model_path = mlflow.artifacts.download_artifacts( | |
| artifact_uri=str(model_uri) | |
| ) | |
| model_path = pathlib.Path(local_model_path) |
| filter_string=f"source_run_id = '{mlflow_run.info.run_id}'" | ||
| ) | ||
|
|
||
| model_uri = logged_models.iloc[0].artifact_location |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add output_format="list" to search_logged_models to get a list instead of DataFrame.
- change logged models ouput to be list
…n new version of mlflow
📝 Description
upgraded mlflow version to 3.0.0
🛠️ Changes Made
changed logic in model retrievel due to new mlflow objects. the models are no longer in list_artifacts function
✅ Checklist
🧪 Testing
ran alll tracker unit tests that is the only package that has this library.
🔗 References
🚨 Breaking Changes?
🔍️ Additional Notes
removes mlflow from the mlrun/mlrun image.