[Docs] MLRun CE docs improvments #8797
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jillnogold
reviewed
Nov 2, 2025
docs/change-log/index.md
Outdated
| #### Documentation | ||
|
|
||
| Improvements to [Set up your environment](../install/remote.md). | ||
| Improvements to [Set up your environment](../setup-guide/remote.md). |
Collaborator
Author
There was a problem hiding this comment.
@jillnogold ones it will be merged i will add the changes to the relevant change logs
jillnogold
reviewed
Nov 2, 2025
docs/cheat-sheet.md
Outdated
|
|
||
| ## MLRun setup | ||
| Docs: [Set up your client environment](./install/remote.md), [Installation and setup guide](./install.md) | ||
| Docs: [Set up your client environment](./setup-guide/remote.md), [Development Guide](setup-guide.md) |
Member
There was a problem hiding this comment.
jillnogold
reviewed
Nov 2, 2025
docs/cheat-sheet.md
Outdated
| Docs: [Set up your client environment](./setup-guide/remote.md), [Development Guide](setup-guide.md) | ||
|
|
||
| ### MLRun server/client overview | ||
| ### MLRun server/client overviewß |
jillnogold
reviewed
Nov 2, 2025
|
|
||
| MLRun has two main components, the service and the client (SDK+UI): | ||
| - MLRun service runs over Kubernetes (can also be deployed using local Docker for demo and test purposes) - see [installation documentation](./install.md) for more information | ||
| - MLRun service runs over Kubernetes (can also be deployed using local Docker for demo and test purposes) - see [MLRun CE installation documentation](setup-guide.md) for more information |
Member
There was a problem hiding this comment.
- MLRun service runs over Kubernetes (can also be deployed using local Docker for demo and test purposes). See the MLRun CE installation documentation for more information.
jillnogold
reviewed
Nov 2, 2025
docs/cheat-sheet.md
Outdated
|
|
||
| ### Remote connection (laptop, CI/CD, etc.) | ||
| Docs: [Configure remote environment](./install/remote.md#configure-remote-environment) | ||
| Docs: [Configure remote environment](./setup-guide/remote.md#configure-remote-environment)ß |
Member
There was a problem hiding this comment.
jillnogold
reviewed
Nov 2, 2025
jillnogold
reviewed
Nov 2, 2025
docs/install-mlrun-ce/index.md
Outdated
| # Installing MLRun CE | ||
|
|
||
|
|
||
| MLRun Community Edition (CE) provides a complete MLOps platform for managing the full machine learning lifecycle. |
Member
There was a problem hiding this comment.
The MLRun Community Edition (CE) provides a complete MLOps platform for managing the full machine learning lifecycle.
jillnogold
reviewed
Nov 2, 2025
|
|
||
| ## Next Steps | ||
|
|
||
| After installation, please check the developemnt notes [page](./mlrun-ce-development-notes.md) for optional development configuration. |
Member
There was a problem hiding this comment.
After installation, check the development notes for optional development configuration.
jillnogold
reviewed
Nov 2, 2025
| - 8Gi | ||
| - Set a default storage class for the kubernetes cluster, in order for the pods to have persistent storage. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#storageclass-objects) for more information. | ||
| - RAM: A minimum of 8Gi is required for running all the initial MLRun components. The amount of RAM required for running MLRun jobs depends on the job's requirements. | ||
| - Please review the [MLRun CE Installation Notes](./mlrun-ce-installation-notes.md) for any additional installation steps you may need to consider. |
Member
There was a problem hiding this comment.
- Review the MLRun CE Installation Notes for any additional installation steps you may need to consider.
jillnogold
reviewed
Nov 2, 2025
| --set kube-prometheus-stack.enabled=false \ | ||
| --set spark-operator.enabled=false \ | ||
| ``` | ||
| You can use your code on a local machine while running your functions on a remote cluster. Refer to [Set up your environment](https://docs.mlrun.org/en/latest//setup-guide/remote.md.html) for more information. |
Member
There was a problem hiding this comment.
You can use your code on a local machine while running your functions on a remote cluster. Refer to Set up your environment for more information.
jillnogold
reviewed
Nov 2, 2025
| @@ -0,0 +1,79 @@ | |||
| # MLRun CE Development Notes | |||
|
|
|||
| This page contains development notes for configure your developemt system. | |||
Member
There was a problem hiding this comment.
This page contains notes for configuring your development system.
jillnogold
reviewed
Nov 2, 2025
jillnogold
reviewed
Nov 2, 2025
| @@ -0,0 +1,79 @@ | |||
| # MLRun CE Development Notes | |||
jillnogold
reviewed
Nov 2, 2025
| - [Configuring the user Jupyter conda environment](#configuring-the-user-jupyter-conda-environment) | ||
| - [Configuring TDengine and Kafka for model monitoring](#configuring-tdengine-and-kafka-for-model-monitoring) | ||
|
|
||
| ## Change Deployemnt and Jobs default PVC |
Member
There was a problem hiding this comment.
Change the deployment and jobs default PVC
jillnogold
reviewed
Nov 2, 2025
|
|
||
| See more details, including additional configuration options, in {py:class}`~mlrun.projects.MlrunProject.set_model_monitoring_credentials`. | ||
|
|
||
| ## Running local spark jobs |
jillnogold
reviewed
Nov 2, 2025
| @@ -0,0 +1,43 @@ | |||
| # MLRun CE Installation Notes | |||
jillnogold
reviewed
Nov 2, 2025
| @@ -0,0 +1,81 @@ | |||
| (mlrun-ce-overview)= | |||
| # MLRun CE Overview | |||
jillnogold
reviewed
Nov 2, 2025
|
|
||
| **In this section** | ||
| - [What is MLRun CE](#what-is-mlrun-ce) | ||
| - [Main Advantages](#mlrun-ce-main-advantages) |
jillnogold
reviewed
Nov 2, 2025
| ## What is MLRun CE | ||
|
|
||
| MLRun CE is an open-source platform that simplifies the entire lifecycle of your AI project. | ||
| By installing the MLRun CE Helm chart on your Kubernetes cluster or local laptop, you get a powerful, integrated environment for development your LLM and ML projects. |
Member
There was a problem hiding this comment.
By installing the MLRun CE Helm chart on your Kubernetes cluster or local laptop, you get a powerful, integrated environment for developing your LLM and ML projects.
jillnogold
reviewed
Nov 2, 2025
| By installing the MLRun CE Helm chart on your Kubernetes cluster or local laptop, you get a powerful, integrated environment for development your LLM and ML projects. | ||
| The platform is built on two main applications - MLRun for MLOps orchestration and Nuclio for serverless computing. | ||
|
|
||
| ### **MLRun: The MLOps Orchestration Framework** |
Member
There was a problem hiding this comment.
MLRun: the MLOps orchestration framework
jillnogold
reviewed
Nov 2, 2025
|
|
||
| ### **MLRun: The MLOps Orchestration Framework** | ||
| MLRun is the MLOps orchestration framework that automates the entire AI pipeline, from data preparation and model training to deployment and management. It automates tasks like model tuning and optimization, enabling you to build scalable and observable AI applications. With MLRun, you can run your batch jobs and your real-time applications over elastic resources and gain end-to-end observability. | ||
|
|
Member
There was a problem hiding this comment.
Nuclio: the serverless engine
jillnogold
reviewed
Nov 2, 2025
| ### **Nuclio: The Serverless Engine** | ||
| Nuclio is a high-performance serverless framework that focuses on data, I/O, and compute-intensive workloads. It is the engine that powers the real-time functions within MLRun. Nuclio allows you to deploy your code as serverless functions, which are highly efficient and can process hundreds of thousands of events per second. It supports various data sources, triggers, and execution over CPUs and GPUs. | ||
|
|
||
| ## MLRun CE Main Advantages |
jillnogold
reviewed
Nov 2, 2025
|
|
||
| ## MLRun CE Main Advantages | ||
|
|
||
| 1. **Open-source MLOps Solution** – MLRun CE is an open-source MLOps platform that you can quickly install on your Kubernetes cluster or local desktop by deploying the mlrun-ce chart. |
Member
There was a problem hiding this comment.
- Open-source MLOps solution: MLRun CE is an open-source MLOps platform that you can quickly install on your Kubernetes cluster or local desktop by deploying the mlrun-ce chart.
jillnogold
reviewed
Nov 2, 2025
|
|
||
| 1. **Open-source MLOps Solution** – MLRun CE is an open-source MLOps platform that you can quickly install on your Kubernetes cluster or local desktop by deploying the mlrun-ce chart. | ||
|
|
||
| 2. **Rapid Project Development** - Allows you to take your code from a Jupyter Notebook or you local IDE to a scalable k8s based platform, with minimal changes. |
Member
There was a problem hiding this comment.
- Rapid project development: Allows you to take your code from a Jupyter Notebook or you local IDE to a scalable k8s based platform, with minimal changes.
jillnogold
reviewed
Nov 2, 2025
| 2. **Rapid Project Development** - Allows you to take your code from a Jupyter Notebook or you local IDE to a scalable k8s based platform, with minimal changes. | ||
| This significantly shortens the time-to-production, enabling faster iteration and business | ||
|
|
||
| 3. **Efficient AI Project Management** – Gives users tools for experiment tracking, hyperparameter tuning, and model selection, allowing you to easily compare experiments, optimize models, and ensure reproducibility. |
Member
There was a problem hiding this comment.
- Efficient AI project management: Gives users tools for experiment tracking, hyperparameter tuning, and model selection, allowing you to easily compare experiments, optimize models, and ensure reproducibility.
# Conflicts: # docs/install-mlrun-ce/kubernetes-install.md
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
jillnogold
reviewed
Dec 30, 2025
docs/architecture.md
Outdated
|
|
||
| When one of the components detailed above is updated, it immediately impacts the feature generation, the model serving pipeline, and the monitoring. MLRun applies versioning to each component, as well as versioning and rolling upgrades across components. | ||
|
|
||
| ## MLRun Non-root user support |
jillnogold
reviewed
Dec 30, 2025
docs/index.md
Outdated
| See the supported data stores, development tools, services, platforms, etc., supported by MLRun's open architecture in **{ref}`ecosystem`**. | ||
|
|
||
| **In this section** | ||
| - [Deployemnt options](#deployment-options) |
jillnogold
reviewed
Dec 30, 2025
docs/index.md
Outdated
| **In this section** | ||
| - [Deployemnt options](#deployment-options) | ||
| - [Set up your client](#set-up-your-client) | ||
| - [Gen AI tasks](#gen-ai-tasks) |
jillnogold
reviewed
Dec 30, 2025
docs/index.md
Outdated
| You can work with your favorite IDE (e.g. PyCharm, VSCode, Jupyter, Colab, etc.). Read how to configure your client against the deployed | ||
| MLRun server in {ref}`install-remote`. | ||
|
|
||
| Once you have installed and configured MLRun, follow the {ref}`quick-start-ml` and additional {ref}`Tutorials and Examples<tutorial>` to learn how to use MLRun to develop and deploy machine learning applications to production. |
Member
There was a problem hiding this comment.
Once you have installed and configured MLRun, follow the {ref}quick-start-ml and additional {ref}Tutorials and Examples<tutorial> to learn how to use MLRun to develop and deploy GenAI and machine learning applications to production.
jillnogold
reviewed
Dec 30, 2025
| --set mlrun.api.extraEnvKeyValue.MLRUN_REDIS__URL=<redis-address> | ||
| ``` | ||
| ## Installing Spark Operator on non mlrun namespace | ||
| # By default Spark Operator jobNamespaces is set to "mlrun" namespace, if you are installing Spark Operator on a different namespace you need to set the jobNamespaces value accordingly: |
Member
There was a problem hiding this comment.
By default Spark Operator jobNamespaces is set to "mlrun" namespace. If you are installing Spark Operator on a different namespace you need to set the jobNamespaces value accordingly:
jillnogold
reviewed
Dec 30, 2025
| ```bash | ||
| --set mlrun.api.extraEnvKeyValue.MLRUN_REDIS__URL=<redis-address> | ||
| ``` | ||
| ## Installing Spark Operator on non mlrun namespace |
Member
There was a problem hiding this comment.
Installing Spark Operator on non-mlrun namespace
jillnogold
reviewed
Dec 30, 2025
| - [Opt out of components](#opt-out-of-components) | ||
| - [Using NFS storage](#using-nfs-storage) | ||
| - [Configuring the online feature store](#configuring-the-online-feature-store) | ||
| - [Installing Spark Operator on non mlrun namespace](#installing-spark-operator-on-non-mlrun-namespace) |
Member
There was a problem hiding this comment.
jillnogold
reviewed
Dec 30, 2025
|
|
||
| ```{admonition} Note | ||
| These configurations are only required for AWS S3 storage, due to the usage of the same S3 protocol in MinIO. For other | ||
| storage options (such as GCS, Azure blobs etc.) only the artifact path needs to be modified, and credentials need to |
Member
There was a problem hiding this comment.
storage options (such as GCS, Azure blobs etc.), just modify the artifact path and provide credentials.
jillnogold
reviewed
Dec 30, 2025
| [**Function storage**](../runtimes/function-storage.md) for more details on auto-mount). This can be done in one | ||
| of following ways: | ||
|
|
||
| * Set the client-side MLRun configuration to disable auto-mount. This disables auto-mount for any function run |
Member
There was a problem hiding this comment.
- Set the client-side MLRun configuration to disable auto-mount. This disables auto-mount for any functions you subsequently run:
theSaarco
requested changes
Dec 30, 2025
Member
theSaarco
left a comment
theSaarco
left a comment
There was a problem hiding this comment.
Some fixes, comments and suggestions.
docs/install-mlrun-ce/index.md
Outdated
| # MLRun CE overview | ||
|
|
||
| MLRun CE is an open-source platform that simplifies the entire lifecycle of your AI project. | ||
| By installing the MLRun CE Helm chart on your Kubernetes cluster or local laptop, you get a powerful, integrated environment for developing your LLM and ML projects. |
Member
There was a problem hiding this comment.
Suggested change
| By installing the MLRun CE Helm chart on your Kubernetes cluster or local laptop, you get a powerful, integrated environment for developing your LLM and ML projects. | |
| By installing the MLRun CE Helm chart on your Kubernetes cluster or local laptop, you get a powerful, integrated environment for developing your GenAI and ML projects. |
A better analogy to "ML" would be "GenAI", not necessarily "LLM".
docs/install-mlrun-ce/index.md
Outdated
|
|
||
| - **Open-source MLOps Solution:** MLRun CE is an open-source MLOps platform that you can quickly install on your Kubernetes cluster or local desktop by deploying the mlrun-ce chart. | ||
|
|
||
| - **Rapid project development:** Allows you to take your code from a Jupyter Notebook or you local IDE to a scalable k8s based platform, with minimal changes. |
Member
There was a problem hiding this comment.
Suggested change
| - **Rapid project development:** Allows you to take your code from a Jupyter Notebook or you local IDE to a scalable k8s based platform, with minimal changes. | |
| - **Rapid project development:** Allows you to take your code from a Jupyter Notebook or your local IDE to a scalable k8s based platform, with minimal changes. |
| (install-on-kubernetes)= | ||
| # Install MLRun CE on Kubernetes | ||
|
|
||
| These instructions install the community edition (CE) on your Kubernetes cluster. This procedure installs an EKS cluster, an EBS volume, an S3 bucket, load balancing, etc. When you complete this procedure, you'll have the Community Edition of MLRun running on your EKS cluster. |
Member
There was a problem hiding this comment.
If it's a local cluster, then there won't be EKS, EBS, S3 etc. This seems to be here by mistake and taken from the AWS procedure.
| - Access to a Kubernetes cluster, version >=1.32. To install MLRun on your cluster, you must have administrator permissions. | ||
| For local installation on Windows or Mac, [Docker Desktop](https://www.docker.com/products/docker-desktop) is recommended. | ||
| - The Kubernetes command-line tool (kubectl) compatible with your Kubernetes cluster is installed. Refer to the [kubectl installation | ||
| instructions](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for more information. |
Member
There was a problem hiding this comment.
If using docker-desktop then you will already have kubectl installed and configured, should we mention that here?
Collaborator
Author
There was a problem hiding this comment.
i added a comment thanks :)
| - The Kubernetes command-line tool (kubectl) compatible with your Kubernetes cluster is installed. Refer to the [kubectl installation | ||
| instructions](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for more information. | ||
| - Helm version >=3.16 CLI is installed. Refer to the [Helm installation instructions](https://helm.sh/docs/intro/install/) for more information. | ||
| - An accessible Docker Registry (such as [Docker Hub](https://hub.docker.com)). The Registry's URL and credentials are consumed by the applications via a pre-created secret. If using Docker Hub, the Registry server is `https://registry.hub.docker.com/`. See the [Docker ID documentation](https://docs.docker.com/docker-id/) for details about creating a user with login that you will configure in the secret. |
Member
There was a problem hiding this comment.
The note about using docker-hub is repeated later in the detailed installation. Should this be here as well?
| name=tsdb_profile_name, | ||
| user="root", | ||
| password="taosdata", | ||
| host=f"tdengine-tsdb.{namespace}.svc.cluster.local", |
Member
There was a problem hiding this comment.
We reference namespace here - should either tell the user what to set there (would usually be mlrun) or avoid using it -> I think if you're using the service in the same ns as your pods then you don't need the ns in the host-name, don't you?
Collaborator
Author
There was a problem hiding this comment.
yes you correct, i deleted kept only the default service name
| ``` | ||
|
|
||
| ## Configuring TDengine and Kafka for model monitoring | ||
| TDengine and Kafka are part of the default CE installations. These are the default TDengine and Kafka installation values. It's recommended to change the user/password. |
Member
There was a problem hiding this comment.
It is indeed recommended, but that should be done in deployment time, doesn't it? Here you just use the user/password that you already configured beforehand.
Collaborator
Author
There was a problem hiding this comment.
I will remove this line, and we can add a an example how to change default passwords in the chart in the installation note
| # Create and register stream profile | ||
| stream_profile = DatastoreProfileKafkaSource( | ||
| name=stream_profile_name, | ||
| brokers=f"kafka-stream.{namespace}.svc.cluster.local:9092", |
gilad-shaham
approved these changes
Jan 6, 2026
jillnogold
pushed a commit
to jillnogold/mlrun
that referenced
this pull request
Jan 7, 2026
### 📝 Description <!-- A short summary of what this PR does. --> <!-- Include any relevant context or background information. --> --- This PR includes changes relevant to MLRun docs, in this PR we divided the installation and setup guide into two tabs. The setup guide explains how to set up your MLRun system and provides additional information about it, such as compatibility support. Then, we created a new tab that contains MLRun CE details like an overview, instructions for installation, and custom installation or development steps. This PR aims to improve the current docs by dividing and creating new pages, cleaning the current installation, and better explaining MLRun CE and its advantages. ### 🛠️ Changes Made <!-- - Key changes (e.g., added feature X, refactored Y, fixed Z) --> --- ### ✅ Checklist - [x] I updated the documentation (if applicable) - [ ] I have tested the changes in this PR - [ ] I confirmed whether my changes are covered by system tests - [ ] If yes, I ran all relevant system tests and ensured they passed before submitting this PR - [ ] I updated existing system tests and/or added new ones if needed to cover my changes - [ ] If I introduced a deprecation: - [ ] I followed the [Deprecation Guidelines](./DEPRECATION.md) - [ ] I updated the relevant Jira ticket for documentation --- ### 🧪 Testing <!-- - How it was tested (unit tests, manual, integration) --> <!-- - Any special cases covered. --> --- ### 🔗 References - Ticket link: https://iguazio.atlassian.net/browse/ML-11313 - Design docs links: - External links: --- ### 🚨 Breaking Changes? - [ ] Yes (explain below) - [x] No <!-- If yes, describe what needs to be changed downstream: --> --- ### 🔍️ Additional Notes <!-- Anything else reviewers should know (follow-up tasks, known issues, affected areas etc.). --> <!-- ### 📸 Screenshots / Logs --> # Conflicts: # docs/projects/git-best-practices.ipynb
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📝 Description
This PR includes changes relevant to MLRun docs, in this PR we divided the installation and setup guide into two tabs.
The setup guide explains how to set up your MLRun system and provides additional information about it, such as compatibility support.
Then, we created a new tab that contains MLRun CE details like an overview, instructions for installation, and custom installation or development steps.
This PR aims to improve the current docs by dividing and creating new pages, cleaning the current installation, and better explaining MLRun CE and its advantages.
🛠️ Changes Made
✅ Checklist
🧪 Testing
🔗 References
🚨 Breaking Changes?
🔍️ Additional Notes