Skip to content

Commit 4ccfcfc

Browse files
zhongxuanwang-nvzhongxuanwang-nv
authored
Merge branch 'develop' into test_llm
2 parents 3ecf5ec + 528ea98 commit 4ccfcfc

File tree

35 files changed

+349
-118
lines changed

35 files changed

+349
-118
lines changed

‎README.md‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,18 @@ With NeMo Agent toolkit, you can move quickly, experiment freely, and ensure rel
5757

5858
## 🚀 Installation
5959

60-
Before you begin using NeMo Agent Toolkit, ensure that you have Python 3.11 or 3.12 installed on your system.
60+
Before you begin using NeMo Agent Toolkit, ensure that you have Python 3.11, 3.12, or 3.13 installed on your system.
61+
62+
### Supported Platforms
63+
64+
| Operating System | Architecture | Python Version | Supported |
65+
|------------------|--------------|---------------|-----------|
66+
| Linux | x86_64 | 3.11, 3.12, 3.13 | ✅ Tested, Validated in CI |
67+
| Linux | aarch64 | 3.11, 3.12, 3.13 | ✅ Tested, Validated in CI |
68+
| macOS | x86_64 | 3.11, 3.12, 3.13 | ❓ Untested, Should Work |
69+
| macOS | aarch64 | 3.11, 3.12, 3.13 | ✅ Tested |
70+
| Windows | x86_64 | 3.11, 3.12, 3.13 | ❓ Untested, Should Work |
71+
| Windows | aarch64 | 3.11, 3.12, 3.13 | ❌ Unsupported |
6172

6273
### Stable Version
6374

@@ -156,7 +167,7 @@ The following diagram illustrates the key components of NeMo Agent toolkit and h
156167

157168
## 🛣️ Roadmap
158169

159-
- [ ] Integrate with [NeMo DataFlywheel](https://github.com/NVIDIA-AI-Blueprints/data-flywheel) for continuous model improvement from production data.
170+
- [x] Integrate with [NeMo DataFlywheel](https://github.com/NVIDIA-AI-Blueprints/data-flywheel) for continuous model improvement from production data.
160171
- [ ] Add support for [Google ADK](https://google.github.io/adk-docs/) framework.
161172
- [ ] Add an agent optimizer to auto-tune hyperparameters and prompts to maximize performance.
162173
- [ ] MCP authorization and streamable HTTP support.

‎docs/source/extend/adding-an-llm-provider.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ async def openai_langchain(llm_config: OpenAIModelConfig, builder: Builder):
179179

180180
from langchain_openai import ChatOpenAI
181181

182-
yield ChatOpenAI(**llm_config.model_dump(exclude={"type"}, by_alias=True))
182+
yield ChatOpenAI(**llm_config.model_dump(exclude={"type", "thinking"}, by_alias=True))
183183
```
184184

185185
Similar to the registration function for the provider, the client registration function can perform any necessary setup actions before yielding the client, along with cleanup actions after the `yield` statement.

‎docs/source/extend/plugins.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ async def openai_langchain(llm_config: OpenAIModelConfig, builder: Builder):
7979

8080
from langchain_openai import ChatOpenAI
8181

82-
yield ChatOpenAI(**llm_config.model_dump(exclude={"type"}, by_alias=True))
82+
yield ChatOpenAI(**llm_config.model_dump(exclude={"type", "thinking"}, by_alias=True))
8383
```
8484

8585
The `wrapper_type` parameter in the decorator specifies the LLM framework that the plugin is compatible with. This instruments the plugin with the appropriate telemetry hooks to enable observability, evaluation, and profiling.

‎docs/source/quick-start/installing.md‎

Lines changed: 53 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,12 @@ This guide will help you set up your NVIDIA NeMo Agent toolkit development envir
2121

2222
## Supported LLM APIs
2323

24-
The following LLM APIs are supported:
24+
The following LLM API providers are supported:
2525

2626
- NIM (such as Llama-3.1-70b-instruct and Llama-3.3-70b-instruct)
2727
- OpenAI
2828
- AWS Bedrock
29+
- Azure OpenAI
2930

3031
## Framework Integrations
3132

@@ -35,6 +36,7 @@ To install these first-party plugin libraries, you can use the full distribution
3536

3637
- `nvidia-nat[agno]` or `nvidia-nat-agno` - [Agno](https://agno.com/)
3738
- `nvidia-nat[crewai]` or `nvidia-nat-crewai` - [CrewAI](https://www.crewai.com/)
39+
- `nvidia-nat[data-flywheel]` or `nvidia-nat-data-flywheel` - [NeMo DataFlywheel](https://github.com/NVIDIA-AI-Blueprints/data-flywheel)
3840
- `nvidia-nat[langchain]` or `nvidia-nat-langchain` - [LangChain](https://www.langchain.com/)
3941
- `nvidia-nat[llama-index]` or `nvidia-nat-llama-index` - [LlamaIndex](https://www.llamaindex.ai/)
4042
- `nvidia-nat[mem0ai]` or `nvidia-nat-mem0ai` - [Mem0](https://mem0.ai/)
@@ -50,16 +52,60 @@ To install these first-party plugin libraries, you can use the full distribution
5052
- `nvidia-nat[zep-cloud]` or `nvidia-nat-zep-cloud` - [Zep](https://www.getzep.com/)
5153

5254

53-
## Prerequisites
55+
## Supported Platforms
56+
57+
| Operating System | Architecture | Python Version | Supported |
58+
|------------------|--------------|---------------|-----------|
59+
| Linux | x86_64 | 3.11, 3.12, 3.13 | ✅ Tested, Validated in CI |
60+
| Linux | aarch64 | 3.11, 3.12, 3.13 | ✅ Tested, Validated in CI |
61+
| macOS | x86_64 | 3.11, 3.12, 3.13 | ❓ Untested, Should Work |
62+
| macOS | aarch64 | 3.11, 3.12, 3.13 | ✅ Tested |
63+
| Windows | x86_64 | 3.11, 3.12, 3.13 | ❓ Untested, Should Work |
64+
| Windows | aarch64 | 3.11, 3.12, 3.13 | ❌ Unsupported |
65+
66+
## Software Prerequisites
5467

5568
NVIDIA NeMo Agent toolkit is a Python library that doesn't require a GPU to run by default. Before you begin using NeMo Agent toolkit, ensure that you meet the following software prerequisites:
5669

57-
- Install [Git](https://git-scm.com/)
58-
- Install [Git Large File Storage](https://git-lfs.github.com/) (LFS)
59-
- Install [uv](https://docs.astral.sh/uv/getting-started/installation/) (version 0.5.4 or later, latest version is recommended)
70+
- [Python](https://www.python.org/) 3.11, 3.12, or 3.13
71+
72+
### Additional Prerequisites for Development
73+
- [Git](https://git-scm.com/)
74+
- [Git Large File Storage](https://git-lfs.github.com/) (LFS)
75+
- [uv](https://docs.astral.sh/uv/getting-started/installation/) (version 0.5.4 or later, latest version is recommended)
76+
77+
## Install from Package
78+
79+
The package installation is recommended for production use.
80+
81+
:::{note}
82+
To run any examples, you need to install the NeMo Agent toolkit from source.
83+
:::
84+
85+
To install the latest stable version of NeMo Agent toolkit, run the following command:
86+
87+
```bash
88+
pip install nvidia-nat
89+
```
90+
91+
NeMo Agent toolkit has many optional dependencies which can be installed with the core package. Optional dependencies are grouped by framework and can be installed with the core package. For example, to install the LangChain plugin, run the following:
92+
93+
```bash
94+
pip install nvidia-nat[langchain] # For LangChain
95+
```
96+
97+
Or for all optional dependencies:
98+
99+
```bash
100+
pip install nvidia-nat[all]
101+
```
102+
103+
The full list of optional dependencies can be found [here](../quick-start/installing.md#framework-integrations).
60104

61105
## Install From Source
62106

107+
Installing from source is required to run any examples provided in the repository or to contribute to the project.
108+
63109
1. Clone the NeMo Agent toolkit repository to your local machine.
64110
```bash
65111
git clone -b main https://github.com/NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit
@@ -84,7 +130,7 @@ NVIDIA NeMo Agent toolkit is a Python library that doesn't require a GPU to run
84130
source .venv/bin/activate
85131
```
86132
:::{note}
87-
Python 3.11 and 3.12 is also supported simply replace `3.13` with `3.11` or `3.12` in the `uv` command above.
133+
Python 3.11 and 3.12 are also supported simply replace `3.13` with `3.11` or `3.12` in the `uv` command above.
88134
:::
89135

90136
1. Install the NeMo Agent toolkit library.
@@ -108,7 +154,7 @@ NVIDIA NeMo Agent toolkit is a Python library that doesn't require a GPU to run
108154
Many of the example workflows require plugins, and following the documented steps in one of these examples will in turn install the necessary plugins. For example following the steps in the `examples/getting_started/simple_web_query/README.md` guide will install the `nvidia-nat-langchain` plugin if you haven't already done so.
109155
:::
110156
111-
In addition to plugins, there are optional dependencies needed for profiling. To install these dependencies, run the following:
157+
In addition to plugins, there are optional dependencies needed for profiling. Installing the `profiling` sub-package is required for evaluation and profiling workflows using `nat eval`. To install these dependencies, run the following:
112158
```bash
113159
uv pip install -e '.[profiling]'
114160
```

‎docs/source/tutorials/create-a-new-workflow.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ async def text_file_ingest_function(config: TextFileIngestFunctionConfig, builde
198198

199199
## Creating the Workflow Configuration
200200

201-
Starting from the `custom_config.yml` file you created in the previous section, replace the two `webpage_query` tools with the new `text_file_ingest` tool. For the data source, you can use a collection of text files located in the `examples/documentation_guides/workflows/text_file_ingest/data` directory that describes [DOCA GPUNetIO](https://docs.nvidia.com/doca/sdk/doca-gpunetio/index.html).
201+
Starting from the `custom_config.yml` file you created in the previous section, replace the two `webpage_query` tools with the new `text_file_ingest` tool. For the data source, you can use a collection of text files located in the `examples/documentation_guides/workflows/text_file_ingest/data` directory that describes [DOCA GPUNetIO](https://docs.nvidia.com/doca/sdk/doca+gpunetio/index.html).
202202

203203
:::{note}
204204
<!-- path-check-skip-next-line -->

‎docs/source/workflows/evaluate.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,20 @@ limitations under the License.
2323

2424
NeMo Agent toolkit provides a set of evaluators to run and evaluate workflows. In addition to the built-in evaluators, the toolkit provides a plugin system to add custom evaluators.
2525

26+
## Prerequisites
27+
28+
In addition to the base `nvidia-nat` package, you need to install the `profiling` sub-package to use the `nat eval` command.
29+
30+
If you are installing from source, you can install the sub-package by running the following command from the root directory of the NeMo Agent toolkit repository:
31+
```bash
32+
uv pip install -e '.[profiling]'
33+
```
34+
35+
If you are installing from a package, you can install the sub-package by running the following command:
36+
```bash
37+
uv pip install nvidia-nat[profiling]
38+
```
39+
2640
## Evaluating a Workflow
2741
To evaluate a workflow, you can use the `nat eval` command. The `nat eval` command takes a workflow configuration file as input. It runs the workflow using the dataset specified in the configuration file. The workflow output is then evaluated using the evaluators specified in the configuration file.
2842

‎docs/source/workflows/mcp/index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@ NeMo Agent toolkit [Model Context Protocol (MCP)](https://modelcontextprotocol.i
2121
* An [MCP client](./mcp-client.md) to connect to and use tools served by remote MCP servers.
2222
* An [MCP server](./mcp-server.md) to publish tools using MCP to be used by any MCP client.
2323

24+
**Note:** MCP client functionality requires the `nvidia-nat-mcp` package. Install it with `uv pip install nvidia-nat[mcp]`.
25+
2426

2527
```{toctree}
2628
:caption: MCP

‎docs/source/workflows/mcp/mcp-client.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,14 @@ You can use NeMo Agent toolkit as an MCP Client to connect to and use tools serv
2323

2424
This guide will cover how to use NeMo Agent toolkit as an MCP Client. For more information on how to use NeMo Agent toolkit as an MCP Server, please refer to the [MCP Server](./mcp-server.md) documentation.
2525

26+
## Installation
27+
28+
MCP client functionality requires the `nvidia-nat-mcp` package. Install it with:
29+
30+
```bash
31+
uv pip install nvidia-nat[mcp]
32+
```
33+
2634
## MCP Client Configuration
2735

2836
The MCP client can connect to MCP servers using different transport types. The choice of transport should match the server's configuration.
@@ -90,7 +98,7 @@ class MCPClientConfig(FunctionBaseConfig, name="mcp_client"):
9098
Configuration for connecting to an MCP server as a client and exposing selected tools.
9199
"""
92100
server: MCPServerConfig = Field(..., description="Server connection details (transport, url/command, etc.)")
93-
tool_filter: dict[str, ToolOverrideConfig] | list[str] | None = Field(
101+
tool_filter: dict[str, MCPToolOverrideConfig] | list[str] | None = Field(
94102
default=None,
95103
description="""Filter or map tools to expose from the server (list or dict).
96104
Can be:

‎docs/source/workflows/mcp/mcp-server.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,8 @@ nat mcp --config_file examples/getting_started/simple_calculator/configs/config.
6262

6363
To list the tools published by the MCP server you can use the `nat info mcp` command. This command acts as a MCP client and connects to the MCP server running on the specified URL (defaults to `http://localhost:9901/mcp` for streamable-http, with backwards compatibility for `http://localhost:9901/sse`).
6464

65+
**Note:** The `nat info mcp` command requires the `nvidia-nat-mcp` package. If you encounter an error about missing MCP client functionality, install it with `uv pip install nvidia-nat[mcp]`.
66+
6567
```bash
6668
nat info mcp
6769
```

‎docs/source/workflows/profiler.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,9 +34,14 @@ Will allow for features such as offline-replay or simulation of workflow runs wi
3434

3535
The NeMo Agent toolkit profiler requires additional dependencies not installed by default.
3636

37-
Install these dependencies by running the following command:
37+
Install these dependencies by running the following command from the root directory of the NeMo Agent toolkit repository:
3838
```bash
39-
uv pip install -e .[profiling]
39+
uv pip install -e ".[profiling]"
40+
```
41+
42+
If you are installing from a package, you need to install the `nvidia-nat[profiling]` package by running the following command:
43+
```bash
44+
uv pip install nvidia-nat[profiling]
4045
```
4146

4247
## Current Profiler Architecture

0 commit comments

Comments
 (0)