Skip to content

Conversation

@willkill07
Copy link
Member

@willkill07 willkill07 commented Aug 14, 2025

Summary

This introduces comprehensive Azure OpenAI support to the NeMo Agent Toolkit, enabling users to leverage Azure OpenAI services for both language models and embeddings across all supported AI frameworks.

Key Features Added

Core Azure OpenAI Support

  • New Azure OpenAI LLM Provider (azure_openai_llm.py)

    • Full configuration support with API key, endpoint, deployment, and model parameters
    • Temperature, top-p, seed, and retry configuration
    • Unified naming conventions with alias support for compatibility
  • New Azure OpenAI Embedder Provider (azure_openai_embedder.py)

    • Dedicated embedder implementation for Azure OpenAI embedding models
    • API key and endpoint configuration
    • Deployment-based model selection

Framework Integrations

Updated all major AI framework plugins to support Azure OpenAI:

  • LangChain - LLM and embedder integration
  • LlamaIndex - LLM and embedder integration with dependency updates
  • CrewAI - LLM integration
  • Semantic Kernel - LLM integration

Documentation Updates

  • Comprehensive LLM documentation with Azure OpenAI configuration examples
  • Embedder documentation with setup instructions and environment variables
  • Configuration examples showing proper Azure deployment setup
  • Environment variable guidance (AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT)

Development & Testing Improvements

  • Enhanced workflow commands with versioned dependency support
  • Improved testing coverage for LangChain and LlamaIndex agents
  • CI/CD updates for proper Azure OpenAI testing
  • Compatibility test fixes and logic improvements

Technical Implementation Details

Configuration Schema

llms:
  azure_openai_llm:
    _type: azure_openai
    azure_deployment: gpt-4o-mini
    
embedders:
  azure_openai_embedder:
    _type: azure_openai
    azure_deployment: text-embedding-3-small

Key Configuration Parameters

  • azure_deployment: Azure OpenAI deployment name
  • azure_endpoint: Base URL for Azure OpenAI endpoint
  • api_version: Azure OpenAI API version (default: 2025-04-01-preview)
  • api_key: Authentication key for Azure OpenAI services

Infrastructure Changes

  • Dependency updates in uv.lock with Azure OpenAI SDK integration
  • Workflow template updates for new Azure OpenAI workflows
  • CI/CD pipeline enhancements for Azure OpenAI testing
  • Registry updates to include new Azure OpenAI providers

Bug Fixes & Improvements

  • Unified naming conventions across embedder and LLM providers
  • Fixed compatibility test logic errors
  • Improved function naming consistency (e.g., azure_openai_embedder_model)

Testing

  • New test coverage for Azure OpenAI LLM and embedder providers
  • Integration tests for LangChain and LlamaIndex with Azure OpenAI
  • Compatibility testing improvements
  • CI/CD pipeline validation

Migration Notes

  • Existing workflows remain unchanged
  • New Azure OpenAI providers are additive and don't affect existing configurations
  • Environment variables follow Azure OpenAI naming conventions
  • Full backward compatibility maintained

This PR significantly expands the NeMo Agent Toolkit's cloud provider ecosystem, offering users enterprise-grade Azure OpenAI integration with comprehensive framework support and documentation.

Closes #533

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@willkill07 willkill07 added feature request New feature or request non-breaking Non-breaking change labels Aug 14, 2025
@willkill07 willkill07 changed the title Add Azure OpenAI LLM provider and client; Unify support for AWS Bedrock Add Azure OpenAI LLM provider and client; Improve support for AWS Bedrock Aug 15, 2025
@willkill07 willkill07 changed the title Add Azure OpenAI LLM provider and client; Improve support for AWS Bedrock Add Azure OpenAI LLM provider and client Aug 15, 2025
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
@willkill07 willkill07 self-assigned this Aug 18, 2025
Copy link
Contributor

@yczhang-nv yczhang-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@willkill07 willkill07 changed the title Add Azure OpenAI LLM provider and client Feature: Azure OpenAI LLM provider and client Aug 19, 2025
@avoroshilov
Copy link
Contributor

Running pytest --run_integration tests/nat/llm_providers -k "test_azure_openai_langchain_agent or test_azure_openai_minimal_agent" initially fails tests (as it should, due to LLMs not configured properly), but when I change to some valid configuration:

    llm_config = AzureOpenAIModelConfig(
        azure_deployment="gpt-4.1",
        azure_endpoint="...",
    )

Both tests pass:

(.venv) D:\Dev\aiq\nat-azure>pytest --run_integration tests/nat/llm_providers -k "test_azure_openai_langchain_agent or test_azure_openai_minimal_agent"
================================================= test session starts =================================================
platform win32 -- Python 3.12.8, pytest-8.4.1, pluggy-1.6.0
rootdir: D:\Dev\aiq\nat-azure
configfile: pyproject.toml
plugins: anyio-4.10.0, langsmith-0.4.14, nvidia-nat-test-1.3.dev13+gc515eb983, asyncio-0.24.0, cov-6.2.1, pytest_httpserver-1.1.3, pretty-1.2.0
asyncio: mode=Mode.AUTO, default_loop_scope=session
collected 8 items / 6 deselected / 2 selected

tests\nat\llm_providers\test_langchain_agents.py .                                                               [ 50%]
tests\nat\llm_providers\test_llama_index_agents.py .                                                             [100%]

================================================== warnings summary ===================================================
tests\conftest.py:109
  D:\Dev\aiq\nat-azure\tests\conftest.py:109: PytestRemovedIn9Warning: Marks applied to fixtures have no effect
  See docs: https://docs.pytest.org/en/stable/deprecations.html#applying-a-mark-to-a-fixture-function
    @pytest.mark.usefixtures("restore_environ")

tests/nat/llm_providers/test_langchain_agents.py::test_azure_openai_langchain_agent
  D:\Dev\aiq\nat-azure\.venv\Lib\site-packages\pydantic\_internal\_config.py:295: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.10/migration/
    warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
Results (19.29s):
         2 passed
         6 deselected
         2 warnings

@willkill07
Copy link
Member Author

willkill07 commented Aug 19, 2025

@avoroshilov thank you for testing!

This verifies that LangChain and Llama-Index integration works.

I have updated the PR's tests to use a supported model OOTB. Instead of explicitly setting azure_endpoint an environment variable is available and documented: AZURE_OPENAI_ENDPOINT.

@willkill07
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 78442f8 into NVIDIA:develop Aug 19, 2025
12 checks passed
saglave pushed a commit to snps-scm13/SNPS-NeMo-Agent-Toolkit that referenced this pull request Sep 2, 2025
This introduces comprehensive Azure OpenAI support to the NeMo Agent Toolkit, enabling users to leverage Azure OpenAI services for both language models and embeddings across all supported AI frameworks.

- **New Azure OpenAI LLM Provider** (`azure_openai_llm.py`)
  - Full configuration support with API key, endpoint, deployment, and model parameters
  - Temperature, top-p, seed, and retry configuration
  - Unified naming conventions with alias support for compatibility

- **New Azure OpenAI Embedder Provider** (`azure_openai_embedder.py`)
  - Dedicated embedder implementation for Azure OpenAI embedding models
  - API key and endpoint configuration
  - Deployment-based model selection

Updated all major AI framework plugins to support Azure OpenAI:
- **LangChain** - LLM and embedder integration
- **LlamaIndex** - LLM and embedder integration with dependency updates
- **CrewAI** - LLM integration
- **Semantic Kernel** - LLM integration

- **Comprehensive LLM documentation** with Azure OpenAI configuration examples
- **Embedder documentation** with setup instructions and environment variables
- **Configuration examples** showing proper Azure deployment setup
- **Environment variable guidance** (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`)

- **Enhanced workflow commands** with versioned dependency support
- **Improved testing coverage** for LangChain and LlamaIndex agents
- **CI/CD updates** for proper Azure OpenAI testing
- **Compatibility test fixes** and logic improvements

```yaml
llms:
  azure_openai_llm:
    _type: azure_openai
    azure_deployment: gpt-4o-mini

embedders:
  azure_openai_embedder:
    _type: azure_openai
    azure_deployment: text-embedding-3-small
```

- `azure_deployment`: Azure OpenAI deployment name
- `azure_endpoint`: Base URL for Azure OpenAI endpoint
- `api_version`: Azure OpenAI API version (default: `2025-04-01-preview`)
- `api_key`: Authentication key for Azure OpenAI services

- **Dependency updates** in `uv.lock` with Azure OpenAI SDK integration
- **Workflow template updates** for new Azure OpenAI workflows
- **CI/CD pipeline enhancements** for Azure OpenAI testing
- **Registry updates** to include new Azure OpenAI providers

- **Unified naming conventions** across embedder and LLM providers
- **Fixed compatibility test logic errors**
- **Improved function naming consistency** (e.g., `azure_openai_embedder_model`)

- New test coverage for Azure OpenAI LLM and embedder providers
- Integration tests for LangChain and LlamaIndex with Azure OpenAI
- Compatibility testing improvements
- CI/CD pipeline validation

- Existing workflows remain unchanged
- New Azure OpenAI providers are additive and don't affect existing configurations
- Environment variables follow Azure OpenAI naming conventions
- Full backward compatibility maintained

This PR significantly expands the NeMo Agent Toolkit's cloud provider ecosystem, offering users enterprise-grade Azure OpenAI integration with comprehensive framework support and documentation.

Closes NVIDIA#533

- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Will Killian (https://github.com/willkill07)
  - Andrey Voroshilov (https://github.com/avoroshilov)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: NVIDIA#643
@willkill07 willkill07 deleted the azure_openai branch October 23, 2025 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]: Azure OpenAI support

3 participants