-
Notifications
You must be signed in to change notification settings - Fork 481
Feature: Azure OpenAI LLM provider and client #643
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Andrey Voroshilov <[email protected]> Signed-off-by: Will Killian <[email protected]>
Co-authored-by: Will Killian <[email protected]> Signed-off-by: Andrey Voroshilov <[email protected]> Signed-off-by: Will Killian <[email protected]>
…rs and LLMs Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
Signed-off-by: Will Killian <[email protected]>
yczhang-nv
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
Running Both tests pass: |
|
@avoroshilov thank you for testing! This verifies that LangChain and Llama-Index integration works. I have updated the PR's tests to use a supported model OOTB. Instead of explicitly setting |
Signed-off-by: Will Killian <[email protected]>
9b0c5c1 to
76c87e1
Compare
Signed-off-by: Will Killian <[email protected]>
|
/merge |
This introduces comprehensive Azure OpenAI support to the NeMo Agent Toolkit, enabling users to leverage Azure OpenAI services for both language models and embeddings across all supported AI frameworks.
- **New Azure OpenAI LLM Provider** (`azure_openai_llm.py`)
- Full configuration support with API key, endpoint, deployment, and model parameters
- Temperature, top-p, seed, and retry configuration
- Unified naming conventions with alias support for compatibility
- **New Azure OpenAI Embedder Provider** (`azure_openai_embedder.py`)
- Dedicated embedder implementation for Azure OpenAI embedding models
- API key and endpoint configuration
- Deployment-based model selection
Updated all major AI framework plugins to support Azure OpenAI:
- **LangChain** - LLM and embedder integration
- **LlamaIndex** - LLM and embedder integration with dependency updates
- **CrewAI** - LLM integration
- **Semantic Kernel** - LLM integration
- **Comprehensive LLM documentation** with Azure OpenAI configuration examples
- **Embedder documentation** with setup instructions and environment variables
- **Configuration examples** showing proper Azure deployment setup
- **Environment variable guidance** (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`)
- **Enhanced workflow commands** with versioned dependency support
- **Improved testing coverage** for LangChain and LlamaIndex agents
- **CI/CD updates** for proper Azure OpenAI testing
- **Compatibility test fixes** and logic improvements
```yaml
llms:
azure_openai_llm:
_type: azure_openai
azure_deployment: gpt-4o-mini
embedders:
azure_openai_embedder:
_type: azure_openai
azure_deployment: text-embedding-3-small
```
- `azure_deployment`: Azure OpenAI deployment name
- `azure_endpoint`: Base URL for Azure OpenAI endpoint
- `api_version`: Azure OpenAI API version (default: `2025-04-01-preview`)
- `api_key`: Authentication key for Azure OpenAI services
- **Dependency updates** in `uv.lock` with Azure OpenAI SDK integration
- **Workflow template updates** for new Azure OpenAI workflows
- **CI/CD pipeline enhancements** for Azure OpenAI testing
- **Registry updates** to include new Azure OpenAI providers
- **Unified naming conventions** across embedder and LLM providers
- **Fixed compatibility test logic errors**
- **Improved function naming consistency** (e.g., `azure_openai_embedder_model`)
- New test coverage for Azure OpenAI LLM and embedder providers
- Integration tests for LangChain and LlamaIndex with Azure OpenAI
- Compatibility testing improvements
- CI/CD pipeline validation
- Existing workflows remain unchanged
- New Azure OpenAI providers are additive and don't affect existing configurations
- Environment variables follow Azure OpenAI naming conventions
- Full backward compatibility maintained
This PR significantly expands the NeMo Agent Toolkit's cloud provider ecosystem, offering users enterprise-grade Azure OpenAI integration with comprehensive framework support and documentation.
Closes NVIDIA#533
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
- Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.
Authors:
- Will Killian (https://github.com/willkill07)
- Andrey Voroshilov (https://github.com/avoroshilov)
Approvers:
- Yuchen Zhang (https://github.com/yczhang-nv)
URL: NVIDIA#643
Summary
This introduces comprehensive Azure OpenAI support to the NeMo Agent Toolkit, enabling users to leverage Azure OpenAI services for both language models and embeddings across all supported AI frameworks.
Key Features Added
Core Azure OpenAI Support
New Azure OpenAI LLM Provider (
azure_openai_llm.py)New Azure OpenAI Embedder Provider (
azure_openai_embedder.py)Framework Integrations
Updated all major AI framework plugins to support Azure OpenAI:
Documentation Updates
AZURE_OPENAI_API_KEY,AZURE_OPENAI_ENDPOINT)Development & Testing Improvements
Technical Implementation Details
Configuration Schema
Key Configuration Parameters
azure_deployment: Azure OpenAI deployment nameazure_endpoint: Base URL for Azure OpenAI endpointapi_version: Azure OpenAI API version (default:2025-04-01-preview)api_key: Authentication key for Azure OpenAI servicesInfrastructure Changes
uv.lockwith Azure OpenAI SDK integrationBug Fixes & Improvements
azure_openai_embedder_model)Testing
Migration Notes
This PR significantly expands the NeMo Agent Toolkit's cloud provider ecosystem, offering users enterprise-grade Azure OpenAI integration with comprehensive framework support and documentation.
Closes #533
By Submitting this PR I confirm: