.Net Bug: Ollama connector (still?) doesn't play nice with Aspire

Describe the bug

I have been using the SK Ollama connector with the .NET Aspire community toolkit Ollama client integration, expecting an easy, configuration-free, plug-and-play experience. Unfortunately, a dependency registration error happens at runtime.

The SK connector requires an OllamaApiClient to be registered when called without any parameter, but the .NET Aspire client integration registers its interface, IOllamaApiClient.

To Reproduce

Register both the SK Ollama connector and the .NET Aspire's community toolkit Ollama client integration in a generic host-powered app:

builder.AddOllamaApiClient("embedding-model");
builder.Services.AddOllamaTextEmbeddingGeneration();

In the .NET Aspire app host, register an Ollama model and reference it in your app:

var embeddingModel = builder.AddOllama("ollama")
    .AddModel("embedding-model", "bge-m3");

builder.AddProject<Projects.HelloSK02_ApiService>("apiservice")
    .WithReference(embeddingModel)
    .WaitFor(embeddingModel);

When requesting an ITextEmbeddingGenerationService in the app, you'll get the following exception:

No service for type 'OllamaSharp.OllamaApiClient' has been registered.

Expected behavior

I would have expected these two components to play nicely, for instance, the SK connector to fallback and look for IOllamaApiClient or the community Aspire component to register the concrete class. I'm not sure who's responsibility it is, but given that the SK connector is still alpha, I thought I would give it a try here.

Workaround

So far I've been doing this to bypass the registration problem:

builder.Services.AddSingleton(sp => (OllamaApiClient)sp.GetRequiredService<IOllamaApiClient>());

Platform

Language: C#
Sources:
- Microsoft.SemanticKernel.Connectors.Ollama: 1.37.0-alpha
- CommunityToolkit.Aspire.OllamaSharp: 9.1.0
AI model: bge-m3
IDE: Rider
OS: Windows

Additional context

It may be related to #10321, but the fix doesn't seem to cover this particular path.

Anthony Simmon (@asimmon)

I´m also facing this bug.

Thanks for the workaround Anthony Simmon (@asimmon)

Mark Wallace (@markwallace-microsoft)

Mark Wallace (@markwallace-microsoft) you just need to add this 3 methods at OllamaServiceCollectionExtensions class to solve this issue:

/// <summary>
/// Add Ollama Text Generation service to the kernel builder.
/// </summary>
/// <param name="services">The target service collection.</param>
/// <param name="connectionName">A name used to retrieve the connection string from the ConnectionStrings configuration section.</param>
/// <param name="serviceId">The optional service ID.</param>
/// <returns>The updated kernel builder.</returns>
public static IServiceCollection AddOllamaTextGeneration(this IServiceCollection services, string connectionName, string? serviceId = null)
{
    Verify.NotNull(services);

    return services.AddKeyedSingleton<ITextGenerationService>(serviceId, (serviceProvider, _) =>
    {
        var client = (OllamaApiClient)serviceProvider.GetKeyedService<IOllamaApiClient>(connectionName);
        return new OllamaTextGenerationService(client.SelectedModel, client);
    }
    );
}

/// <summary>
/// Add Ollama Chat Completion service to the kernel builder.
/// </summary>
/// <param name="services">The target service collection.</param>
/// <param name="connectionName">A name used to retrieve the connection string from the ConnectionStrings configuration section.</param>
/// <param name="serviceId">The optional service ID.</param>
/// <returns>The updated kernel builder.</returns>
public static IServiceCollection AddOllamaChatCompletion(this IServiceCollection services, string connectionName, string? serviceId = null)
{
    ArgumentException.ThrowIfNullOrEmpty(nameof(services));

    return services.AddKeyedSingleton<IChatCompletionService>(serviceId, (serviceProvider, _) =>
    {
        var loggerFactory = serviceProvider.GetService<ILoggerFactory>();

        IOllamaApiClient apiClient = serviceProvider.GetKeyedService<IOllamaApiClient>(connectionName);
        var builder = ((IChatClient)apiClient)
            .AsBuilder()
            .UseFunctionInvocation(loggerFactory, config => config.MaximumIterationsPerRequest = MaxInflightAutoInvokes);

        if (loggerFactory is not null)
        {
            builder.UseLogging(loggerFactory);
        }

        return builder.Build(serviceProvider).AsChatCompletionService(serviceProvider);
    });

    return services;
}

/// <summary>
/// Add Ollama Text Embeddings Generation service to the kernel builder.
/// </summary>
/// <param name="services">The target service collection.</param>
/// <param name="connectionName">A name used to retrieve the connection string from the ConnectionStrings configuration section.</param>
/// <param name="serviceId">The optional service ID.</param>
/// <returns>The updated kernel builder.</returns>
public static IServiceCollection AddOllamaTextEmbeddingGeneration(this IServiceCollection services, string connectionName, string? serviceId = null)
{
    ArgumentException.ThrowIfNullOrEmpty(nameof(services));

    services.AddKeyedSingleton<ITextEmbeddingGenerationService>(serviceId, (serviceProvider, _) =>
    {
        var loggerFactory = serviceProvider.GetService<ILoggerFactory>();

        IOllamaApiClient apiClient = serviceProvider.GetKeyedService<IOllamaApiClient>(connectionName);
        var builder = ((IEmbeddingGenerator<string, Embedding<float>>)apiClient).AsBuilder();

        if (loggerFactory is not null)
        {
            builder.UseLogging(loggerFactory);
        }

        return builder.Build(serviceProvider).AsTextEmbeddingGenerationService(serviceProvider);
    });

    return services;
}

And you can also improve OllamaKernelBuilderExtensions class too:

/// <summary>
/// Add Ollama Chat Completion service to the kernel builder.
/// </summary>
/// <param name="builder">The kernel builder.</param>
/// <param name="connectionName"></param>
/// <param name="serviceId">The optional service ID.</param>
/// <returns>The updated kernel builder.</returns>
public static IKernelBuilder AddOllamaChatCompletion(this IKernelBuilder builder, string connectionName, string? serviceId = null)
{
    ArgumentException.ThrowIfNullOrEmpty(nameof(builder));

    builder.Services.AddOllamaChatCompletion(connectionName, serviceId);

    return builder;
}

/// <summary>
/// Add Ollama Text Generation service to the kernel builder.
/// </summary>
/// <param name="builder">The kernel builder.</param>
/// <param name="connectionName"></param>
/// <param name="serviceId">The optional service ID.</param>
/// <returns>The updated kernel builder.</returns>
public static IKernelBuilder AddOllamaTextGeneration(this IKernelBuilder builder, string connectionName, string? serviceId = null)
{
    ArgumentException.ThrowIfNullOrEmpty(nameof(builder));
    builder.Services.AddOllamaTextGeneration(connectionName, serviceId);
    return builder;
}

/// <summary>
/// Add Ollama Text Embeddings Generation service to the kernel builder.
/// </summary>
/// <param name="builder">The kernel builder.</param>
/// <param name="connectionName"></param>
/// <param name="serviceId">The optional service ID.</param>
/// <returns>The updated kernel builder.</returns>
public static IKernelBuilder AddOllamaTextEmbeddingGeneration(this IKernelBuilder builder, string connectionName, string? serviceId = null)
{
    ArgumentException.ThrowIfNullOrEmpty(nameof(builder));
    builder.Services.AddOllamaTextEmbeddingGeneration(connectionName, serviceId);
    return builder;
}

.Net: Add missing Ollama Connector Aspire Friendly Extensions (#10775)

.Net: Add missing Ollama Connector Aspire Friendly Extensions (micros…

Navigation Menu

Repository navigation

.Net Bug: Ollama connector (still?) doesn't play nice with Aspire #10532

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

.Net Bug: Ollama connector (still?) doesn't play nice with Aspire #10532

Description

Activity

Caldas commented on Feb 21, 2025

Caldas commented on Feb 21, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions