[API Proposal]: AsyncConfigurationProvider

Background and motivation

As it currently stands, configuration sources have no way of being asynchronous (Highlighted in #36018).
These new set of types should allow for users to implement asynchronous configuration providers with no breaking changes to the way the existing API's function.

API Proposal

namespace Microsoft.Extensions.Configuration;

public interface IAsyncConfigurationBuilder : IConfigurationBuilder
{
    Task<IAsyncConfigurationBuilder> AddAsync(IAsyncConfigurationSource source);
    Task<IAsyncConfigurationRoot> BuildAsync();
}

public interface IAsyncConfigurationManager : IConfigurationManager, IAsyncConfigurationBuilder
{
}

public interface IAsyncConfigurationProvider : IConfigurationProvider
{
    Task LoadAsync();
}

public interface IAsyncConfigurationRoot : IConfigurationRoot
{
    Task ReloadAsync();
}

public interface IAsyncConfigurationSource : IConfigurationSource
{
    Task<IAsyncConfigurationProvider> BuildAsync(IAsyncConfigurationBuilder builder);
}

public static class IConfigurationManagerExtensions
{
    public static IAsyncConfigurationManager? AsAsyncManager(this IConfigurationManager configurationManager);
}

namespace Microsoft.Extensions.Configuration;

public class AsyncConfigurationBuilder : ConfigurationBuilder, IAsyncConfigurationBuilder
{
    public Task<IAsyncConfigurationBuilder> AddAsync(IAsyncConfigurationSource source);
    public Task<IAsyncConfigurationRoot> BuildAsync();
}

public class AsyncConfigurationProvider : ConfigurationProvider, IAsyncConfigurationProvider
{
    public virtual Task LoadAsync();
}

public class AsyncConfigurationRoot : IAsyncConfigurationRoot, IDisposable
{
    public AsyncConfigurationRoot(IList<IConfigurationProvider> providers);

    public IEnumerable<IConfigurationProvider> Providers;
    public string? this[string key];
    public IEnumerable<IConfigurationSection> GetChildren();
    public IChangeToken GetReloadToken();
    public IConfigurationSection GetSection(string key);
    public void Reload();
    public Task ReloadAsync();
    public void Dispose();
}

namespace Microsoft.Extensions.Configuration;

-public sealed class ConfigurationManager : IConfigurationManager, IConfigurationRoot, IDisposable
+public sealed class ConfigurationManager : IAsyncConfigurationManager, IAsyncConfigurationRoot, IDisposable
{
+    Task<IAsyncConfigurationBuilder> IAsyncConfigurationBuilder.AddAsync(IAsyncConfigurationSource source);
+    Task<IAsyncConfigurationRoot> IAsyncConfigurationBuilder.BuildAsync();
}

API Usage

Using AsyncConfigurationBuilder:
Calling builder.Build(); will only load synchronous sources.
Calling builder.BuildAsync() will load both sync & async sources.

internal class Program
{
    static async Task Main(string[] args)
    {
        var builder = new AsyncConfigurationBuilder();
        builder.Add(new MyConfigurationSource());
        await builder.AddAsync(new MyAsyncConfigurationSource());
        var configuration = await builder.BuildAsync();
    }
}


public class MyConfigurationSource : IConfigurationSource
{
    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        return new MyConfigurationProvider();
    }
}

public class MyConfigurationProvider : ConfigurationProvider
{
    public override void Load()
    {
        Data["Key"] = "Value";
    }
}

public class MyAsyncConfigurationSource : IAsyncConfigurationSource
{
    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        throw new NotImplementedException();
    }

    public async Task<IAsyncConfigurationProvider> BuildAsync(IAsyncConfigurationBuilder builder)
    {
        await Task.Delay(200); //Simulate some IO work.
        return new MyAsyncConfigurationProvider();
    }
}

public class MyAsyncConfigurationProvider : AsyncConfigurationProvider
{
    public override async Task LoadAsync()
    {
        await Task.Delay(1000); //Simulate some IO work.
        Data["Key"] = "Value";
    }
}

Using one of the generic host API's:

internal class Program
{
    static async Task Main(string[] args)
    {
        HostApplicationBuilder hostBuilder = Host.CreateApplicationBuilder(args);
        hostBuilder.Configuration.Add(new MyConfigurationSource());

        var asyncConfigManager = hostBuilder.Configuration.AsAsyncManager();
        if (asyncConfigManager is not null)
        {
            await asyncConfigManager.AddAsync(new MyAsyncConfigurationSource());
        }

        var host = hostBuilder.Build();
        host.Run();
    }
}


public class MyConfigurationSource : IConfigurationSource
{
    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        return new MyConfigurationProvider();
    }
}

public class MyConfigurationProvider : ConfigurationProvider
{
    public override void Load()
    {
        Data["Key"] = "Value";
    }
}

public class MyAsyncConfigurationSource : IAsyncConfigurationSource
{
    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        throw new NotImplementedException();
    }

    public async Task<IAsyncConfigurationProvider> BuildAsync(IAsyncConfigurationBuilder builder)
    {
        await Task.Delay(200); //Simulate some IO work.
        return new MyAsyncConfigurationProvider();
    }
}

public class MyAsyncConfigurationProvider : AsyncConfigurationProvider
{
    public override async Task LoadAsync()
    {
        await Task.Delay(1000); //Simulate some IO work.
        Data["Key"] = "Value";
    }
}

Alternative Designs

None.

The alternative design looks better but did you also look at all of the consumers of the Load method to understand what else would need to be changed to async to make this work in practice? What about the generic hosting APIs?

@jackbond

The current model is fundamentally incompatible with that @jackbond and if we do that then we should make a different type. The current configuration assumes you can access all of the keys in memory. What you're asking for is asynchronous access to individual keys. If we go that route then we should build a new API that doesn't overlap with the existing configuration provider.

The alternative design looks better but did you also look at all of the consumers of the Load method to understand what else would need to be changed to async to make this work in practice? What about the generic hosting APIs?

Ok thanks for the feedback, I'm on xmas holiday at the moment, ill revisit this in the new year.

I don’t understand why you need asynchronous access to individual values and how did you get all of the keys in memory. IMO that’s another interface and a different API completely.

Right, and I don't have a problem with having a new abstraction for stores like keyvault and azure app configuration. Seems very much like #36035.

If that's the goal of asynchrony, then I would opt for not polluting the existing API and designing a new one.

For example, even reading a local json config file should be an async operation, it's doing async IO after all.

Loading everything up front (async or sync) is fine, and why the original API proposal (though I would change the existing interfaces and classes instead of making new ones) is easier to reason about with the existing configuration model.

Anything that requires asynchronous access to values should be a different abstraction. So maybe this IAsyncConfigurationProvider proposal should be about accessing values asynchronously. Instead of loading all keys up front somehow (Do most of those systems have a cheap way to query all keys? Do we even want that?).

I think these providers would be inappropriate for things like configuration binding, which want to know all of the keys (in the entire object hierarchy) before binding. This API would be used directly in the places that need access to individual values at the moment they are queried, without caching from the source.

Well I can't speak for all providers, but Azure.Data.AppConfiguration.ConfigurationClient has a method GetConfigurationSettingsAsync which allows you to read all the keys. It also loads all the values, but some of those values are merely links to the underlying KeyVault. We delay load those, and use .Result (which always bother me) in TryGet.

Right, to make general abstractions, we need to make sure the pattern holds broadly. I'm making an assumption that enumerating keys without values isn't common, but if we can find places where it is, then those assumptions can change.

But at the same time, I do miss the old Microsoft days when the attitude was, "If you don't support our pattern, it's your problem" :)

That's ironic because the reason the azure configuration provider and keyvault provider is not ideal today is because the existing abstraction doesn't work well with the existing model😄

😄 True enough, although this feels more like an inhouse dispute, where IConfiguration simply hasn't received the modernization attention it deserves.

As one of the creators of this abstraction, that's not the case.

For example, how is the FileInfo class setting its Exists property without sneaking in an async operation?

FileProviders was never really meant to exist outside of the ASP.NET Core use case of being a readonly file system for web applications. I don't think it has any of the appropriate features for a general purpose file system abstraction (as much as people want it to be).

ServiceCollection should support BuildServiceProviderAsync

This doesn't make any sense to me and would make everything less efficient.

It's hard to do async all the way down when some libraries stop you dead in your tracks.

Agreed. It's a pain, but everything shouldn't be async or we can just throw performance out the window.

How about this path

Introduce a new AsyncConfigurationBuilder alternative to the current subsystem. It can still implement the same IConfigurationBuilder interface, but it could also be extended with the necessary async method variants.

A path where users can explicitly use the async variant instead of the sync one, with the host.

The host detects if the async one is in use, and uses its async methods rather than the sync methods.

The goal here is to think of a way to allow an async alternative builder / provider to be defined and swapped into use for those who need it.

The async builder, could work with a new (extended?) async provider definition.

The issue is point 3.
We need to add Task based methods to the IConfigurationManager but doing so would be a breaking change.
So the only available option is to create a superseding interface that implements IConfigurationManager and adds the async based methods. Lets call this interface IAsyncConfigurationManager.
We now want to be able to use this in the generic host APIs. If we take a look at IHostApplicationBuilder we can see that it defines the configuration via a property of the type IConfigurationManager. We cant change or add to this interface as it would also be a breaking change.

TLDR;

Cant change or add to: IConfigurationManager, can only inherit and extend.
Generic host API's such as IHostApplicationBuilder expose their configuration building via IConfigurationManager.
The generic host APIs cant be changed or added to, can only inherit and extend.
inheriting and extending it would mean carbon copies of all 3 API's just to allow async configuration. I don't think that is a good user experience.

Sounds like we just need to introduce an AsyncHost and start over all the way down the Host stack.

Which is probably a good idea, because it's really dumb that HostBuilder and WebHostBuilder don't implement the same interface.

WebHostBuilder is not getting updates, WebApplicationBuilder is what we’ve been evolving, similarly Host.CreateApplicationBuilder (those implement the same interface )

Id like feedback on this idea:

We move ahead with IAsyncConfigurationBuilder and the other async interface types for the direct usage of configuration builder and the ability to register, and asynchronously load providers.

We also leave the generic host API's as they are (exposing IConfigurationBuilder).

Use case 1 - Using ConfigurationBuilder directly:
Users now instanciate AsyncConfigurationBuilder and load their synchronous and asynchronous providers as expected and build returns an IAsyncConfigurationRoot (an extension of ConfigurationRoot with the ability to asyncronously reload the data soruces).

Use case 2 - Using one of the generic host API's:
Users instanciate the relevant host builder.
They can either call .Configuration.Add(...) to add their synchronous provider types, or they can cast the host configuration manager to IAsyncConfigurationManager to access the relevant async API's

This is what I have in my current PoC:

Granted, I'm fully aware that casting the ConfigurationManager isnt discoverable, expected or a nice user experience, but it does solve all of the breaking change issues without having to provide a set of new generic host API's.
I'm hoping that seeing as its more of an advanced use case, that if its documented well, it will be sufficient?

Some thoughts/questions:

Instead of casting ApplicationHistBuilder.Cinfiguration, could we add an IAsyncConfigurationBuilder AsyncConfiguration property? The two properties could even point to the same instance potentially. Still a slightly awkward API, but much more discoverable than casting IMO.
Does AsyncConfigurationBuilder.AddAsync() really need to be async? I thought all I/O happened when you actually build the builder; and adding a configuration source would basically just synchronously add it to some private collection.

Some thoughts/questions:

Instead of casting ApplicationHistBuilder.Cinfiguration, could we add an IAsyncConfigurationBuilder AsyncConfiguration property? The two properties could even point to the same instance potentially. Still a slightly awkward API, but much more discoverable than casting IMO.

Does AsyncConfigurationBuilder.AddAsync() really need to be async? I thought all I/O happened when you actually build the builder; and adding a configuration source would basically just synchronously add it to some private collection.

I'm happy to have it as an option and see what others think, I don't feel strongly either way but I think I would prefer the cast as A duplicate property would be confusing. If I were a user discovering them, I personally would assume that the two properties were isolated from each other. Sorry scratch what I said, adding a property to the existing interface isnt possible as it would be a breaking change. It means that anyone who has their own type that implements the interface now has to implement said property. Maybe, we could introduce an extension method on the existing property type to expose the new interface? Something like .AsAsyncManager()

await host.Configuration.AsAsyncManager().AddAsync(...);

Unfortunately yes. Here's why:
- When using the AsyncConfigurationBuilder directly, its not needed as the data is loaded upon the BuildAsync() call.
- When using ConfigurationManager (the type that the generic host API's expose), the manager itself is its own builder and configuration root. This means that when you add a data source, it immediate loads the data upon the .Add/.AddAsync call. This is done to let the user access the configuration values when configuring other parts of the host before it is built (think DI etc).

Example:

        //Using the config before calling build.
        HostApplicationBuilder hostBuilder = Host.CreateApplicationBuilder(args);
        hostBuilder.Configuration.Add(...);

        hostBuilder.Services.Configure<MyType>(() => hostBuilder.Configuration...);

        var host = hostBuilder.Build();

It means that anyone who has their own type that implements the interface now has to implement said property

Not necessarily, if we use a default property implementation on the interface. Though idk if the implementation would really be simple enough to be "default". Your extension method idea would work too.

the manager itself is its own builder and configuration root. This means that when you add a data source, it immediate loads the data

I see, thanks for clarifying. I suppose ConfigurationManager.Add() could be modified to call Add() and then wait on BuildAsync(), rather than waiting on an .AddAsync() call. This would not be a breaking API change, just a change to internal logic. Though that would mean potentially repeated calls to AsyncConfigurationBuilder.BuildAsync() and idk if that's allowed.

It means that anyone who has their own type that implements the interface now has to implement said property

Not necessarily, if we use a default property implementation on the interface. Though idk if the implementation would really be simple enough to be "default". Your extension method idea would work too.

the manager itself is its own builder and configuration root. This means that when you add a data source, it immediate loads the data

I see, thanks for clarifying. I suppose ConfigurationManager.Add() could be modified to call Add() and then wait on BuildAsync(), rather than waiting on an .AddAsync() call. This would not be a breaking API change, just a change to internal logic. Though that would mean potentially repeated calls to AsyncConfigurationBuilder.BuildAsync() and idk if that's allowed.

Note that in my PoC, I've also added IAsyncConfigurationSource to allow for asynchronous construction of the provider. The await not only loads the data from the source, but it also constructs the source provider asynchronously and adds it to the managers internal sources. Part of this feature extension is to allow users to implement the interface on their own types and asynchronously construct a source if needed.

        private async Task AddSourceAsync(IAsyncConfigurationSource source)
        {
            IAsyncConfigurationProvider provider = await source.BuildAsync(this).ConfigureAwait(false);

            provider.Load();
            await provider.LoadAsync().ConfigureAwait(false);
            _changeTokenRegistrations.Add(ChangeToken.OnChange(provider.GetReloadToken, RaiseChanged));

            _providerManager.AddProvider(provider);
            RaiseChanged();
        }

If we forget about IAsyncConfigurationSource for a second. You mean it would look something like this?

        HostApplicationBuilder hostBuilder = Host.CreateApplicationBuilder(args);
        hostBuilder.Configuration.Add(new MyAsyncConfigurationSource());

        await hostBuilder.Configuration.BuildAsync(); //Build before usage.
        hostBuilder.Services.Configure<MyType>(() => hostBuilder.Configuration...); //Usage

        var host = hostBuilder.Build();

If so, I think that would lead to confusion as adding an IConfigurationSource would result in available access straight away, but adding an IAsyncConfigurationSource would mean your data was missing until you called BuildAsync on the configuration.

I've updated the proposal to reflect what has been discussed so far.

@davidfowl

WebHostBuilder is not getting updates, WebApplicationBuilder is what we’ve been evolving, similarly Host.CreateApplicationBuilder (those implement the same interface )

I will look into this, thanks @davidfowl! Maybe my Host-based extensions have all been supporting the wrong interfaces.

If no one has any comments, can this be marked as ready for review?

@rosebyte

@rosebyte Did you mean to mark this one as .NET 11, or the issue above that you marked as a duplicate?

@rosebyte Did you mean to mark this one as .NET 11, or the issue above that you marked as a duplicate?

Its probably that they don't want to mark this issue as planned for .NET 11 as they already have a separate design proposal and don't want to confuse people. I noticed Microsoft.Testing.Platform already has a IConfigurationProvider.LoadAsync() so my guess is that this issue gets closed and the .NET team add their own mechanisms (which I have no issues with).

Either way, if this gets addressed in .NET 11 then fantastic. 🚀

@KieranDevvs

Hello, @KieranDevvs is right. We’ll keep this issue, as this is the API proposal, and it’s better to have a single issue for the feature. I can’t promise it will be in net11.0, as it’s already March and priorities can change.

Navigation Menu

Repository navigation

[API Proposal]: AsyncConfigurationProvider #79193

Background and motivation

API Proposal

API Usage

Alternative Designs

21 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[API Proposal]: AsyncConfigurationProvider #79193

Description

Background and motivation

API Proposal

API Usage

Alternative Designs

Activity

davidfowl commented on Dec 3, 2022

davidfowl commented on Dec 20, 2022

KieranDevvs commented on Dec 21, 2022

davidfowl commented on Dec 23, 2022

davidfowl commented on Dec 24, 2022

davidfowl commented on Dec 24, 2022

davidfowl commented on Dec 24, 2022

davidfowl commented on Dec 24, 2022

davidfowl commented on Dec 24, 2022

21 remaining items

KieranDevvs commented on Apr 11, 2025

robertmclaws commented on Apr 12, 2025

davidfowl commented on Apr 12, 2025

KieranDevvs commented on Apr 12, 2025

Rabadash8820 commented on Apr 12, 2025

KieranDevvs commented on Apr 12, 2025

Rabadash8820 commented on Apr 12, 2025

KieranDevvs commented on Apr 12, 2025

KieranDevvs commented on Apr 13, 2025

robertmclaws commented on Apr 13, 2025

KieranDevvs commented on Apr 15, 2025

robertmclaws commented on Mar 2, 2026

KieranDevvs commented on Mar 2, 2026

rosebyte commented on Mar 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions