Concurrent insert can cause AsSplitQuery to fail to populate unrelated child collections

When materialising entities via a query with split querying enabled, it's possible to encounter missing child entities purely by concurrently inserting unrelated data into the same tables. This can happen even when there have been no changes to the affected parent/child rows and ordering of the returned rows is completely stable. EF runs the expected queries and it does appear to successfully retrieve the child entities from the database, but it discards them.

Minimal consistent repro that sneakily inserts an unrelated record in between executing the first and second split queries (.NET 8.0, EF Core v8.0.5, SQL Server):

Program.cs:

using Microsoft.EntityFrameworkCore;

namespace EFTest;

public class Widget
{
    public Widget(int id, IEnumerable<Child> children)
    {
        Id = id;
        Children = children.ToHashSet();
    }

    // EF calls this when materialising the Widget
    private Widget(int id)
    {
        Id = id;

        // Strategically add another Widget between loading Widget 1 and loading its Children
        using var dbContext = new MyDbContext();
        dbContext.Widgets.Add(new(id: 2, children: [new("New")]));
        dbContext.SaveChanges();
    }

    public int Id { get; private init; }

    public HashSet<Child> Children { get; private init; } = [];

    public override string ToString() =>
        $"Widget {Id} with children: [{string.Join(',', Children.Select(c => c.Name))}]";
}

public record Child(string Name)
{
    public int WidgetId { get; private set; }
}

public class MyDbContext() : DbContext(new DbContextOptionsBuilder().UseSqlServer(ConnectionString).Options)
{
    private const string ConnectionString = "<SQL Server connection string goes here>";

    public DbSet<Widget> Widgets { get; private init; } = null!;

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Widget>(e =>
        {
            e.Property(widget => widget.Id).ValueGeneratedNever();

            e.OwnsMany<Child>(widget => widget.Children);
        });
    }
}

public static class Program
{
    public static async Task Main()
    {
        await using (var dbContext = new MyDbContext())
        {
            await dbContext.Database.EnsureCreatedAsync();
            await dbContext.Widgets.ExecuteDeleteAsync();

            dbContext.Widgets.Add(new(id: 1, children: [new("A"), new("B"), new("C")]));

            await dbContext.SaveChangesAsync();
        }

        await using (var dbContext = new MyDbContext())
        {
            var widgets = await dbContext.Widgets.AsSplitQuery().OrderByDescending(w => w.Id).ToListAsync();

            Console.WriteLine(string.Join('\n', widgets));
        }
    }
}

Add your connection string (MyDbContext.ConnectionString), generate an initial migration (dotnet ef migrations add Initial), then run the program.

Received output:

Widget 1 with children: []

Expected output:

Widget 1 with children: [A,B,C]

The repro no longer exhibits the issue if you remove the OrderByDescending(w => w.Id), but you can make the issue reappear by simply giving "Widget 2" an ID of 0. It seems that the moment the "parentless" (from the split query's perspective, anyhow) child row is encountered during enumeration, EF stops stitching together child entities with parent entities entirely. This means that depending on the order of the results you may experience child entities missing for all, or some, or no parent entities.

I acknowledge that the docs say that for split querying data consistency is not guaranteed without using serialisable/snapshotting transactions, but I found this scenario a little too unexpected and couldn't find any existing reports of this same scenario, so I thought I'd at least ask whether this is expected. The thing that makes this so unexpected is that the right data is being loaded by EF and it has enough information to stitch things together, but the presence of an additional row causes it not to do so. I'd expect these results if I was deleting "Widget 1" in between the first and second query such that EF sees the parent but no children, and I was fully willing to accept those consequences when not using serialisable/snapshotting transactions, but I didn't expect that I could make "Widget 1"s children disappear (for a single query) by merely inserting "Widget 2" at the wrong time even though "Widget 1" and all its related data was untouched the whole time.

Provider and version information

EF Core version: v8.0.5
Database provider: Microsoft.EntityFrameworkCore.SqlServer (I was able to reproduce the same behaviour with Npgsql.EntityFrameworkCore.PostgreSQL as well)
Target framework: .NET 8.0
Operating system: I've reproduced this behaviour on Windows, Linux & macOS
IDE: I've reproduced this behaviour in both Visual Studio and Rider

Sorry for taking so long to answer this.

As you noted, the moment you're doing split queries without serializable/snapshot isolation, you're exposed to all kinds of inconsistent data scenarios; this is one of the reasons split queries aren't the default loading mode. But I do agree that the behavior you're describing sounds odd and could indicate a bug in EF's materialization logic... I'll place this in the backlog for further investigation, but it's unlikely we'll get to this any time soon, as similar such inconsistencies may appear in any case (even if a bit less unexpected).

In EF Core 2.1, which used split queries by default, there was not such problem.

I'm not sure at this point exactly how things worked in EF Core 2.1, but what you're reporting really is the expected behavior when doing split query; without a higher isolation level (e.g. Snapshot), there are no consistency guarantees, so this sort of error can occur. As this is a concurrency issue and therefore highly sensitive to timing, my guess is that EF Core 2.1 had exactly the same behavior, but the issue simply didn't manifest.

Out of curiosity, can you please post the fully SQLs for the queries generated by both EF Core 2.1 and 8.0, so we can compare them?

@adebelyi thanks for the added detail.

We can see, that when 2.1 loading child collections, it does not include Menu id in select output as 8.0 do

This is something that isn't clear to me (but as I said, I'm unfamiliar with how EF 2.1 split queries worked). I'm probably missing something, but the principal (Menu) key must be projected in the 2nd split query which returns the children, otherwise how is EF to know which child belongs to which principal?

I do agree that a more thorough investigation is needed here at some point, and that there may be a possible bug in EF (at the very least the odd behavior originally signaled above).

I just wrote another repro for this after it caused us a production outage.
https://github.com/torutek/EfTest
It inserts a new row between the queries of a split query (this is hooked in the LogTo method in EfTestContext).

Data looks like Person *-* Team -* Vehicle.

Logs look like:
EF Queries out Person
EF Queries out Team
~ We race and insert a new team with a new vehicle ~
Ef Queries out Vehicle

I did a little bit of stepping through EF to see what is happening.
https://github.com/dotnet/efcore/blob/main/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.ClientMethods.cs#L566

It looks like in here when it reads the Vehicle that is on a Team it doesn't know about it continues iterating through the teams query results looking for the matching team, but never finding a match. This causes it to discard all vehicles from that point onwards in the query.

This feels like something that should be able handled better by EF, this certainly isn't the behavior we expected based on the documentation.
This makes SplitQuery very dangerous to use without using snapshots. You risk not receiving a bunch of unrelated data from a query because new data was inserted at the same time.

Is there any indication of when this issue occurs? I would love to be able to detect how often this happens so I can the transaction tradeoff

I have a similar problem where FirstOrDefaultAsync with split query return empty children
While ToListAsync return correct result
Same query and same include but different result
I have been trying to reproduce in a minimal example but it is not happening there only in our project which is too complex to share
Is there any suggestion or maybe someone had a similar problem

@roji

Has there been any further investigation of this issue? It doesn’t appear that this behavior is typical for any type of inconsistency one would expect to be intrinsic for this type of scenario (data changing in between split queries). Given that split queries generally perform better than single queries, and the workaround is to manually wrap every query in a transaction, which is impractical if one wants to enable globally, it feels like it could be worth understanding a little more clearly @roji

Please correct me if this is an inaccurate representation of the current situation.

See below for a possible fix plan.

Note that a full fix here rlies on #23775, which would provide us with comparability. Note also that we have plans for a possible rewrite of our shaper architecture (to make it non-generated), it may be better to tackle this after that (unsure).

Fix plan

Fix: Split query silently drops child collections on concurrent insert (#33826)

Problem

When using AsSplitQuery(), a concurrent insert of an unrelated parent+children between the
parent query and child query execution causes all child entities for all remaining parents
to be silently dropped.

The correct data is fetched from the database — EF just discards it during materialization.

Root Cause

PopulateSplitIncludeCollection iterates through the child data reader, which is shared across
all parents and forward-only. When it encounters a child row whose parent identifier doesn't match
the current parent, it sets HasNext = true (preserving the row for the next parent) and returns.

The problem is that this preserved row may be an orphan — a child of a concurrently-inserted
parent that doesn't exist in the parent result set. Since no future parent will ever match it, the
cursor gets permanently stuck on the orphan row: every subsequent parent sees the same
non-matching row, sets HasNext = true, and returns with 0 children.

Example:

Parents (ASC):  Blog(10), Blog(30)
Children (ASC): Post(BlogId=5_orphan), Post(BlogId=10), Post(BlogId=10), Post(BlogId=30)

PopulateSplit(Blog 10): reads Post(BlogId=5), no match → HasNext=true → 0 children ✗
PopulateSplit(Blog 30): HasNext=true, Post(BlogId=5), no match → HasNext=true → 0 children ✗
                        Cursor stuck on orphan — all children lost!

Solution: Ordering-Aware Orphan Skipping

Both parent and child queries are ordered by the same key (user orderings + identifier tiebreakers,
injected via AppendOrderingInternal in SelectExpression). Parents are processed in this order.

When a non-matching child row is encountered, we compare its full ordering key with the current
parent's ordering key:

Child sorts BEFORE current parent → the child's parent would have appeared earlier in the
parent stream. Since we've already passed it and didn't see it, this is an orphan. Skip it.
Child sorts AFTER current parent → the child's parent may appear later. Save it
(HasNext = true) — this is the existing behavior.

This is essentially a merge-join between the parent stream and child stream, both ordered by the
same key.

Parents (ASC):  Blog(10), Blog(30)
Children (ASC): Post(BlogId=5_orphan), Post(BlogId=10), Post(BlogId=10), Post(BlogId=30)

PopulateSplit(Blog 10):
  Post(BlogId=5): no match, ordering key 5 < 10 → sorts before → SKIP orphan
  Post(BlogId=10): match → add ✓
  Post(BlogId=10): match → add ✓
  Post(BlogId=30): no match, ordering key 30 > 10 → sorts after → save
PopulateSplit(Blog 30):
  Saved Post(BlogId=30): match → add ✓

No buffering. No extra queries. Near-zero overhead (comparison only on non-matching rows, which
are rare without concurrent inserts).

Why compare the full ordering key (not just identifiers)?

When the user specifies OrderBy on a non-identifier column (e.g., .OrderBy(b => b.Name)), the
final ordering is ORDER BY Name ASC, Id ASC. Comparing only Id would give wrong results:

Parents (by Name ASC, Id ASC):  Blog(Id=10, Name="Alpha"), Blog(Id=5, Name="Charlie")
Children (same ordering):       Post(BlogId=10), Post(BlogId=5)

Processing Blog 10: sees Post(BlogId=5), comparing Id: 5 < 10 → would skip!
But Blog 5 sorts AFTER Blog 10 (because "Charlie" > "Alpha"). This is a REAL future parent.

Comparing the full ordering key ("Charlie", 5) vs ("Alpha", 10) correctly determines that
Blog 5 sorts after Blog 10, so the row should be saved, not skipped.

Ordering comparison mechanism

Use Comparer<object>.Default.Compare() which delegates to IComparable on the boxed values.
This works for all common key types (int, long, Guid, string, DateTime, etc.). For types that
don't implement IComparable, fall back to the existing behavior (save via HasNext = true).

A separate issue (#23775) tracks adding IComparable support to ValueComparer; once that
lands, this code can be updated to leverage it.

Implementation

1. Add ordering key data to `SplitCollectionInfo`

File: src/EFCore.Relational/Query/SqlExpressions/SelectExpression.Helper.cs

Add to the SplitCollectionInfo struct:

ParentOrderingKey (Expression) — extracts ordering key from parent data reader
ChildOrderingKey (Expression) — extracts ordering key from child data reader
OrderingDirections (bool[]) — IsAscending per ordering column

Generate these alongside the existing identifier accessors in SelectExpression.ApplyProjection,
using a new GetOrderingKeyAccessor helper (similar to GetIdentifierAccessor but operating on
SelectExpression.Orderings). Only include the parent-level orderings (not inner collection
orderings that are appended later). Capture the parent ordering count before inner orderings are
appended to distinguish them.

2. Add ordering key data to `RelationalSplitCollectionShaperExpression`

File: src/EFCore.Relational/Query/RelationalSplitCollectionShaperExpression.cs

Add matching properties:

ParentOrderingKey (Expression)
ChildOrderingKey (Expression)
OrderingDirections (bool[])

Update the constructor, VisitChildren, and Update to include these new properties.

3. Add `ParentOrderingKey` to `SplitQueryCollectionContext`

File: src/EFCore.Relational/Query/Internal/SplitQueryCollectionContext.cs

Add a ParentOrderingKey property (object[]) alongside the existing ParentIdentifier.

4. Update `InitializeSplitIncludeCollection` / `InitializeSplitCollection`

File: ...ShaperProcessingExpressionVisitor.ClientMethods.cs

These methods are called once per parent, reading from the parent data reader.

Accept a new parentOrderingKey parameter (Func<QueryContext, DbDataReader, object[]>)
Extract the parent's ordering key and store it in SplitQueryCollectionContext.ParentOrderingKey

5. Update code-gen for `InitializeSplit*` calls

File: ...ShaperProcessingExpressionVisitor.cs

In the expression generation code (around lines 1080-1102 for Include, 1409-1423 for Collection):

Compile ParentOrderingKey from RelationalSplitCollectionShaperExpression into a lambda
Pass it as an additional argument to the InitializeSplit* method calls

6. Update `PopulateSplitIncludeCollection[Async]` / `PopulateSplitCollection[Async]`

File: ...ShaperProcessingExpressionVisitor.ClientMethods.cs

All 4 methods get two new parameters:

childOrderingKey (Func<QueryContext, DbDataReader, object[]>)
orderingDirections (bool[])

In the while loop, when CompareIdentifiers returns false:

if (!CompareIdentifiers(identifierValueComparers, parentIdentifier, childId))
{
    // Compare full ordering keys to determine if child is an orphan or a future parent's child
    var cmp = CompareOrderingKeys(
        splitQueryCollectionContext.ParentOrderingKey,
        childOrderingKey(queryContext, dbDataReader),
        orderingDirections);

    if (cmp > 0)
    {
        // Child sorts before current parent → orphan or already-processed parent → skip
        dataReaderContext.HasNext = null;
        continue;
    }

    // Child sorts after (or equal, which shouldn't happen) → save for next parent
    dataReaderContext.HasNext = true;
    return;
}

Add a new static helper:

private static int CompareOrderingKeys(object[] left, object[] right, bool[] ascending)
{
    for (var i = 0; i < left.Length; i++)
    {
        var cmp = Comparer<object>.Default.Compare(left[i], right[i]);
        if (cmp != 0)
        {
            return ascending[i] ? cmp : -cmp;
        }
    }
    return 0;
}

7. Update code-gen for `PopulateSplit*` calls

File: ...ShaperProcessingExpressionVisitor.cs

In the expression generation code (around lines 1107-1143 for Include, 1432-1461 for Collection):

Compile ChildOrderingKey from RelationalSplitCollectionShaperExpression into a lambda
Pass childOrderingKeyLambda and orderingDirections as additional arguments

8. Update MethodInfo references

File: ...ShaperProcessingExpressionVisitor.cs

The PopulateSplitIncludeCollectionMethodInfo, PopulateSplitIncludeCollectionAsyncMethodInfo,
PopulateSplitCollectionMethodInfo, PopulateSplitCollectionAsyncMethodInfo static fields
reference the method signatures. These need to be updated to match the new signatures (additional
parameters).

Similarly for InitializeSplitIncludeCollectionMethodInfo and
InitializeSplitCollectionMethodInfo.

9. Handle `IComparable` unavailability gracefully

In CompareOrderingKeys, wrap the Comparer<object>.Default.Compare() call to handle types that
don't implement IComparable. If comparison throws, return 0 (which falls through to the existing
HasNext = true behavior — no skip, same as current code).

10. Add regression test

File: test/EFCore.Relational.Specification.Tests/Query/AdHocQuerySplittingQueryTestBase.cs

Add a test that:

Seeds a parent entity (e.g., Blog(Id=10)) with children (Posts)
Uses a DbCommandInterceptor to inject a concurrent insert (e.g., Blog(Id=1) with a Post)
just before the child query executes
Executes Blogs.Include(b => b.Posts).AsSplitQuery().ToListAsync()
Asserts that Blog 10 still has its posts

Add a second test variant with OrderByDescending(b => b.Id) (the original issue reporter's
scenario).

Consider a third variant with OrderBy(b => b.Name) (non-identifier ordering) to cover the full
ordering key comparison path.

Files to Modify

File	Changes
`SelectExpression.Helper.cs`	Add ordering key fields to `SplitCollectionInfo`
`SelectExpression.cs`	Generate ordering key accessors in `ApplyProjection`; add `GetOrderingKeyAccessor` helper
`RelationalSplitCollectionShaperExpression.cs`	Add ordering key properties, update constructor/Update/VisitChildren
`SplitQueryCollectionContext.cs`	Add `ParentOrderingKey` property
`ShaperProcessingExpressionVisitor.cs`	Update code-gen for Initialize/Populate calls with ordering key lambdas
`ShaperProcessingExpressionVisitor.ClientMethods.cs`	Update Initialize/Populate method signatures + orphan skip logic
`AdHocQuerySplittingQueryTestBase.cs`	Add regression tests

Performance Considerations

Zero overhead in the common case (no concurrent inserts): the ordering comparison only
executes when CompareIdentifiers returns false, which means a non-matching child row was
encountered. In normal operation, non-matching rows only appear at parent boundaries (which
triggers the existing HasNext = true path with one additional comparison call).
Ordering key extraction overhead: the childOrderingKey lambda is called only on
non-matching rows. The parentOrderingKey lambda is called once per parent during
InitializeSplit* (same as parentIdentifier).
Projection overhead: ordering columns are added to the child query's SELECT. Some of these
(identifier columns) are already projected. User ordering columns (e.g., Name) may add
additional projected columns. This is a small overhead on the SQL side.
No buffering, no extra queries, no delayed TTFB.

Edge Cases

No user ordering (default): ordering is solely on identifier columns ASC. The ordering key
and identifier are the same columns. The comparison is trivially correct.
User ordering on identifier DESC: AppendOrderingInternal deduplicates, so DESC is
preserved. The orderingDirections array captures this correctly.
User ordering on non-identifier column: full ordering key comparison handles this correctly
(see "Why compare the full ordering key" section above).
Composite primary keys: ordering key includes all PK columns. Lexicographic comparison
respects the column order and direction.
Types without IComparable: CompareOrderingKeys catches the exception and returns 0,
falling back to existing behavior (no orphan skipping).
cmp == 0 with non-matching identifier: theoretically impossible (ordering key includes
identifiers as tiebreakers), but handled safely by falling through to HasNext = true.
Multiple split collections: each collection has its own data reader, ordering key, and
SplitQueryCollectionContext. The fix applies independently per collection.
Nested split collections (split collection within a split collection): each level has its
own PopulateSplit* call with its own ordering key. The fix applies at each level.

@roji

@roji just wondering if this is planned for the .net 11 release timeline

@devony

@devony that's unclear at this point.

Fixes #33826

Handle out-of-order split include rows during concurrent inserts (#38676

docs: record why PendingModelChangesWarning is suppressed

Add migration designer chain tests and document timestamp suppression (…

Navigation Menu

Repository navigation

Concurrent insert can cause AsSplitQuery to fail to populate unrelated child collections #33826

Provider and version information

15 remaining items

Fix: Split query silently drops child collections on concurrent insert (#33826)

Problem

Root Cause

Solution: Ordering-Aware Orphan Skipping

Why compare the full ordering key (not just identifiers)?

Ordering comparison mechanism

Implementation

1. Add ordering key data to `SplitCollectionInfo`

2. Add ordering key data to `RelationalSplitCollectionShaperExpression`

3. Add `ParentOrderingKey` to `SplitQueryCollectionContext`

4. Update `InitializeSplitIncludeCollection` / `InitializeSplitCollection`

5. Update code-gen for `InitializeSplit*` calls

6. Update `PopulateSplitIncludeCollection[Async]` / `PopulateSplitCollection[Async]`

7. Update code-gen for `PopulateSplit*` calls

8. Update MethodInfo references

9. Handle `IComparable` unavailability gracefully

10. Add regression test

Files to Modify

Performance Considerations

Edge Cases

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Concurrent insert can cause AsSplitQuery to fail to populate unrelated child collections #33826

Description

Provider and version information

Activity

roji commented on Jul 19, 2024

roji commented on Aug 26, 2024

roji commented on Aug 27, 2024

danzel commented on Nov 18, 2024

erick-thompson commented on Jan 16, 2025

Ahmedrz89 commented on Mar 25, 2025

devony commented on Apr 21, 2025

15 remaining items

roji commented on Mar 10, 2026

Fix: Split query silently drops child collections on concurrent insert (#33826)

Problem

Root Cause

Solution: Ordering-Aware Orphan Skipping

Why compare the full ordering key (not just identifiers)?

Ordering comparison mechanism

Implementation

1. Add ordering key data to SplitCollectionInfo

2. Add ordering key data to RelationalSplitCollectionShaperExpression

3. Add ParentOrderingKey to SplitQueryCollectionContext

4. Update InitializeSplitIncludeCollection / InitializeSplitCollection

5. Update code-gen for InitializeSplit* calls

6. Update PopulateSplitIncludeCollection[Async] / PopulateSplitCollection[Async]

7. Update code-gen for PopulateSplit* calls

8. Update MethodInfo references

9. Handle IComparable unavailability gracefully

10. Add regression test

Files to Modify

Performance Considerations

Edge Cases

devony commented on May 19, 2026

roji commented on May 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

1. Add ordering key data to `SplitCollectionInfo`

2. Add ordering key data to `RelationalSplitCollectionShaperExpression`

3. Add `ParentOrderingKey` to `SplitQueryCollectionContext`

4. Update `InitializeSplitIncludeCollection` / `InitializeSplitCollection`

5. Update code-gen for `InitializeSplit*` calls

6. Update `PopulateSplitIncludeCollection[Async]` / `PopulateSplitCollection[Async]`

7. Update code-gen for `PopulateSplit*` calls

9. Handle `IComparable` unavailability gracefully