The default querying code pattern of elasticContext.Query<MyDocument>().ToList() causes 3 separate Large Object Heap allocations when the result set is over 8000ish items. They are caused by usage of List<T>, which is backed by a T[]. When the result set is more than 8000ish items, List<T> allocates a 16000 item array, which takes up more than 85000 bytes, which ends up being placed on the LOH (and thus in the gen 2 GC bucket). All items that are saved in that array also end up in the gen 2 GC bucket, meaning there's a lot of transient memory that can't be cleaned up for a long time used up by the large number of objects, plus heap fragmentation resulting from the long-lived array instances.
The 3 arrays end up being:
- The deserialized hits collection in https://github.com/CenturyLinkCloud/ElasticLINQ/blob/dev/Source/ElasticLINQ/Response/Model/Hits.cs#L27. This one is particularly bad because it pins a bunch of
JObject instances, which end up being very expensive to keep around.
- The materialized list of deserialized
MyDocument types coming from https://github.com/CenturyLinkCloud/ElasticLINQ/blob/dev/Source/ElasticLINQ/Response/Materializers/ListHitsElasticMaterializer.cs#L53.
- The result of the final
ToList() call in the default querying code pattern.
Problem 3 can be solved by user code, for example by just enumerating over the result instead of stashing it in a list, or using a different collection such as LinkedList<T>.
Problem 2 should also be relatively easily solved by using a different collection type if the result set is large enough. LinkedList<T> might be a convenient candidate, tough it will end up allocating even more small objects for each node in the list. A custom IReadOnlyList<T> implementation that wraps a List<List<T>> (with the inner lists limited to say 5000 items) might be more efficient overall.
Problem 1 is harder to solve. The interface is public, so swapping out the implementation type is not possible (tough might still be something that could be done in a 2.0 release). A possible improvement would be to have the ElasticQueryProvider null-out the Hit list once it's done materializing the result. That way at least the Hit instances and all their related JObjects have a chance of getting cleaned up earlier.
I'm going to submit a couple of pull requests to try to address these issues soon, but wanted to provide some background on them first and see if anyone has any better ideas.
The default querying code pattern of
elasticContext.Query<MyDocument>().ToList()causes 3 separate Large Object Heap allocations when the result set is over 8000ish items. They are caused by usage ofList<T>, which is backed by aT[]. When the result set is more than 8000ish items,List<T>allocates a 16000 item array, which takes up more than 85000 bytes, which ends up being placed on the LOH (and thus in the gen 2 GC bucket). All items that are saved in that array also end up in the gen 2 GC bucket, meaning there's a lot of transient memory that can't be cleaned up for a long time used up by the large number of objects, plus heap fragmentation resulting from the long-lived array instances.The 3 arrays end up being:
JObjectinstances, which end up being very expensive to keep around.MyDocumenttypes coming from https://github.com/CenturyLinkCloud/ElasticLINQ/blob/dev/Source/ElasticLINQ/Response/Materializers/ListHitsElasticMaterializer.cs#L53.ToList()call in the default querying code pattern.Problem 3 can be solved by user code, for example by just enumerating over the result instead of stashing it in a list, or using a different collection such as
LinkedList<T>.Problem 2 should also be relatively easily solved by using a different collection type if the result set is large enough.
LinkedList<T>might be a convenient candidate, tough it will end up allocating even more small objects for each node in the list. A customIReadOnlyList<T>implementation that wraps aList<List<T>>(with the inner lists limited to say 5000 items) might be more efficient overall.Problem 1 is harder to solve. The interface is public, so swapping out the implementation type is not possible (tough might still be something that could be done in a 2.0 release). A possible improvement would be to have the
ElasticQueryProvidernull-out theHitlist once it's done materializing the result. That way at least theHitinstances and all their relatedJObjects have a chance of getting cleaned up earlier.I'm going to submit a couple of pull requests to try to address these issues soon, but wanted to provide some background on them first and see if anyone has any better ideas.