There are to many infrastructure (reflection and lambda compilation) overheads when we many times executing a test with different data, In our case we have the one NUnit test that receives about 20k complex data records (many level hierarchy with object, collection etc) from the test data provider.
I've checked execution pats with dotTrace and found that about 90% of execution time take infrastructure tasks such as reflection calls and lambdas compilations. It happens because every comparison prepares information about comparing objects from scratch.
As I understand some reflection and lambda caches should be added to avoid this issue.