-
Notifications
You must be signed in to change notification settings - Fork 66
Labels
apache-arrowLow level Apache Arrow tasksLow level Apache Arrow tasksengine-capabilityInternal engine featuresInternal engine featuresperf-optPerformance and optimizationPerformance and optimization
Milestone
Description
Related to #1085. cc @albertlockett
I think the additional arrow compression stuff may not be behaving quite as expected. I was running the demo earlier this week on an image prior to this change. Testing today from main I noticed the cpu use went through the roof on any arrow exporter, and I think it's due to the default enabled zstd compression introduced in 1085. See screenshot below.
The lower cpu use is with what was previously the default behavior (payload_compression: none). On the right, everything is the same except removing that statement.

albertlockett
Metadata
Metadata
Assignees
Labels
apache-arrowLow level Apache Arrow tasksLow level Apache Arrow tasksengine-capabilityInternal engine featuresInternal engine featuresperf-optPerformance and optimizationPerformance and optimization
Type
Projects
Status
Done