Skip to content

Memory consumption when retrieving huge result data #253

@fenjen

Description

@fenjen

When querying the InfluxDB for large results, the okio client seems to hold the entire response in local byte arrays.

I'm querying data with com.influxdb.client.QueryApi.query(String, BiConsumer<Cancellable, FluxRecord>, Consumer<? super Throwable>, Runnable). The result data is huge, but the Consumer aggregates it efficiently. Yet my JVM stops frequently because the okio client seems to hold references to the entire response. The heap dump analyzer shows variables that contain the whole original CSV data.

Also I can't cancel the request until the whole response was read into memory, because the BiConsumer is given result data too late.

See attached image for details.
okio-heap-consumption

Specifications:

  • Client Version: 2.2.0
  • InfluxDB Version: 2.0.4
  • JDK Version: 11
  • Platform: all

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions