Skip to content

Streaming Inserts #41439

@alexey-milovidov

Description

@alexey-milovidov

Use case

A client doing a HTTP POST request and sending a potentially infinite stream of data for INSERT.

How to implement

We already support this.
But there are the following nuances:

  1. The input format should be able to form blocks of data for INSERT not only by the threshold on the number of rows or bytes but also by timeout. For example, to generate a block every second even if the stream of data is smaller.

  2. When the connection is unexpectedly closed, it should parse and process the remaining data instead of treating it as an error.

Additional context

With these changes we will be able to get InfluxDB line protocol for free:
https://docs.influxdata.com/influxdb/v1.8/write_protocols/line_protocol_tutorial/

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions