-
Notifications
You must be signed in to change notification settings - Fork 56
more efficient stream parsing #212
Copy link
Copy link
Closed
Description
Hello,
I continuously stream massive amounts of market data from ClickHouse and after profiling my app I discovered that most ticks are spent in stream parsing, i..e:
const rows = await this.#client.query({
query: `
SELECT
*
FROM...
`})
const stream = rows.stream()
stream.on('data', async (rows) => {
for (const row of rows) {
const [
event,
time,
...
] = await row.json()
})I briefly checked the module source code, and looks like quite expensive JSON.parse() is used internally even when we call text().
My question would be **is there a faster way to parse incoming stream maybe by using some raw output format? ** Or am I using reading streams wrong?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels