Keep buffer for long text during transformation#357
Keep buffer for long text during transformation#357qnighy wants to merge 1 commit intoinikulin:masterfrom qnighy:buffer-safekeeping
Conversation
In RewritingStream, buffers are sometimes dropped too early to get raw HTML source. It leads to long text node being truncated (only a suffix being emitted). This commit introduces "buffer safekeeping" extension to keep a buffer at a specified location. It allows RewritingStream to always retrieve correct raw HTML source. Fixes #292.
|
Hi |
|
This is currently blocked by #362. Once that PR is merged, here is the approach I would take instead: This issue is caused by (1) will always waste a bit of memory, as parsed chunks will only be dropped prior to the next token being generated. We might also want to look into dropping parsed chunks whenever we add to the buffer in the preprocessor, to reduce the impact of this change. (2) is a much bigger architectural change. It would reliably drop parsed chunks as soon as they are emitted, but might have other unintended consequences (eg. reduced perf). To fix this specific issue, I would lean towards implementing (1). |
Fixes #357 Co-Authored-By: Masaki Hara <[email protected]>
|
Fixed in #432 |
3 participants
In RewritingStream, buffers are sometimes dropped too early to get raw HTML source. It leads to long text node being truncated (only a suffix being emitted).
This PR introduces "buffer safekeeping" extension to keep a buffer at a specified location. It allows RewritingStream to always retrieve correct raw HTML source.
Fixes #292.