fix(pgwire): resolve intermittent NullPointerException during INSERTs#5788
Merged
bluestreak01 merged 2 commits intomasterfrom Jun 27, 2025
Merged
fix(pgwire): resolve intermittent NullPointerException during INSERTs#5788bluestreak01 merged 2 commits intomasterfrom
bluestreak01 merged 2 commits intomasterfrom
Conversation
The `put()` method was creating a large number of duplicate entries in INSERT caches used by the PGWire protocol. This was triggered when PGWire would `peek()` at a cached entry and later re-`put()` it; The old logic would fail to find the existing entry unless it was at the head of its block, resulting in a duplicate. This commit changes `put()` to scan the entire block for a matching key. The new behavior is: 1. If an identical (key, value) pair exists, it is promoted to the MRU position. 2. If a (key, null) entry exists (left by `poll()`), its slot is reused for the new value to avoid a key allocation. 3. Otherwise, a new entry is inserted. This change introduces a loop to scan the block, which adds a small seek cost. However, given the small block sizes used in practice (e.g., the default of 4 for the INSERT cache), this cost is negligible and is an acceptable trade-off for correctness and the elimination of duplicate entries.
puzpuzpuz
approved these changes
Jun 27, 2025
Contributor
Author
|
an unrelated test failure, fixed by #5790 |
bluestreak01
requested changes
Jun 27, 2025
Member
bluestreak01
left a comment
There was a problem hiding this comment.
PR description must be user-centric. No one outside of our developers know what SimpleAssociativeCache is and what is it used for
bluestreak01
approved these changes
Jun 27, 2025
Contributor
[PR Coverage check]😍 pass : 33 / 34 (97.06%) file detail
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR provides a definitive fix for a bug in the PGWire INSERT cache that could cause intermittent
NullPointerExceptionerrors. It resolves the underlying issue that was only partially mitigated before.Symptoms
When executing SQL INSERT statements via the PostgreSQL wire protocol, the server would occasionally fail with one of the following errors in the logs:
or
Affected Versions
This issue is present in QuestDB versions from 8.2 up to and including 8.3.3.
Context & Root Cause
While PR #5706 prevented these specific errors from appearing, it did not fix the core problem: the INSERT cache was creating duplicate entries and when one entry was closed on eviction, it also closed the other entry which was left in a cache and on usage it would throw the internal error. This PR corrects the cache logic, fully resolving the bug.
Implementation details
The
put()method was creating a large number of duplicate entries in INSERT caches used by the PGWire protocol. This was triggered when PGWire wouldpeek()at a cached entry and later re-put()it; The old logic would fail to find the existing entry unless it was at the head of its block, resulting in a duplicate.This commit changes
put()to scan the entire block for a matching key. The new behavior is:poll()), its slot is reused for the new value to avoid a key allocation.This change introduces a loop to scan the block, which adds a small seek cost. However, given the small block sizes used in practice (e.g., the default of 4 for the INSERT cache), this cost is negligible and is an acceptable trade-off for correctness and the elimination of duplicate entries.