Skip to content

fix(pgwire): resolve intermittent NullPointerException during INSERTs#5788

Merged
bluestreak01 merged 2 commits intomasterfrom
jh_simply_associative_cache_dups
Jun 27, 2025
Merged

fix(pgwire): resolve intermittent NullPointerException during INSERTs#5788
bluestreak01 merged 2 commits intomasterfrom
jh_simply_associative_cache_dups

Conversation

@jerrinot
Copy link
Copy Markdown
Contributor

@jerrinot jerrinot commented Jun 26, 2025

Summary

This PR provides a definitive fix for a bug in the PGWire INSERT cache that could cause intermittent NullPointerException errors. It resolves the underlying issue that was only partially mitigated before.

Symptoms

When executing SQL INSERT statements via the PostgreSQL wire protocol, the server would occasionally fail with one of the following errors in the logs:

Internal error. Exception type: NullPointerException

or

Cannot invoke "io.questdb.cairo.sql.InsertOperation.createMethod(io.questdb.griffin.SqlExecutionContext, io.questdb.cairo.pool.WriterSource)" because "this.insertOp" is null

Affected Versions

This issue is present in QuestDB versions from 8.2 up to and including 8.3.3.

Context & Root Cause

While PR #5706 prevented these specific errors from appearing, it did not fix the core problem: the INSERT cache was creating duplicate entries and when one entry was closed on eviction, it also closed the other entry which was left in a cache and on usage it would throw the internal error. This PR corrects the cache logic, fully resolving the bug.


Implementation details

The put() method was creating a large number of duplicate entries in INSERT caches used by the PGWire protocol. This was triggered when PGWire would peek() at a cached entry and later re-put() it; The old logic would fail to find the existing entry unless it was at the head of its block, resulting in a duplicate.

This commit changes put() to scan the entire block for a matching key. The new behavior is:

  1. If an identical (key, value) pair exists, it is promoted to the MRU position.
  2. If a (key, null) entry exists (left by poll()), its slot is reused for the new value to avoid a key allocation.
  3. Otherwise, a new entry is inserted.

This change introduces a loop to scan the block, which adds a small seek cost. However, given the small block sizes used in practice (e.g., the default of 4 for the INSERT cache), this cost is negligible and is an acceptable trade-off for correctness and the elimination of duplicate entries.

The `put()` method was creating a large number of duplicate entries
in INSERT caches used by the PGWire protocol. This was triggered when PGWire
would `peek()` at a cached entry and later re-`put()` it; The old
logic would fail to find the existing entry unless it was at the head
of its block, resulting in a duplicate.

This commit changes `put()` to scan the entire block for a matching key.
The new behavior is:
1. If an identical (key, value) pair exists, it is promoted to the MRU position.
2. If a (key, null) entry exists (left by `poll()`), its slot is reused for the new value
   to avoid a key allocation.
3. Otherwise, a new entry is inserted.

This change introduces a loop to scan the block, which adds a small seek cost.
However, given the small block sizes used in practice (e.g., the default of 4 for the INSERT cache),
this cost is negligible and is an acceptable trade-off for correctness
and the elimination of duplicate entries.
@jerrinot jerrinot added Bug Incorrect or unexpected behavior Postgres Wire Issues or changes relating to Postgres wire protocol Core Related to storage, data type, etc. labels Jun 26, 2025
@jerrinot
Copy link
Copy Markdown
Contributor Author

an unrelated test failure, fixed by #5790

Copy link
Copy Markdown
Member

@bluestreak01 bluestreak01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description must be user-centric. No one outside of our developers know what SimpleAssociativeCache is and what is it used for

@jerrinot jerrinot changed the title fix(core): correct put() logic in SimpleAssociativeCache fix(pgwire): Resolve intermittent NullPointerException during INSERTs Jun 27, 2025
@jerrinot jerrinot changed the title fix(pgwire): Resolve intermittent NullPointerException during INSERTs fix(pgwire): resolve intermittent NullPointerException during INSERTs Jun 27, 2025
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 33 / 34 (97.06%)

file detail

path covered line new line coverage
🔵 io/questdb/std/SimpleAssociativeCache.java 33 34 97.06%

@bluestreak01 bluestreak01 merged commit c15ad6a into master Jun 27, 2025
47 checks passed
@bluestreak01 bluestreak01 deleted the jh_simply_associative_cache_dups branch June 27, 2025 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Incorrect or unexpected behavior Core Related to storage, data type, etc. Postgres Wire Issues or changes relating to Postgres wire protocol

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants