Initial implementation of `SCYLLA_USE_METADATA_ID` feature by nikagra · Pull Request #599 · scylladb/java-driver

nikagra · 2025-09-05T11:48:50Z

Issue scylla-driver#81

repared statements invalidation used to be completely broken, you can read details on it here
Recently core merged PR that fixes problems for select statements.
After that PR, if driver supply SCYLLA_USE_METADATA_ID at the startup, when statements is prepared server hands out result queries metadata hash.
When driver executes this query it sends metadata id alongside with the request.
Server checks driver metadata id and what it has on its side, if there is any difference it will send new metadata and metadata id.
Driver suppose to pick it up and update metadata and metadata id on it's side, before deserializing response, solving issue with having outdated metadata on driver side.
Unfortunately it is solved only for SELECT statements, UPDATE/INSERT will be solved on separate occasion.

Core changes are in, we need to start working on driver side.

To Be Done:

parse SCYLLA_USE_METADATA_ID protocol extension from SUPPORTED frame;
- if present in SUPPORTED, send it in the STARTUP frame;
if the extension is negotiated:
- store the metadata ID, same way it is done for CQL v5
- read METADATA_CHANGED (=3) flag in result metadata;
- if the flag is set, read the new metadata ID (short bytes) and update it on prepare statement
if received new result metadata that is different from the cached one:
- use the new metadata for deserialization instead of the old one;
- replace the old cached metadata with the new one.

Resolve skip metadata flag

Default value of skip metadata flag should be treated as safest option
Which means that if SCYLLA_USE_METADATA_ID was negotiated or CQL v5 is used then it is true, result metadata is skipped.
In other cases it is false.

Fixes

Enhancing forked native-protocol to support optional protocol options (PR#2) in FrameCodec, Codec's and SubCodec's.
Negotiating SCYLLA_USE_METADATA_ID extension during protocol initialization Options step.
Replacing FrameEncoder and FrameDecoder with updated ProtocolFeatures in pipeline after feature is negotiated.
These protocol features will be used during Frame encoding and decoding on native-protocol side

core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java

core/src/main/java/com/datastax/oss/driver/internal/core/protocol/MetadataIdInfo.java

core/src/test/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandlerTest.java

bom/pom.xml

core/src/main/java/com/datastax/oss/driver/internal/core/channel/ChannelFactory.java

dkropachev

Small changes, please cleanup commit history after you done.

core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java

dkropachev · 2025-10-02T14:21:44Z

There is definitely a problem, I have restarted it three times, got same result should_not_struggle_to_fill_pools failed in all runs, it does not happen on other branches.

…ture negotiation during driver startup 🚀 # Conflicts: # core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java

…Features` to `FrameEncoder` and `FrameDecoder` constructors. Updating integration tests.

# Conflicts: # core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java # core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java

…Level.LOCAL_ONE` in `ProfileIT`.

…s` (used in `OsgiGeoTypesIT` & `OsgiGraphIT`).

Co-authored-by: Dmitry Kropachev <[email protected]>

…lding of protocol features to `ProtocolFeatureStore`.

…lifying unit tests.

dkropachev · 2025-10-02T17:35:54Z

There is definitely a problem, I have restarted it three times, got same result should_not_struggle_to_fill_pools failed in all runs, it does not happen on other branches.

FYI, this problem exists on main branch, it is not specific to this PR.

nikagra force-pushed the scylla-4.x-use-metadataId branch from 3760023 to 22254de Compare September 5, 2025 12:03

nikagra marked this pull request as ready for review September 16, 2025 13:24

dkropachev mentioned this pull request Sep 16, 2025

Implement ProtocolFeatureStore #605

Merged

nikagra force-pushed the scylla-4.x-use-metadataId branch from 40d49d1 to 2feab3c Compare September 17, 2025 16:53

dkropachev self-requested a review September 18, 2025 15:04

dkropachev changed the title ~~[#567] Initial implementation of SCYLLA_USE_METADATA_ID feature neg…~~ [#567] Initial implementation of SCYLLA_USE_METADATA_ID feature Sep 18, 2025

dkropachev requested changes Sep 18, 2025

View reviewed changes

dkropachev force-pushed the scylla-4.x branch from 1979ab1 to a4e60af Compare September 20, 2025 13:33

nikagra force-pushed the scylla-4.x-use-metadataId branch from 86585ce to b428ac1 Compare September 24, 2025 10:31

nikagra requested a review from dkropachev September 24, 2025 15:02

nikagra force-pushed the scylla-4.x-use-metadataId branch from 9122cf0 to 27d7a77 Compare September 24, 2025 16:05

dkropachev requested changes Sep 25, 2025

View reviewed changes

core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java Outdated Show resolved Hide resolved

core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java Outdated Show resolved Hide resolved

nikagra self-assigned this Sep 26, 2025

nikagra changed the title ~~[#567] Initial implementation of SCYLLA_USE_METADATA_ID feature~~ Initial implementation of SCYLLA_USE_METADATA_ID feature Sep 26, 2025

nikagra force-pushed the scylla-4.x-use-metadataId branch from 27d7a77 to 850cc81 Compare September 29, 2025 14:59

dkropachev reviewed Oct 2, 2025

View reviewed changes

core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java Outdated Show resolved Hide resolved

dkropachev approved these changes Oct 2, 2025

View reviewed changes

nikagra force-pushed the scylla-4.x-use-metadataId branch from 61eac1b to 28b25fe Compare October 2, 2025 16:52

nikagra added 11 commits October 2, 2025 19:02

[scylladb#567] Initial implementation of SCYLLA_USE_METADATA_ID fea…

964c0f3

…ture negotiation during driver startup 🚀 # Conflicts: # core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java

[scylladb#597] Updating extension negotiation logic. Adding `Protocol…

321537a

…Features` to `FrameEncoder` and `FrameDecoder` constructors. Updating integration tests.

[scylladb#597] Fixing unit tests.

70e0a64

[scylladb#597] Enhancing protocol feature handling.

51a442b

# Conflicts: # core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java # core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java

[scylladb#597] Changes following rebase.

661e579

[scylladb#597] Switching native-protocol dependency.

3fffb35

[scylladb#597] Fixing stubbing for update query in ProfileIT.

1794208

[scylladb#597] Fixing more stubbing for update query and `Consistency…

29e3840

…Level.LOCAL_ONE` in `ProfileIT`.

[scylladb#597] Switching simulacron-native-server dependency.

c695b60

[scylladb#597] Updating groupId of native-protocol in `BundleOption…

0ad6d0e

…s` (used in `OsgiGeoTypesIT` & `OsgiGraphIT`).

[scylladb#597] Adding javadoc. Removing redundant comment.

00d052d

nikagra and others added 3 commits October 2, 2025 19:02

[scylladb#597] Updating MetadataIdInfo method names.

69dbbd7

Co-authored-by: Dmitry Kropachev <[email protected]>

[scylladb#596] Refactoring functions that update pipeline. Moving bui…

d19e902

…lding of protocol features to `ProtocolFeatureStore`.

[scylladb#596] Setting default EMPTY value for featureStore, simp…

3e967d7

…lifying unit tests.

nikagra force-pushed the scylla-4.x-use-metadataId branch from 28b25fe to 3e967d7 Compare October 2, 2025 17:02

dkropachev merged commit 4b53d10 into scylladb:scylla-4.x Oct 2, 2025
8 of 11 checks passed

nikagra deleted the scylla-4.x-use-metadataId branch October 2, 2025 17:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial implementation of `SCYLLA_USE_METADATA_ID` feature#599

Initial implementation of `SCYLLA_USE_METADATA_ID` feature#599
dkropachev merged 14 commits intoscylladb:scylla-4.xfrom
nikagra:scylla-4.x-use-metadataId

nikagra commented Sep 5, 2025 •

edited by dkropachev

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkropachev left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkropachev commented Oct 2, 2025

Uh oh!

dkropachev commented Oct 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

nikagra commented Sep 5, 2025 • edited by dkropachev Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue scylla-driver#81

To Be Done:

Resolve skip metadata flag

Fixes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkropachev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkropachev commented Oct 2, 2025

Uh oh!

dkropachev commented Oct 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

nikagra commented Sep 5, 2025 •

edited by dkropachev

Loading