[#596] Initial implementation of `SCYLLA_USE_METADATA_ID` protocol feature negotiation by nikagra · Pull Request #663 · scylladb/java-driver

nikagra · 2025-09-24T16:17:02Z

Issue java-driver#596

repared statements invalidation used to be completely broken, you can read details on it here
Recently core merged PR that fixes problems for select statements.
After that PR, if driver supply SCYLLA_USE_METADATA_ID at the startup, when statements is prepared server hands out result queries metadata hash.
When driver executes this query it sends metadata id alongside with the request.
Server checks driver metadata id and what it has on its side, if there is any difference it will send new metadata and metadata id.
Driver suppose to pick it up and update metadata and metadata id on it's side, before deserializing response, solving issue with having outdated metadata on driver side.
Unfortunately it is solved only for SELECT statements, UPDATE/INSERT will be solved on separate occasion.

Core changes are in, we need to start working on driver side.

To Be Done:

parse SCYLLA_USE_METADATA_ID protocol extension from SUPPORTED frame;
- if present in SUPPORTED, send it in the STARTUP frame;
if the extension is negotiated:
- store the metadata ID, same way it is done for CQL v5
- read METADATA_CHANGED (=3) flag in result metadata;
- if the flag is set, read the new metadata ID (short bytes) and update it on prepare statement
if received new result metadata that is different from the cached one:
- use the new metadata for deserialization instead of the old one;
- replace the old cached metadata with the new one.

Resolve skip metadata flag

Default value of skip metadata flag should be treated as safest option
Which means that if SCYLLA_USE_METADATA_ID was negotiated or CQL v5 is used then it is true, result metadata is skipped.
In other cases it is false.

Changes

Negotiating SCYLLA_USE_METADATA_ID extension during protocol initialization Options step.
Refactoring ProtocolFeature enum to ProtocolFeatures class enhancing isSUpportedBy method.
Introducing and storing in the channel ProtocolFeatureStore instance, which stores information about optional protocol features.
Enhancing Message to load ProtocolFeatureStore from channel and to later use it for encoding and decoding (in encoders and decoders).

dkropachev · 2025-09-25T16:01:18Z

driver-core/src/main/java/com/datastax/driver/core/Host.java

    /**
     * Gets invoked when the listener is registered with a cluster, or at cluster startup if the
-     * listener was registered at initialization with {@link
-     * com.datastax.driver.core.Cluster#register(Host.StateListener)}.


Please move these kind of changes into a separate commit with proper message to it that explains why it is done

This should've been some automated change, but what tool has performed it. I will revert the changes

I still see them, please remove them from the PR.

Still see them

driver-core/src/main/java/com/datastax/driver/core/Connection.java

driver-core/src/main/java/com/datastax/driver/core/DefaultResultSetFuture.java

driver-core/src/main/java/com/datastax/driver/core/Message.java

driver-core/src/main/java/com/datastax/driver/core/SessionManager.java

driver-core/src/main/java/com/datastax/driver/core/Host.java

driver-core/src/main/java/com/datastax/driver/core/ProtocolFeatureStore.java

driver-core/src/test/java/com/datastax/driver/core/PreparedStatementInvalidationTest.java

driver-core/src/main/java/com/datastax/driver/core/Host.java

dkropachev · 2025-10-14T10:29:44Z

driver-core/src/main/java/com/datastax/driver/core/Host.java

    /**
     * Gets invoked when the listener is registered with a cluster, or at cluster startup if the
-     * listener was registered at initialization with {@link
-     * com.datastax.driver.core.Cluster#register(Host.StateListener)}.


I still see them, please remove them from the PR.

driver-core/src/main/java/com/datastax/driver/core/Message.java

@dkropachev

…ature negotiation. Introducing `ProtocolFeatureStore`. [scylladb#596] Initial implementation of SCYLLA_USE_METADATA_ID feature. [scylladb#596] Fixing NPE when accessing sharding info from empty ProtocolFeatureStore. [scylladb#596] Storing `ProtocolFeatureStore` in `ProtocolEncoder`/`ProtocolDecoder`. Getting read of sharing it via `Channel` attributes. [scylladb#596] Addressing missing `ProtocolFeatureStore` in a `statement` param of `executeAsync` to fix ITs. [scylladb#596] Adjusting the way feature store is handled in `DefaultResultSetFuture`. Removing feature store from its constructor. [scylladb#596] Moving channel attribute management for feature store to utility methods of `ProtocolFeatureStore`. [scylladb#596] Setting `Host`'s feature store separately from `Channel`. Code clean up Apply suggestion from @dkropachev Co-authored-by: Dmitry Kropachev <[email protected]> [scylladb#596] `ProtocolEncoder` clean up.

@dkropachev

…validationTest`. Registering for closure resources implementing `AutoCloseable` interface in flaky tests class. [scylladb#596] Improving Scylla test coverage in `PreparedStatementInvalidationTest`. Apply suggestion from @dkropachev Rearranging assertions and minor improvements in `should_never_update_statement_id_for_conditional_updates` Co-authored-by: Dmitry Kropachev <[email protected]>

nikagra requested a review from dkropachev September 24, 2025 16:17

nikagra self-assigned this Sep 24, 2025

dkropachev reviewed Sep 25, 2025

View reviewed changes

nikagra force-pushed the scylla-3.x-use-metadataId branch from 55b46e4 to 559cd95 Compare September 29, 2025 10:25

nikagra marked this pull request as ready for review October 3, 2025 17:14

nikagra requested a review from Bouncheck October 3, 2025 17:16

dkropachev reviewed Oct 3, 2025

View reviewed changes

nikagra commented Oct 8, 2025

View reviewed changes

driver-core/src/main/java/com/datastax/driver/core/SessionManager.java Outdated Show resolved Hide resolved

nikagra force-pushed the scylla-3.x-use-metadataId branch 3 times, most recently from 1349c00 to 2cbb7bf Compare October 10, 2025 11:26

dkropachev reviewed Oct 10, 2025

View reviewed changes

dkropachev force-pushed the scylla-3.x-use-metadataId branch 2 times, most recently from e2971d2 to 82e78f0 Compare October 11, 2025 22:14

nikagra force-pushed the scylla-3.x-use-metadataId branch from 3daac00 to ff4451f Compare October 13, 2025 13:10

nikagra commented Oct 13, 2025

View reviewed changes

driver-core/src/main/java/com/datastax/driver/core/Host.java Show resolved Hide resolved

nikagra force-pushed the scylla-3.x-use-metadataId branch 4 times, most recently from 0d9e088 to 3b34d0c Compare October 13, 2025 21:18

dkropachev requested changes Oct 14, 2025

View reviewed changes

nikagra force-pushed the scylla-3.x-use-metadataId branch from 70b2368 to ca5672c Compare October 14, 2025 12:01

dkropachev changed the title ~~[#596] Initial implementation of SCYLLA_USE_METADATA_ID protocol fe…~~ [#596] Initial implementation of SCYLLA_USE_METADATA_ID protocol feature negotiation Oct 14, 2025

nikagra force-pushed the scylla-3.x-use-metadataId branch 2 times, most recently from 30edf34 to 9334677 Compare October 14, 2025 13:14

dkropachev force-pushed the scylla-3.x-use-metadataId branch from 288d8bc to c1d0287 Compare October 14, 2025 14:39

dkropachev approved these changes Oct 14, 2025

View reviewed changes

nikagra and others added 2 commits October 14, 2025 13:13

dkropachev force-pushed the scylla-3.x-use-metadataId branch from c1d0287 to 76a5354 Compare October 14, 2025 17:13

dkropachev merged commit fa6236e into scylladb:scylla-3.x Oct 14, 2025
6 of 7 checks passed

nikagra deleted the scylla-3.x-use-metadataId branch October 15, 2025 13:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#596] Initial implementation of `SCYLLA_USE_METADATA_ID` protocol feature negotiation#663

[#596] Initial implementation of `SCYLLA_USE_METADATA_ID` protocol feature negotiation#663
dkropachev merged 2 commits intoscylladb:scylla-3.xfrom
nikagra:scylla-3.x-use-metadataId

nikagra commented Sep 24, 2025 •

edited by dkropachev

Loading

Uh oh!

dkropachev Sep 25, 2025

Uh oh!

nikagra Sep 26, 2025

Uh oh!

dkropachev Oct 14, 2025

Uh oh!

dkropachev Oct 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkropachev Oct 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

nikagra commented Sep 24, 2025 • edited by dkropachev Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue java-driver#596

To Be Done:

Resolve skip metadata flag

Changes

Uh oh!

dkropachev Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

nikagra Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

dkropachev Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

dkropachev Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkropachev Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

nikagra commented Sep 24, 2025 •

edited by dkropachev

Loading