Skip to content

refactor(store): integrate store grpc module#27

Merged
imbajin merged 6 commits intohugegraph:3.7.1-pd-storefrom
JisoLya:ref-store-grpc
Sep 5, 2025
Merged

refactor(store): integrate store grpc module#27
imbajin merged 6 commits intohugegraph:3.7.1-pd-storefrom
JisoLya:ref-store-grpc

Conversation

@JisoLya
Copy link
Copy Markdown
Member

@JisoLya JisoLya commented Sep 3, 2025

Purpose of the PR

  • close #xxx

Main Changes

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • xxx

Does this PR potentially affect the following parts?

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

Summary by CodeRabbit

  • 新功能
    • 新增 QueryService,支持双向流式查询及同步查询/计数,提供聚合、扫描、索引、去重、排序、采样等控制项。
    • HgStoreState 增加 getPeers 接口,用于获取分片同伴信息。
    • 新增 TTL 清理请求消息类型,支持按图/表/分片/ID 进行清理。
  • 变更
    • ScanPartitionRequest 的 Request 增加 batchSize 参数以控制批量大小。
    • 移除 HgStoreSession 的 Count RPC 及相关聚合消息。

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 3, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

在多个 gRPC Proto 文件中新增与调整接口:为 ScanPartitionRequest.Request 增加 batchSize 字段;新增独立的查询协议与 QueryService(含流式与非流式查询);新增 TTLCleanRequest;移除 HgStoreSession 的 Count RPC 与 Agg 消息;为 HgStoreState 增加 getPeers RPC 及相关请求/响应消息。

Changes

Cohort / File(s) Change Summary
分区扫描请求批量参数
hugegraph-store/hg-store-grpc/src/main/proto/graphpb.proto
在 ScanPartitionRequest 的内嵌 Request 中新增字段:int32 batchSize = 12
查询协议与服务新增
hugegraph-store/hg-store-grpc/src/main/proto/query.proto
新增查询相关枚举与消息(AggregationType、AggregateFunc、ScanType、ScanTypeParam、Index、DeDupOption、QueryRequest、QueryResponse),新增 QueryService:双向流式 rpc query,以及非流式 rpc query0、count
TTL 清理请求新增
hugegraph-store/hg-store-grpc/src/main/proto/store_common.proto
新增消息 TTLCleanRequest{ string graph=1; int32 partitionId=2; string table=3; repeated bytes ids=4; }
会话服务精简
hugegraph-store/hg-store-grpc/src/main/proto/store_session.proto
移除 rpc Count(ScanStreamReq) returns (Agg);删除消息 Agg(Header header=1; int64 count=2);移除对 store_stream_meta.proto 的 import
状态服务扩展
hugegraph-store/hg-store-grpc/src/main/proto/store_state.proto
新增 rpc getPeers(PartitionRequest) returns (PeersResponse);新增消息 PartitionRequest{id=1} 与 PeersResponse{peers=1}

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Client
  participant QueryService
  participant Store

  rect rgb(235, 245, 255)
    note over Client,QueryService: 双向流式查询(新)
    Client->>QueryService: stream QueryRequest
    QueryService->>Store: 执行扫描/索引/聚合
    Store-->>QueryService: 扫描结果分批
    QueryService-->>Client: stream QueryResponse(is_ok, data, is_finished)
    Client-->>QueryService: 流结束
  end

  rect rgb(240, 255, 240)
    note over Client,QueryService: 非流式查询(query0 / count)
    Client->>QueryService: QueryRequest
    QueryService->>Store: 执行查询/计数
    Store-->>QueryService: 结果/计数
    QueryService-->>Client: QueryResponse
  end
Loading
sequenceDiagram
  autonumber
  actor Admin
  participant HgStoreState
  participant Partition

  rect rgb(255, 248, 235)
    note over Admin,HgStoreState: 获取分区 Peers(新)
    Admin->>HgStoreState: getPeers(PartitionRequest{id})
    HgStoreState->>Partition: 查询副本信息
    Partition-->>HgStoreState: Peers 列表
    HgStoreState-->>Admin: PeersResponse{peers}
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

我把查询装进小篮子,跳一跳就成双向流,
扫一行,聚一下,结果像胡萝卜成串溜。
旧数数悄然退场,TTL 扫把轻轻挥手,
同伴在哪?getPeers 一问便知由。
扑扇耳朵,代码田里又添一畦新秋。

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Sep 3, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Sep 3, 2025

@codecov-ai-reviewer review

@codecov-ai

This comment has been minimized.

@JisoLya
Copy link
Copy Markdown
Member Author

JisoLya commented Sep 3, 2025

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 3, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@dosubot dosubot bot added the enhancement New feature or request label Sep 3, 2025
@codecov-ai

This comment has been minimized.

@JisoLya JisoLya changed the title Ref store grpc module refactor: integrate store grpc module Sep 3, 2025
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (8)
hugegraph-store/hg-store-grpc/src/main/proto/graphpb.proto (1)

47-49: 新增字段命名与类型建议:batchSize → batch_size,且与全局分区类型保持一致

  • 当前新增字段为 int32 batchSize = 12; 建议统一采用 protobuf 样式:snake_case,即 batch_size。
  • 语义上“批大小”应非负,且在本工程其他 proto 中分区/计数多用无符号整型(例如同文件的 partition_id 为 uint32)。建议评估是否使用 uint32 并在实现侧约束范围(>0 才生效,=0 表示使用默认)。
  • 另外,请明确与 Reply/seq_no 的背压与窗口语义关系:batch_size 是否仅限制每次返回的元素数,还是作为流控窗口?建议加注释避免误用。

可选修正示例(仅示意字段命名与注释,不影响已分配的 tag 12):

-    int32 batchSize = 12;
+    // Max items per response chunk; 0 uses server default.
+    uint32 batch_size = 12;
hugegraph-store/hg-store-grpc/src/main/proto/store_state.proto (2)

35-36: getPeers 返回类型过于宽泛,建议结构化返回

当前 PeersResponse 仅有 string peers,缺乏结构化信息(如地址、角色、是否 leader、term 等)。建议返回 repeated 字段或定义 Peer 消息,便于客户端解析与演进。

示例:

-rpc getPeers(PartitionRequest) returns (PeersResponse){}
+rpc getPeers(PartitionRequest) returns (PeersResponse) {}

并将响应改为:

-message PeersResponse{
-  string peers = 1;
-}
+message Peer {
+  string id = 1;
+  string address = 2;
+  bool is_leader = 3;
+}
+message PeersResponse {
+  repeated Peer peers = 1;
+}

75-80: 分区 ID 命名与类型对齐建议

  • PartitionRequest.id 为 int32,而其他文件(graphpb.proto)使用 uint32 partition_id。为避免混淆,建议统一命名为 partition_id,类型用 uint32。

应用示例(不改变 tag):

-message PartitionRequest{
-  int32 id = 1;
-}
+message PartitionRequest {
+  uint32 partition_id = 1;
+}
hugegraph-store/hg-store-grpc/src/main/proto/query.proto (5)

26-32: RPC 命名一致性与语义清晰度

  • query0 命名不直观;建议采用 query_unary 或 simple_query。
  • count 与 store_session.proto 移除的 Count 流式聚合存在命名重叠语义,请确认迁移文档并避免歧义。

48-54: 枚举命名与注释不一致:NO_SCAN

NO_SCAN 的注释为 “only scan index”,与名称相矛盾。建议更名为 INDEX_ONLY 或修正文档,避免误导。

-  NO_SCAN = 3; // only scan index
+  INDEX_ONLY = 3; // only scan index

55-63: ScanTypeParam 字段语义需要更明确的编码约定

  • key_start/key_end/id_prefix 为 bytes,但未说明编码(大端/小端、是否包含 label/方向、是否带 code 前缀)。建议补充注释以确保多语言一致解析。
  • scan_boundary/code 的取值范围与含义(闭区间/开区间)需明确。

76-114: QueryRequest 字段一致性与边界约束

  • 字段编号无须递增,但 21、23、24、25 的顺序混乱,建议在注释中标注迁移历史并使用 reserved 声明空洞编号,防止未来误用。
  • property 与 null_property 组合语义可能冲突(既指定属性又声明只返回 key),建议定义优先级或互斥校验。
  • sample_factor 应明确取值范围 (0,1] 并在服务端校验;limit/offset 建议定义上限。
  • load_property_from_index/check_ttl 的开关需说明对性能与一致性的影响。

116-123: 跨文件类型引用与包名

QueryResponse.data 使用 Kv,依赖 store_common.proto。建议为 store_common.proto 增加 package 声明(例如 package common_pb;),并在引用处使用未限定名或限定名以避免冲突。同时需要确保 Java 包(option java_package)与 proto 包的区别被正确处理。

如需,我可以提交一份最小可编译的 buf.yaml/proto 包名与 import 路径修正补丁。

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 26b91e4 and 9888d37.

📒 Files selected for processing (5)
  • hugegraph-store/hg-store-grpc/src/main/proto/graphpb.proto (1 hunks)
  • hugegraph-store/hg-store-grpc/src/main/proto/query.proto (1 hunks)
  • hugegraph-store/hg-store-grpc/src/main/proto/store_common.proto (1 hunks)
  • hugegraph-store/hg-store-grpc/src/main/proto/store_session.proto (0 hunks)
  • hugegraph-store/hg-store-grpc/src/main/proto/store_state.proto (2 hunks)
💤 Files with no reviewable changes (1)
  • hugegraph-store/hg-store-grpc/src/main/proto/store_session.proto
🧰 Additional context used
🪛 Buf (1.55.1)
hugegraph-store/hg-store-grpc/src/main/proto/query.proto

20-20: import "store_common.proto": file does not exist

(COMPILE)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: pd
  • GitHub Check: Analyze (java)
  • GitHub Check: build-server (hbase, 11)

@JisoLya
Copy link
Copy Markdown
Member Author

JisoLya commented Sep 3, 2025

@codecov review

@JisoLya JisoLya changed the title refactor: integrate store grpc module refactor(store): integrate store grpc module Sep 3, 2025
@imbajin
Copy link
Copy Markdown

imbajin commented Sep 3, 2025

@codecov-ai-reviewer review

@codecov-ai

This comment has been minimized.

@imbajin imbajin requested a review from Copilot September 3, 2025 07:43
@codecov-ai

This comment has been minimized.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR integrates the store gRPC module by adding a new QueryService for streaming queries and updating existing message structures. The changes support bidirectional streaming queries, TTL cleanup operations, and peer discovery functionality.

  • Adds QueryService with bidirectional streaming support for queries, counts, and aggregations
  • Introduces TTL cleanup functionality with new message types
  • Updates existing services to support peer discovery and batch size configuration

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
store_state.proto Adds getPeers RPC and supporting message types for partition peer discovery
store_session.proto Removes Count RPC and Agg message, cleans up imports
store_common.proto Adds TTLCleanRequest message for TTL cleanup operations
query.proto New file defining QueryService with streaming query capabilities and comprehensive query options
graphpb.proto Adds batchSize parameter to ScanPartitionRequest for batch size control

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +78 to +79
message PeersResponse{
string peers = 1;
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'peers' field returns a string, but peer information would be better represented as a structured format (e.g., repeated string or a structured message with host/port fields) to avoid requiring string parsing on the client side.

Suggested change
message PeersResponse{
string peers = 1;
message PeerInfo {
string host = 1;
uint32 port = 2;
}
message PeersResponse{
repeated PeerInfo peers = 1;

Copilot uses AI. Check for mistakes.
@imbajin
Copy link
Copy Markdown

imbajin commented Sep 3, 2025

Seer had some issues with your request. Please try again.

@codecov-ai-reviewer review

@codecov-ai

This comment has been minimized.

Comment on lines +101 to +102
uint32 offset = 25; // offset

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sample_factor field (line 102) should have validation constraints. Sampling factors should be between 0 and 1, and the comment should clarify this requirement. Consider using a more descriptive name like 'sampling_rate'.

Suggested change
uint32 offset = 25; // offset
double sampling_rate = 31; // Sampling rate between 0.0 and 1.0 (exclusive of 0, inclusive of 1)

Did we get this right? 👍 / 👎 to inform future reviews.

@JisoLya JisoLya requested a review from Pengzna September 5, 2025 02:55
@imbajin imbajin merged commit 6bce417 into hugegraph:3.7.1-pd-store Sep 5, 2025
3 of 14 checks passed
Tsukilc pushed a commit that referenced this pull request Sep 23, 2025
* refactor: integrate store-grpc module

* ref: change comments to en

* reformat&add comments

---------

Co-authored-by: Copilot <[email protected]>
Tsukilc pushed a commit that referenced this pull request Sep 23, 2025
* refactor: integrate store-grpc module

* ref: change comments to en

* reformat&add comments

---------

Co-authored-by: Copilot <[email protected]>
imbajin pushed a commit that referenced this pull request Oct 26, 2025
* refactor: integrate store-grpc module

* ref: change comments to en

* reformat&add comments

---------

Co-authored-by: Copilot <[email protected]>
imbajin pushed a commit that referenced this pull request Oct 29, 2025
* refactor: integrate store-grpc module

* ref: change comments to en

* reformat&add comments

---------

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants