Skip to content

fix HBase PrefixFilter bug#2364

Merged
simon824 merged 6 commits intoapache:masterfrom
haohao0103:hao-dev
Nov 29, 2023
Merged

fix HBase PrefixFilter bug#2364
simon824 merged 6 commits intoapache:masterfrom
haohao0103:hao-dev

Conversation

@haohao0103
Copy link
Copy Markdown
Collaborator

@haohao0103 haohao0103 commented Nov 27, 2023 •
edited by javeme
Loading

Purpose of the PR

Main Changes

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • xxx

Does this PR potentially affect the following parts?

  • Nope
  • Dependencies (add/update license info)
  • Modify configurations
  • The public API
  • Other affects (typed here)

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

@javeme javeme changed the title fix #2177 fix HBase PrefixFilter bug Nov 27, 2023
javeme
javeme reviewed Nov 27, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@javeme javeme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution

@codecov
Copy link
Copy Markdown

codecov bot commented Nov 27, 2023 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c0ea21e) 65.88% compared to head (90780fc) 65.87%.
Report is 2 commits behind head on master.

Additional details and impacted files 
@@             Coverage Diff              @@
##             master    #2364      +/-   ##
============================================
- Coverage     65.88%   65.87%   -0.02%     
  Complexity      826      826              
============================================
  Files           507      507              
  Lines         42073    42076       +3     
  Branches       5831     5832       +1     
============================================
- Hits          27720    27716       -4     
- Misses        11595    11608      +13     
+ Partials       2758     2752       -6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JackyYangPassion JackyYangPassion mentioned this pull request Nov 27, 2023
Closed
JackyYangPassion
JackyYangPassion approved these changes Nov 27, 2023
View reviewed changes
...er/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseSessions.java Outdated
*/
default R scan(String table, byte[] prefix) {
return this.scan(table, prefix, true, prefix);
final Scan scan = new Scan().setRowPrefixFilter(prefix);
Copy link
Copy Markdown
Contributor

@JackyYangPassion JackyYangPassion Nov 27, 2023 •
edited by imbajin
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

    public Scan setRowPrefixFilter(byte[] rowPrefix) {
        if (rowPrefix == null) {
            this.setStartRow(HConstants.EMPTY_START_ROW);
            this.setStopRow(HConstants.EMPTY_END_ROW);
        } else {
            this.setStartRow(rowPrefix);
            this.setStopRow(this.calculateTheClosestNextRowKeyForPrefix(rowPrefix));
        }

        return this;
    }

calculateTheClosestNextRowKeyForPrefix(rowPrefix)

The core of this method accurately calculates stop row, thereby solving the problem of edge table query seek HFile filtering invalid key-value cells, thus reducing subgraph query p99.

Copy link
Copy Markdown
Member

@imbajin imbajin Nov 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculateTheClosestNextRowKeyForPrefix

calculateTheClosestNextRowKeyForPrefix() doesn't include for now?

Copy link
Copy Markdown
Collaborator Author

@haohao0103 haohao0103 Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setRowPrefixFilter will call calculateTheClosestNextRowKeyForPrefix method

javeme
javeme previously approved these changes Nov 27, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@javeme javeme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

JackyYangPassion
JackyYangPassion requested changes Nov 28, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@JackyYangPassion JackyYangPassion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some ci error need to fix

JackyYangPassion
JackyYangPassion approved these changes Nov 29, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@JackyYangPassion JackyYangPassion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JackyYangPassion
Copy link
Copy Markdown
Contributor

JackyYangPassion commented Nov 29, 2023 •
edited
Loading

ci-cassandra failure is not related to this PR.

log

Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [/127.0.0.1:9042] Cannot connect))

@simon824 simon824 merged commit d376b02 into apache:master Nov 29, 2023
VGalaxies pushed a commit to VGalaxies/incubator-hugegraph that referenced this pull request Jan 12, 2024
@haohao0103 @VGalaxies
fix HBase PrefixFilter bug (apache#2364)
422a8cd 
* apache#2177

* apache#2177

* apache#2177

* apache#2177

* apache#2177

* apache#2177
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@imbajin imbajin imbajin left review comments

@JackyYangPassion JackyYangPassion JackyYangPassion approved these changes

@simon824 simon824 simon824 approved these changes

@javeme javeme javeme left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug] Due to a bug in HBase PrefixFilter, the performance of xhop subgraph query has been degraded.

5 participants

@haohao0103 @JackyYangPassion @javeme @imbajin @simon824