Feature Description (功能描述)

• mailing list link: https://lists.apache.org/thread/w92mrt8b8n93yvxfqy9hvy17p355hscg
• preview release: https://github.com/hugegraph/hugegraph/releases/tag/pd-store-tmp
• deployment guide: https://github.com/apache/incubator-hugegraph/wiki/HugeGraph-Distributed-(pd%E2%80%90store)-Version-Deployment-Guide
• project tracking: HugeGraph PD-Store Tasks
• sub summary issue:
  • [Summary] integrate new module PD into hugegraph #2481
  • [Summary] integrate new module Store into hugegraph #2482
  • [Summary] integrate new backend Hstore into hugegraph #2483
  • [Summary] Dropping Support for Java 8: Compatibility Breaks Ahead #2499
  • [Summary] Feedback on the trial of HugeGraph Distributed (pd-store) Version (BETA) #2512

Background

Currently, the architecture of the community version of HugeGraph is still in version 1.0. In the internal version 2.0 of HugeGraph, we have the following design goals:

• Support trillion-scale data storage and tens of thousands of graph storages.
• Support multi-active, high availability, dynamic scalability, and automated operation and maintenance.
• Maximize read and write performance.

Based on these goals, we have designed a distributed architecture that supports graph data partitioning and multiple replicas, and separates storage from computation for flexible scaling.

In version 2.0, in addition to the hugegraph-server, we also introduce two additional components: hugegraph-pd and hugegraph-store

The responsibilities of these two components are as follows:

• hugegraph-pd: pd stands for placement driver, which can be simply understood as a meta server responsible for service discovery, partition information storage, and node scheduling.
• hugegraph-store: as a new built-in storage backend, it uses RocksDB as the distributed backend storage foundation.

We will gradually merge the internal version 2.0 of HugeGraph into the community version. Therefore, the first step is to integrate the hugegraph-pd and hugegraph-store modules into this repository.

Tasks

We will reorganize commit messages and merge changes ahead of the pd-store branch into the master branch:

• pd-grpc, pd-common, pd-client (~8k loc)
• pd-core (~9k loc)
• pd-service (~11k loc)
• pd-dist, pd-test (~1k loc)

• store-grpc, store-common, store-client (~13k loc)
• store-rocksdb (~4k loc)
• store-core (~14k loc)
• store-node (~11k loc)
• store-dist, store-test, store-cli (~3k loc)

• server-hstore (~3k loc)
• server-core (~12k loc)

flowchart TD
  A(hg-pd-grpc)
  B(hg-pd-common)
  C(hg-pd-client)
  D(hugegraph-core)
  E(hugegraph-hstore)
  F(hg-store-grpc)
  G(hg-store-common)
  H(hg-store-client)
  J(hg-pd-core)
  K(hg-pd-service)
  L(hg-pd-dist)
  M(hg-store-cli)
  N(hg-store-rocksdb)
  O(hg-store-core)
  P(hg-store-node)
  Q(hg-store-dist)
  B-->A
  C-->B
  D-.->C
  E-->D
  H-->F
  H-->G
  H-->C
  D-->G
  E-->H
  O-->D
  J-->B
  K-->J
  L-->K
  M-->H
  N-->G
  O-.->H
  O-->N
  P-->O
  Q-->P

Project Structure

# new project structure
                   - core/api/test/...
          - server - hbase/rocksdb/mysql/...
          -        - ...
hugegraph - pd - pd submodules
          -
          - store - store submodules