Skip to content

[Bug] [connector-clickhouse] Clickhouse sharding_key 配置没有按预期进行 #8161

@supernovaYe

Description

@supernovaYe

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

生产的Clickhouse集群超过30台机器,但实际只分发了其中8台,进行了本地测试,发现获取分发shard的代码没有按预期执行

SeaTunnel Version

2.3.8

SeaTunnel Config

env {
  # You can set SeaTunnel environment configuration here
  parallelism = 1
  job.mode = "BATCH"
  checkpoint.interval = 10000
  spark.executor.instances = 1
  spark.executor.cores = 1
  spark.executor.memory = "1g"
  spark.master = local
  job.name = "SeaTunnel"
}

source {
    FakeSource {
        row.num = 1
        parallelism = 1
        schema = {
          fields {
            event_name = string
          }
        }
        result_table_name = "fake"
    }
}

sink {
  Clickhouse {
    host = "xxx.xxx.xxx.xxx:8123"
    database = "db"
    table = "table"
    username = "superNova"
    password = "********"
    bulk_size = 10000
    clickhouse.socket_timeout = 50000
    split_mode = true
    sharding_key = "event_name"
  }
}

Running Command

seatunnel-spark-connector-v2-example
SeaTunnelApiExample class

Error Exception

no exception

Zeta or Flink or Spark Version

spark

Java or Scala Version

java 1.8
scala 2.11.12

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions