Skip to content

[Bug] [clickhouse] CK连接器任务抽取完毕后无法正常停止 #7897

@YOMO-Lee

Description

@YOMO-Lee

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

当我通过clickhouse连接器抽取数据的时候,觉得速度比较慢,想调整并行度(虽然看到官网提示clickhouse连接器目前不支持并行),把parallelism 调整为2,这个时候当任务抽取完毕的时候,该任务并不会停止
已经验证过,必现,当我把并行度改为1的时候就没问题

虽然不支持并行度,但是期望调整 parallelism 参数并不会影响任务的正常结束
另外期望能开发该连接器,使其支持并行
可以提供思路,我来改

SeaTunnel Version

2.3.7

SeaTunnel Config

env {
  parallelism = 2
  job.mode = "BATCH"
}


source{
    Clickhouse {
    host = ""
    database = "fdc"
    sql = "select * from fdc.trace_data WHERE traceTime > '2024-10-24 04:00:00' and traceTime < '2024-10-24 04:00:20'"
    username = "reader"
    password = "reader@123"
    server_time_zone = "UTC"
    result_table_name = "fdc.trace_data"
    clickhouse.config = {
	  "max_memory_usage": "128000000000",
	  "max_bytes_before_external_group_by": "50000000000",
      "socket_timeout": "300000"
    }
  }
}


sink {
    jdbc {
        url = "jdbc:oceanbase:///ods"
        driver = "com.oceanbase.jdbc.Driver"
        user = "root"
        password = "Bh%Yac{C9l-~#?~dq[}bSQO5ng8Rr?w"
        compatible_mode = "mysql"
        generate_sink_sql = true
        database = ods
        table = trace_data
        data_save_mode = APPEND_DATA
    }
}

Running Command

./bin/seatunnel.sh --config job/ck2ob.config -e local

Error Exception

任务抽取完毕,但是停不下来

Zeta or Flink or Spark Version

zeta

Java or Scala Version

java

Screenshots

119180a7b9e22a7afeadc960f80a42a

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions