[Bug] [connector-v2] Dependency Conflict #7528
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
The jar packages in the lib directory have dependency conflicts. For example, when running the s3 plugin, there are multiple Hadoop dependent jar files (hive jar files) that cause errors. When uploading other JDBC drivers, running the es plugin results in HTTP dependency jar conflicts. How to resolve such dependency conflicts
SeaTunnel Version
dev,2.3.8
SeaTunnel Config
{
"env": {
"execution.parallelism": "1",
"job.mode": "BATCH"
},
"source": [
{
"plugin_name":"jdbc",
"url":"jdbc:postgresql://******:5432/oceandatum?currentSchema=dw_label",
"password":"******",
"user":"******",
"driver":"org.postgresql.Driver",
"query": "SELECT \"id\",\"name\",\"label_column_fact\" FROM \"label_group_a\"",
"result_table_name": "fake"
}
],
"transform": [
{"plugin_name": "sql",
"source_table_name": "fake",
"result_table_name": "fake2",
"query": "SELECT concat(id,'$test') AS id,name AS label,concat('{','\"id\":\"',id,'\",','\"name\":\"',name,'\",','\"label_column_fact\":\"',label_column_fact,'\"','}') AS property FROM fake "
}
],
"sink": [
{
"plugin_name": "elasticsearch",
"hosts":["******:9200"],
"index":"relation",
"primary_keys":["id"]
}
]
}
Running Command
bin/seatunnel.sh -c demo.jsonError Exception
Caused by: java.io.IOException: org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider instantiation exception: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
Zeta or Flink or Spark Version
No response
Java or Scala Version
1.8
Screenshots
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct