-
Notifications
You must be signed in to change notification settings - Fork 566
Closed
Labels
good first issueGood for newcomersGood for newcomersstage/staleIssues or PRs that had no activity for a long timeIssues or PRs that had no activity for a long timetype/umbrella
Description
Describe the proposal
Currently there are two sort connectors that can be running on both flink1.13 and flink1.15 which are starrocks and PostGreSQL
We should provide flink 1.15 support on all connectors that can only run on flink 1.13
They are :
- mysql-cdc
- elasticSearch
- hdfs
- iceberg
- jdbc
- kafka
- pulsar
- tubemq
- mongoDb-cdc
- sqlServer-cdc
- redis
- doris
- hbase
- hudi
- kudu
- oracle-cdc
- hive
How to tackle down
Take hudi connector as an Example
- Please make sure that you have read and understand this post : https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sourcessinks/
- Please make sure that you have read and understand every line of code in hudi connector on flink 1.13 (If things are not clear to you please present a issue or discussion on https://github.com/apache/inlong)
- Most of the systems have their flink connectors on their github page, for instance hudi (https://github.com/apache/hudi/blob/master/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java), we will import this dependency and add our inlong-specific features (metric, audit) on them.
- Make sure you add license after modify the TableFactory in /licenses/inlong-sort-connectors
- Run some tests on flink1.15 and make sure your connector works
- Modify inlong-distribution and make sure the connectors will be compiled just fine
Make reference for these two connectors:
#8235
#8193
Task list
- [Feature][Sort] Add Iceberg source on flink 1.15 #8643
- [Feature][Sort] Add Kafka connector on Flink 1.15 #8358
- [Feature][Sort] Add Pulsar connector based on flink 1.15 #8695
- [Feature][Sort] Add sqlserver connector of flink 1.15 #8999
- [Feature][Sort] Add HBase connector on flink 1.15 #9009
- [Feature] Add Mysql connector on flink 1.15 #8967
- [Feature][Sort] Add TubeMQ connector on flink 1.15 #8922
- [Feature][Sort] Add StarRocks connector on Flink 1.15 #8065
- [Feature][Sort] Add PostgreSQL source connector on flink 1.15 #8062
- [Feature] Add hudi connector on flink 1.15 #8994
- [Feature][Sort]Add mongodb connector on flink 1.15 #8959
- [Feature][Sort] Add Hive connector on flink 1.15 #9010
- [Feature][Sort] Add Redis connector on flink 1.15 #8948
- [Feature] [Sort] Add JDBC connector on Flink 1.15 #8464
InLong Component
InLong Sort
Are you willing to submit PR?
- Yes, I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
good first issueGood for newcomersGood for newcomersstage/staleIssues or PRs that had no activity for a long timeIssues or PRs that had no activity for a long timetype/umbrella