Skip to content

[Improve] [Zeta][IMAP store] need improve the HDFS configuration #6494

@liunaijie

Description

@liunaijie

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

https://github.com/apache/seatunnel/blob/dev/seatunnel-engine/seatunnel-engine-storage/imap-storage-plugins/imap-storage-file/src/main/java/org/apache/seatunnel/engine/imap/storage/file/config/HdfsConfiguration.java
We use hazelcast imap to store the job status, and it will lose when cluster restart as it's only in memory, so we need persistence the data to avoid missing history data after restart.
Now we can store them to hdfs, localfile, oss. But i find the HDFS configuration only setted the fs.defaultFS parameter.
We can improve it add more parameter like hdfs_site_file_path, some config related kerberos

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions