bug: file:/// is not supported on services-hdfs #6555
Description
Describe the bug
I found file:/// namenode path is not supported on services-hdfs when I tested datafusion-comet using opendal to access a hadoop fake filesystem. This is because we trimmed trailing / of namenode, causing file:/// to become file:.
opendal/core/src/services/hdfs/backend.rs
Line 79 in dd242e0
error:
25/09/02 16:50:42 INFO core/src/lib.rs: Comet native library version 0.10.0 initialized
hdfsBuilderConnect(forceNewInstance=0, nn=fake:, port=0, kerbTicketCachePath=(NULL), userName=(NULL)) error:
UnknownHostException: fakejava.lang.IllegalArgumentException: java.net.UnknownHostException: fake
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:466)
at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:134)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:374)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:308)
at org.apache.hadoop.hdfs.DistributedFileSystem.initDFSClient(DistributedFileSystem.java:202)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:278)
at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:275)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:275)
at org.apache.comet.Native.executePlan(Native Method)
Steps to Reproduce
There is a test fake filesystem that only replaces the prefix with fake:///: https://github.com/apache/datafusion-comet/blob/13cb948dc5ae5376531563318962d8ee38ae4286/spark/src/test/java/org/apache/comet/hadoop/fs/FakeHDFSFileSystem.java#L28
Expected Behavior
Supports scheme like: fake:///
Additional Context
No response
Are you willing to submit a PR to fix this bug?
- Yes, I would like to submit a PR.