HadoopTableOperations might swallow IOException when doing commit #1349
Description
We met a condition that it failed to update version-hint.text when doing commit due to IOException, but the exception is swallowed.
The table is created by HadoopCatalog, by a user with proper permissions to access the directory, so the initial v1.metadata.json and version-hint.text are generated correctly.
After that, another job tries to update the table, with another user but without proper permissions. The subsequent v#.metadata.json could be generated correctly, but it fails to update version-hint.text so its content is always 1.
We noticed that failure by seeing no data when trying to read that Iceberg table and found the following exception in the log:
java.io.FileNotFoundException: xxxxx/version-hint.text (Permission denied)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:222)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:209)
at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:295)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:328)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:398)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789)
at org.apache.iceberg.hadoop.HadoopTableOperations.writeVersionHint(HadoopTableOperations.java:273)
at org.apache.iceberg.hadoop.HadoopTableOperations.commit(HadoopTableOperations.java:162)
at org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$5(BaseTransaction.java:344)
at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:403)
at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:212)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:188)
at org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:329)
at org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:220)