Repository navigation
Commit 0911a80
[fix](tvf) Keep file modification time in TVF file listings
### What problem does this PR solve?
Issue Number: None
Related PR: #62023, #66684
Problem Summary: Since #62023 moved TVF file listing onto the filesystem SPI,
`ExternalFileTableValuedFunction#parseFile` builds each `TBrokerFileStatus` from a
`FileEntry` without its modification time, although the S3, HDFS, Azure and local listings
all fill `FileEntry#modificationTime`. Every `s3()` / `hdfs()` TVF range therefore reaches
BE with mtime 0.
BE keys cached blocks of external files by `path:mtime`. With the file cache on, which
becomes the default, an object overwritten in place under the same key keeps serving the
blocks cached from its old content. branch-4.x does not have this problem: its listing goes
through `BrokerUtil.parseFile`, which copies the listed modification time. This sets the
modification time again.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- `run-fe-ut.sh --run` on `ExternalFileTableValuedFunctionTest` (new
`testListedFilesCarryTheirModificationTime`), `TVFScanNodeTest`,
`SessionVariablesTest`, `VariableMgrTest`: 40 tests, 0 failures
- `mvn checkstyle:check` on fe-core: 0 violations
- Behavior changed: No (TVF ranges carry the listed modification time, as on branch-4.x)
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>1 parent 747dd54 commit 0911a80
2 files changed
Lines changed: 68 additions & 2 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris/tablefunction
- test/java/org/apache/doris/tablefunction
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
184 | 184 | | |
185 | 185 | | |
186 | 186 | | |
187 | | - | |
188 | | - | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
189 | 193 | | |
190 | 194 | | |
191 | 195 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
30 | 36 | | |
31 | 37 | | |
32 | 38 | | |
| 39 | + | |
| 40 | + | |
33 | 41 | | |
34 | 42 | | |
35 | 43 | | |
36 | 44 | | |
37 | 45 | | |
38 | 46 | | |
39 | 47 | | |
| 48 | + | |
40 | 49 | | |
41 | 50 | | |
42 | 51 | | |
| |||
45 | 54 | | |
46 | 55 | | |
47 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
48 | 110 | | |
49 | 111 | | |
50 | 112 | | |
| |||
0 commit comments