You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 95ab40c
Browse filesBrowse the repository at this point in the historyBrowse files
fix(operator): name the file each line came from when a file scan was asked to
A file scan with Extract and Include Filename on could not run at all in any
attribute type that reads line by line, which is the default. The two halves
disagreed about how wide a row is: the schema prepends a `filename` column
whenever the flag is set, while the line-by-line branch emitted the value alone,
so the first tuple was one field against two columns and could not be built.
Include Filename is only offered once Extract is on, so every configuration
reaching it is one the panel invites.
The branch now pairs each entry with its name, as the single-value branch
already did. Both exports follow: their `&& isSingle` was there to mirror the
defect, and the spec pinning that a line-mode export carries no filename column
now pins the column it does carry.
Closesapache#8598
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Copy file name to clipboardExpand all lines: common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/file/FileScanOpDesc.scala
+6-5Lines changed: 6 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -141,10 +141,10 @@ class FileScanOpDesc
141
141
(""*8, "_fn", "_f.read()", "_f")
142
142
}
143
143
144
-
//Match the platform (FileScanUtils.createTuplesFromFile): its line-by-line
145
-
//branch ignores outputFileName and emits only the value, so the filename
Copy file name to clipboardExpand all lines: common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/file/FileScanSourceOpDesc.scala
Copy file name to clipboardExpand all lines: common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/file/FileScanUtils.scala
Copy file name to clipboardExpand all lines: common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/file/FileScanOpDescSpec.scala
+12-3Lines changed: 12 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -191,11 +191,20 @@ class FileScanOpDescSpec extends AnyFlatSpec with BeforeAndAfter {
Copy file name to clipboardExpand all lines: common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/file/FileScanSourceOpDescSpec.scala
+30Lines changed: 30 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -369,6 +369,36 @@ class FileScanSourceOpDescSpec extends AnyFlatSpec with BeforeAndAfter {
0 commit comments