Repository navigation
Commit 75d80c7
[SPARK-51079] Added constructor for ArrowSerializer for backward compatibility
### What changes were proposed in this pull request?
Previously, [PR](#49790) extends ArrowSerializer class to support large variable types in pandas UDF. This PR makes it more backward-compatible by adding a new constructor for cases where the flag for large variable is irrelevant (default to false).
### Why are the changes needed?
To achieve versionless in spark connect, we need to be careful about changing a client API in Scala. For example, the change of an interface like this could affect Scala UDF referring to this class (as it is part of the spark connect client package) but running with an older spark connect client.
### How was this patch tested?
N/A. It's a new constructor.
Closes #50513 from haiyangsun-db/SPARK-51079.
Authored-by: Haiyang Sun <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>1 parent 24f600f commit 75d80c7
1 file changed
Lines changed: 6 additions & 0 deletions
File tree
- sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/arrow
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
55 | 61 | | |
56 | 62 | | |
57 | 63 | | |
| |||
0 commit comments