Skip to content

Commit 237a65d

Browse files
jfw-ppintkathole
authored andcommitted
fix: build Spark DataFrame from Arrow with schema and empty handling(feast-dev#5594)
Signed-off-by: Jacob Weinhold <[email protected]>
1 parent 611ead0 commit 237a65d

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • sdk/python/feast/infra/compute_engines/spark

‎sdk/python/feast/infra/compute_engines/spark/nodes.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,6 @@ def execute(self, context: ExecutionContext) -> DAGValue:
9090
else:
9191
spark_df = self.spark_session.createDataFrame(arrow_table.to_pandas())
9292

93-
9493
return DAGValue(
9594
data=spark_df,
9695
format=DAGFormat.SPARK,
@@ -103,6 +102,7 @@ def execute(self, context: ExecutionContext) -> DAGValue:
103102
},
104103
)
105104

105+
106106
class SparkAggregationNode(DAGNode):
107107
def __init__(
108108
self,

0 commit comments

Comments
 (0)