Partial upsert let users choose to update only specific columns and ignore the rest, and apply complex merging logic based on the existing row and the new row that shares the same primary key.

Pinot allows you to run any function using Apache Groovy scripts. To support complex use cases (e.g merge previous and new values based on conditions), we proposed to enhance partial upsert with groovy transformation function.

a few challenges to consider

• 1. limitation of Groovy function regex, currently the groovy evaluator doesn't support multi-line groovy script.
• 2. FunctionEvaluator interface currently only take one GenericRow as input.
• 2.1 For partial upsert we need to access both old row and the new row.
• 2.2 How to represent the argument and how to bind the arguments with values for both previous row and the new row.
• 3. Integrate Groovy function in the partial upsert handler