You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Native UDFs: config to disable or restrict library loading #6175
CometNativeUDF.register loads any shared library at any path the caller names, on the driver and
then on every executor, with the full privileges of the JVM process. There is no configuration an
operator can set to turn the feature off or to limit which paths may be loaded.
Options:
spark.comet.nativeUdf.enabled, checked in register on the driver and in the planner on the
executor, so a plan built elsewhere cannot bypass it. Whether it defaults to on (the feature is
already opt-in by API call) or off (loading native code should need an operator's consent) is
the main question for this issue.
spark.comet.nativeUdf.allowedPaths, a list of directory prefixes. Paths would be canonicalized
before the check so .. and symlinks cannot escape it.
The executor-side check matters as much as the driver-side one: the library path travels in the NativeScalarUdf proto, so the executor must not trust that the driver checked it.
Activity