-
Notifications
You must be signed in to change notification settings - Fork 238
Description
Since Feathr is on Maven now (https://search.maven.org/artifact/com.linkedin.feathr/feathr_2.12), we should consider using Maven as the source to submit Spark jobs,rather than using the public wasb path, since it's a bit hard to maintain and doesn't distribute well (like don't have mirrors etc., and will be slow).
Databricks supports maven based library: https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/libraries#--library
Synapse can be achieved by using this config: spark.jars.packages. See (https://spark.apache.org/docs/3.2.1/configuration.html) for more details.