Create inventory of service principals and direct files access in Azure for a Spark session #310
Description
In Azure, data access is authorized using service principals using Spark session settings.
Spark session level settings
spark.conf.set("fs.azure.account.auth.type","OAuth")
spark.conf.set("fs.azure.account.oauth.provider.type","org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
spark.conf.set("fs.azure.account.oauth2.client.id","")
spark.conf.set("fs.azure.account.oauth2.client.secret",dbutils.secrets.get(scope="",key=""))
spark.conf.set("fs.azure.account.oauth2.client.endpoint","https://login.microsoftonline.com//oauth2/token")
We need a feature in the tool to create an inventory of all service principals and direct files/mount points that are currently being used in the workspace along with the objects in teh spark session