-
Notifications
You must be signed in to change notification settings - Fork 101
IndexError on access_jobs task #428
Copy link
Copy link
Closed
Labels
step/assessmentgo/uc/upgrade - Assessment Stepgo/uc/upgrade - Assessment Step
Description
The access_jobs task threw following err on test workspace:
IndexError
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
File ~/.ipykernel/1440/command--1-2145852919:18
15 entry = [ep for ep in metadata.distribution("databricks_labs_ucx").entry_points if ep.name == "runtime"]
16 if entry:
17 # Load and execute the entrypoint, assumes no parameters
---> 18 entry[0].load()()
19 else:
20 import databricks_labs_ucx
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/runtime.py:253, in main()
252 def main():
--> 253 trigger(*sys.argv)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/framework/tasks.py:93, in trigger(*argv)
90 cfg = WorkspaceConfig.from_file(Path(args["config"]))
91 logging.getLogger("databricks").setLevel(cfg.log_level)
---> 93 current_task.fn(cfg)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/runtime.py:105, in assess_jobs(cfg)
103 ws = WorkspaceClient(config=[cfg.to](http://cfg.to/)_databricks_config())
104 crawler = JobsCrawler(ws, RuntimeBackend(), cfg.inventory_database)
--> 105 crawler.snapshot()
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/assessment/crawlers.py:552, in JobsCrawler.snapshot(self)
551 def snapshot(self) -> list[JobInfo]:
--> 552 return self._snapshot(self._try_fetch, self._crawl)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/framework/crawlers.py:244, in CrawlerBase._snapshot(self, fetcher, loader)
242 raise err
243 logger.debug(f"[{self._full_name}] crawling new batch for {self._table}")
--> 244 loaded_records = list(loader())
245 self._append_records(loaded_records)
246 return loaded_records
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/assessment/crawlers.py:496, in JobsCrawler._crawl(self)
494 all_jobs = list(self._ws.jobs.list(expand_tasks=True))
495 all_clusters = {c.cluster_id: c for c in self._ws.clusters.list()}
--> 496 return self._assess_jobs(all_jobs, all_clusters)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/assessment/crawlers.py:538, in JobsCrawler._assess_jobs(self, all_jobs, all_clusters_by_id)
536 if cluster_config.init_scripts:
537 for init_script_info in cluster_config.init_scripts:
--> 538 init_script_data = _get_init_script_data(self._ws, init_script_info)
539 if not init_script_data:
540 continue
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/labs/ucx/assessment/crawlers.py:89, in _get_init_script_data(w, init_script_info)
87 def _get_init_script_data(w, init_script_info):
88 if init_script_info.dbfs:
---> 89 file_api_format_destination = init_script_info.dbfs.destination.split(":")[1]
90 if file_api_format_destination:
91 try:
IndexError: list index out of range
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
step/assessmentgo/uc/upgrade - Assessment Stepgo/uc/upgrade - Assessment Step
Type
Projects
Status
No status