Added parallel fetching of registered model identifiers to speed-up assessment workflow#691
Merged
Added parallel fetching of registered model identifiers to speed-up assessment workflow#691
Conversation
HariGS-DB
had a problem deploying
to
account-admin
GitHub Actions
Failure
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #691 +/- ##
==========================================
- Coverage 81.16% 80.26% -0.90%
==========================================
Files 38 38
Lines 3875 4069 +194
Branches 718 751 +33
==========================================
+ Hits 3145 3266 +121
- Misses 571 639 +68
- Partials 159 164 +5 ☔ View full report in Codecov by Sentry. |
Contributor
Author
|
Screenshot of test_workspace_access.test_workspace_acces_e2e. |
nfx
requested changes
Dec 14, 2023
HariGS-DB
had a problem deploying
to
account-admin
GitHub Actions
Failure
Contributor
Author
|
Changes done. |
nfx
requested changes
Dec 14, 2023
HariGS-DB
had a problem deploying
to
account-admin
GitHub Actions
Failure
nfx
requested changes
Dec 14, 2023
HariGS-DB
had a problem deploying
to
account-admin
GitHub Actions
Failure
nfx
changed the title
nfx
changed the title
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `workspace_group_name` and `account_group_name` to `make_ucx_group` fixture ([#664](#664)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added filters to run only cloud specific task ([#681](#681)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Cleanup README.md ([#695](#695)). * Cleanup installer framework and speed up test execution ([#711](#711)). * Decouple group configuration from `install.py` ([#714](#714)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed for secret scope apply task always raises ValueError ([#683](#683)). * Fixed some flaky integration tests ([#700](#700)). * More `mypy` chores ([#697](#697)). * Moved `ExternalLocations` and `Mounts` to `locations` module ([#692](#692)). * New CLI command for workspace mapping ([#678](#678)). * No more `mypy` chores after this ([#699](#699)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Some `mypy` chores ([#696](#696)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)). * Updated installation documentation to use Databricks CLI ([#686](#686)).
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `workspace_group_name` and `account_group_name` to `make_ucx_group` fixture ([#664](#664)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added filters to run only cloud specific task ([#681](#681)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Cleanup README.md ([#695](#695)). * Cleanup installer framework and speed up test execution ([#711](#711)). * Decouple group configuration from `install.py` ([#714](#714)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed for secret scope apply task always raises ValueError ([#683](#683)). * Fixed some flaky integration tests ([#700](#700)). * More `mypy` chores ([#697](#697)). * Moved `ExternalLocations` and `Mounts` to `locations` module ([#692](#692)). * New CLI command for workspace mapping ([#678](#678)). * No more `mypy` chores after this ([#699](#699)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Some `mypy` chores ([#696](#696)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `workspace_group_name` and `account_group_name` to `make_ucx_group` fixture ([#664](#664)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added filters to run only cloud specific task ([#681](#681)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Cleanup README.md ([#695](#695)). * Cleanup installer framework and speed up test execution ([#711](#711)). * Decouple group configuration from `install.py` ([#714](#714)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed for secret scope apply task always raises ValueError ([#683](#683)). * Fixed some flaky integration tests ([#700](#700)). * More `mypy` chores ([#697](#697)). * Moved `ExternalLocations` and `Mounts` to `locations` module ([#692](#692)). * New CLI command for workspace mapping ([#678](#678)). * No more `mypy` chores after this ([#699](#699)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Some `mypy` chores ([#696](#696)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
Closed
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `workspace_group_name` and `account_group_name` to `make_ucx_group` fixture ([#664](#664)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added filters to run only cloud specific task ([#681](#681)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Cleanup README.md ([#695](#695)). * Cleanup installer framework and speed up test execution ([#711](#711)). * Decouple group configuration from `install.py` ([#714](#714)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed for secret scope apply task always raises ValueError ([#683](#683)). * Fixed some flaky integration tests ([#700](#700)). * More `mypy` chores ([#697](#697)). * Moved `ExternalLocations` and `Mounts` to `locations` module ([#692](#692)). * New CLI command for workspace mapping ([#678](#678)). * No more `mypy` chores after this ([#699](#699)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Some `mypy` chores ([#696](#696)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
Closed
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `databricks labs ucx validate-external-locations` command for cli ([#715](#715)). * Added `workspace_group_name` and `account_group_name` to `make_ucx_group` fixture ([#664](#664)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added filters to run only cloud specific task ([#681](#681)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Cleanup README.md ([#695](#695)). * Cleanup installer framework and speed up test execution ([#711](#711)). * Decouple group configuration from `install.py` ([#714](#714)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed for secret scope apply task always raises ValueError ([#683](#683)). * Fixed some flaky integration tests ([#700](#700)). * More `mypy` chores ([#697](#697)). * Moved `ExternalLocations` and `Mounts` to `locations` module ([#692](#692)). * New CLI command for workspace mapping ([#678](#678)). * No more `mypy` chores after this ([#699](#699)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Some `mypy` chores ([#696](#696)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx ensure-assessment-run` to CLI commands ([#708](#708)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `databricks labs ucx validate-external-locations` command for cli ([#715](#715)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed secret scope apply task to raise ValueError ([#683](#683)). * Fixed legacy table ACL ownership migration and other integration testing issues ([#722](#722)). * Fixed some flaky integration tests ([#700](#700)). * New CLI command for workspace mapping ([#678](#678)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
Merged
nfx
added a commit
that referenced
this pull request
Dec 21, 2023
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx ensure-assessment-run` to CLI commands ([#708](#708)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `databricks labs ucx validate-external-locations` command for cli ([#715](#715)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed secret scope apply task to raise ValueError ([#683](#683)). * Fixed legacy table ACL ownership migration and other integration testing issues ([#722](#722)). * Fixed some flaky integration tests ([#700](#700)). * New CLI command for workspace mapping ([#678](#678)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
FastLee
pushed a commit
that referenced
this pull request
Jan 19, 2024
* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([#682](#682)). * Added `databricks labs ucx ensure-assessment-run` to CLI commands ([#708](#708)). * Added `databricks labs ucx installations` command ([#679](#679)). * Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([#680](#680)). * Added `databricks labs ucx validate-external-locations` command for cli ([#715](#715)). * Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([#653](#653)). * Added cluster override and handle case of write protected DBFS ([#610](#610)). * Added cluster policy selector in the installer ([#655](#655)). * Added detailed UCX pre-requisite information to README.md ([#689](#689)). * Added interactive wizard for `databricks labs uninstall ucx` command ([#657](#657)). * Added more granular error retry logic ([#704](#704)). * Added parallel fetching of registered model identifiers to speed-up assessment workflow ([#691](#691)). * Added retry on workspace listing ([#659](#659)). * Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([#650](#650)). * Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([#661](#661)). * Changed the fields of the table "Tables" to lower case ([#684](#684)). * Enabled integration tests for `EXTERNAL` table migrations ([#677](#677)). * Enforced `mypy` validation ([#713](#713)). * Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([#705](#705)). * Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([#654](#654)). * Fixed secret scope apply task to raise ValueError ([#683](#683)). * Fixed legacy table ACL ownership migration and other integration testing issues ([#722](#722)). * Fixed some flaky integration tests ([#700](#700)). * New CLI command for workspace mapping ([#678](#678)). * Reduce server load for getting workspace groups and their members ([#666](#666)). * Throwing ManyError on migrate-groups tasks ([#710](#710)). * Updated installation documentation to use Databricks CLI ([#686](#686)). Dependency updates: * Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([#651](#651)). * Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([#687](#687)). * Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([#712](#712)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change description:
This PR makes changes to the model listing process. Currently the model listing is done serially which takes time in workspace with large number of registered models. This changes uses python threads to parallelize model listing to speed up the inventory process for models
Changes details:
Updates to generic.py.model_listing() to use Threads.gather to parallelize get_model function
It accepts num_threads from config.
Also updated the relevant test cases