Skip to content

Comments

Refresh models, add a filter to only include the GH catalog#40

Merged
tonybaloney merged 2 commits intotonybaloney:mainfrom
cmbrose:cmbrose/refresh-model
Jun 10, 2025
Merged

Refresh models, add a filter to only include the GH catalog#40
tonybaloney merged 2 commits intotonybaloney:mainfrom
cmbrose:cmbrose/refresh-model

Conversation

@cmbrose
Copy link
Contributor

@cmbrose cmbrose commented Jun 6, 2025

Refreshing models and was digging into #19 at the same time. Turns out that mistral-large is Retired as of April 15, which explains the error (see here).

I can't find any way to add a deprecation filter to our https://api.catalog.azureml.ms/asset-gallery/v1.0/models API call when fetching models, so instead I opted to also fetch the GH Models catalog and intersect the lists (the Azure catalog has much richer data so wanted to keep that around still).

Annoyingly, this filter actually causes us to drop models that technically are supported (e.g. Cohere Embed 4), but are not in the catalog. As an alternative approach we could add a filter against Azure's deprecation page, but it seems like we should be in parity with the GH Models catalog anyways.

| Cohere-command-r-plus-08-2024 | ❌ | ✅ | text | text |
| DeepSeek-R1 | ❌ | ❌ | text | text |
| DeepSeek-V3 | ❌ | ❌ | text | text |
| DeepSeek-R1-0528 | ❌ | ❌ | text | text |
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DeepSeek-V3 is gone?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the catalog filter and changed this PR to just be a model refresh which brough V3 back. It's still available in azureml, but is currently deprecated and set to retire in August

@cmbrose cmbrose marked this pull request as draft June 8, 2025 10:54
@cmbrose
Copy link
Contributor Author

cmbrose commented Jun 8, 2025

@tonybaloney - I reverted the change to add the GH catalog filter, this is just a model refresh now with existing logic. I'll file an issue so we can talk about this more, as I was debugging I found out that it's more complicated than I originally thought 😅

@cmbrose cmbrose marked this pull request as ready for review June 8, 2025 11:02
@tonybaloney
Copy link
Owner

it's more complicated

yeah, I'm learning that too

@tonybaloney tonybaloney merged commit 71f67e7 into tonybaloney:main Jun 10, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants