Skip to content

Comments

Add async support#24

Merged
tonybaloney merged 7 commits intotonybaloney:mainfrom
cmbrose:cmbrose/async
May 26, 2025
Merged

Add async support#24
tonybaloney merged 7 commits intotonybaloney:mainfrom
cmbrose:cmbrose/async

Conversation

@cmbrose
Copy link
Contributor

@cmbrose cmbrose commented May 19, 2025

Adds --async support for all models. Adds GitHubAsyncModels as a new model helper class to handle that. Also adds a _Shared base class between GitHubAsyncModels and GitHubModels to avoid some duplication.

The sync and async models are registered with the same display text, so they don't show up twice in llm models. This is how the openai plugin does it for example.

$ llm models -q github
GitHub Models: github/AI21-Jamba-1.5-Large
GitHub Models: github/AI21-Jamba-1.5-Mini
GitHub Models: github/Codestral-2501
GitHub Models: github/Cohere-command-r
GitHub Models: github/Cohere-command-r-08-2024
...

The exact same models appear for --async since they all support async now.

$ llm models -q github --async
GitHub Models: github/AI21-Jamba-1.5-Large
GitHub Models: github/AI21-Jamba-1.5-Mini
GitHub Models: github/Codestral-2501
GitHub Models: github/Cohere-command-r
GitHub Models: github/Cohere-command-r-08-2024

@tonybaloney tonybaloney merged commit 9bcb138 into tonybaloney:main May 26, 2025
1 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants