Repository navigation
CompleteRequestSettings is insufficiently abstract #2735
Description
Note: I could also file this as a bug as many models do not work with SK and the Oobabooga and HuggingFace connectors because of this problem.
When I started using other completion APIs, e.g. Oobabooga, HuggingFace, the design around Microsoft.SemanticKernel.AI.TextCompletion.CompleteRequestSettings gets inconvenient quickly because behind those APIs the parameters vary, and the SK implementation is specifically shaped around OpenAI API.
Seems like this one design element is a big blocker to the vision of a pluggable system.
A simple (but not terribly robust) solution would be to get rid of CompleteRequestSettings altogether and allow the use of a name/value collection instead.
Reacted by Roger Barreto
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
markwallace-microsoft commentedon Sep 7, 2023
Shawn Callegari (@shawncal) I have been working on a solution for this issue, so I can take this one.
douglasware commentedon Sep 8, 2023
I've added support for Function Calling in my fork this week and it's pretty obvious that the library painted itself into a corner here with ChatRequestSettings, CompleteRequestSettings, and ISKFunction. You are going to be forced to do a big refactor soon and break some stuff.
Is there a branch I can watch to follow along?
markwallace-microsoft commentedon Sep 11, 2023
Doug Ware (@douglasware) I'll a link to the PR once I have something to share. I expect to have something later this week.
markwallace-microsoft commentedon Sep 15, 2023
Doug Ware (@douglasware) Here's the PR #2829. I still have more testing to do but any feedback is much appreciated.
markwallace-microsoft commentedon Sep 19, 2023
Doug Ware (@douglasware) Could you take a look at this comment #2829 (comment) to see does this change make sense
.Net: Option #3 - Refactor to support generic LLM request settings (u…
.Net: Option #3 - Refactor to support generic LLM request settings (u…