Skip to content

Pinecone retrieval improvements - #148

Merged
okhat merged 1 commit into
stanfordnlp:mainfrom
msnidal:feature/pinecone-retrieval-improvements
Oct 9, 2023
Merged

okhat merged 1 commit into
stanfordnlp:mainfrom
msnidal:feature/pinecone-retrieval-improvements

Conversation

@msnidal

@msnidal msnidal commented Oct 9, 2023

Copy link
Copy Markdown
Contributor

Firstly thank you for the great library. I was interested in using the Pinecone retrieval component and ran into several issues, including:

  • Hardcoded api/env placeholders
  • Undeclared dependency (reasonably to not force pinecone install for other users)
  • Lack of support for multiple queries (used only results of first embedding)
  • Deprecated openai embedding kwargs
    etc.

This aims to address these issues by overhauling the pinecone retrieval class in many ways: fixing the hardcoded keys, including the dependency as optional so you can use pip install dspy-ai[pinecone], and implementing similar behaviour to dsp.retrieveEnsemble to allow search against a query list, along with other code quality improvements.

@okhat

okhat commented Oct 9, 2023

Copy link
Copy Markdown
Collaborator

This is really cool, thanks @msnidal

@drawal1 how does this look to you, will it still support what you need

@drawal1 drawal1 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes it robust and flexible. Love it! I will test it later to see if it needs any modifications for my use case.
Note: retriever documentation needs to be updated here

@okhat
okhat merged commit d711ab0 into stanfordnlp:main Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants