Skip to content

Add source Reference metadata to Chat response #143

Description

@gjreda

A Chat response currently only includes the message text. Ideally, we'd like to cite the References that were used to create the message response.

We should update the response to include metadata about the source References.

  • Update ingest to use pypdf for text extraction on all PDFs so that text for grobid success/fails are parsed in a consistent way. This will allow us to also include page numbers on our Reference metadata.
  • Add page number to Chunk metadata
  • Add reference metadata to ranker context here so that our ranker query returns appropriate metadata
  • Update chat to include relevant reference metadata in in prompt
  • Update LLM response to include citations of appropriate references (might need to play around with LLM prompt)
  • Update CLI params to allow for n_choices to be utilized
  • Update python/readme.md

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions