Repository navigation
Warn when extract() receives a string naming an existing file - #597
Closed
inamdarmihir wants to merge 1 commit into
Closed
inamdarmihir wants to merge 1 commit into
inamdarmihir wants to merge 1 commit into
Conversation
|
Your branch is 1 commits behind git fetch origin main
git merge origin/main
git pushNote: Enable "Allow edits by maintainers" to allow automatic updates. |
Collaborator
|
Thanks for taking a look at #249, @inamdarmihir. After reviewing the tradeoffs, we want to avoid adding filesystem |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
extract()treats any string as literal text. When a file path is passed astext_or_documents, the path string itself becomes the document sent to the model, which then tends to map it onto the schema (e.g. avenueextraction returning a directory name) or hallucinate content, and can leak local paths and usernames into results (#249).This makes
extract()emit aUserWarningwhen the string is a single-line path of an existing file, telling the user to read the file first. Behavior is otherwise unchanged: the string is still processed as literal text, so no existing caller breaks.The check is deliberately narrow so ordinary text never triggers it or hits the filesystem unnecessarily: it only runs for strings of 1 to 4096 characters with no newline, and uses
os.path.isfile(so directories and nonexistent paths don't warn). The warning usesstacklevel=3so it points at the caller oflx.extract, not the wrapper. The docstring fortext_or_documentsnow states that non-URL strings are never treated as paths.This is an independent change from #489, which was closed without merging.
Fixes #249
Other
How Has This Been Tested?
Added
FilePathInputWarningTestintests/init_test.py: absolute and relative existing paths warn (message contains the path); plain text, a nonexistent path, a directory, an empty string, multi-line text starting with a real path, a string with a null byte, and aDocumentlist do not warn; strings over the length limit skip theisfilecall; the warning is attributed to the caller's file.The 3 failures are in
tests/create_provider_plugin_test.py(offline setuptools install) and fail identically on an unmodifiedmain.Checklist:
Code of conduct.
Contributing
page, and I either signed the Google
Individual CLA
or am covered by my company's
Corporate CLA.
issue(s) and we have agreed upon the general approach.
issue(s) that documentation elsewhere needs updating.
pylintover the affected code.