Skip to content

rows_from_file() raises confusing error if file-like object is not in binary mode #520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
simonw opened this issue Jan 2, 2023 · 3 comments
Closed

rows_from_file() raises confusing error if file-like object is not in binary mode #520

simonw opened this issue Jan 2, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@simonw
Copy link
Owner

simonw commented Jan 2, 2023

I got this error:

  File "/Users/simon/Dropbox/Development/openai-to-sqlite/openai_to_sqlite/cli.py", line 27, in embeddings
    rows, _ = rows_from_file(input)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/simon/.local/share/virtualenvs/openai-to-sqlite-jt4obeb2/lib/python3.11/site-packages/sqlite_utils/utils.py", line 305, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
                  ^^^^^^^^^^^^^^^^^^^

From this code:

@cli.command()
@click.argument(
    "db_path",
    type=click.Path(file_okay=True, dir_okay=False, allow_dash=False),
)
@click.option(
    "-i",
    "--input",
    type=click.File("r"),
    default="-",
)
def embeddings(db_path, input):
    "Store embeddings for one or more text documents"
    click.echo("Here is some output")
    db = sqlite_utils.Database(db_path)
    rows, _ = rows_from_file(input)
    print(list(rows))

The error went away when I changed it to type=click.File("rb").

This should either be called out in the documentation or rows_from_file() should be fixed to handle text-mode files in addition to binary files.
@simonw simonw added the bug Something isn't working label Jan 2, 2023
@simonw
Copy link
Owner Author

simonw commented Jan 3, 2023

The documentation here does at least say the following: https://sqlite-utils.datasette.io/en/3.30/python-api.html#reading-rows-from-a-file

  • fp (BinaryIO) -- a file-like object containing binary data

@mcarpenter
Copy link
Contributor

Hey, isn't this essentially the same issue as #448 ?

@simonw
Copy link
Owner Author

simonw commented May 8, 2023

Hey, isn't this essentially the same issue as #448 ?

Yes it is, good catch!

@simonw simonw mentioned this issue May 8, 2023
Closed
@simonw simonw closed this as completed in dab2388 May 8, 2023
simonw added a commit that referenced this issue May 8, 2023
@simonw
Release 3.31
b3b100d 
Refs #509, #517, #520, #525, #527, #532, #534, #536, #537, #538, #539
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@simonw @mcarpenter