Skip to content

Mechanism for forcing column-type, over-riding auto-detection #100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
simonw opened this issue Apr 16, 2020 · 3 comments
Closed

Mechanism for forcing column-type, over-riding auto-detection #100

simonw opened this issue Apr 16, 2020 · 3 comments
Labels
enhancement New feature or request

Comments

@simonw
Copy link
Owner

simonw commented Apr 16, 2020

As seen in dogsheep/github-to-sqlite#27 (comment) - there's a problem where you insert a record with a None value for a column and that column is created as TEXT - but actually you intended it to be an INT (as later examples will demonstrate).

Some kind of mechanism for over-riding the detected types of columns would be useful here.

@simonw simonw added the enhancement New feature or request label Apr 16, 2020
@simonw
Copy link
Owner Author

simonw commented Apr 16, 2020

The syntax for explicitly creating a table looks like this:

db["cats"].create({
    "id": int,
    "name": str,
    "weight": float,
}, pk="id")

So using a dictionary of column name -> Python type here would be good.

I think it becomes a new optional argument to .insert_all() and .insert() and .upsert_all() and .upsert(). Question is, what should that argument be called?

Options:

  • columns
  • types
  • type_overrides
  • extra_columns

I like the first two best. I'm leaning towards columns - it would serve two purposes: you can use it to ensure extra columns are created, and you can use it to specify the type on extra OR deteced columns.

Maybe something like this:

db["authors"].insert_all(
    [{"id": 1, "name": "Sally", "score": "2"}],
    pk="id",
    not_null={"name", "score"},
    defaults={"score": 1},
    columns={"score": int, "weight": float}
)

@simonw
Copy link
Owner Author

simonw commented Apr 16, 2020

Reminder: the current list of arguments for those methods is:

def insert_all(
self,
records,
pk=DEFAULT,
foreign_keys=DEFAULT,
column_order=DEFAULT,
not_null=DEFAULT,
defaults=DEFAULT,
batch_size=DEFAULT,
hash_id=DEFAULT,
alter=DEFAULT,
ignore=DEFAULT,
replace=DEFAULT,
extracts=DEFAULT,
conversions=DEFAULT,
upsert=False,
):

@simonw
Copy link
Owner Author

simonw commented Apr 17, 2020

I'm going with columns={"score": int}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant