I haven't looked into that bit yet. Thanks for the advice!

I think this is at a good state for a serious review now.

I haven't tackled oracle support at all, I still need to add documentation, there's some cleanup to do and I suspect there's more tests that will be useful to add. But I think this demonstrates the general approach works.

@hannseman Thanks for catching that!

I just tried this PR on a local project and adding a new field with db_default set will cause the default to be dropped.

class TestModel(models.Model):
    foo = models.CharField(
        max_length=255, db_default="hej", null=True, blank=True
    )

Adding the field foo to TestModel will generate the following SQL:

BEGIN;
--
-- Add field foo to testmodel
--
ALTER TABLE "shipping_testmodel" ADD COLUMN "foo" varchar(255) DEFAULT 'hej' NULL;
ALTER TABLE "shipping_testmodel" ALTER COLUMN "foo" DROP DEFAULT;
COMMIT;

Is this a problem in the current version of the PR? I can't replicate the issue now.

Is this a problem in the current version of the PR? I can't replicate the issue now.

Looks better! Nice!

I also tried to add a migration which moves a field from null=True to null=False, for example from:

models.CharField(max_length=255, db_default="hej", null=True, blank=True)

to

models.CharField(max_length=255, db_default="hej", null=False, blank=True)

This gives me this prompt:

You are trying to change the nullable field 'foo' on testmodel to non-nullable without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Ignore for now, and let me handle existing rows with NULL myself (e.g. because you added a RunPython or RunSQL operation to handle NULL values in a previous data migration)
 3) Quit, and let me add a default in models.py

Not really sure how we want to handle this case as there could be NULL values in the column set explicitly by the user. We should handle it somehow at least since selecting option 1 and for example setting the prompted default to"prompteddefault" will generate the following migration:

BEGIN;
--
-- Alter field foo on testmodel
--
ALTER TABLE "shipping_testmodel" ALTER COLUMN "foo" SET DEFAULT 'prompteddefault';
UPDATE "shipping_testmodel" SET "foo" = 'hej' WHERE "foo" IS NULL;
ALTER TABLE "shipping_testmodel" ALTER COLUMN "foo" SET NOT NULL;
ALTER TABLE "shipping_testmodel" ALTER COLUMN "foo" DROP DEFAULT;
COMMIT;

I guess when using db_default we could skip the prompt and be fine with:

UPDATE "shipping_testmodel" SET "foo" = 'hej' WHERE "foo" IS NULL;
ALTER TABLE "shipping_testmodel" ALTER COLUMN "foo" SET NOT NULL;

There's also an issue when adding a new field with null=False, for example:

bar = models.CharField(max_length=255, db_default="hej", null=False)

This gives this prompt:

You are trying to add a non-nullable field 'bar' to testmodel without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Quit, and let me add a default in models.py
Select an option:

Thanks @hannseman! I'll get on those today!

@hannseman Those should be fixed now.

@charettes Do you have any ideas how to handle the bulk create on sqlite better? Or is my current implementation essentially the best we can do?

I had an idea how to enable bulk create to work fully on sqlite last night, which worked. Instead of using the DEFAULT keyword in sql, I can get the db_default expression from the field and pass that to the database.

I'm still only doing this on sqlite where the new supports_default_keyword_in_insert database feature is False. I think the DEFAULT keyword is cleaner where supported. But the sqlite implementation would work on all databases I think, so we could go that way too.

@hannseman Those should be fixed now.

Cool! Looks much better now!

Thanks @hannseman! Your test-driving has been very helpful for finding areas that need more work!

Closing due to inactivity.

@ian-foote Feel-free to send a new patch when you have time (maybe during the DjangoCon EU sprints 😉), thanks.