Skip to content

Fixed #28821 -- Made QuerySet.bulk_create() work for multi-table inheritance when possible.#13763

Closed
jdufresne wants to merge 1 commit intodjango:masterfrom
jdufresne:multi-table-bulk-insert
Closed

Fixed #28821 -- Made QuerySet.bulk_create() work for multi-table inheritance when possible.#13763
jdufresne wants to merge 1 commit intodjango:masterfrom
jdufresne:multi-table-bulk-insert

Conversation

@jdufresne
Copy link
Copy Markdown
Member

Copy link
Copy Markdown
Member

@charettes charettes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch

I think the docs should mention that the current implementation of bulk_create requires one extra query per parent as one could assume that the implementation is smart enough to rely on INSERT INTO child ... FROM SELECT id AS parent_ptr_id FROM (INSERT INTO parent ... RETURNING id) which can be achieved in a single query.

@charettes
Copy link
Copy Markdown
Member

Just to make it clear I think we should definitely move along with this patch even if it doesn't support single query MTI bulk_create for now as it addresses a relatively old limitation of this feature.

A follow up ticket that adds support for single query multi-table INSERTs and make bulk_create and Model.save rely on it when possible could then be created if one doesn't already exists.

@jdufresne
Copy link
Copy Markdown
Member Author

Thanks for taking a look, Simon. I added "An additional SQL query is executed per concrete model in the inheritance hierarchy." to the querysets.txt docs. Please let me know if you think it could be improved, expanded, or tweaked.

Copy link
Copy Markdown
Member

@charettes charettes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a comment regarding Model.from_db but outside out that it might be worth adding a test for direct multi table inheritance e.g.

class A(models.Model):
    pass

class B(models.Model):
    pass

class C(A, B):
    pass

@jdufresne
Copy link
Copy Markdown
Member Author

it might be worth adding a test for direct multi table inheritance e.g.

Thanks! This suggested test exposed a bug. I'm looking into resolving it.

@jdufresne
Copy link
Copy Markdown
Member Author

I have implemented the suggested change and included the suggested test.

The precise test is slightly different than the suggested one due to avoid conflicts in PR fields which triggers a system check failure:

bulk_create.MultipleParents: (models.E005) The field 'id' from parent model 'bulk_create.nofields' clashes with the field 'id' from parent model 'bulk_create.twofields'.

Copy link
Copy Markdown
Member

@felixxm felixxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdufresne Thanks 👍

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use local_concrete_fields 🤔 Also, the current solutions doesn't work with recursive parents, e.g.

Pizzeria.objects.bulk_create([
    Pizzeria(name='Vegan Pizza House', rank=33),
])

crashes when we add the Ranked model:

class Place(models.Model):
    name = models.CharField(max_length=100)

    class Meta:
        abstract = True


class Ranked(models.Model):
    rank = models.IntegerField(null=True)


class Restaurant(Ranked, Place):
    pass


class Pizzeria(Restaurant):
    pass

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I'm going to make an attempt at finishing this PR, but I'm not super familiar with Django innards. Do you have any suggestions towards resolving this problem?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dralley The suggestion is already in the comment: "I think we should use local_concrete_fields", but it's only a suggestion. I didn't check this.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I thought it was referring to two separate issues.

Alright then, I'll make a new PR and we can proceed from there

@dralley
Copy link
Copy Markdown

dralley commented Feb 12, 2021

I'd be glad to test this (or attempt to test it) on our codebase once it's ready. We have a lot of multi-table inheritance.

@dralley
Copy link
Copy Markdown

dralley commented Mar 2, 2021

@jdufresne Do you plan to pick this work back up?

@jdufresne
Copy link
Copy Markdown
Member Author

@dralley I am no longer working on this. Please feel free to pick up the remaining work if you're motivated to do so. I will close this now to avoid any future confusion.

@jdufresne jdufresne closed this Mar 2, 2021
@mbrav
Copy link
Copy Markdown

mbrav commented Jul 15, 2021

ping

@dralley
Copy link
Copy Markdown

dralley commented Jul 15, 2021

@mbrav I haven't hada lot of time. I do hope to pick it back up eventually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants