Skip to content

Fix issue with wrong argument type when using prefetch_primary_key?#49692

Merged
rafaelfranca merged 1 commit intorails:mainfrom
mguan2020:branchi
Nov 10, 2023
Merged

Fix issue with wrong argument type when using prefetch_primary_key?#49692
rafaelfranca merged 1 commit intorails:mainfrom
mguan2020:branchi

Conversation

@mguan2020
Copy link
Contributor

@mguan2020 mguan2020 commented Oct 18, 2023

Motivation / Background

This Pull Request has been created in order to fix issue #49657.

Detail

This Pull Request changes the logic of the insert method in database_statements.rb such that if a list of returning columns is specified, a list of values corresponding to the columns will be returned. Previously, it was possible for an integer to be returned even if the list of returning columns was not nil.

A test file create_new_entry.rb is provided to ensure the correct functionality of the insert method.

The sample code provided by @nicoco007 is now functioning correctly.

Additional information

Checklist

Before submitting the PR make sure the following are checked:

  • This Pull Request is related to one change. Changes that are unrelated should be opened in separate PRs.
  • Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
  • Tests are added or updated if you fix a bug or add a feature.
  • CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

Copy link
Contributor

@nvasilevski nvasilevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!! I think this PR could benefit from a few changes specifically around the test

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be a bit excessive to have a whole separate file for this particular test case
I'd say we can find a place for this scenario somewhere around existing create tests - https://github.com/rails/rails/blob/f5a1075e71b79eb7eb0f4f6323468ef9c7286ee9/activerecord/test/cases/persistence_test.rb#L499

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! In my modified PR, the test is moved into persistence_test.rb under a new method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it's understood that by reaching assert true line without failing we imply that Post.create! was successful and no exception was raised. But generally I find an explicit assertion like assert_nothing_raised { Post.create! } to be even more reader friendly.

However, what I find even more friendly is presence of a clear assertion. In our case we expect that next_sequence_value is what will define the primary key value for us, so perhaps let's add an explicit expectation by making next_sequence_value constant and asserting for the actual value?

For example:

def next_sequence_value
  123_456
end

def test_create_prefetches_primary_key_value
  post = assert_nothing_raised { Post.create! }
  assert_equal 123_456, post.id
end

We can use one of the existing AR test models, Post for example - https://github.com/rails/rails/blob/f5a1075e71b79eb7eb0f4f6323468ef9c7286ee9/activerecord/test/models/post.rb#L3
but we could define a subclass which defines the prefetch behavior:

  class PostWithPrefetchedPk < Post
    class << self
      def prefetch_primary_key?
        true
      end

      def next_sequence_value
        Random.rand(1000000)
      end
    end
  end

Or perhaps it's unnecessary to inherit from the existing one and our post can be a standalone model like

  class PostWithPrefetchedPk < ActiveRecord::Base
    class << self
      def prefetch_primary_key?
        true
      end

      def next_sequence_value
        Random.rand(1000000)
      end
    end
  end
end

but with a unique name to avoid conflicts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great suggestions! The test can definitely be made more meaningful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After doing some further research, it appears the assert_nothing_raised is deprecated (source: https://apidock.com/ruby/Test/Unit/Assertions/assert_nothing_raised).

Would it be okay to just add an explicit expectation for next_sequence_value?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's okay to omit assert_nothing_raised since we will have a much better explicit assertion for the primary key value but just wanted to mention that it must be a wrong apidock link since assert_nothing_raised at least the one defined in Rails itself is not deprecated and available
https://api.rubyonrails.org/classes/ActiveSupport/Testing/Assertions.html#method-i-assert_nothing_raised

def assert_nothing_raised
yield.tap { assert(true) }
rescue => error
raise Minitest::UnexpectedError.new(error)
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the reply. What the article is saying is strange because I see assert_nothing_raised in other parts of the Rails codebase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the potential fix should wrap the id_value into an array like Array(id_value) since this is what _create_record expects. Not saying that the current solution is wrong, I haven't looked close enough

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds interesting. I'll look into this further.

From what I have though, my fix appears to be passing the tests.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just my 2 cents: I think the question here boils down to "what do we do if returning and id_value are both passed?" IMO, the way it's been done here makes the most sense – I would personally expect that the method return the values for all columns specified in returning, not just the ID on its own in an array. I guess that technically opens up another question though: should the ID in the array when returning is passed be id_value or just be whatever the database returned? I personally would expect it to just be what the DB returned untouched, as is done here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue I can see here is that insert() is a public method and technically Rails 7.1 established that id_value will take precedence over any other parameter passed and by changing it to favor returning: we are technically changing the API which can't be done in a bugfix.

However, we can put it as if Rails 7.1 made a mistake by favoring id_value and it supposed to favor returning: from the beginning. Favoring returning: in Rails 7.1 wouldn't have been a breaking change since existing usages will continue pass returning: nil keeping the priority of id_value

So personally I think we can go with this fix! We may want to adjust the documentation for the method though.

Speaking a bit ahead I feel like there is an option for future version of Rails to deprecate passing id_value
It feels excessive to pass id_value to this method which is not used for anything other than a return value.
Consumers can just use the id_value argument directly without relying on it to be returned.
But that's a different story

Copy link
Contributor Author

@mguan2020 mguan2020 Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand your concern about changing the API. Changing the API seems to be necessary in order to fix this issue.

I like your idea of adjusting documentation of this API to make it more clear for users what it's returning.

I'm looking at creating a new PR to update documentation.

Comment on lines 509 to 512
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new model can point to the existing posts table by defining self.table_name = "posts"
Usually there is no need to have a new table.

However, I wonder if in this we actually do want a new table with special handling for the id column. I assume we would want to drop the auto-increment property from our table since the value for the primary key will be pre-fetched. I assume this will represent a more realistic pk prefetching setup. However I have to admit I'm not sure how a real world db schema would look in this case.

But the schema definition needs to go to
https://github.com/rails/rails/blob/main/activerecord/test/schema/schema.rb#L974

Suggested change
ActiveRecord::Schema.define do
create_table :post_with_prefetched_pks, force: true do |t|
end
end

Copy link
Contributor Author

@mguan2020 mguan2020 Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about removing the explicit schema definition.

I agree with your point about the new table being a more realistic setup, but for now I will follow your suggestion and have the new model point to the existing posts table.

Edit: After removing the schema definition, all checks are passing. Thank you for the suggestion!!

Copy link
Contributor

@nvasilevski nvasilevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍
The only small concern I have is that we may be changing the API of the insert() method but at the same time we could put it in a way that returning: (introduced in Rails 7.1) should have always taken precedence over id_value and returning id_value first was a bug.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue I can see here is that insert() is a public method and technically Rails 7.1 established that id_value will take precedence over any other parameter passed and by changing it to favor returning: we are technically changing the API which can't be done in a bugfix.

However, we can put it as if Rails 7.1 made a mistake by favoring id_value and it supposed to favor returning: from the beginning. Favoring returning: in Rails 7.1 wouldn't have been a breaking change since existing usages will continue pass returning: nil keeping the priority of id_value

So personally I think we can go with this fix! We may want to adjust the documentation for the method though.

Speaking a bit ahead I feel like there is an option for future version of Rails to deprecate passing id_value
It feels excessive to pass id_value to this method which is not used for anything other than a return value.
Consumers can just use the id_value argument directly without relying on it to be returned.
But that's a different story

@rafaelfranca rafaelfranca merged commit 10b36be into rails:main Nov 10, 2023
rafaelfranca added a commit that referenced this pull request Nov 10, 2023
Fix issue with wrong argument type when using prefetch_primary_key?
davinlagerroos pushed a commit to umn-asr/oracle-enhanced that referenced this pull request Feb 21, 2024
See rails/rails#48241
and rails/rails#49692

48241 added support for setting additional db-controlled attributes via
a returning clause. As part of these changes, that PR removed the
[setting of the in-memory object's id](https://github.com/rails/rails/pull/48241/files#diff-11b42664eb9834972953ecd5725c75fd6020af6d4ee5da37ef6d8e0c146f7773L1242)
in Persistance#_create_record. Instead, those changes expect some
`Column`s to be true for `auto_incremented_by_db?` so they can be
matched with the id value (and possibly other database values) returned
by insert and then used to update attributes on the in memory
activerecord object in [`_insert_record`](https://github.com/rails/rails/pull/48241/files#diff-11b42664eb9834972953ecd5725c75fd6020af6d4ee5da37ef6d8e0c146f7773R1248).

Unless an adapter adds the ability to identify some Column as `auto_incremented_by_db?`
the insert happens but the newly created object does not get its id value
updated from nil to the new id value. This commit updates `column_defintions`
and `Column` to identify the primary key column and make it selectable by
`auto_incremented_by_db?` so the corresponding attribute gets updated
with the id_value from the `insert`.

49692 makes further changes to support id_value in `insert` but I still need to
explicity pass `returning: nil` and wrap the return from `super` in Array
to get the id to update in tests.

Limitations:

This approach will probably need to be reconsidered if the adapter adds
support for composite primary keys, but for now this gets `SomeModel.create`
to set the id after the record is inserted and fixes many current test
failures.

PR 48241 also supports updating more via `returning` but this commit
does not add that feature. It looks like the seeds of an
implementation are there in `DatabaseStatements#sql_for_insert` and
`DatabaseStatements.exec_insert` but the goal of this commit is to fix
setting id during create.

Performance:

Using a left join to bring in the primary key since its performance
is pretty close to the performance of the original query and better
than using a WITH clause:

using benchmark-ips:

Warming up --------------------------------------
old_column_definitions
                         1.000 i/100ms
with clause column_definitions
                         1.000 i/100ms
left join column_definitions
                         1.000 i/100ms
Calculating -------------------------------------
old_column_definitions
                         10.093 (± 9.9%) i/s -     51.000 in   5.070277s
with clause column_definitions
                          9.031 (± 0.0%) i/s -     46.000 in   5.106131s
left join column_definitions
                          9.919 (± 0.0%) i/s -     50.000 in   5.049687s

Comparison:
old_column_definitions:       10.1 i/s
left join column_definitions:        9.9 i/s - same-ish: difference falls within error
with clause column_definitions:        9.0 i/s - 1.12x  slower
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants