-
Notifications
You must be signed in to change notification settings - Fork 148
Improved relationship serializer options #32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved relationship serializer options #32
Conversation
spec/shared/examples/object_serializer_class_methods_examples.rb
Outdated
Show resolved
Hide resolved
| attributes :compensation | ||
|
|
||
| has_one :account | ||
| has_one :account, serializer: EmployeeAccountSerializer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This here was extremely unclear what the behavior should have been. There is a property called account, but the class is defined as EmployeeAccount. Should it assume AccountSerializer because of the property name? Should it read the object and see that it's an EmployeeAccount and therefore use EmployeeAccountSerializer?
FYI, the existing behavior (and the updated behavior) is that has_one :account will resolve to AccountSerializer. This has not changed. What has changed is that it now requires the serializer to exist. But, the reality is that you probably won't have an AccountSerializer when the class is EmployeeAccount. To me, this looks like a bug here in the EmployeeSerializer, where we should expect it to be EmployeeAccountSerializer and therefore the relationship would somehow need to specify the serializer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks a bit messy, but it seems like the account serializer was never defined and actually not used except in two tests (lines 144-151). I think we can just replace the account with something that already has a defined serializer, like move the has_one :photo into the EmployeeSerializer and update the tests to reflect photo instead of the account. Plus we delete a couple of extra lines.
|
@stas Hey, just wondering if you've had a chance to consider this one. I believe there are some important improvements here moving forward. I'd love to help address any concerns you may have. |
stas
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First of all, apologies for the delay in reviewing this. 🙈
@christophersansone I left some notes I'd love to have your input on before we move on with this.
Non-PR-related concerns (cc @jopotts @kpheasey):
- Should we set up a default set of fixtures, the fact that every test-case defines it's own set of fixtures seems, at least, hard to follow. I prefer us to have a default set of models/fixtures we all work and reuse?! If so, could we start with this test-case to build this fixtures library please 🙏
- Shall we adopt the
jsonapi-rspec1 gem to help DRY up these tests instead of having these custom shared examples?! The gem is not maintained despite having a really nice API and a bunch of forks. We could even get in touch with thejsonapi-rbcrew and see if we can take it over.
| attributes :compensation | ||
|
|
||
| has_one :account | ||
| has_one :account, serializer: EmployeeAccountSerializer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks a bit messy, but it seems like the account serializer was never defined and actually not used except in two tests (lines 144-151). I think we can just replace the account with something that already has a defined serializer, like move the has_one :photo into the EmployeeSerializer and update the tests to reflect photo instead of the account. Plus we delete a couple of extra lines.
|
I agree that the tests and fixtures can use some improvement. I opened #33 to discuss having more, clearer tests for each relationship option, but it sounds like you have some ideas to broaden it even further. I will defer to you on how to better structure everything related to tests, but I am open to whatever makes sense. |
| end | ||
| end | ||
|
|
||
| def initialize_static_serializer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@christophersansone I'm trying to understand the reasoning behind all this caching and extra attributes. Correct me if I'm wrong, but a relationship will have just one serializer klass, which we can calculate upon the relationship initialization DSL (say as part of the create_relationship). Same is true for the polymorphic case. So why do we need the extra logic here, seems overly complicated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct me if I'm wrong, but a relationship will have just one serializer klass...
This is true in some cases but not all. A relationship's serializer can either be static or dynamic.
When it is static, it is determined once up front and will not change. This is the most common use case:
belongs_to :actor
has_many :movies
belongs_to :genre, serializer: CustomGenreSerializerThe serializer is considered dynamic when it may change based on object or params. This occurs in the following ways:
has_many :movies, polymorphic: true
has_many :actors, polymorphic: { ComedyActor => :custom_comedy, DramaActor => :custom_drama }
has_many :genres, serializer: -> (object, params) {
...
}
Therefore, the role of initialize_static_serializer is to determine if the relationship's serializer can be static, and if so, which serializer it is. It determines this once and then caches it forever. If it determines that the serializer is dynamic, then it returns nil, stating that there is no static serializer.
Then, if there is no static serializer, it knows that it needs to dynamically determine the serializer. In some cases, like polymorphic: true, it simply looks at the object class, computes the serializer by name once and memoizes it in @serializers_for_name. Other times it haas more work to do, such as when options[:serializer] is a Proc.
The bottom line is this. If options[:serializer] is specified, that takes precedence over everything. The developer is specifically stating which serializer to use (or how to determine it if a Proc is specified). If not, then it needs to infer it, based on whether options[:polymorphic] is specified, by the relationship name, etc.
Is it complicated? Yes. Is it overly complicated? I don't think so, given that it all needed to be backwards compatible with options[:record_type] and options[:polymorphic]. I basically just consolidated all the logic into one place. Now to me the flow is quite simple:
- Up front, it determines if the serializer is static or dynamic.
- For every record being serialized, the relationship calls
serializer_forto determine which serializer to use. If the serializer is static, it returns the static serializer immediately. If not, it performs the dynamic check (which also does what it can to memoize the results for performance purposes).
To get an idea of how much more consolidated the logic is, look in the original implementation about how many checks for polymorphic are all over the place. Look at how many times it needs to iteratively call serializer.to_s.constantize or object.class.name.demodulize.underscore.to_sym.to_s.constantize. And consider how much simpler the Relationship API is for determining the serializer of any given object.
Finally, to address the second part of your question:
...which we can calculate upon the relationship initialization DSL (say as part of the create_relationship)...
If the serializer is indeed static, then it is indeed determined "up front". It is not done in create_relationship for two reasons:
-
As stated earlier, if
options[:serializer]is specified, then it takes precedence over any other serializer determination such aspolymorphic. Therefore, we don't want to create the relationship with its serializer defined asoptions[:serializer] || compute_serializer(). We could still compute it increate_relationship, but it would have to be as a separate variable. To me this is more complicated, not less, and at that point I would rather have the relationship handle it. -
Before, the
serializerpassed intoRelationshipwas a symbol, which still has to be constantized. And it was constantized often. Now, the static serializer determined by the relationship is the actual serializer class, and THAT is memoized. Because of this, we cannot necessarily determine this when the Relationship is created, because the serializer class may not be loaded yet. Therefore, it is determined "up front", i.e. the first time it is attempted.
Hopefully this answers all your questions.
|
Hi @christophersansone, Declare a And in the response you will see, the Here's how the response looks like, Hopefully it will help you to fix the bug. |
|
@mmkarim Thanks for the report! Can you confirm that this is not occurring in the master branch here? I noticed some things in the existing code along the lines of properly resolving the IDs. It seems like If a Pet FWIW, If you can confirm that this problem is in |
|
@christophersansone I tried with the master branch. It's working correctly there. |
|
@mmkarim I think this is what's going on:
I'll try to make this backward compatible with the old behavior, in case others run into this. The right solution here is this:
Lastly, can you help me understand what you are expecting to accomplish when both As a rule of thumb, specifying |
|
@stas I created @mmkarim I believe I fixed your issue by giving |
|
Hi @christophersansone, The issue with the This is the error I'm getting, though I'm passing the serializer already, |
|
Hi @christophersansone - great work. @mmkarim it would be good to have a test to show the failures, and prevent future regression. Not sure how easy that would be though. And about @stas's comment about the fixtures and improved tests, I agree but as @christophersansone said, it should be separate from this PR. #33 is the start of that. |
|
Also it will be great opportunity to specify I have got custom id ( |
stas
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@christophersansone let's get this merged! 🙇♂️
@ivan-garanin please move the conversation into #45, thank you!
|
@christophersansone I'd like to get this merged, please let me know if you're ok me picking up the work where you left. Ty 🙇♂️ |
|
@stas So sorry for dropping the ball. I had some time over the holidays but have been swamped since. I've been using it in production since then and it's been working great. I did find a minor race condition that I resolved, but other than that, it hasn't changed. I resolved the merge conflict here in Github (just the README). I thought it would just create a new commit in my branch, but apparently not. I would have just rebased instead had I known. Anyway, I think it's good to go. Is there anything else that needs to be done prior to merging? Feel free to ask or to just take the ball and run with it, whatever works best! |
|
Thanks @christophersansone 🙇♂️ |
…zer/jsonapi-serializer), instead of fast_jsonapi (https://github.com/Netflix/fast_jsonapi). Also contains a small fix in the GrapeSwagger parser to use the relationship `key` instead of `record_type`, which may be caused by jsonapi-serializer/jsonapi-serializer#32 TODO: A couple of the specs are commented out for now.
|
Hello, def relationships_example(relationship_data)
# data = { id: 1, type: relationship_data[:record_type] }
data = { id: 1, type: Dry::Inflector.new.singularize(relationship_data[:key]).to_sym }The original implementation used But also, the model documentation generated for my API by Grape Swagger started having relationship types as null too. E.g.: {
"data": {
"id": 1,
"type": "group",
"attributes": {
"name": "Example string",
"cost-center": "Example string",
"description": "Example string",
"created-at": "Example string",
"updated-at": "Example string"
},
"relationships": {
"users": {
"data": [
{
"id": 1,
"type": null
}
]
}
}
}
}PS: Such an issue doesn't exist with the original Netflix codebase |
|
@everlast240 Hi there! Yes we made some changes to
Now, the |
|
@everlast240, actually I'd be happy to review an example and maybe put it into a test case if you're willing to share it. @christophersansone thanks for helping us understand the internals 🙇 |
Overview
This PR implements the following changes:
serializeroption as aProc, when the serializer may be dynamic or requires some custom logic. Resolves Allow a relationship's serializer to be computed based on the object #29. Example:When
record_typeis not specified on a relationship, the outputtedtypeis now always determined from its serializer'sset_type.When
record_typeis not specified on a relationship, the relationship's serializer must always exist (so that it can get thetype).Is it a breaking change for anyone?
There are two edge cases where this can result in different behavior than before:
A
record_typeis not specified, and the old way to infer the type does not match that of the relationship's serializer. In this case, this was likely invalid JSON:API output (see Polymorphic hash does not select the proper serializer for included records #30), so this change now prevents this from occurring.A
record_typeis not specified, and the relationship's serializer is not defined. This should not really be happening in practice... if a relationship cannot be serialized, there is no sense in defining the relationship to begin with.TL;DR
The new serializer option behaves just like the
Procdefinitions for attributes and conditionals. This gives the developer full control over determining the serializer to use based on any possible use case.Because the
Procmay output different serializers, and different serializers means having differenttypevalues, I needed a way to get the correct types somehow. I chose to use the serializer'sset_typevalue to offer a Single Source of Truth. This resolves some uncertainty as to whatrecord_typedoes, whether it chooses the serializer, whattypeis being outputted, etc (#14). It also resolves #30, where thetypein therelationshipshash may differ from thetypein theincludedlist. With this change, thetypewill always be consistent and simplifies the API because the developer no longer needs to define it in both the serializer and on all relationships with viarecord_type.There were very few changes that I had to make to the existing specs... but there were changes. I want to specifically point them out and encourage you to review the spec changes. I annotated the changeset with comments for further clarity as well.
Because the
typeis now resolving differently, it is possible that the output may change for some people. When the object itself is serialized, asdataor withinincluded, thetypealways comes from the serializer'sset_type. That has not changed at all. What has changed is that therelationshipshash now uses that same value fromset_type, as opposed to inferring it by the property name. Therefore, the only cases where the output would be different after this change than before this change is if the relationships hash contained a differenttypethan the record itself, which to me is a bug in the output.As a result of the previous point, it now requires a serializer to be defined for each class being outputted, so that it can properly resolve the
typekey. In other words, previously, you could define a relationship without actually having a serializer defined for those items, and it would infer thetypefrom the property name. I can’t imagine that this is an actual problem in practice, because it does no good to have a relationship defined without somehow being able to serialize those records. I only mention it because there were some specs that defined relationships without defining the serializers.How do these changes impact performance?
I tried to compare the rspec performance benchmarks between the
masterbranch and this branch. When the difference is microseconds, it is really hard to get perfectly consistent readings on my computer (late 2019 MacBook Pro). Initial load appears to be consistently affected because of the up front serializer resolution, but we are talking about 0.3ms. After that, performance is quite similar. The tests with larger recordsets were often faster with the new implementation, though it is not consistent enough to state one way or the other. I am confident saying that the performance impact is negligible on initial render and at least as fast on subsequent renders.What should happen to
record_type?The
record_typeoption is still usable and will override the default behavior of looking up the serializer. To me, the negatives outweigh the positives at this point. It adds to the API surface and overlaps other functionality, causing confusion. And worse, because it can result in thetypein therelationshipshash being different than the serialized record itself, it can result in invalid output. I believe it should be deprecated, but it still "works" for now.