This repository was archived by the owner on Feb 20, 2026. It is now read-only.
fix: Handle excludeLarge properties for arrays properly#1265
Merged
danieljbruce merged 44 commits into1242-to-main-2from Aug 28, 2024
Merged
fix: Handle excludeLarge properties for arrays properly#1265danieljbruce merged 44 commits into1242-to-main-2from
danieljbruce merged 44 commits into1242-to-main-2from
Conversation
encoding problem
…pis/nodejs-datastore into 1242-write-tests-on-save-with-fix # Conflicts: # test/gapic-mocks/commit.ts
product-auto-label
bot
added
size: l
daniel-sanche
approved these changes
Aug 26, 2024
src/index.ts
Outdated
| @@ -1125,6 +1118,25 @@ class Datastore extends DatastoreRequest { | |||
| // This was replaced with a more efficient mechanism in the top-level | |||
| // `excludeFromIndexes` option. | |||
There was a problem hiding this comment.
is this TODO still relevant? Does the top-level excludeFromIndexes work the same way as these changes?
Contributor
Author
There was a problem hiding this comment.
I think this TODO can be removed. There is currently code here that adds excludeFromIndexes in the top level and I added code that provides excludeFromIndexes everywhere else based on the path that gets generated. To avoid test failures, the code for adding excludeFromIndexes in the top level is still needed so I don't think this TODO is relevant anymore.
…-datastore into 1242-write-tests-on-save-with-fix # Conflicts: # test/gapic-mocks/commit.ts
product-auto-label
bot
added
size: m
danieljbruce
added a commit
that referenced
this pull request
Aug 28, 2024
… of nested fields (#1266) * test: Restrict types inside the save function (#1264) * Add interfaces that restrict values passed to save * Add interfaces for save We need help from the compiler so these interfaces will be useful for determining the shape of the data. * Modify current interfaces to make a closer match The interfaces should match the actual data types that get passed in. * key and data in PrepareEntityObjectResponse option These properties are actually optional * Document all the interfaces added for the save fn * Add two tests showing name/value should be provide * Make the SaveNonArrayData more specific. save doesn’t encode strings or ints or any values that don’t have keys properly so we should restrict the interface to what it does include. * Add header * Test should be more flexible * Add a class for elements with ToString * Define the toString class inline * Eliminate links to points in code * test: Add tests that describe how save should encode excludeFromIndexes for a complex case (#1263) * Write a sample test for what save output should be * Add a test for a bunch of complex excludeFromIndex * Change the replace function so it works everywhere * Modify the expected value Make it contain longStringArray * Correct the test for evaluating arrays * Break up the entityToEntityProto into separate par * Fix the tests to work with new addExcludeFromIndex * Revert "Break up the entityToEntityProto into separate par" This reverts commit 60dabd7. * Revert "Fix the tests to work with new addExcludeFromIndex" This reverts commit ba4e82b. * Remove source code changes * Skip the test that looks at arrays * Remove some of the wildcard indexes This is in excludeFromIndexes - replace them with literals. * Remove another boolean value from excludeFromIndex * Eliminate duplicate expectedMutations * Eliminate duplicate runTest function * Begin to set up the async tests * Pack all tests into the async framework * Delete tests addressed by async * Get rid of test functions and inline everything * Add comment * Add comments describing each test case * Remove only * Add another test looks at name/value not in array * Add a test for excludeLarge properties and name/va * Fix the test so that the array case matches Matches the single case * should pass the right properties for an array * Rename the test * Change name to entityName * Add two tests that capture the array encoding encoding problem * Change expected output of arrays * Remove only * Add 2 more tests to ensure behaviour is preserved * Correct test - it should apply excludeFromIndexes * Update the test for the nested value * fix: Handle excludeLarge properties for arrays properly (#1265) * Write a sample test for what save output should be * Add a test for a bunch of complex excludeFromIndex * Change the replace function so it works everywhere * Modify the expected value Make it contain longStringArray * Correct the test for evaluating arrays * Break up the entityToEntityProto into separate par * Fix the tests to work with new addExcludeFromIndex * Break up the entityToEntityProto into separate par * Fix the tests to work with new addExcludeFromIndex * Revert "Break up the entityToEntityProto into separate par" This reverts commit 60dabd7. * Revert "Fix the tests to work with new addExcludeFromIndex" This reverts commit ba4e82b. * Remove source code changes * Skip the test that looks at arrays * Remove some of the wildcard indexes This is in excludeFromIndexes - replace them with literals. * Remove another boolean value from excludeFromIndex * Eliminate duplicate expectedMutations * Eliminate duplicate runTest function * Begin to set up the async tests * Pack all tests into the async framework * Delete tests addressed by async * Get rid of test functions and inline everything * Add comment * Add comments describing each test case * Remove only * Add another test looks at name/value not in array * Add a test for excludeLarge properties and name/va * Fix the test so that the array case matches Matches the single case * should pass the right properties for an array * Rename the test * Change name to entityName * Add two tests that capture the array encoding encoding problem * Change expected output of arrays * Remove only * Add 2 more tests to ensure behaviour is preserved * Make argument more specific for excludeFromIndexes * Use excludeLargeProperties for array/non-array cas * Fix the test to include entity proto * Correct mistakes in initial implementation * Don’t skip any of the tests * Remove only * Add check for excludeLargeProperties * Remove TODO * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR contains the fix for solving this issue where
excludeLargeProperties: truedoesn't work properly for arrays. It also ensures that the fix doesn't break existing use cases.Background
This PR is a successor to this other PR and contains a fix that addresses the failing test cases in the other PR.
Changes
entityToEntityProtoused to have functionality that would addexcludeFromIndexes: truein the right places, but we move this code into a separateaddExcludeFromIndexesfunction because we want to reuse it for arrays.savethe code that callsfindLargeProperties_is moved so that it is only used as is for non array data. For the array data thisfindLargeProperties_code is used per element to populate theexcludeFromIndexeslist in the correct manner.excludeFromIndexeslist is correctly calculated we can use theaddExcludeFromIndexesfunctionality that we pulled out to applyexcludeFromIndexes: truein the right places.Next Steps