Skip to content

feat: enforce LIST_PAGINATION_ENABLED#2401

Merged
dimas-b merged 2 commits intoapache:mainfrom
dimas-b:paging-flag
Aug 20, 2025
Merged

feat: enforce LIST_PAGINATION_ENABLED#2401
dimas-b merged 2 commits intoapache:mainfrom
dimas-b:paging-flag

Conversation

@dimas-b
Copy link
Contributor

@dimas-b dimas-b commented Aug 19, 2025

The enforcement of the LIST_PAGINATION_ENABLED flag was missed in #1938. This change make the flag effective as discussed in #2296.

Note: this causes a change in the default Polaris behaviour (no pagination by default) with respect to the previous state of main. However, there is no behaviour change with respect to 1.0.0 or 1.0.1 as previous releases did not have #1938.

return PageTokenUtil.decodePageRequest(serializedPageToken, requestedPageSize);
@Nullable String serializedPageToken,
@Nullable Integer requestedPageSize,
Supplier<Boolean> shouldDecodeToken) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a Supplier? None of the call sites is really "dynamic".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not called when the token is null, so it avoids touching the config look up in most cases... I'm fine with an unconditional config lookup too. What is preferable from your POV?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, neither getResolvedCatalogEntity nor getConfig are really simple.
Mind making this at least a Predicate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Predicate is a one-argument function, but here we just need to have a deferred getter for a simple boolean 🤔

Updated to BooleanSupplier. WDYT?

The enforcement of the LIST_PAGINATION_ENABLED flag was missed in apache#1938.
This change make the flag effective as discussed in apache#2296.

Note: this causes a change in the default Polaris behaviour (no pagination
by default) with respect to the previous state of `main`. However, there
is no behaviour change with respect to 1.0.0 or 1.0.1 as previous releases
did not have apache#1938.
@github-project-automation github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Aug 20, 2025
@dimas-b dimas-b merged commit a0a2b87 into apache:main Aug 20, 2025
12 checks passed
@github-project-automation github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Aug 20, 2025
@dimas-b dimas-b deleted the paging-flag branch August 20, 2025 18:19
dimas-b added a commit to dimas-b/polaris that referenced this pull request Aug 20, 2025
Correct method args (follow-up to apache#2401)
@dimas-b dimas-b mentioned this pull request Aug 20, 2025
dimas-b added a commit that referenced this pull request Aug 20, 2025
Correct method args (follow-up to #2401)
snazy added a commit to snazy/polaris that referenced this pull request Nov 20, 2025
* feat: enforce LIST_PAGINATION_ENABLED (apache#2401)

* feat: enforce LIST_PAGINATION_ENABLED

The enforcement of the LIST_PAGINATION_ENABLED flag was missed in apache#1938.
This change make the flag effective as discussed in apache#2296.

Note: this causes a change in the default Polaris behaviour (no pagination
by default) with respect to the previous state of `main`. However, there
is no behaviour change with respect to 1.0.0 or 1.0.1 as previous releases
did not have apache#1938.

* Add PolarisMetaStoreManager.loadEntities (apache#2290)

* Add PolarisMetaStoreManager.loadEntities

currently `PolarisMetaStoreManager.listEntities` only exposes a limited
subset of the underlying `BasePersistence.listEntities` functionality.

most of the callers have to post-process the `EntityNameLookupRecord` of
`ListEntitiesResult` and call `PolarisMetaStoreManager.loadEntity`
on the individual items sequentually to transform and filter them.

this is bad for the following reasons:

- suboptimal performance as we run N+1 queries to basically load every
  entity twice from the persistence backend
- suffering from race-conditions when entities get dropped between the
  `listEntities` and `loadEntity` call
- a lot of repeated code in all the callers (of which only some are
  dealing with the race-condition by filtering out null values)

as a solution we add `PolarisMetaStoreManager.loadEntities` that takes
advantage of the already existing `BasePersistence` methods.
we rename one of the `listEntities` methods to `loadEntities` for
consistency.

since many callers dont need paging and want the result as a list, we
add `PolarisMetaStoreManager.loadEntitiesAll` as a convenient wrapper.

we also remove the `PolarisEntity.nameAndId` method as callers who only
need name and id should not be loading the full entity to begin with.

note we rework `testCatalogNotReturnedWhenDeletedAfterListBeforeGet`
from `ManagementServiceTest` because the simulated race-condition
scenario can no longer happen.

* review: remove filter + transformer params

* review: add TODO in listPolicies

* review: improve javadoc

* chore: fix Page javadoc (apache#2412)

Correct method args (follow-up to apache#2401)

* Add 1.0.1-incubating release blog post (apache#2403)

* This change fixes: (apache#2395)

* use [email protected] for announcement
* publish Docker images only when the vote passed (we are not suppose to publish any public artifacts before the vote is completed)
* update the vote email accordingly
* remove blog post link in the release announcement email to let this to the discretion of the release manager

* chore(deps): update actions/setup-java action to v5 (apache#2414)

* chore(test): Restore PolarisAccessManager (apache#2413)

Restore `PolarisAccessManager` (with default implementation
of `IcebergTokenAccessManager`) which was added as an extension
point for running Polaris tests in downstream build environments
under apache#789, but was mistakenly removed in apache#2343

* Update changelog prior to 1.1.0 release (apache#2406)

* fix(deps): update quarkus platform and group to v3.25.4 (apache#2279)

* NoSQL: merge related changes

* Last merged commit 82cd416

---------

Co-authored-by: Dmitri Bourlatchkov <[email protected]>
Co-authored-by: Christopher Lambert <[email protected]>
Co-authored-by: JB Onofré <[email protected]>
Co-authored-by: Mend Renovate <[email protected]>
Co-authored-by: Pierre Laporte <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants