feat(rpc): paginate EXPERIMENTAL_view_state queries by vaporif · Pull Request #15743 · near/nearcore

vaporif · 2026-05-16T15:54:56Z

Closes #15612.
Adds from_key and limit request fields and a next_key response field to view_state to support pagination. Added MAX_VIEW_STATE_PAGE_ITEMS and MAX_VIEW_STATE_PAGE_BYTES near usage, so this might be moved somewhere if needed.

Follow-up issue

include_proof can't be combined with pagination, so that request is rejected. A proof has to cover the trie path from the root down to the cursor, but a resumed page jumps to the cursor with a NO_SIDE_EFFECTS seek that doesn't record that path. Recording it means changing the trie layer, which is out of scope here. Needs a separate issue and update TODO(issue_number) here after creation.

vaporif · 2026-05-16T15:56:35Z

    fn parse(value: Value) -> Result<Self, RpcParseError> {
-        Params::parse(value)
+        let request: Self = Params::parse(value)?;
+        // TODO(#15612): a resumed page seeks with AccessOptions::NO_SIDE_EFFECTS, so it


needs issue number replaced here

Trisfald

Thanks for the PR, looks solid overall.

There are just a few comments I'd like you to take a look at before merging.

Trisfald · 2026-05-18T08:57:32Z

-                        });
-                    }
+        let paginated = limit.is_some() || from_key.is_some();
+        debug_assert!(!(paginated && include_proof), "include_proof unsupported with pagination");


The debug_assert! here is a no-op in release.
I think an user can hit this condition from any query request with include_proof: true and from_key_base64 set.

I suggest replacing the assert with a real error (ViewStateError::ProofUnsupportedWithPagination or similar), to avoid the panic in debug and the broken proof in release.

Worth adding a regression test too. Similar to test_experimental_view_state_proof_with_pagination_rejected but using client.query(...) with QueryRequest::ViewState.

Trisfald · 2026-05-18T09:11:24Z

+            Some(from_key) => {
+                let mut full = query[..acc_sep_len].to_vec();
+                full.extend_from_slice(from_key);
+                iter.seek(Bound::Excluded(full))?;


If a caller passes from_key < prefix, the seek may land on a key below the prefix region, the starts_with(&query) check fails, and we return an empty page even when matching keys exist.

Two possible improvements:

Validating from_key.starts_with(prefix) and returning an error

Documenting the precondition with a /// doc comment on the from_key field in RpcViewStateRequest and QueryRequest::ViewState

added a dedicated error

Trisfald · 2026-05-18T09:18:05Z

Could you also add a CHANGELOG.md entry under ## [unreleased] → ### Non-protocol Changes?

…to paginated-view-state

Trisfald

LGTM, thanks!

Copilot

Pull request overview

Adds pagination to the view_state / EXPERIMENTAL_view_state JSON-RPC queries to address the 50 KB state-size limit that prevents inspecting larger accounts. The request gains optional from_key_base64 and limit fields, and the response returns a next_key cursor to fetch the following page. Paginated callers opt out of the legacy per-account size gate and instead get per-page item and byte caps (10,000 items / 50 KB). Combining pagination with include_proof is rejected for now because the resumed seek does not record the trie path needed for a proof.

Changes:

New optional from_key/limit request fields and next_key response field on ViewStateResult / QueryRequest::ViewState, threaded through the trie viewer and runtime adapter.
Per-page item/byte caps in TrieViewer::view_state, with seek-based resume that strips out-of-prefix or out-of-account rows, plus two new ViewStateError variants.
JSON-RPC validation rejecting include_proof + pagination and from_key not starting with prefix; OpenAPI/OpenRPC specs updated and integration/e2e tests added.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
runtime/runtime/src/state_viewer/mod.rs	Implements paginated iteration with item/byte caps and cursor production.
runtime/runtime/src/state_viewer/errors.rs	Adds `ProofUnsupportedWithPagination` and `FromKeyOutsidePrefix` error variants.
runtime/runtime/src/adapter.rs	Extends `ViewRuntimeAdapter::view_state` with `from_key`/`limit`.
chain/chain/src/runtime/mod.rs	Threads new params through `NightshadeRuntime::view_state` and `QueryRequest::ViewState` handling.
chain/chain/src/runtime/errors.rs	Maps the new viewer errors into `QueryError` (currently as `InternalError`).
chain/jsonrpc-primitives/src/types/view_state.rs	Adds `from_key`/`limit` fields on `RpcViewStateRequest`.
chain/jsonrpc/src/api/mod.rs	New `validate_view_state_pagination` helper shared by RPC parsers.
chain/jsonrpc/src/api/view_state.rs	Calls the shared validator from `EXPERIMENTAL_view_state` parsing.
chain/jsonrpc/src/api/query.rs	Calls the shared validator for `query`'s ViewState variant; structured path uses `None`/`None`.
chain/jsonrpc/src/lib.rs	Forwards `from_key`/`limit` from request DTO into `QueryRequest::ViewState`.
core/primitives/src/views.rs	Adds `next_key` to `ViewStateResult` and `from_key`/`limit` to the `ViewState` query variant.
chain/jsonrpc/openapi/openapi.json / openrpc.json	Spec updates for new request/response fields.
chain/jsonrpc/jsonrpc-tests/tests/rpc_query.rs	Integration tests covering paginated success and validation rejections.
integration-tests/src/tests/runtime/state_viewer.rs	Extensive trie-viewer pagination test suite (boundaries, prefix, byte cap, isolation, size-limit bypass).
integration-tests/src/user/{rpc_user,runtime_user}.rs, integration-tests/src/env/test_env.rs, test-loop-tests/src/tests/sharded_rpc.rs	Pass `None`/`None` for the new fields at existing call sites.
CHANGELOG.md	Documents the new pagination feature.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-05-19T18:15:17Z

Codecov Report

❌ Patch coverage is 82.64463% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.92%. Comparing base (e18ad71) to head (f79bd97).

Files with missing lines	Patch %	Lines
runtime/runtime/src/state_viewer/mod.rs	87.71%	4 Missing and 3 partials ⚠️
chain/jsonrpc/src/api/query.rs	69.23%	2 Missing and 2 partials ⚠️
chain/chain/src/runtime/errors.rs	0.00%	2 Missing ⚠️
integration-tests/src/env/test_env.rs	0.00%	2 Missing ⚠️
integration-tests/src/user/rpc_user.rs	0.00%	2 Missing ⚠️
chain/chain/src/runtime/mod.rs	91.66%	0 Missing and 1 partial ⚠️
chain/jsonrpc/src/api/mod.rs	95.45%	1 Missing ⚠️
chain/jsonrpc/src/api/view_state.rs	87.50%	0 Missing and 1 partial ⚠️
integration-tests/src/user/runtime_user.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #15743   +/-   ##
=======================================
  Coverage   69.92%   69.92%           
=======================================
  Files         939      939           
  Lines      197783   197879   +96     
  Branches   197783   197879   +96     
=======================================
+ Hits       138290   138360   +70     
- Misses      54871    54888   +17     
- Partials     4622     4631    +9

Flag	Coverage Δ
pytests-nightly	`1.13% <0.00%> (-0.01%)`	⬇️
unittests	`69.34% <82.64%> (+<0.01%)`	⬆️
unittests-nightly	`69.58% <82.64%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jancionear

LGTM, just left some nits, feel free to ignore them.

Thanks for the contribution!

jancionear · 2026-05-19T13:06:52Z

+        let request: Self = Params::parse(value)?;
+        super::validate_view_state_pagination(
+            request.prefix.as_slice(),
+            request.from_key.as_ref().map(|k| k.as_slice()),
+            request.limit,
+            request.include_proof,
+        )?;
+        Ok(request)


I wonder if parsing is the right place for this validation code. We already validate in the query handler itself. Doesn't really hurt to do it though.

jancionear · 2026-05-19T13:10:21Z

+            err @ (node_runtime::state_viewer::errors::ViewStateError::ProofUnsupportedWithPagination
+            | node_runtime::state_viewer::errors::ViewStateError::FromKeyOutsidePrefix) => {
+                Self::InternalError { error_message: err.to_string(), block_height, block_hash }
+            }


Could be nice to have separate error types for that, but not strictly necessary. And I guess this should get caught earlier by the parsing code, so those errors won't really be visible to users.

jancionear · 2026-05-19T18:17:40Z

+    for i in 0..5 {
+        entries.push((
+            alice_account(),
+            format!("akey{i}").into_bytes(),


cspell complains that akey is an unknown word. Please add an ignore statement (something like // cspell:ignore akey).

We need the CI green before merging

…to paginated-view-state

vaporif added 3 commits May 16, 2026 16:19

add viewstate pagination logic

951575b

add tests

6bfdfb2

regen spec

8f94d55

vaporif requested review from a team and frol as code owners May 16, 2026 15:54

vaporif requested a review from wacban May 16, 2026 15:54

Merge branch 'master' into paginated-view-state

9993b0d

vaporif commented May 16, 2026

View reviewed changes

shreyan-gupta requested a review from jancionear May 17, 2026 03:02

Trisfald requested changes May 18, 2026

View reviewed changes

vaporif added 3 commits May 18, 2026 14:24

adress review comments

5f8d2f3

Merge branch 'paginated-view-state' of github.com:vaporif/nearcore in…

66e4ab5

…to paginated-view-state

Merge branch 'master' into paginated-view-state

90fb220

vaporif requested a review from Trisfald May 18, 2026 13:26

Merge branch 'master' into paginated-view-state

4ea0379

Trisfald approved these changes May 18, 2026

View reviewed changes

jancionear requested a review from Copilot May 19, 2026 12:54

Copilot started reviewing on behalf of jancionear May 19, 2026 12:55 View session

Copilot AI reviewed May 19, 2026

View reviewed changes

Comment thread chain/chain/src/runtime/errors.rs

jancionear approved these changes May 19, 2026

View reviewed changes

jancionear reviewed May 19, 2026

View reviewed changes

vaporif added 5 commits May 19, 2026 20:05

renames

fd21fd2

Merge branch 'master' into paginated-view-state

ef2df7d

more renames

ac5e9f2

Merge branch 'paginated-view-state' of github.com:vaporif/nearcore in…

542c578

…to paginated-view-state

Merge branch 'master' into paginated-view-state

f79bd97

jancionear approved these changes May 20, 2026

View reviewed changes

jancionear added this pull request to the merge queue May 20, 2026

Merged via the queue into near:master with commit d679c60 May 20, 2026
32 checks passed

Uh oh!

Conversation

vaporif commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Follow-up issue

Uh oh!

vaporif May 16, 2026

Choose a reason for hiding this comment

Uh oh!

Trisfald left a comment

Choose a reason for hiding this comment

Uh oh!

Trisfald May 18, 2026

Choose a reason for hiding this comment

Uh oh!

vaporif May 18, 2026

Choose a reason for hiding this comment

Uh oh!

Trisfald May 18, 2026

Choose a reason for hiding this comment

Uh oh!

vaporif May 18, 2026

Choose a reason for hiding this comment

Uh oh!

Trisfald commented May 18, 2026

Uh oh!

Trisfald left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

codecov Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jancionear left a comment

Choose a reason for hiding this comment

Uh oh!

jancionear May 19, 2026

Choose a reason for hiding this comment

Uh oh!

jancionear May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jancionear May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vaporif commented May 16, 2026 •

edited

Loading

codecov Bot commented May 19, 2026 •

edited

Loading

jancionear May 19, 2026 •

edited

Loading