)

### What problem does this PR solve?

Issue Number: close #69926

Problem Summary:

Hive JSON queries that only need partition columns can return zero rows
even when the selected files contain records. For example, `SELECT * ...
WHERE dt = 'p1'` returns data while `SELECT COUNT(*) ... WHERE dt =
'p1'` returns zero. The V2 JSON reader derived its row count from the
output block, which has no columns after physical field pruning.
Writer-backed reader tests reproduce the same row-count loss for empty
WAL projections.

Track accepted JSON records independently of materialized columns,
including batch progress, array expansion, error rollback, and
ignore-malformed NULL rows. Preserve JSON-path record selection for
empty projections, and let the existing TableReader materialize
partition values using the returned row count.

For WAL, save the deserialized source row count before materialization
transfers column ownership, then pass it through the existing filter
pipeline. This preserves the row count when no file columns are
requested. The empty-projection reproduction is at the reader interface;
production replay reachability of an empty projection remains
unverified.

Add ten JSON BE tests covering empty file schemas and projections,
arrays, JSON roots/paths, malformed input, batch boundaries, EOF, and
actual TableReader partition materialization. Add eight writer-backed
WAL cases covering empty, single-column, and reordered full projections
across multiple blocks, with and without schema discovery, plus repeated
EOF reads. Add a WAL SQL regression that forces two original async-load
transactions to abort, then verifies row counts, NULLs, aggregates,
filters, and actual values after replay. Add a Hive SQL regression using
the existing partitioned JSON fixture for COUNT, partition projections,
grouping, and file-column filters with external COUNT pushdown enabled
and disabled.

### Release note

Fix incorrect zero counts and missing rows when querying only partition
columns of Hive JSON tables. Preserve logical row counts for empty WAL
reader projections.

### Check List (For Author)

- Test
- [x] Unit Test: 166 ASAN tests passed, including JSON reader,
TableReader, WAL V2 reader, WAL reader/writer, and CSV/Text
empty-projection controls. Before the JSON fix, nine of ten new tests
failed; before the WAL fix, both empty-projection cases failed while six
projection controls passed.
- [x] WAL SQL regression: `test_group_commit_wal_row_count` passed twice
on isolated Doris 4.1.5 with scanner V2 enabled. Generated the `.out`
file with `run-regression-test.sh`, then reran for result comparison.
Execution on master is pending CI.
- [x] Hive SQL regression: added `test_hive_json_partition_only`; Groovy
compilation passed. Full execution against the external Hive fixture is
pending CI.
- Behavior changed:
- [x] Yes. Empty JSON and WAL file projections preserve logical row
counts. Hive JSON partition-only queries produce the correct rows and
aggregates.
- Does this need documentation?
    - [x] No.

…/JSON/WAL fixes (#70185)

### What problem does this PR solve?

Backport the following master PRs to `branch-4.2`, in master merge
order:

1. #69672 — Preserve source values and alias semantics in ES scans.
2. #69971 — Add session controls for Parquet page indexes and data-page
size.
3. #70155 — Repair ES source predicate regression cases.
4. #70079 — Fix ORC condition-cache seeks past filtered stripe suffixes.
5. #70154 — Preserve JSON and WAL rows with empty file projections.

The ES changes are adapted to the existing 4.2 FE planner and BE source
locations. Mapping safety, scan-wide read mode, local residuals, native
`esquery` handling and alias routing retain the upstream behavior
without introducing the master connector framework. Added adapter tests
cover both scan protocols, aliases, missing-value negations and native
residual rejection.

Preserve existing branch-specific ES regression coverage. Bring in the
previously absent predicate suite, adapting its plan assertions to 4.2's
`LOCAL_PREDICATES` output and retaining LIMIT result checks. Parquet
variables use the 4.2 annotation interface and the original optional
Thrift field IDs. The ORC dependency is pinned to the same fixed commit
as #70079.

The ES implementation also preserves source numeric and JSONB predicate
semantics after scan-wide fallback. Numeric-token rounding must not make
a remote predicate discard a row matching the value Doris materializes;
empty JSONB objects remain nonnull even when ES indexes no leaf.
Docvalue eligibility and pushdown are retained.

Mapped date formats and their order are carried per concrete backing
index through the existing property map to both scan protocols.
Calendar/epoch alternatives now distinguish basic_date from
epoch_millis, for strings and source numeric tokens. Numeric docvalues
and older FE numeric requests retain their epoch decoder. Typed alias
selection consistently uses the requested mapping type in schema
inference and planning. The pinned ORC decoder's libdeflate dependency
is included on ARM, matching master.

### Release note

Fix ES source-value filtering, mapped dates and typed alias query
correctness, ORC condition-cache seek crashes/missing rows and its ARM
link dependency, and JSON/WAL row counts for empty file projections. Add
Parquet output page-index and data-page-size session controls.

### Validation

- FE compilation and Checkstyle passed. ES validation: 81 tests (48
datasource/planner tests and 33 legacy ES tests), 0 failures/errors, 1
external-environment skip. The 9 Parquet session-variable tests passed.
- Rebuilt the pinned ORC dependency under AddressSanitizer: 56 focused
tests passed, including 32 filtered-stripe-suffix seek cases.
- Compiled all 10 affected Doris C++ implementation/test translation
units and the ORC fixture generator. The regenerated fixture is
byte-for-byte identical to the committed fixture.
- clang-format 16 checks passed for all affected Doris C++ sources and
headers. Modified Groovy suites compiled successfully.
- The ES reader/parser implementations and test translation units
compiled. The shared DATETIMEV2 date-format algorithm passed 21
standalone ASAN checks. CMake dependency checks passed for aarch64,
arm64 and x86_64; an actual ARM link was not run locally.
- Added `test_es_source_review_cases.groovy` with SQL-to-SQL checks
against an internal reference table for numeric/JSONB source predicates,
LIMIT, epoch-only formats, and explicit legacy DATETIMEV2 basic_date
columns. It covers both scanners and parallel-scroll settings, waits for
legacy metadata readiness, and restores session settings. The suite
compiled; no golden files were handwritten.
- The applicable ES fixes are synchronized to master in #70205. Master
already links libdeflate on every architecture.
- Full Doris BE unit-test execution and live external SQL regression
were not performed locally; CI validation is requested.

### Check List (For Author)

- Test
  - [x] Unit Test
  - [x] Regression tests included
- Behavior changed:
  - [x] Yes; the upstream behavior is backported to 4.2.
- Does this need documentation?
- [x] Parquet controls are documented by the upstream documentation PR:
apache/doris-website#4581

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label

…ections #70154 (#70180)

Cherry-picked from #70154

Co-authored-by: Gabriel <[email protected]>

)

### What problem does this PR solve?

Backport three master fixes to `branch-4.1`: avoid unnecessary
database-object loading for SCHEMATA scans, prevent ORC condition-cache
seek crashes or missing rows, and preserve JSON/WAL row counts when no
file columns are projected.

The commits retain their original authors and are applied in master
merge order:

1. #70010 — Avoid database-object
enumeration when SCHEMATA only needs names.
2. #70079 — Advance safely past
SARG-filtered ORC stripe suffixes during condition-cache seeks.
3. #70154 — Track logical JSON
records independently of projected columns and capture WAL row counts
before column ownership is transferred.

Branch adaptations:

- Keep Thrift field 7 reserved and add the optional `include_db_ids`
flag at upstream field 8 with its default `true`. Preserve existing
privilege checks, pattern matching and schema-name formatting for both
enumeration paths.
- Use the upstream fixed ORC dependency revision
`5962bb071d6345cc29c5353d64d85a203525e4d9`, which descends from the
branch's existing revision.
- Resolve the JSON reader conflicts by retaining 4.1's error handling
and adding logical row accounting without importing unrelated
missing-field/load-rejection changes. Remove the rooted-array pre-count
that consumes the ondemand cursor before JSON-path materialization;
retain the upstream rooted-array coverage.

### Validation

- The branch's FE production and test sources compiled with the existing
Java 8 target settings. All **7 targeted FE tests passed**, including
the existing `FrontendServiceImplTest#testGetDBNames`, the
100,000-namespace case, filtering and legacy-request compatibility. FE
Checkstyle passed.
- A fresh ASAN build of the exact pinned ORC dependency passed **56
focused tests**, including **32 filtered-stripe-suffix boundary cases**.
- All **7 affected C++ implementation/test/generator translation units
compiled**. Clang-format 16 passed for all **8 affected C++
source/header files**.
- The ORC fixture generator reproduced the committed fixture byte for
byte. All **3 added Groovy suites compiled**.
- Full Doris BE unit-test execution and external SQL regression
execution remain for CI; they were not run locally for this backport.

### Release note

Improve SCHEMATA enumeration for large catalogs. Fix potential crashes
or missing rows during ORC condition-cache seeks and incorrect JSON/WAL
row counts with empty file projections.

### Check List (For Author)

- Test
  - [x] Unit tests and focused dependency tests
- [x] Regression tests included and compiled; full SQL execution pending
CI
- Behavior changed:
- [x] Yes. Apply the upstream fixes while preserving branch-specific
validation and protocol compatibility.
- Does this need documentation?
  - [x] No.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Add branch pick label

### ARM64 link dependency follow-up

The ORC revision enables libdeflate ZLIB decoding on every architecture.
Remove the old x86-only guard around `add_thirdparty(deflate)` so ARM64
BE links also receive the required library, matching master. An adjacent
comment records this dependency invariant.

Validation: probes include each branch's actual
`be/cmake/thirdparty.cmake` and select `aarch64`, `arm64`, `x86` and
`x86_64` dependency configurations. Before the fix, both ARM
configurations fail to link with missing libdeflate symbols while both
x86 controls pass. After the fix, all eight repository/configuration
combinations link and pass a DEFLATE compression/decompression round
trip. These probes run natively on x86_64 and validate CMake dependency
selection; a native ARM64/full BE build remains for CI. Clang-format 16
passed for the PR-affected C++ files.