Repository navigation
[test](regression) Make test_http_tvf independent of HuggingFace and loopback hosts - #70232
Conversation
…loopback hosts ### Background The `http()` table-valued function reads CSV, JSON, Parquet, and ORC files from HTTP(S) URLs; for an `hf://` URI the FE expands the HuggingFace path into download URLs. `external_table_p0/tvf/test_http_tvf` (group `p0`, so every P0 run executes it) serves most of its fixtures from an HTTP server it starts inside the regression runner, and builds their URLs from `httpTvfHost`, else `externalEnvIp`, which is `127.0.0.1` in every pipeline config. Its last seven queries read public HuggingFace datasets through `hf://`. ### What problem does this PR solve? Issue Number: None Related PR: apache#65877, apache#66013, apache#66027 (branch-4.1 fixes by @hello-stephen, ported here), apache#65299 (added the in-runner HTTP server and its host lookup), apache#60915 (moved the suite into P0 and relaxed sql15 / sql16 to `count > 0`) PR Type: Test Problem Summary: On master the suite still depends on huggingface.co and on HuggingFace's mutable `main` branches, and it advertises the runner as `127.0.0.1`, which a BE on another host resolves to itself. This PR ports the three branch-4.1 fixes: a loopback or wildcard `externalEnvIp` falls back to the runner's host address (an explicit `httpTvfHost` is still used as given), the seven `hf://` reads use byte-identical mirrors in the regression OSS bucket pinned to a dataset revision, and the mirrored `prompts.csv` skips its header line. With the mirror pinned, sql15 / sql16 assert the exact count 204 again. ### Details - **Failure:** None on master CI today: P0 passed 66 of 66 runs on 2026-11-04, though the HuggingFace queries took about 146 of the suite's 149 seconds (build 1092587). On branch-4.1 the suite failed with `Connection refused` on `http://127.0.0.1:<port>` with remote BEs (apache#65877), with `java.net.SocketTimeoutException: Connect timed out` to HuggingFace (apache#66013), and then with sql15 getting 205 instead of 204 on the mirror (apache#66027). - **Cause:** The suite assumed that every BE runs on the runner's host, true in P0 only, and that huggingface.co is reachable with fixed `main` branches: `prompts.csv` on `fka/awesome-chatgpt-prompts` `main` now has five columns and is over six times the 204-record file the golden came from. The mirrored CSV keeps a header line that `hf://` hid, hence `skip_lines = 1`. - **Stability evidence:** No query reaches huggingface.co any more. The only remote host left is `doris-regression-hk.oss-cn-hongkong.aliyuncs.com` (public reads), which P0 already uses in `query_p0/test_parquet_dict` (28 of 28 recent runs passed). Since master's P0 does not fail today, there is no before-and-after failure rate. ### Testing - `-forceGenOut` run of the suite on a local 1 FE + 1 BE cluster from master `4530f761fa1` (the FE carries local patches that don't touch the HTTP TVF), with P0's `externalEnvIp = 127.0.0.1` and no `httpTvfHost`: the runner advertised `http://192.168.1.164:<port>`, and the 22 blocks generated for sql01 to sql18 and both `replaced_*` checks are byte-identical to master's `.out`, with sql15 / sql16 now after `replaced_after`, where they run. sql19 then hit the 900 s query timeout: this machine downloads the 20 MB Parquet mirror at about 10 KB/s. - The suite without sql19 to sql21, in compare mode against the committed `.out`: passed. With `skip_lines` removed from sql15 it fails with `Expect cell is: 204 But real is: 205`. - sql19 to sql21 did not run locally. Each mirrored Parquet file has the same size as on HuggingFace and a byte-identical footer (schema, row groups, statistics), and each footer's minimum `text` is the first row the `.out` expects. This PR's P0 is their first run against the mirror on master. ### Notes for Reviewers - Apart from a one-line comment on the loopback check, the suite now matches branch-4.2's except for master's own parts: the `p0` label (apache#60915) and the `mutable/` cases (apache#68847). - `hf://` loses its only end-to-end coverage (real HuggingFace URLs, tree-API glob expansion, `@revision`); `HFUtilsTest` still covers parsing, URL conversion, and glob detection. - sql16, sql20, and the second sql21 now repeat sql15 / sql19, since plain HTTP has no globbing; they stay so that the goldens and branch-4.x line up. - In P0 the runner now advertises its host address instead of `127.0.0.1`; both work there, but branch-4.x labels the suite `p2`, so P0 runs this fallback for the first time. - `ai_p0/test_ai_profile` copied the old host lookup and has the same latent loopback problem; it is not changed here. ### Release note None ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Partially verified: sql19 to sql21 not run locally (mirrors checked against HuggingFace by size and Parquet footer); this PR's P0 runs them ### AI Disclosure - Tool: Claude Code (Claude Opus 5.5) - Generated: the port of the three branch-4.1 changes onto master's suite, the `.out` update (generated by the framework), and this description - Human review: not stated - Prompts: The author asked to port branch-4.1 apache#65877, apache#66013, and apache#66027 to master as part of moving branch-4.2's changes back to master, and, after seeing that the mirrors drop the only `hf://` end-to-end coverage, chose to port all three as on branch-4.2. Co-authored-by: Dongyang Li <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
Automated static review of apache/doris PR #70232 at head 6ed3394. The change is aimed at removing mutable HuggingFace dependencies and fixing remote-BE loopback routing, but the revised P0 suite has three verified problems.
The regression no longer exercises the HTTP TVF's hf:// dispatch, revision parsing, or HFUtils.expandGlob() patterns; the former cases are repeated exact HTTPS reads. The mandatory P0 suite also adds two roughly 20 MB public Parquet downloads even though the head commit records a 900-second timeout at about 10 KB/s and says sql19-sql21 were not run locally. Finally, the new default fallback can select an IPv6 address while the server listens on IPv4 0.0.0.0 and the URL is built without IPv6 brackets.
Checkpoint conclusions: the task's reliability goal is only partially achieved; the accepted findings are inline below. The change is limited to the regression suite and its generated golden, with no persistence, transaction, Thrift/Protobuf, or production configuration change. No new concurrency or locking path is introduced; the relevant lifecycle is the test HTTP server and its host selection. The FE/BE HTTP call chain was traced through HttpTableValuedFunction, HttpUtils, and HttpFileReader. The pre-existing duplicate sql21 tags are consumed deterministically and are not reported. No additional user focus was supplied. Existing inline threads and live PR comments are empty.
This review is static-only as required by the review contract: no builds or regression tests were executed by this invocation. The download-timeout evidence comes from the current head commit's recorded validation, and the HTTP HEAD/range behavior was checked through source inspection and the review ledger.
Review converged after 2 rounds: both delegated reviewers returned no new valuable findings in round 2, all changed files were covered, and every candidate was accepted, dismissed with evidence, or deduplicated.
Review converged after 2 round(s).
| http( | ||
| "uri" = "hf://datasets/fka/awesome-chatgpt-prompts/blob/main/prompts.csv", | ||
| "format" = "csv" | ||
| "uri" = "${stableRemoteDataUrl}/awesome-chatgpt-prompts/61229c8c07ea12c6e219b2b2b728f23f2842901b/prompts.csv", |
There was a problem hiding this comment.
[P2] Preserve HF/glob integration coverage when moving fixtures to the mirror. This replacement turns the old hf:// cases into repeated exact HTTPS reads, so the P0 suite no longer exercises HttpTableValuedFunction's hf:// dispatch, HFUtils.expandGlob() (*, **, [1]), or @revision parsing; sql15/sql16 and sql19/sql20 now assert the same file. Keep deterministic HF/glob cases, or add a hermetic equivalent, instead of dropping all end-to-end coverage.
There was a problem hiding this comment.
The facts are right, and this is the trade-off the description lists under "Notes for Reviewers": P0 no longer drives hf:// end to end, and sql16, sql20 and the second sql21 now repeat sql15 / sql19. The duplicates stay so that the suite and its goldens match branch-4.1 and branch-4.2, where #65877, #66013 and #66027 come from; the sql15 to sql21 blocks of the .out are identical on all three branches.
Two points on scope:
- Not all
hf://coverage is gone. FE UTHFUtilsTestcovers@revisionparsing, URL conversion and glob matching (*,**,[...]), andtestGlobExpansionExceptions(fe/fe-core/src/test/java/org/apache/doris/tablefunction/HFUtilsTest.java:406-413) expandshf://datasets/fka/awesome-chatgpt-prompts/blob/main/*against the live HuggingFace tree API on every FE UT run (the last 300 runs passed). What this PR drops is the multi-level listing (*/...,**),@revisionagainst a real non-main branch, and BE reads ofresolve/URLs. - A hermetic equivalent can't be built from the suite. The endpoint is the constant
DEFAULT_ENDPOINT = "https://huggingface.co"(fe/fe-core/src/main/java/org/apache/doris/tablefunction/HFUtils.java:66), and nothing in the URI or the TVF properties changes it, so pointinghf://at the in-runner server needs an FE change (a configurable endpoint). That is a separate change, not part of this test-only port. Deterministichf://cases against huggingface.co would keep the dependency this PR removes: on branch-4.1 the suite failed withConnect timed outto HuggingFace ([fix](regression) Mirror HTTP TVF fixtures in regression OSS #66013), and on master the HuggingFace part takes about 146 of the suite's roughly 150 seconds.
So I'm keeping this part as ported.
| select * from | ||
| http( | ||
| "uri" = "hf://datasets/stanfordnlp/imdb@main/plain_text/test-00000-of-00001.parquet", | ||
| "uri" = "${stableRemoteDataUrl}/stanfordnlp-imdb/main/plain_text/test-00000-of-00001.parquet", |
There was a problem hiding this comment.
[P1] Avoid making the mandatory P0 suite depend on unbounded 20 MB downloads. The head commit reports sql19 hitting the 900 s query timeout at roughly 10 KB/s, and sql19-sql21 were not run locally; each query causes FE metadata checks and BE range reads against the new OSS object. A throttled CI region can therefore stall or fail the whole P0 run. Use a small/local fixture or otherwise enforce a bounded, validated artifact for these cases.
There was a problem hiding this comment.
This doesn't add downloads, and this PR's P0 run shows no stall.
- The same files were downloaded before. On master, sql19, sql20 and the second sql21 each read
stanfordnlp/imdb@main/plain_text/test-00000-of-00001.parquet(20.5 MB) from huggingface.co, and the first sql21 (*/*.parquet) expands to all three Parquet files of the repo (20.5 + 21.0 + 42.0 MB). That is about 145 MB per run from HuggingFace. After this PR the four reads (test_http_tvf.groovy:394-424) fetch about 82 MB from the regression OSS bucket. - This PR's P0 ran them (build 1092831 at 6ed3394):
test_http_tvfpassed in 8.9 s, with sql19, sql20 and the two sql21 queries taking 1.8, 1.3, 1.7 and 1.3 s. The 39 P0 runs before it, with the HuggingFace version, took 149 to 170 s (median 151 s). - The 900 s / 10 KB/s in the commit message is the author's workstation, where the BE's traffic went through a local proxy; a direct GET of the same object from that machine runs at about 600 KB/s. It is not a measurement of CI.
- The reads are bounded: each query is capped by
query_timeout(900 s by default,fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:1239), which is what ended the local run, and the FE size probe uses a 10 s connect and 30 s read timeout (fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/manager/HttpUtils.java:180-181). P0 runs 14 suites in parallel, so a slow suite takes one worker, not the whole run. - The bucket isn't new to P0:
query_p0/test_parquet_dictreads from it (regression-test/suites/query_p0/test_parquet_dict.groovy:23) and passed its last 400 P0 / cloud_p0 runs. - A local fixture doesn't work for these files: an added file over 1 MB fails the large-file check on master (
.github/workflows/lfs-warning.yml:56-67,:91-97), and a smaller substitute would change the sql19 to sql21 goldens, which are identical on branch-4.1 and branch-4.2.
So no change here; I've added the P0 numbers to the PR description.
| if (httpServerHost == null || httpServerHost.trim().isEmpty() | ||
| || InetAddress.getByName(httpServerHost).isLoopbackAddress() | ||
| || InetAddress.getByName(httpServerHost).isAnyLocalAddress()) { | ||
| httpServerHost = InetAddress.getLocalHost().getHostAddress() |
There was a problem hiding this comment.
[P2] Keep the fallback address compatible with the listener and URL syntax. With the default loopback config, this branch can select an IPv6 result from getLocalHost(), but the server is bound only to IPv4 0.0.0.0 and the URL is built without IPv6 brackets. On an IPv6-resolving runner every local HTTP TVF query becomes unreachable or malformed; choose a reachable IPv4 address or bind and format IPv6 explicitly.
There was a problem hiding this comment.
The listener part doesn't hold, and the URL part is an existing limitation with an override.
HttpServer.create(new InetSocketAddress("0.0.0.0", 0), 0)(test_http_tvf.groovy:32) is dual-stack on a JVM with IPv6. The JDK opens an AF_INET6 channel, binds the IPv4 wildcard as::, and clearsIPV6_V6ONLY(OpenJDK 17:ServerSocketChannelImpl.java:108-109,Net.java:559-567,net_util_md.c:281-298,Net.c:267-277). On JDK 17.0.19 the server listens asIPv6 *:<port>and answers bothhttp://127.0.0.1:<port>/andhttp://[::1]:<port>/. With-Djava.net.preferIPv4Stack=truename lookups are IPv4-only as well, so in neither mode can the listener andgetLocalHost()disagree on the address family.getLocalHost()returns an IPv6 address only if the host name has no IPv4 address, since the JDK puts IPv4 first whilejava.net.preferIPv6Addressesis unset, or if the JVM sets that property. On P0 it returned172.16.0.146(build 1092831:Start local HTTP server for http tvf test: http://172.16.0.146:45365) and the suite passed.ai_p0/test_ai_structured_outputhas usedgetLocalHost().getHostAddress()unconditionally with the same listener and URL form (regression-test/suites/ai_p0/test_ai_structured_output.groovy:30,:50,:55) and passed its last 200 P0 / cloud_p0 runs.- The brackets are missing for an IPv6 literal, but that predates this PR: master already builds
http://${host}:${port}fromhttpTvfHost,externalEnvIpor this samegetLocalHost()fallback (test_http_tvf.groovy:151-158at 19dfb87). A runner whose name resolves only to IPv6 can sethttpTvfHost = "[addr]", which is used as given (test_http_tvf.groovy:151-152).
No pipeline runs on a runner whose name resolves only to IPv6, so I'm keeping the branch-4.x logic unchanged here.
|
/review |
There was a problem hiding this comment.
Automated static review of apache/doris PR #70232 at head 6ed3394.
The change is limited to the HTTP TVF regression suite and its generated golden. It improves fixture stability by pinning the former HuggingFace reads to an OSS mirror and adds a fallback for loopback externalEnvIp values, but one existing blocking finding still applies. Existing inline P1 comment 4474784302 remains valid: sql19-sql21 still send multi-megabyte reads to a public OSS object from the mandatory P0 suite, with no local fallback or test-specific transfer bound. The FE performs metadata HEAD requests and the BE performs range reads; a slow or unavailable endpoint can consume the query timeout and fail the P0 run. I carry that existing comment by ID and do not repost it.
The two existing P2 threads also remain known context and are not duplicated: 4474784015 covers the loss of end-to-end hf:// dispatch, revision parsing, and glob expansion when the old cases become repeated exact HTTPS reads; 4474784544 covers the IPv6/address-format risk in the getLocalHost() fallback and URL construction. The round-2 delegated recheck found no distinct additional defect, and the generated sql15-sql21 output pairing, duplicate qt_sql21 tags, server stop lifecycle, and regression DSL were otherwise covered.
Checkpoint conclusions: the reliability goal is only partially achieved because the existing P1 remains; the modification is focused on tests and does not change persistence, transactions, Thrift/Protobuf, production configuration, or a new concurrency path. The relevant lifecycle is the test HttpServer, which is stopped in finally. The FE/BE HTTP TVF path was inspected through HttpTableValuedFunction, HttpUtils, and HttpFileReader. All changed paths and existing threads were reviewed. No user-provided focus was supplied.
This invocation was static-only as required: no builds or regression tests were executed. The conclusions come from the authoritative diff, current-head source inspection, the current-head commit description, and the live PR comments. Review converged after 2 rounds: the latest delegated reviewer returned NO_NEW_VALUABLE_FINDINGS, and every risk item was accepted as an existing finding, deduplicated, or dismissed with evidence.
Review converged after 2 round(s).
Review converged after 2 round(s).
Existing P0/P1 findings confirmed for this head: #70232 (comment)
Background
The
http()table-valued function reads CSV, JSON, Parquet, and ORC files from HTTP(S) URLs; for anhf://URI the FE expands the HuggingFace path into download URLs.external_table_p0/tvf/test_http_tvf(groupp0, so every P0 run executes it) serves most of its fixtures from an HTTP server it starts inside the regression runner, and builds their URLs fromhttpTvfHost, elseexternalEnvIp, which is127.0.0.1in every pipeline config. Its last seven queries read public HuggingFace datasets throughhf://.What problem does this PR solve?
Issue Number: None
Related PR: #65877, #66013, #66027 (branch-4.1 fixes by @hello-stephen, ported here), #65299 (added the in-runner HTTP server and its host lookup), #60915 (moved the suite into P0 and relaxed sql15 / sql16 to
count > 0)PR Type: Test
Problem Summary:
On master the suite still depends on huggingface.co and on HuggingFace's mutable
mainbranches, and it advertises the runner as127.0.0.1, which a BE on another host resolves to itself. This PR ports the three branch-4.1 fixes: a loopback or wildcardexternalEnvIpfalls back to the runner's host address (an explicithttpTvfHostis still used as given), the sevenhf://reads use byte-identical mirrors in the regression OSS bucket pinned to a dataset revision, and the mirroredprompts.csvskips its header line. With the mirror pinned, sql15 / sql16 assert the exact count 204 again.Details
Connection refusedonhttp://127.0.0.1:<port>with remote BEs ([branch-4.1][fix](regression) Avoid loopback HTTP TVF host #65877), withjava.net.SocketTimeoutException: Connect timed outto HuggingFace ([fix](regression) Mirror HTTP TVF fixtures in regression OSS #66013), and then with sql15 getting 205 instead of 204 on the mirror ([fix](regression) Skip mirrored CSV header in HTTP TVF #66027).mainbranches:prompts.csvonfka/awesome-chatgpt-promptsmainnow has five columns and is over six times the 204-record file the golden came from. The mirrored CSV keeps a header line thathf://hid, henceskip_lines = 1.doris-regression-hk.oss-cn-hongkong.aliyuncs.com(public reads), which P0 already uses inquery_p0/test_parquet_dict(28 of 28 recent runs passed). Since master's P0 does not fail today, there is no before-and-after failure rate.Testing
-forceGenOutrun of the suite on a local 1 FE + 1 BE cluster from master4530f761fa1(the FE carries local patches that don't touch the HTTP TVF), with P0'sexternalEnvIp = 127.0.0.1and nohttpTvfHost: the runner advertisedhttp://192.168.1.164:<port>, and the 22 blocks generated for sql01 to sql18 and bothreplaced_*checks are byte-identical to master's.out, with sql15 / sql16 now afterreplaced_after, where they run. sql19 then hit the 900 s query timeout: this machine's BE reached the 20 MB Parquet mirror through a local proxy at about 10 KB/s (a direct GET from the same machine runs at about 600 KB/s)..out: passed. Withskip_linesremoved from sql15 it fails withExpect cell is: 204 But real is: 205.textis the first row the.outexpects. This PR's P0 (build 1092831) then ran the whole suite against the mirror: it passed in 8.9 s, with sql19, sql20 and the two sql21 queries taking 1.3 to 1.8 s each, and the runner advertisedhttp://172.16.0.146:<port>; the 39 P0 runs before it, with the HuggingFace version, took 149 to 170 s.Notes for Reviewers
p0label ([Test](regression) Normalize labels for external table suites #60915) and themutable/cases ([fix](filecache) Keep HTTP files out of the file cache and key external files by storage #68847).hf://loses its only end-to-end coverage (real HuggingFace URLs, tree-API glob expansion,@revision);HFUtilsTeststill covers parsing, URL conversion, glob matching, and one live*expansion against the HuggingFace tree API.127.0.0.1; both work there, but branch-4.x labels the suitep2, so P0 runs this fallback for the first time.ai_p0/test_ai_profilecopied the old host lookup and has the same latent loopback problem; it is not changed here.Release note
None
Check List (For Author)
AI Disclosure
.outupdate (generated by the framework), and this descriptionhf://end-to-end coverage, chose to port all three as on branch-4.2.