Skip to content

[feat] Utilize standardized app_base_url throughout e2e tests#13679

Merged
sfc-gh-bnisco merged 1 commit intodevelopfrom
01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_
Feb 12, 2026
Merged

[feat] Utilize standardized app_base_url throughout e2e tests#13679
sfc-gh-bnisco merged 1 commit intodevelopfrom
01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_

Conversation

@sfc-gh-bnisco
Copy link
Copy Markdown
Collaborator

@sfc-gh-bnisco sfc-gh-bnisco commented Jan 22, 2026

Describe your changes

  • Standardize e2e URL construction by switching tests from app_port/http://localhost:{app_port} to app_base_url + build_app_url(...) for navigation, request calls, route interception, and URL assertions.
  • Ensure direct WebSocket connections derive ws:// vs wss:// from app_base_url (so the same tests work for https/external runs, not just local http).
  • Document the “no localhost hardcoding” rule in e2e guidance (e2e_playwright/AGENTS.md, .github instructions, and Cursor rules).

Testing Plan

  • Existing tests have been updated and are passing
  • Tests can now run against external deployments by configuring the appropriate CLI flags or environment variables (tested in SiS vNext)

Contribution License Agreement

By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.

@snyk-io
Copy link
Copy Markdown
Contributor

snyk-io bot commented Jan 22, 2026

Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jan 22, 2026

✅ PR preview is ready!

Name Link
📦 Wheel file https://core-previews.s3-us-west-2.amazonaws.com/pr-13679/streamlit-1.54.0-py3-none-any.whl
📦 @streamlit/component-v2-lib Download from artifacts
🕹️ Preview app pr-13679.streamlit.app (☁️ Deploy here if not accessible)

Copy link
Copy Markdown
Collaborator Author

sfc-gh-bnisco commented Jan 22, 2026

@sfc-gh-bnisco sfc-gh-bnisco added change:feature PR contains new feature or enhancement implementation impact:internal PR changes only affect internal code security-assessment-completed labels Jan 22, 2026 — with Graphite App
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from a1cf074 to 62eec2f Compare January 23, 2026 21:05
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_url_runner_in_e2e_playwright branch from 9e9dfa9 to 24b4488 Compare January 23, 2026 21:05
@sfc-gh-bnisco sfc-gh-bnisco changed the title [feat] Utilize standardized app_base_url throughout e2e tests" [feat] Utilize standardized app_base_url throughout e2e tests Jan 26, 2026
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_url_runner_in_e2e_playwright branch from 24b4488 to 07b02c8 Compare January 26, 2026 21:03
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from 62eec2f to 1bccb97 Compare January 26, 2026 21:03
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jan 26, 2026

📉 Frontend coverage change detected

The frontend unit test (vitest) coverage has decreased by 0.0000%

  • Current PR: 86.8200% (13884 lines, 1829 missed)
  • Latest develop: 86.8200% (13884 lines, 1829 missed)

✅ Coverage change is within normal range.

📊 View detailed coverage comparison

@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch 3 times, most recently from a86c1f8 to 6e57b44 Compare January 27, 2026 00:01
@sfc-gh-bnisco sfc-gh-bnisco requested a review from Copilot January 27, 2026 00:38
@sfc-gh-bnisco sfc-gh-bnisco added the ai-review If applied to PR or issue will run AI review workflow label Jan 27, 2026
@github-actions github-actions bot removed the ai-review If applied to PR or issue will run AI review workflow label Jan 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Summary

This PR refactors the E2E test infrastructure to utilize a standardized app_base_url fixture and build_app_url() helper function throughout the test suite. This enables running E2E tests against externally hosted apps (via --external-app-url CLI option) instead of only local http://localhost:{port} URLs.

Key changes:

  • Adds app_base_url fixture (module scope) that returns either http://localhost:{app_port} or an externally provided URL
  • Adds build_app_url() helper function for proper URL composition with path, query, and fragment parameters
  • Updates 29 test files to replace hardcoded localhost URLs with the new standardized approach
  • Updates documentation in .cursor/rules/e2e_playwright.mdc, .github/instructions/e2e_playwright.instructions.md, and e2e_playwright/AGENTS.md

Code Quality

The implementation is well-structured and follows good practices:

Strengths:

  1. Well-designed build_app_url function (conftest.py lines 472-522): The function properly handles URL parsing with urllib.parse, preserves base URL query parameters, and correctly composes paths. The logic handles edge cases like:

    • Absolute vs relative paths
    • Base URLs with/without trailing slashes
    • Query parameter merging from both base URL and provided params
    • Fragment handling
  2. Helpful utility functions: ensure_nonempty_path() (lines 525-532) and normalize_url_for_compare() (lines 535-541) are useful additions for consistent URL handling.

  3. Clear WebSocket URL conversion (web_server_test.py lines 40-46): The app_ws_url() helper properly converts HTTP/HTTPS schemes to WS/WSS.

Minor observations:

  1. The app_ws_url() function in web_server_test.py could potentially be moved to conftest.py if other test files need WebSocket URL construction in the future. However, this is not a blocking issue as it's currently only used in that file.

  2. The PR consistently uses app_base_url: str parameter type annotation, which is good for type safety.

Test Coverage

This PR is a refactoring effort that updates existing tests rather than adding new functionality to test. The changes are appropriate because:

  1. All test files that previously used hardcoded f"http://localhost:{app_port}/..." patterns have been updated to use build_app_url(app_base_url, ...)
  2. The changes preserve the original test logic while enabling external app testing
  3. Tests that only need browser context (like test_xsrf_cookie_format) appropriately don't require app_base_url

The documentation updates in all three files (.cursor/rules/e2e_playwright.mdc, .github/instructions/e2e_playwright.instructions.md, e2e_playwright/AGENTS.md) provide clear guidance on the new URL handling pattern:

  • Never hardcode localhost URLs
  • Use app_base_url for navigation
  • Use build_app_url() for URL composition

Backwards Compatibility

No breaking changes. This PR only modifies test infrastructure and does not affect the Streamlit library's public API or behavior. All changes are internal to the E2E testing framework.

The app_port fixture is still available for tests that need the raw port number, ensuring backward compatibility with any existing test patterns that require it.

Security & Risk

No security concerns identified.

  1. The URL handling properly uses urllib.parse for safe URL manipulation
  2. The external app URL is only used when explicitly provided via CLI option or environment variable
  3. The changes don't expose any new attack vectors or sensitive information

Low regression risk:

  • The refactoring is mechanical (replacing string concatenation with function calls)
  • The build_app_url() function has clear, deterministic behavior
  • Tests can still run locally with the same behavior as before

Recommendations

The PR is well-implemented and ready for merge. The following are minor suggestions for future consideration (not blocking):

  1. Consider adding unit tests for build_app_url(): While the function is exercised through E2E tests, explicit unit tests would document its expected behavior and catch regressions faster.

  2. Consider moving app_ws_url() to conftest.py: If other test files need WebSocket URL construction in the future, having this helper in conftest.py would be more consistent.

  3. Documentation completeness: The documentation could mention that app_port fixture is still available for cases where only the port number is needed.

Verdict

APPROVED: This PR is a well-executed refactoring that standardizes URL handling across E2E tests, enabling external app testing while maintaining backward compatibility. The code quality is high, documentation is updated, and there are no security concerns or breaking changes.


This is an automated AI review using opus-4.5-thinking. Please verify the feedback and use your judgment.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Standardizes E2E test URL construction by introducing an app_base_url + build_app_url(...) approach (instead of hardcoded http://localhost:{app_port}), and adds infrastructure to target apps embedded in external host pages via iframes.

Changes:

  • Replace hardcoded localhost URL building across multiple E2E tests with app_base_url + build_app_url(...).
  • Extend Playwright fixtures/utilities to support external host pages + iframe targeting (app_target, new CLI options).
  • Broaden a few shared helpers to work with iframe contexts (FrameLocator).

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
e2e_playwright/websocket_reconnects_test.py Switch metrics/media URL building to app_base_url + build_app_url.
e2e_playwright/web_server_test.py Replace localhost hardcoding; add helper for ws URL derivation.
e2e_playwright/st_video_test.py Update media routing + navigation to use app_base_url.
e2e_playwright/st_text_test.py Make tooltip test iframe-compatible via app_target/FrameLocator.
e2e_playwright/st_sidebar_flicker_test.py Use build_app_url for query-based navigation.
e2e_playwright/st_main_layout_test.py Use build_app_url when navigating in custom server fixture.
e2e_playwright/st_image_test.py Update media routing + navigation to use app_base_url.
e2e_playwright/st_file_uploader_test.py Update upload routing + navigation to use app_base_url.
e2e_playwright/st_download_button_test.py Update media routing + navigation to use app_base_url.
e2e_playwright/st_context_test.py Assert against app_base_url instead of localhost+port.
e2e_playwright/st_chat_input_test.py Build navigation URLs with build_app_url rather than parsing app_port.
e2e_playwright/st_audio_test.py Update media routing + navigation to use app_base_url.
e2e_playwright/shared/app_utils.py Allow tooltip helpers to accept FrameLocator contexts.
e2e_playwright/multipage_apps_v2/mpa_v2_basics_test.py Replace localhost hardcoding in multipage URL assertions/navigation.
e2e_playwright/multipage_apps/mpa_basics_test.py Replace localhost hardcoding in multipage URL assertions/navigation.
e2e_playwright/mega_tester_app_test.py Use app_target for iframe-compatible assertions; remove direct navigation call.
e2e_playwright/hostframe_app_test.py Align iframe context URL expectations with app_base_url; add hostframe-load test.
e2e_playwright/host_config_test.py Use app_base_url/build_app_url for host-config routing/navigation.
e2e_playwright/host_config_bypass_test.py Replace localhost base URL usage with app_base_url.
e2e_playwright/forward_msg_cache_test.py Replace localhost navigation with build_app_url(app_base_url, ...).
e2e_playwright/custom_components/component_errors_test.py Replace localhost routing/navigation with app_base_url/build_app_url.
e2e_playwright/conftest.py Add build_app_url, app_target, and CLI/fixtures for external host+iframe mode; generalize load/run waits.
e2e_playwright/config_static_serving_test.py Replace localhost hardcoding for static asset requests.
e2e_playwright/compilation_error_dialog_test.py Use build_app_url for manual navigation.
e2e_playwright/basic_app_test.py Replace localhost navigation with build_app_url(app_base_url, ...).
e2e_playwright/auth_test.py Replace localhost URL waits with build_app_url(app_base_url, ...).
e2e_playwright/AGENTS.md Document app_base_url/build_app_url guidance and “no localhost hardcoding”.
.github/instructions/e2e_playwright.instructions.md Same guidance for contributors/agents.
.cursor/rules/e2e_playwright.mdc Same guidance for Cursor rules.
Comments suppressed due to low confidence (1)

e2e_playwright/shared/app_utils.py:951

  • The expect_help_tooltip signature now allows FrameLocator, but the docstring still documents app as a Page only. Updating the parameter docs to reflect the accepted types will make it clearer how to use this helper in iframe-targeted tests.
def expect_help_tooltip(
    app: Locator | Page | FrameLocator,
    element_with_help_tooltip: Locator,
    tooltip_text: str | re.Pattern[str],
) -> None:
    """Expect a tooltip to be displayed when hovering over the help symbol of an element.

@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from 6e57b44 to fe1a970 Compare January 27, 2026 00:48
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_url_runner_in_e2e_playwright branch from 07b02c8 to 212927b Compare January 27, 2026 00:48
@sfc-gh-bnisco sfc-gh-bnisco requested a review from Copilot January 27, 2026 01:01
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 4 comments.

@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_url_runner_in_e2e_playwright branch from 212927b to 6ca20fe Compare January 27, 2026 19:33
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from fe1a970 to ca84594 Compare January 27, 2026 19:33
@sfc-gh-bnisco sfc-gh-bnisco requested a review from Copilot January 27, 2026 19:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 4 comments.

Comment on lines +44 to +45
- Never hardcode `http://localhost:{app_port}` in tests or fixtures.
- Use `app_base_url` for navigation, routing, and URL assertions.
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bullet says to never hardcode http://localhost:{app_port} in “tests or fixtures”, but app_base_url in e2e_playwright/conftest.py intentionally constructs http://localhost:{app_port} as the default fallback. Consider clarifying the rule to apply to test modules (or “outside conftest”), e.g. “Don’t build localhost URLs in test files; use app_base_url/build_app_url instead.”

Suggested change
- Never hardcode `http://localhost:{app_port}` in tests or fixtures.
- Use `app_base_url` for navigation, routing, and URL assertions.
- Never hardcode `http://localhost:{app_port}` in test modules (outside `conftest.py`).
- Use `app_base_url` for navigation, routing, and URL assertions instead of building localhost URLs directly in tests.

Copilot uses AI. Check for mistakes.
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from 2cd96c1 to ef72adc Compare January 29, 2026 05:51
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_tests branch 2 times, most recently from 5f43c64 to 5c81234 Compare January 29, 2026 06:00
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch 2 times, most recently from 081ef48 to cf2273e Compare January 29, 2026 19:16
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_tests branch 2 times, most recently from 1968bfb to 696cc5b Compare January 29, 2026 20:48
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from cf2273e to 53f11ff Compare January 29, 2026 20:48
@sfc-gh-bnisco sfc-gh-bnisco marked this pull request as ready for review January 29, 2026 22:06
@sfc-gh-bnisco sfc-gh-bnisco requested a review from a team as a code owner January 29, 2026 22:06
Copy link
Copy Markdown
Collaborator

@lukasmasuch lukasmasuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 2, 2026

Summary

This PR standardizes Playwright e2e tests on app_base_url/build_app_url and documents the no-localhost guidance across e2e instructions, improving support for external app runs.

Code Quality

Two remaining localhost assumptions undermine the new URL-handling guidance and will break when running against external app URLs or base paths.

AUTH_SECRETS_TEMPLATE = """
[auth]
redirect_uri = "http://localhost:{app_port}/oauth2callback"
cookie_secret = "your_cookie_secret_here"
expose_tokens = ["id", "access"]

[auth.testprovider]
client_id = "test-client-id"
client_secret = "test-client-secret"
server_metadata_url = "http://localhost:{oidc_server_port}/.well-known/openid-configuration"
"""
@pytest.fixture(scope="module")
def prepare_secrets_file(app_port: int, oidc_server_port: int):
    """Fixture that inject the correct port to auth_secrets.toml file redirect_uri."""
    # Read in the file
    rendered_secrets = AUTH_SECRETS_TEMPLATE.format(
        app_port=app_port, oidc_server_port=oidc_server_port
    )
    # First test without file upload config
    with iframed_app.page.expect_request(lambda request: request.method == "PUT") as r:
        file_chooser.set_files(files=files[0])

    url = r.value.url
    headers = r.value.all_headers()
    response = r.value.response()
    assert response is not None
    assert response.status == 204  # Upload successful
    assert url.startswith("http://localhost")
    assert "_stcore/upload_file" in url
    assert "header1" not in headers

Test Coverage

Changes are confined to e2e tests and guidance; no new unit tests are expected. However, external/embedded test runs will still fail or mis-route in the two locations above until app_base_url is used consistently.

Backwards Compatibility

No runtime behavior changes; these are test-only updates.

Security & Risk

Low risk overall. The main regression risk is e2e runs against externally hosted apps (or base paths) still failing due to remaining hardcoded localhost URLs.

Recommendations

  1. Update auth_test.py to build redirect_uri from app_base_url (and thread app_base_url into prepare_secrets_file) instead of http://localhost:{app_port}.
  2. Replace the hostframe_app_test.py upload URL assertion with an app_base_url-derived check (e.g., build_app_url(app_base_url, path="/_stcore/upload_file/") prefix) so it works with external/base-path hosts.

Verdict

CHANGES_REQUESTED: A couple of remaining localhost hardcodes conflict with the PR’s new external-compatible URL handling and will break external app runs until updated.


This is an automated AI review using gpt-5.2-codex-high. Please verify the feedback and use your judgment.

@github-actions github-actions bot added the do-not-merge PR is blocked from merging label Feb 2, 2026
@sfc-gh-bnisco
Copy link
Copy Markdown
Collaborator Author

I addressed the requested changes by AI in this latest push.

@sfc-gh-bnisco sfc-gh-bnisco removed the do-not-merge PR is blocked from merging label Feb 4, 2026
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_tests branch from 696cc5b to 8c8547b Compare February 4, 2026 21:34
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from 53f11ff to 27dda05 Compare February 4, 2026 21:34
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from 27dda05 to d365ccf Compare February 12, 2026 00:42
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_add_external_tests branch from 8c8547b to 1ebc6fd Compare February 12, 2026 00:42
@sfc-gh-bnisco
Copy link
Copy Markdown
Collaborator Author

sfc-gh-bnisco commented Feb 12, 2026

Merge activity

  • Feb 12, 8:25 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Feb 12, 9:28 PM UTC: Graphite rebased this pull request as part of a merge.
  • Feb 12, 9:48 PM UTC: @sfc-gh-bnisco merged this pull request with Graphite.

@sfc-gh-bnisco sfc-gh-bnisco changed the base branch from 01-22-_feat_add_external_tests to graphite-base/13679 February 12, 2026 21:09
@sfc-gh-bnisco sfc-gh-bnisco changed the base branch from graphite-base/13679 to develop February 12, 2026 21:26
@sfc-gh-bnisco sfc-gh-bnisco force-pushed the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch from d365ccf to 757f876 Compare February 12, 2026 21:27
@sfc-gh-bnisco sfc-gh-bnisco merged commit 6704284 into develop Feb 12, 2026
43 checks passed
@sfc-gh-bnisco sfc-gh-bnisco deleted the 01-22-_feat_utilize_standardized_app_base_url_throughout_e2e_tests_ branch February 12, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:feature PR contains new feature or enhancement implementation impact:internal PR changes only affect internal code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants