Repository navigation
Page.content() omits shadow DOM, but locators pierce it — selectors that look unique silently aren't (revives #30816) #42794
Description
Problem
Page.content() only serializes light DOM — shadow root contents are invisible to it, which is expected per the DOM serialization spec. But Playwright's own CSS-based locators pierce open shadow roots by default when matching at runtime.
That mismatch is a real footgun for anything that builds selectors or automation from page.content()'s output (scrapers, selector generators, AI browser agents, test recorders): a CSS path that looks unique in the light-DOM-only HTML can still match a second element inside an open shadow root — silently, since matching succeeds and no error is raised. The result is a click (or any locator action) landing on the wrong element, indistinguishable from a correct one.
Minimal repro
html = """<html><body>
<div id="host"><button onclick="mark('light')">Light</button></div>
<script>
window.hits = [];
function mark(v) { window.hits.push(v) }
host.attachShadow({mode: 'open'}).innerHTML =
"<button onclick=\"window.hits.push('shadow')\">Shadow</button>";
</script>
</body></html>"""
page.set_content(html)
# Looks unique in page.content() — the shadow button is invisible to it.
assert page.content().count("<button") == 1
# But Playwright's own locator pierces the shadow root and finds BOTH:
assert page.locator("#host > button").count() == 2 # not 1A tool that parses page.content(), sees one <button>, and hands back #host > button as "the unique selector" has just produced one that silently matches two live elements.
This was already proposed, and closed for inactivity — not rejected on merit
#30816 asked for exactly this (page.content({ serializableShadowRoots: true }) via Element.getHTML({ serializableShadowRoots: true })) and was closed after ~16 months with:
"If you have additional information not present in this issue that you think will help prioritizing it, please open a new issue and reference this one. More support or clarity on its necessity may prompt a review."
This is that — a concrete failure mode instead of a general "let the crawler grab everything," plus proof it's buildable.
Already built and validated, independently
Hit this building zerodom (an MCP server generating verified-unique CSS selectors for AI agents), and implemented exactly what #30816 proposed — walking the DOM for open shadow roots, serializing them via Element.getHTML({ serializableShadowRoots: true, shadowRoots }), and feeding the result back into selector generation so it can see (and correctly disambiguate against) shadow content it would otherwise be blind to.
Validated across 111 real sites — 1,334 of 10,756 audited selectors had to be scoped against open shadow roots (121 of 129 on shoelace.style alone), every one resolving to exactly one live element once shadow content was actually visible.
Ask
Not asking for that exact implementation — just that Page.content() (or a documented alternative) makes shadow-root content visible, so tools built on it stop silently producing selectors that look correct and aren't. Happy to share the full implementation/tests, or help however's useful given how content() is actually implemented across Chromium/Firefox/WebKit.
Activity
yury-s commentedon Sep 21, 2026
SYED HUSNAIN KHALID (@DevHusnainAi) can you share an actual use case where this is useful?
Hronom commentedon Sep 21, 2026
A concrete browser-agent use case is a coding agent interacting with a component-library control rendered inside an open shadow root: it receives a page snapshot or HTML representation, derives a locator, clicks or types, and then reads back the application state. If the evidence surface omits shadow content while live locators pierce it, uniqueness proven from the serialized representation is unsound. That is a target-identity problem, not only a scraper-completeness problem.
A useful contract could be either:
For generated selectors, preserve the host/shadow-root path and document generation rather than flattening a shadow descendant into an ordinary light-DOM CSS path. Invalidate the locator when the host or shadow root is replaced. Tests could include one light-DOM control plus one same-shaped control in an open root, nested open roots, host replacement, a closed root, and a locator generated before navigation followed by a fresh read-back.
I maintain Hronaut, a local visible Browser/MCP workspace, and this is the same target-identity/read-back boundary used in its evaluator material. I have not run the proposed zerodom implementation or Playwright’s patch, so I’m not claiming a result. AI-assisted note.
pavelfeldman commentedon Sep 21, 2026
Yevhen Tienkaiev (@Hronom) Sounds like we are missing
browser_html(ref)tool / cli for your use case, not this feature.Hronom commentedon Sep 22, 2026
That split makes sense. For an agent, the important part is not making
browser_html(ref)an all-purpose DOM serializer; it is making the observation surface explicit. If it returns the ref/generation, whether open shadow roots were included or omitted, and the traversal/serialization mode, the agent can avoid claiming selector uniqueness from incomplete evidence. In the minimal repro, I would expect the tool to either return both candidates or mark#host > buttonas unresolved/ambiguous; after an action, an independent application-state read should verify which target changed. That keeps a snapshot from being mistaken for live execution evidence. I maintain Hronaut, a local visible Browser/MCP workspace, and this is the same boundary we care about. Happy to test a small contract fixture if that tool shape is useful. AI-assisted.