[Bug]: whitespace-only inline elements are removed from cleaned_html, merging words in raw_markdown (prose, not only <pre>/<code>) #2328

crawl4ai version

0.9.3 (Docker image, REST /crawl and /md)

Expected Behavior

Text whose words are split into inline elements keeps its word spacing in raw_markdown, as it does when the markdown is generated from the raw HTML.

Current Behavior

remove_empty_elements_fast (content_scraping_strategy.py) removes whitespace-only inline elements outside <pre>/<code>, and the whitespace inside them goes with them. With the default content_source="cleaned_html", raw_markdown then merges the words: Hi there becomes Hithere.

Real-world trigger: pages whose scripts split text into per-character or per-word spans for animation. Today https://example.com rewrites its paragraph into one <span> per character with each space in its own <span> </span>. Through /md or /crawl (default cleaned_html), the paragraph comes back as Thisdomainisforuseindocumentationexampleswithoutneedingpermission.. Text on the same page that is not split into spans keeps its spaces.

Is this reproducible?

Yes

Inputs Causing the Bug

POST /crawl
{
  "urls": ["raw:<html><body><p><span>H</span><span>i</span><span> </span><span>t</span><span>h</span><span>e</span><span>r</span><span>e</span></p><p>plain text stays fine</p></body></html>"],
  "crawler_config": {"type": "CrawlerRunConfig", "params": {"cache_mode": "bypass",
    "markdown_generator": {"type": "DefaultMarkdownGenerator", "params": {"content_source": "cleaned_html"}}}}
}

Output

cleaned_html no longer contains the <span> </span>:
<p><span>H</span><span>i</span><span>t</span><span>h</span><span>e</span><span>r</span><span>e</span></p>

Root cause

In remove_empty_elements_fast, a childless element whose stripped text has fewer than word_count_threshold words is removed. A whitespace-only element has zero words, so it is always removed. The function preserves the removed element's .tail, but not its own .text, so the space disappears. #1181 hit the same thing inside code and was fixed (v0.7.8) by skipping elements inside <pre>/<code>. Prose split into inline elements has no such guard. #2110 reports the same family of failure in PruningContentFilter on the fit_markdown path.

Suggested fix

When removing an element whose text is whitespace-only, keep the whitespace: append a single space to the previous sibling's .tail (or to parent.text), as is already done for .tail. Alternatively, never remove whitespace-only inline phrasing elements (span, em, strong, b, i, …).

Workaround

DefaultMarkdownGenerator(content_source="raw_html") (via /crawl; /md cannot set it). On the pages we checked it keeps the same words and links as cleaned_html and restores the spacing. fit_html also keeps spacing, but it drops links and pruned content.