Skip to content

fix!: preserve DOCTYPE internal subset #498

Merged
karfau merged 121 commits into
xmldom:masterfrom
shunkica:internal-subset
Jul 10, 2023
Merged

fix!: preserve DOCTYPE internal subset #498
karfau merged 121 commits into
xmldom:masterfrom
shunkica:internal-subset

Conversation

@shunkica

@shunkica shunkica commented Jun 15, 2023

Copy link
Copy Markdown
Collaborator

Add support for parsing the internal subset of a DOCTYPE and saving it as a string in DocumentType.internalSubset.

BREAKING CHANGE: Many documents that were previously accepted by xmldom, esecially non well-formed ones are no longer accepted. Some issues that were formerly reported as errors are now a fatalError.

fixes #117, #497

Spec: XML DOM L2 Core

@shunkica

This comment was marked as outdated.

@shunkica

This comment was marked as outdated.

@karfau karfau added the spec:DOM-Level-2 https://www.w3.org/TR/DOM-Level-2-Core/ label Jun 15, 2023
@karfau

karfau commented Jun 15, 2023

Copy link
Copy Markdown
Member

Thx, I will have a closer look over the weekend.

@shunkica

This comment was marked as outdated.

@karfau karfau changed the title Parse the internal subset of a DOCTYPE feat: preserve internal subset of a DOCTYPE Jun 16, 2023
@karfau karfau changed the title feat: preserve internal subset of a DOCTYPE feat: preserve internal subset of DOCTYPE Jun 16, 2023
Comment thread lib/sax.js Outdated
Comment thread lib/sax.js Outdated
@karfau

karfau commented Jul 9, 2023

Copy link
Copy Markdown
Member

The regular expressions no longer have a chars property.
THe tests for the EntityDecl are good enough as a first step.

I managed to split the methods again, so they have a managable/testable amount of complexity.
Let me know if the approach I picked works for you. It's not a class, but I would say it's close enough.

The rino example is currently failing, so I have to fix that.

The only other thing missing is to add some unit tests for those new methods and for the new error messages, before the PR can be landed.

Comment thread test/sax/parseUtils.test.js Fixed
@karfau

karfau commented Jul 9, 2023

Copy link
Copy Markdown
Member

Rhino doesn't seem to support named groups, which we are now using to get understandable results for ExternalIDs...

But I will risk having to change the regular expression and code to make it compatible in a later PR, once it's merged and published, just to see if anybody complains.

@karfau

karfau commented Jul 9, 2023

Copy link
Copy Markdown
Member

@shunkica From my perspective I would land this. Let me know whether you agree or disagree :)

Great endurance on this contribution, and I'm looking forward to the next one :)

@shunkica

Copy link
Copy Markdown
Collaborator Author

@karfau I agree. Let's proceed with merging, and we'll tackle any issues that arise in subsequent updates.
Thanks for the hard work.

@karfau karfau changed the title fix: preserve internal subset of DOCTYPE fix!: preserve DOCTYPE internal subset Jul 10, 2023
@karfau karfau merged commit 726b471 into xmldom:master Jul 10, 2023
@karfau

karfau commented Jul 10, 2023

Copy link
Copy Markdown
Member

landed, I will be able to make a new beta release on the next tag in the upcoming days.

@karfau

karfau commented Jul 13, 2023

Copy link
Copy Markdown
Member

released as 0.9.0-beta.9 under the next version tag

@shunkica shunkica deleted the internal-subset branch July 13, 2023 10:04
karfau added a commit that referenced this pull request Oct 4, 2023
which is a regression compared to 0.8.x, most likely introduced as part
of #498

- add check to `parseDoctypeCommentOrCData`
- drop redundant and broken `Element.appendChild` implementation
- `hasInsertableNodeType` now checks for `CharacterData` nodes instead
of only text nodes
- align ParseError and DOMException in how they are extending Error
- wrap `DOMException`s in `ParseError` in sax parser
- move custom errors to own module
  - and allow current and modern constructor arguments for DOMException
stevenobiajulu added a commit to stevenobiajulu/xmldom that referenced this pull request Mar 6, 2026
Per XML spec §2.6, PI data extends to immediately before '?>'.
Trailing whitespace inside the PI boundary is content, not separator.
Remove \s* before $ in parseInstruction regex so it is no longer stripped.

Addresses the trailing-whitespace sub-issue from xmldom#42, backporting xmldom#498
behaviour to the 0.8.x line.
karfau pushed a commit that referenced this pull request Mar 7, 2026
**Target branch**: `release-0.8.x`

### What

Remove `\s*` from the `parseInstruction` regex in `lib/sax.js` so that
trailing whitespace inside a processing instruction is preserved instead
of silently stripped.

### Why

Per [XML spec §2.6](https://www.w3.org/TR/xml/#sec-pi), PI data is
everything between the mandatory separator whitespace after the target
and the closing `?>`. Trailing whitespace inside the PI boundary is
content — there is no rule to strip it. Conforming parsers (sax-js,
libexpat) preserve it.

This was already fixed on `master`/`0.9.x` as a side-effect of the large
DOCTYPE rewrite in PR #498 (22k lines). This PR is the minimal,
non-breaking backport for the maintained `0.8.x` line.

### How

`parseInstruction` builds a substring that already excludes `?>`, so `$`
anchors immediately before it. The `\s*` before `$` was greedily
consuming any trailing whitespace from PI data before passing it to
`domBuilder.processingInstruction`. Removing it — while keeping `*?` on
the data group to minimise diff — is the complete fix.

```js
// before
source.substring(start, end).match(/^<\?(\S*)\s*([\s\S]*?)\s*$/)
// after
source.substring(start, end).match(/^<\?(\S*)\s*([\s\S]*?)$/)
```

Five existing snapshots in
`test/xmltest/__snapshots__/not-wf.test.js.snap` were updated: they
captured the old buggy behaviour (trailing space stripped from
XML-declaration-like PIs in the not-well-formed corpus). The updated
snapshots reflect the now-correct output.

### Scope

Addresses the trailing-whitespace sub-issue from #42, backporting #498
behaviour to 0.8.x.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spec:DOM-Level-2 https://www.w3.org/TR/DOM-Level-2-Core/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trailing whitespace in processing instructions not preserved To what degree should xmldom support DTD processing support for Entity/ EntityReference?

4 participants