It spits out a lot of errors and I would be interested to hear if BCD should be using different fragment ids, or if webref is missing these fragment ids, or if something else is going on. Please see the CI failure for the results.

I'd say that the good news is that, in most cases, it seems that "something else is going on" ;)

Main categories of errors I see:

1. Fragments in ids extracts are percent-encoded. That does not seem to be the case for URLs used in BCD. If percent-encoding seems wrong, we could perhaps change that (I'm always confused as to when that is needed, or good practice). Otherwise, for comparison purpose, some preprocessing is going to be required, see inline.
2. The URL used in BCD may have the filename, e.g., index.html in https://webassembly.github.io/spec/js-api/index.html#dom-globaldescriptor-mutable. URLs in the ids extracts don't have the filename (except for multipage specs!). The index.json file in Webref contains a nightly.filename property for each spec that could be used to create the URL variants with the filename if needed. Alternatively, it could perhaps be a good idea to drop that filename in BCD data?
3. Many URLs in BCD use the series URL, whereas ids extracts in Webref are per specification. For example, you'll have https://drafts.csswg.org/css-logical/#position-properties in BCD, while you'll find https://drafts.csswg.org/css-logical-1/#position-properties in Webref. To find the right level in Webref, you'll need to look at entries in index.json in Webref with the same series.nightlyUrl as the URL (without fragment) used in BCD, then select the entry whose shortname is equal to series.currentSpecification.
4. The code does not handle the case where a spec does not define any ID. That does happen for some WebGL extensions referenced by BCD, such as WebGL EXT_disjoint_timer_query
5. For IETF RFCs produced by the HTTP WG, Webref prefers the httpwg.org URL because its rendering is slightly more user-friendly, whereas BCD seems to use www.rfc-editor.org URLs. The latter URL appears as the canonical url in Webref, so it should be relatively easy to find what you want if BCD wants to keep using that origin.

And then there are actual broken links in BCD, such as https://tc39.es/proposal-temporal/#sec-get-temporal.zoneddatetime.prototype.timezone. There are also "outdated" URLs, such as https://tc39.es/ecma262/multipage/additional-ecmascript-features-for-web-browsers.html#sec-object.prototype.__defineGetter__, which redirects to https://tc39.es/ecma262/multipage/fundamental-objects.html#sec-object.prototype.__defineGetter__ that appears in Webref.

There may be a few other error cases to dig into.

Fantastique François!! 🎉
Thanks for the very useful review comment! I've updated the script :) Now we're down to just 269 problems found! :)

What I see now:

• We should change rfc-editor urls to httpwg urls
• Filenames should be omitted
• Quite a few legit broken fragment links that need to be fixed in BCD (yay, these are the ones I want to chase with this exercise)

Something I would like for you to take a look:

• There are about 22 links to HTML multipage fragments and upon spot checking they work. Maybe these are missing in webref or what am I missing?

There are about 22 links to HTML multipage fragments and upon spot checking they work. Maybe these are missing in webref or what am I missing?

As far as I can tell, all of them are examples of what I called outdated links: they work, but that's because the HTML spec has logic in place to redirect past fragments to their new page. Each time, the content referenced by the link moved to another page of the HTML spec and would better be targeted using the new fragment to avoid a redirect.

For example, clicking on https://html.spec.whatwg.org/multipage/browsing-the-web.html#dom-beforeunloadevent-returnvalue makes you load the browsing-the-web.html page, which includes some JavaScript that detects the fragment, knows it no longer exists in that page, and redirects you to the nav-history-apis.html page where the content was moved. The final URL is https://html.spec.whatwg.org/multipage/nav-history-apis.html#dom-beforeunloadevent-returnvalue. That final URL appears in Webref. Ideally, BCD would always use such final URLs to avoid redirects that consume a bit of time, bandwidth and energy.

For links to IDL terms, specs may follow two distinct patterns:

1. Definition in the IDL block, referenced by the prose. For example, the definition of pictureInPictureEnabled appears in an IDL block, while the "definition" in prose is actually a reference.
2. Definition in prose, referenced by the IDL block. For example, the definition of vendorId appears in prose, while the IDL block contains a reference

I don't think there's broad agreement on which approach is the right one. I note that BCD does not seem to be consistent either, and contains links that target either the IDL block or the prose. Typically, among links that fail validation, there are both:

• links to the prose when first pattern is used, e.g., api.Document.exitPictureInPicture targets https://w3c.github.io/picture-in-picture/#ref-for-dom-document-exitpictureinpicture
• links to the IDL block when the second pattern is used, e.g., api.DeviceOrientationEvent.absolute targets https://w3c.github.io/deviceorientation/#ref-for-dom-deviceorientationevent-absolute

Question is: from a BCD perspective, what would you like links to target for IDL terms? The IDL block or the prose?

Practically speaking, I can easily extract the reference in the IDL block when the second pattern is used. I do not see an easy way to extract the "main" reference in prose when the first pattern is used. Perhaps a rule such as "first normative occurrence right after the IDL block" could work though.

As usual, targeting references that are not qualified in any way is not fantastic: api.ShadowRoot.pictureInPictureElement currently targets https://w3c.github.io/picture-in-picture/#ref-for-dom-documentorshadowroot-pictureinpictureelement①⑤ and I suspect the intended link is rather to https://w3c.github.io/picture-in-picture/#ref-for-dom-documentorshadowroot-pictureinpictureelement①③ (no doubt the spec got updated in the meantime and added two more references before this one). That's not the only case, on top of attachShadow that I reported in my previous comment, links to deviceorientation seem mostly bogus at first sight. Did I mention that these reference links are brittle and dangerous already? ;)

We usually use anchors in the form of #dom-interface-member and not #ref-for-interface-member(number).

So, #23958 (comment) and ~25% of the failures should get fixed by #27293.

We usually use anchors in the form of #dom-interface-member and not #ref-for-interface-member(number).

Ah, that certainly works for me ;) I assumed that you would want to treat these anchors like the ones that point at "for web developers" sections, meaning that you were more looking at pointing at a specific place than at always following what the underlying spec uses as main definition anchor.

I think it is fine but I would certainly welcome if spec authors could agree on consistency for #dom-interface-member anchors as well as recognize the importance of these main definition anchors for projects like BCD.

Down to 207 issues! Seems like lots of problems with SVG and WebGL at this point.

WebGL1 problems should disappear once a new version of Reffy gets released (that's pending release of a new version of webidl2.js with support for the new async_iterable generic type).

For SVG links, the specs need an overhaul. I'm reluctant to spend time in Reffy to try to make sense of IDs in these specs in the meantime (for example, for IDL term definitions, we would need to parse the IDL to make sense of the definitions, but the IDL in SVG Paths is currently invalid, so we would need to patch it first). May I suggest skipping checks for links that target https://svgwg.org/? The links look good and the SVG specs don't really change for now.

@Elchi3, the remaining ~100 links seem to be mainly things that need to be reviewed one by one. I see links where specs could perhaps be updated. Others where BCD probably should. There will no doubt be a few ones that need to be handled as exceptions to the rule. I'd be happy to hop on the phone and go through them with you. Feel free to ping me if you would fancy that!

Thanks @tidoust! Now we're down to 53! I will look through them and will probably have questions for you :)

This pull request has merge conflicts that must be resolved before it can be merged.

This pull request has merge conflicts that must be resolved before it can be merged.