Repository navigation
fix(router): support a wildcard in the middle of a path - #5243
Conversation
TrieRouter stored `/assets*` as a literal segment named `assets*`, so it never matched anything. RegExpRouter, PatternRouter and LinearRouter all support the pattern, so whether such a route worked depended on which router SmartRouter happened to settle on. The prefix is now registered on the parent node at insert time and matched against the rest of the path, which is what a trailing `*` means in the other routers. No params are captured, matching them as well. The shared router cases cover it. LinearRouter and PatternRouter skip the nested case, which they get wrong independently of this change.
LinearRouter and PatternRouter both dropped two characters when a route ended in `*`, which removed the last character of the segment as well as the star. `/a/b*` became the prefix `/a/`, so it matched `/a/c`, `/a/` and everything else below `/a/`, and `/assets*` matched `/asset`. Slicing one character keeps the intended prefix. Because that prefix now retains its trailing slash for `/a/b/*`, LinearRouter gets an explicit check so that form still matches `/a/b`, and PatternRouter anchors the wildcard with `(?:/|$)` so `/assets/*` no longer matches `/assetsfoo`. The shared router cases cover both, and the two skips added for this in the previous commit are no longer needed.
RegExpRouter matches `/x*/y` against `/xzz/y`, taking one or more characters within a single part. TrieRouter and PatternRouter did not support the form at all, and LinearRouter accepted `/x/y` and `/x/z/y`, letting the wildcard take nothing or run across a `/`. TrieRouter records the prefix on the parent node and continues from the matching part. PatternRouter compiles the prefix followed by `[^/]+`. LinearRouter now stops the wildcard at the end of its part and requires it to take at least one character. All four routers agree across the shared router cases.
4f92d59 to
7b64ab1
Compare
|
Hi @usualoma What do you think of these changes? We recently changed some router behaviors, but I have been thinking we have not said anything about wildcard-related behaviors. For example, we don't say that the route https://hono.dev/docs/api/routing So, we can refuse these changes because it's not in the spec. But on the other hand, we should remove inconsistencies. |
|
Hi @Kjubikstronk, Thank you for putting this PR together. As @yusukebe pointed out, I agree that eliminating behavioral differences between routers is desirable. I think the expected behavior should be as follows:
A wildcard appended to a non-empty prefix should be able to consume zero or more characters. When it appears in the middle of a route, it should stay within that path segment; when it is route-final, it may consume the rest of the path, including A standalone wildcard segment is slightly different: when another segment follows it, it should consume at least one character. Therefore, This keeps the zero-character behavior of prefix wildcards consistent with the existing route-final Also, since this is not an urgent fix and necessarily changes existing route-matching behavior, it may be better to land it as part of v5. We could describe it in the v5 release notes as something like: “Improved consistency in routing behavior across router implementations.” |
|
@yusukebe What do you think about the proposed routing behavior described above? Separately, I believe we can implement it with only a very small increase in code size, particularly for the default RegExpRouter + TrieRouter combination. |
|
Hi @usualoma I checked your behavior rule, and it makes sense. Some behaviors may confuse users, but it's fine to describe the rule on the website.
Agree! Let's do it. Can you create another PR for this? Then I'll label it |
|
That table is a better answer than this PR. I only went after the one case where the routers disagreed, and the distinction you draw — a wildcard with a prefix consuming zero or more characters within its segment, a standalone Happy to close this in favour of the v5 PR. If it would save @usualoma time I can put the table into tests and implement against it instead, but it's your change to make and I don't want to get in the way of it. |
|
Hi @Kjubikstronk, |
A wildcard with a non-empty prefix now consumes zero or more characters and stays inside its own segment, and a standalone `*` segment consumes at least one character when a segment follows it. Either still consumes the rest of the path when it ends the route. The routers disagreed in different ways. RegExpRouter translated every mid-path `*` to [^/]+, so a prefixed one could not consume zero characters. TrieRouter only registered a prefixed wildcard as a pattern when it ended the route, so a mid-path one was stored as a literal key and never continued to the next segment. PatternRouter escaped it to a literal asterisk. LinearRouter let a standalone `*` match an empty segment. The spec sheet from honojs#5243 is now a table in the shared router tests, so all five routers are held to it.
7b64ab1 to
5f9da9d
Compare
|
Done — the table is now a case list in the shared router tests, so all five routers run it. Each router was failing it differently:
18 failures before, all green now, with the rest of the suite unchanged. One thing I deliberately left out: |
|
@Kjubikstronk Sorry, I missed one more inconsistency: routes such as The expected behavior would be:
Whether Something like the following should align the routers: diff --git a/src/router/pattern-router/router.ts b/src/router/pattern-router/router.ts
index 0b4755e..a9fa7cc 100644
--- a/src/router/pattern-router/router.ts
+++ b/src/router/pattern-router/router.ts
@@ -24,7 +24,9 @@ export class PatternRouter<T> implements Router<T> {
? `/(?<${match[1]}>${match[2] || '[^/]+'})`
: part === '/*'
? '/[^/]+'
- : part.replace(/[.\\+[^\]$()]/g, '\\$&').replace(/\*/g, '[^/]*')
+ : part.endsWith('*')
+ ? part.slice(0, -1).replace(/[.\\+*[^\]$()]/g, '\\$&') + '[^/]*'
+ : part.replace(/[.\\+*[^\]$()]/g, '\\$&')
}
)
diff --git a/src/router/reg-exp-router/node.ts b/src/router/reg-exp-router/node.ts
index 2dce7ac..18c88c1 100644
--- a/src/router/reg-exp-router/node.ts
+++ b/src/router/reg-exp-router/node.ts
@@ -67,9 +67,11 @@ export class Node {
? token === '*'
? i === len - 1
? ['', '', ONLY_WILDCARD_REG_EXP_STR] // '*' matches to all the trailing paths
- : tokens[i - 1] === '/'
- ? ['', '', LABEL_REG_EXP_STR] // a standalone '*' takes a whole segment
- : ['', '', PREFIXED_WILDCARD_REG_EXP_STR] // 'x*' may take nothing
+ : tokens[i + 1] !== '/'
+ ? null
+ : tokens[i - 1] === '/'
+ ? ['', '', LABEL_REG_EXP_STR] // a standalone '*' takes a whole segment
+ : ['', '', PREFIXED_WILDCARD_REG_EXP_STR] // 'x*' may take nothing
: null
: token === '/*'
? ['', '', TAIL_WILDCARD_REG_EXP_STR] // '/path/to/*' is /\\/path\\/to(?:|/.*)$/ |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## v5 #5243 +/- ##
=====================================
Coverage ? 79.77%
=====================================
Files ? 130
Lines ? 10082
Branches ? 2213
=====================================
Hits ? 8043
Misses ? 2039
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
`/x*y` and `/x*y/index` were still handled differently between routers. PatternRouter and RegExpRouter read the `*` as a wildcard wherever it appeared in the segment, so `/x*y` matched `/xay`; TrieRouter and LinearRouter already required it to end the segment. Both now ignore a `*` with anything after it in the same segment, which is what the other two were already doing. The four cases from the spec sheet are in the shared tests.
|
Applied, thanks — your diffs were right. Both routers now ignore a Worth noting the other two already behaved that way: TrieRouter's All 17 cases pass on all five routers, full suite green. |
|
Hi @Kjubikstronk,
Right, I had overlooked this point in my previous comment. I took another look at it, and rather than extending In particular, these two patterns only partially overlap: app.use('/x/*', middleware)
app.get('/x*/y', handler)
Because the overlap is conditional, associating I would therefore like to make a The resulting RegExpRouter behavior would be:
The result would not depend on registration order. For reference, this is the change I have in mind: diff --git a/src/router/reg-exp-router/node.ts b/src/router/reg-exp-router/node.ts
index aed623d3..8955467f 100644
--- a/src/router/reg-exp-router/node.ts
+++ b/src/router/reg-exp-router/node.ts
@@ -98,13 +98,18 @@ export class Node {
nextNode = node.#children[regexpStr]
if (!nextNode) {
- if (regexpStr !== ONLY_WILDCARD_REG_EXP_STR && regexpStr !== TAIL_WILDCARD_REG_EXP_STR) {
+ if (
+ regexpStr !== ONLY_WILDCARD_REG_EXP_STR &&
+ (regexpStr !== TAIL_WILDCARD_REG_EXP_STR ||
+ node.#children[PREFIXED_WILDCARD_REG_EXP_STR])
+ ) {
for (const k in node.#children) {
if (
// a single-char pattern coexists with single-char literals as a literal does
(regexpStr.length > 1 || k.length > 1) &&
k !== ONLY_WILDCARD_REG_EXP_STR &&
- k !== TAIL_WILDCARD_REG_EXP_STR
+ (k !== TAIL_WILDCARD_REG_EXP_STR ||
+ regexpStr === PREFIXED_WILDCARD_REG_EXP_STR)
) {
throw PATH_ERROR
}With this approach, we do not need to expand The minified bundle increase for this additional conflict check is about 20 bytes compared with the current PR. Would you mind updating the PR in this direction? |
RegExpRouter cannot represent `/x*/y` and `/x/*` together: they overlap only conditionally, so associating middleware with the whole route would run it for paths it does not cover. Throw UnsupportedPathError for that pair regardless of registration order and let SmartRouter fall back to TrieRouter.
|
Done, implemented as you described. I checked every row of your table in both registration orders and they all come out the way you predicted, including Added three cases to Full test suite passes. |
|
Hi @Kjubikstronk, From my perspective, this PR is now complete for its intended scope, and the current implementation looks good to me. |
|
Thanks for working through all of that with me — appreciate the patience on the back and forth. |
|
Hi @Kjubikstronk, Thanks again. During a final check, I found one regression, so I used the “Maintainers are allowed to edit this pull request” option and pushed a small follow-up fix directly to this branch.
['/', ':tenant', '/', '*', '/*']The I changed the check to: tokens[i + 1][0] !== '/'and added regression coverage for:
|
|
Thanks for catching that and fixing it on the branch. I had only thought about the next token being exactly |
|
v5 has moved since this was labelled, and #5318 touches the same trie-router gate and test block. Should I rebase this onto v5 and update the description to the zero-or-more behaviour it has now? |
|
I rebased and updated the migration guide. I'll merge into v5. Thanks! |
Rebased onto
mainnow that #5236 and #5252 have landed. This was stacked on two earlier PRs of mine covering the trailing-wildcard cases; those are superseded and closed, so this is a single commit against currentmainand only covers the mid-path form.Sitting on top of the refactored trie made it much smaller — six lines of source instead of the twenty-five it took before.
The gap
RegExpRouter supports a wildcard in the middle of a path. The other three do not, each differently:
/x*/y/xz/y/x*/y/x/y/a/b*/c/a/bz/cTaking RegExpRouter as the reference, a mid-path
*takes one or more characters and stays inside its own part./x/ydoes not match because the wildcard would take nothing, and/x/z/ydoes not because it would cross a/. That is different from a trailing*, which takes anything including/and may be empty.The change
prefix*form only when it was the last part (nextP === undefined). Registering it everywhere, and continuing the walk when the part is longer than the prefix, is enough — the existing per-part matching already prevents it crossing a/.*into the compiled regex, so it looked for a literal asterisk. Aprefix*part now compiles to the escaped prefix followed by[^/]+./from the current position, which let the wildcard take zero characters. It now rejects a zero-length take.Verification
Cross-router matrix over 13 route shapes × 25 paths, covering mid-path, trailing, bare
*, params and static routes:Mid-path wildcardadded to the shared router cases, covering the rest-of-part match, the at-least-one-character rule, that it does not run past its part, and the same below the root. All four routers pass with noskipentries.pnpm vitest --project main: 123 files, 3880 passed, 28 skipped, 0 failed.tsc -p tsconfig.spec.json, ESLint and Prettier clean.Earlier, before the rebase, I measured the trie hot path since this adds a condition inside
search: 1.32–1.35 M matches/sec against 1.34–1.36 without it, within run-to-run noise. The condition here is a comparison on a value already loaded, so the shape is the same.