Repository navigation
Commit d40856b
feat(helpers): add rule-level validate (like test) (#866)
* feat(helpers): add per-call validate hook to findRule
Allow callers to reject a rule result and continue to the next rule
without throwing, so logo XSS fallthrough can skip hostile candidates.
Co-authored-by: Cursor <[email protected]>
* fix(helpers): scope validate hook and forward it through nested rules
`validate` applied to every property, so a validator written for one
field silently nulled unrelated ones. It now accepts an object keyed by
property name; the function form still runs for every property.
A throwing validator aborted the whole property through get-data's
catch. It now rejects only the candidate that triggered it and the
search resumes at the next rule, with the error logged under
DEBUG=metascraper:find-rule.
metascraper-video and metascraper-audio rebuilt the args object for
their nested findRule calls, so validate never reached rules behind an
iframe and per-candidate fallthrough did not happen there.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VSEyFT3dw16tyvhRGVAWrx
* refactor(video,audio): keep param destructuring in iframe rules
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VSEyFT3dw16tyvhRGVAWrx
* refactor(helpers): share withIframe for audio/video
Deduplicate nested iframe probing so validate keeps one code path,
and tighten docs/tests from review feedback.
Co-authored-by: Cursor <[email protected]>
* refactor(helpers): move validate onto rules like test
Scrape-time validate mixed a second control plane into the call.
Bundle/rule validate matches test and needs no args forwarding.
Co-authored-by: Cursor <[email protected]>
* fix(metascraper): keep pkgName on inline mergeRules
loadRules already copied bundle pkgName; mergeRules dropped it so
inline bundles lost debug names and pkgName could become a prop key.
Co-authored-by: Cursor <[email protected]>
* feat(helpers): pass find-rule debug logger to validate
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VSEyFT3dw16tyvhRGVAWrx
* refactor: simplify validate plumbing and its tests
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VSEyFT3dw16tyvhRGVAWrx
* refactor: drop redundant meta guard and duplicated iframe tests
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VSEyFT3dw16tyvhRGVAWrx
* test(helpers): require package entry like sibling tests
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VSEyFT3dw16tyvhRGVAWrx
---------
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>1 parent 9c5e77b commit d40856b
13 files changed
Lines changed: 682 additions & 129 deletions
File tree
- packages
- metascraper-audio
- src
- test
- metascraper-helpers
- src
- test
- metascraper-video
- src
- test
- metascraper
- src
- test
- types
- unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
61 | 92 | | |
62 | 93 | | |
63 | 94 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
| 9 | + | |
| 10 | + | |
13 | 11 | | |
14 | 12 | | |
15 | 13 | | |
| |||
71 | 69 | | |
72 | 70 | | |
73 | 71 | | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
| 72 | + | |
104 | 73 | | |
105 | 74 | | |
106 | 75 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | | - | |
| 8 | + | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
71 | 93 | | |
72 | 94 | | |
73 | 95 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
548 | 548 | | |
549 | 549 | | |
550 | 550 | | |
551 | | - | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
552 | 565 | | |
553 | 566 | | |
| 567 | + | |
554 | 568 | | |
555 | 569 | | |
556 | 570 | | |
557 | 571 | | |
558 | 572 | | |
559 | 573 | | |
560 | 574 | | |
561 | | - | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
562 | 587 | | |
563 | | - | |
| 588 | + | |
564 | 589 | | |
565 | 590 | | |
566 | 591 | | |
| |||
624 | 649 | | |
625 | 650 | | |
626 | 651 | | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
627 | 685 | | |
628 | 686 | | |
629 | 687 | | |
| |||
676 | 734 | | |
677 | 735 | | |
678 | 736 | | |
679 | | - | |
| 737 | + | |
| 738 | + | |
680 | 739 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
0 commit comments