|
| 1 | +# Native duplication defaults |
| 2 | + |
| 3 | +Measured at `a16d4372376bf18eea3dff4cbe85e15fc68e7fc0` with jscpd 5.4.2 on macOS arm64. |
| 4 | +The scans include native tests and Objective-C headers. They respect `.gitignore`, which excludes |
| 5 | +Swift build caches, Android build output, and packaged helpers. No source exclusions were added. |
| 6 | + |
| 7 | +## Reproduce the comparison |
| 8 | + |
| 9 | +The table records source commit `a16d4372376bf18eea3dff4cbe85e15fc68e7fc0`. |
| 10 | +It is an experiment record, not a target for later revisions. Files below the token minimum do |
| 11 | +not contribute to jscpd's analyzed-file count. Percentages are its duplicated-line metric, |
| 12 | +not unique lines to remove. |
| 13 | + |
| 14 | +Use jscpd 5.4.2. To replay the recorded inputs without changing this checkout, extract only |
| 15 | +tracked source from that commit: |
| 16 | + |
| 17 | +```sh |
| 18 | +pnpm install --frozen-lockfile |
| 19 | +pnpm exec jscpd --version |
| 20 | +native_dupes_source="$(mktemp -d "${TMPDIR:-/tmp}/agent-device-native-dupes.XXXXXX")" |
| 21 | +git archive a16d4372376bf18eea3dff4cbe85e15fc68e7fc0 -- apple android linux \ |
| 22 | + packages/platform-apple/src/core/__tests__/display-inventory.test.ts \ |
| 23 | + packages/platform-apple/src/__tests__/simctl-facade.test.ts | tar -x -C "$native_dupes_source" |
| 24 | +``` |
| 25 | + |
| 26 | +The version command must report `jscpd 5.4.2`. The commands below fix the worker count at two. |
| 27 | +`--no-gitignore` lets the tracked-source archive be scanned independently of its temporary path; |
| 28 | +the archive contains no ignored build output. Each row has a native scan and a separate scan of |
| 29 | +the two TypeScript tests. Reports go to `.tmp/native-dupes-prototypes/`. |
| 30 | + |
| 31 | +| Mode | Tokens / lines | Native clones | Duplicated lines | Duplication | Duo pair clones | Duo panel fragments | |
| 32 | +| --- | --- | ---: | ---: | ---: | ---: | ---: | |
| 33 | +| mild | 50 / 5 | 85 | 873 | 1.80% | 4 | 0 | |
| 34 | +| weak | 50 / 5 | 82 | 857 | 1.77% | 4 | 0 | |
| 35 | +| weak | 70 / 5 | 18 | 228 | 0.47% | 2 | 0 | |
| 36 | +| weak | 100 / 8 | 0 | 0 | 0.00% | 0 | 0 | |
| 37 | +| weak | 30 / 3 | 401 | 2971 | 6.12% | 16 | 0 | |
| 38 | +| weak | 20 / 3 | 860 | 5435 | 11.20% | 31 | 2 | |
| 39 | +| weak + consistent identifier normalization | 50 / 5 | 196 | 2150 | 4.44% | 5 | 0 | |
| 40 | +| weak + gaps up to 8 lines | 30 / 3 | 384 | 2960 | 6.10% | 13 | 0 | |
| 41 | + |
| 42 | +Exact commands, in table order: |
| 43 | + |
| 44 | +```sh |
| 45 | +# mild-50-5 |
| 46 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode mild --min-tokens 50 --min-lines 5 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/mild-50-5-native |
| 47 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode mild --min-tokens 50 --min-lines 5 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/mild-50-5-duo |
| 48 | + |
| 49 | +# weak-50-5 |
| 50 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 50 --min-lines 5 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-50-5-native |
| 51 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 50 --min-lines 5 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-50-5-duo |
| 52 | + |
| 53 | +# weak-70-5 |
| 54 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 70 --min-lines 5 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-70-5-native |
| 55 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 70 --min-lines 5 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-70-5-duo |
| 56 | + |
| 57 | +# weak-100-8 |
| 58 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 100 --min-lines 8 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-100-8-native |
| 59 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 100 --min-lines 8 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-100-8-duo |
| 60 | + |
| 61 | +# weak-30-3 |
| 62 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 30 --min-lines 3 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-30-3-native |
| 63 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 30 --min-lines 3 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-30-3-duo |
| 64 | + |
| 65 | +# weak-20-3 |
| 66 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 20 --min-lines 3 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-20-3-native |
| 67 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 20 --min-lines 3 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/weak-20-3-duo |
| 68 | + |
| 69 | +# renamed-50-5 |
| 70 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 50 --min-lines 5 --ignore-identifiers=consistent --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/renamed-50-5-native |
| 71 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 50 --min-lines 5 --ignore-identifiers=consistent --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/renamed-50-5-duo |
| 72 | + |
| 73 | +# gap-30-3 |
| 74 | +pnpm exec jscpd "$native_dupes_source/apple" "$native_dupes_source/android" "$native_dupes_source/linux" --format swift,objectivec,kotlin,java,python --formats-exts objectivec:m,h --mode weak --min-tokens 30 --min-lines 3 --max-gap-lines 8 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/gap-30-3-native |
| 75 | +pnpm exec jscpd "$native_dupes_source/packages/platform-apple/src/core/__tests__/display-inventory.test.ts" "$native_dupes_source/packages/platform-apple/src/__tests__/simctl-facade.test.ts" --format typescript --mode weak --min-tokens 30 --min-lines 3 --max-gap-lines 8 --workers 2 --no-gitignore --absolute --reporters json --output .tmp/native-dupes-prototypes/gap-30-3-duo |
| 76 | + |
| 77 | +rm -rf "$native_dupes_source" |
| 78 | +``` |
| 79 | + |
| 80 | +The measured native runs took about 170–195 ms each, including Node startup, using two workers. |
| 81 | +This is a single timing sample per variant, not a performance benchmark. Linux execution belongs |
| 82 | +to the Repo Guards & Compatibility CI job. |
| 83 | + |
| 84 | +## Selected policy |
| 85 | + |
| 86 | +Use weak mode with a minimum of 50 tokens and 5 lines. Weak mode removes whitespace and comments |
| 87 | +from matching. The default mild mode adds three findings without improving either motivating |
| 88 | +example. At 70 tokens, all seven Java findings disappear, including repeated snapshot capture and |
| 89 | +error response handling. At 100 tokens and 8 lines, the native report is empty. At 30 tokens and |
| 90 | +3 lines, imports, small guards, and repeated test setup dominate a much larger review backlog. |
| 91 | +Identifier normalization also expands that backlog and still misses the Duo panels. |
| 92 | + |
| 93 | +The selected scan analyzes 218 files and reports 82 clones: 68 Swift, 7 Java, 7 Objective-C, and |
| 94 | +0 Python. There are no Kotlin source files in the measured trees; an executable fixture proves |
| 95 | +Kotlin detection. The Objective-C extension mapping includes `.h`, because those headers contain |
| 96 | +executable geometry code as well as declarations. Five header findings are declaration/implementation |
| 97 | +pairs. Existing findings also include deliberately similar tests and Swift package manifests. |
| 98 | +The baseline records these findings without declaring them defects or requiring extraction. |
| 99 | + |
| 100 | +jscpd 5.4.2 has a [built-in fingerprint baseline](https://jscpd.dev/guides/baseline), introduced |
| 101 | +in 5.1.0. A custom comparison script or percentage ceiling is unnecessary. The baseline records |
| 102 | +content fingerprints and clone multiplicities. It tolerates line shifts and fails on a third copy |
| 103 | +of an accepted pair. A new clone fails even if another clone is removed and the total stays equal. |
| 104 | +The executable policy tests cover each language, headers, line shifts, extra copies, replacement |
| 105 | +clones, missing baselines, empty scans, and removed-clone pruning. |
| 106 | + |
| 107 | +The rollout starts with a report-only measurement and saves that result in |
| 108 | +`native-dupes-baseline.json`. `check:native-dupes` then rejects new fingerprints or increased |
| 109 | +multiplicities. It scans all native files rather than filtering by changed files. This keeps |
| 110 | +unchanged files available as comparison targets and needs no Git history in CI. `check:affected` |
| 111 | +selects it for non-documentation changes under the roots in `.jscpd-native.json`. It does not |
| 112 | +filter by extension, so new scanner formats cannot escape selection. Some metadata changes also |
| 113 | +select this inexpensive scan. Configuration and baseline changes select the full check set. |
| 114 | +`--changed` is unsuitable for PR enforcement because it filters by working-tree status, which is |
| 115 | +empty after a commit. The normal check never updates the baseline. |
| 116 | + |
| 117 | +After rebasing onto `21f3f50a90f21bf8f994e742cbb7dd295f255d6c`, the initial baseline was |
| 118 | +refreshed from main's native source: 82 clones, 854 duplicated lines (1.78%), and 218 analyzed |
| 119 | +files. Three fingerprints were replaced by upstream native changes. The table above remains the |
| 120 | +historical comparison at `a16d4372376bf18eea3dff4cbe85e15fc68e7fc0`. |
| 121 | + |
| 122 | +## Motivating cases and limits |
| 123 | + |
| 124 | +The closed-Duo fixtures differ at the measured revision. `display-inventory.test.ts` includes |
| 125 | +bounds, orientation, and unique IDs absent from `simctl-facade.test.ts`. At 50 / 5, jscpd finds |
| 126 | +shared imports and the separate single-panel fixture, but no shared closed-Duo panel fragment. |
| 127 | +At 20 / 3, it finds two 26-token panel fragments, at `simctl-facade.test.ts:36–40` and `46–50` |
| 128 | +against `display-inventory.test.ts:76–80` and `93–97`. It also reports 860 native clones. Enabling |
| 129 | +gap merging cannot recover fragments below its seed token minimum. TypeScript remains owned by |
| 130 | +Fallow; this experiment does not add a second TypeScript enforcement policy. |
| 131 | + |
| 132 | +No matrix variant finds a clone in the checked-in `Fold.m`. A separate 10-token / 2-line probe |
| 133 | +finds two tiny repeated fragments (lines 165–167 against 230–232, and 214–216 against 218–220). |
| 134 | +Those are not evidence that it would catch the historical seven-line case; that original duplicated |
| 135 | +revision was not replayed. Keep small or reimplemented logic in code review. Renamed or partially |
| 136 | +edited copies can evade the selected exact-token policy. Reported blocks need human assessment, |
| 137 | +especially test setup and Objective-C declarations. |
| 138 | + |
| 139 | +## Inspect or change the baseline |
| 140 | + |
| 141 | +Run `pnpm native-dupes:report` to inspect findings without failing on new clones. Both report and |
| 142 | +check write `.tmp/native-dupes/jscpd-report.json`; each clone has an `isNew` field. CI uploads the |
| 143 | +check report as `native-dupes-report` even when the check fails. |
| 144 | + |
| 145 | +When a check reports a new clone, compare both locations and remove the repeated block when they |
| 146 | +share an owner. If retaining duplication is intentional, review its source before running |
| 147 | +`pnpm native-dupes:baseline`. Review the baseline diff together with the source change. Do not |
| 148 | +regenerate it merely to clear a failing check. A detector upgrade or a change to matching options |
| 149 | +also needs a fresh comparison and baseline review. Missing baselines and empty scans fail closed. |
| 150 | + |
| 151 | +The normal check does not prune stale fingerprints. If a clone disappears while its fingerprint |
| 152 | +remains in the baseline, the same block can be reintroduced up to its saved multiplicity without |
| 153 | +failing. When removing a clone, run `pnpm native-dupes:baseline` in the same PR. jscpd's |
| 154 | +`--update-baseline` replaces the saved fingerprints with those found by that scan, removing absent |
| 155 | +fingerprints and reducing obsolete multiplicities. Review that the refresh removes or reduces |
| 156 | +entries; investigate any additions. The executable pruning test proves that removal passes, |
| 157 | +an unpruned allowance accepts reintroduction, and a refreshed baseline rejects reintroduction. |
0 commit comments