Skip to content

Commit a945faa

Browse files
manovotnyclaude
andauthored
fix(migrate): address findings from the pre-release smoke test (#604)
* fix(migrate): address findings from the pre-release smoke test - Report users created without an extra or unverified email or phone Clerk refused once the user existed: the summary lists them, `--json` adds them to `result.warnings`, the user's line records `refused`, and `runs <id>` shows them under warnings. Before, only users.ndjson said so. - A secret key Clerk refuses (401) stops the import with Clerk's error, rather than a stand-in instance that sent the run checks off course. A run undone in full no longer blocks a stand-in import with "did not confirm which instance". - Replace a gateway's HTML error page with "Clerk returned HTTP <status> with no error details" in import, undo and runs breakdowns. - A refused dry run says so after the checks, with the --allow-partial command (`refused: true` in --json). - Explain the APP_ID placeholder in fix commands (`checks.placeholders` in --json). - `--json` carries the notes the human summary adds to Clerk's messages, such as the development-instance user limit (`result.notes`). Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): only call an attach refused when Clerk refused the identifier A 401 says the key stopped working, and an exhausted 429 says nothing about the email or phone, so neither is recorded as `refused` or warned about. Also lists `refused` among the users.ndjson fields, splits the `runs <id>` README sentence, and rewrites the changeset in short sentences. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): leave an attach pending when the failure isn't about the identifier A 401 or a 429 that outlasted its retries says nothing about the email or phone, so the attach stays pending and a re-run finishes it, rather than being dropped with only a note. Also types `createUser`'s response and adds the Oxford commas the README and changeset were missing. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): restore createUser's untyped response, which tsc needs Typing it made tsc report a use before assignment on a path it can't follow. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): leave an attach pending on a 403 that isn't about the phone A 403 can mean the key isn't permitted, which says nothing about the email or phone, so the attach stays pending for a re-run. A country Clerk doesn't support (`unsupported_country_code`) is still a refusal of the phone. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): report dropped phones in runs <id>, and match the refused dry run to the real one A first phone Clerk refused on the create now lands on the user's line as `phoneRefused`, so `runs <id>` warns about it the way the import summary does. A refused dry run now gives the real refusal's advice and next steps, including the earlier run's undo or continue. The README also says which attach errors stay pending. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): address Roy's second review of the final-testing fixes - An attach Clerk answers `form_identifier_exists` looks the user up first: when the user already holds it, an earlier attach with no answer landed, so it counts as attached, not refused. A failed lookup leaves it pending. - A refused first phone's reason goes on the retry's `creating` line too, so a continue that adopts the user keeps `phoneRefused` and its note. - A created user's first line carries its notes, so a stop before the second line doesn't lose why the phone was dropped. - A refused `--dry-run --json` carries the real refusal's `advice` and `steps`. - `runs <id>` lists those users under "Imported without an email or phone", and both commands share the warning prefixes. - Tests for a 429 attach that outlasts the retries, the taken-by-this-user lookup, the adopted phone refusal, and the SMS note on a refused attach. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): address Roy's third review of the final-testing fixes - An email the instance's allowlist or blocklist refuses (`not_allowed_access`) is a refusal, not a key problem left pending. - A taken phone counts as attached when the user holds it in E.164, whatever the source wrote; emails are trimmed. - A lone reserved phone a retry already counts as refused isn't also pending, so a continue that adopts the user warns about it once. - An attach left pending on a Clerk error notes why, once across continues, and `runs <id>` lists those users. - `runs <id>` drops its warnings once the run is undone in full. - A refused dry run prints its next steps in agent mode too, and a real refusal's `--json` carries `advice` and `steps`. - A key Clerk rejects names where it came from, with Clerk's long message. - Tests for the continue's phone wiring, the dry run's next steps, and a 403 for the country; README and changeset brought up to date. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(migrate): group refusals without the identifier, and word an HTML 401 by status - A refusal's summary key drops the email or phone Clerk's message names, so `not_allowed_access` refusals group as one warning instead of one per user. - A 401 whose body is a proxy's HTML page reads "Clerk returned HTTP 401 with no error details" instead of the markup. - Spread the modes into `test.each`, per the testing rule. Co-Authored-By: Claude Opus 5.5 <[email protected]> * refactor(migrate): address Roy's nits on the final-testing fixes - One `adoptedLines` map of `creating` lines replaces the `adoptedReserved`, `adoptedPending` and `adoptedPhoneRefused` options. - `RegExp.escape`, no spread for an unset body, notes and the key source each built once, and a one-line warning loop without a status check that never skipped anything. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Bump. --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
1 parent 4b196c7 commit a945faa

16 files changed

Lines changed: 1434 additions & 183 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"clerk": patch
3+
---
4+
5+
`clerk migrate import` now warns about users it created without an email or phone Clerk refused, in the summary, `--json`, and `clerk migrate runs <id>`, which also warns about users imported without their first phone. An attach that fails on the key or a `429` that outlasts the retries stays pending for a re-run, with a note saying why, and one Clerk calls taken counts as attached when the user already holds it. A secret key Clerk rejects gets Clerk's error and says where the key came from. A refused dry run says so and gives the same next steps as the real run. A run undone in full no longer blocks an import when Clerk can't name the instance. A gateway's HTML error page no longer fills the error breakdown. The checks explain how to replace `APP_ID` in fix commands, and `--json` carries the notes the summary prints. `clerk migrate` is still experimental and needs `CLERK_EXPERIMENTAL=migrate`.

‎packages/cli-core/src/commands/migrate/README.md‎

Lines changed: 56 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -139,11 +139,11 @@ leaves `.gitignore` alone.
139139

140140
Each run is a folder named for its ID, `YYYYMMDD-HHmmss-xxxx`:
141141

142-
| File | Contents |
143-
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
144-
| `run.json` | Kind, status, start and finish times, the target, the source, the file and its sha256, an import's flags, and the counts |
145-
| `users.ndjson` | One line per user outcome: `sourceId`, `clerkId`, `status`, and `reason`, `error`, `code`, `pending` or `passwordDropped` when present |
146-
| `lock` | The PID of the process writing the run, while it runs |
142+
| File | Contents |
143+
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
144+
| `run.json` | Kind, status, start and finish times, the target, the source, the file and its sha256, an import's flags, and the counts |
145+
| `users.ndjson` | One line per user outcome: `sourceId`, `clerkId`, `status`, and `reason`, `error`, `code`, `pending`, `reserved`, `refused`, `phoneRefused`, or `passwordDropped` when present |
146+
| `lock` | The PID of the process writing the run, while it runs |
147147

148148
On a `failed` line, `code` is the HTTP status, such as `422` or `429`, or
149149
`unknown`. On a `skipped` line it gives the reason in machine form:
@@ -154,9 +154,10 @@ asked to skip.
154154

155155
A user's status is `creating`, `created`, `failed`, `skipped`, `deleted` or
156156
`exported`. The last line for each `sourceId` wins. A `429` retry, an extra
157-
email or phone that did not attach, a first phone Clerk refused (which the
158-
summary also counts), and a validation failure all land in the line's `error`
159-
field.
157+
email or phone Clerk refused, a first phone Clerk refused (which the summary
158+
also counts), and a validation failure all land in the line's `error` field. So
159+
does why an attach was left `pending`, when Clerk said: "Not added yet", which a
160+
continue replaces rather than repeats.
160161

161162
`creating` is written as a user's `POST /v1/users` goes out. It stays the latest
162163
status when no answer says whether the create landed: an abort, a network error,
@@ -779,8 +780,10 @@ the flag: it continues this run and creates them. A skip for the file's own data
779780
something to retry: fix the file, which makes a new run.
780781

781782
`--json` returns `{ target, run, resume, checks, result }`. When a run stops
782-
before importing, it carries one of `dryRun: true`, `refused: true`,
783-
`consent: "required"` or `nothingToImport: true` in place of `result`, and a
783+
before importing, it carries `dryRun: true`, `refused: true`,
784+
`consent: "required"` or `nothingToImport: true` in place of `result` (a
785+
refused dry run carries both `dryRun` and `refused`, and a refusal carries
786+
`advice` and `steps`), and a
784787
file already imported in full returns `alreadyImported: true`. With
785788
`--require-password`, `withoutPassword` counts the users it left out before the
786789
checks, so on a first import `checks.total` plus it is the file's size. A
@@ -807,8 +810,11 @@ instance ID; the latest matching import run decides what happens:
807810
Clerk cannot name the instance (`GET /v1/instance` failed, often from rate
808811
limiting right after a large import) and a run of this file exists under a real
809812
instance ID, it exits 2 rather than starting over: try again, or pass
810-
`--new-run`. A continue whose run changed while it waited at the prompt, from
811-
an undo or another continue, exits 2 too, with nothing written.
813+
`--new-run`. A run undone in full holds no users, so it doesn't count. A key
814+
Clerk refuses (`401`) is not a failed lookup: the import stops, naming where the
815+
key came from and Clerk's error, and so do `migrate undo` and
816+
`migrate export clerk`. A continue whose run changed while it waited at the
817+
prompt, from an undo or another continue, exits 2 too, with nothing written.
812818

813819
A continued run also finishes what the last one left open:
814820

@@ -913,7 +919,12 @@ store: …`), passwords a source had to drop, emails Clerk refuses (malformed,
913919
Any reject stops the import, and it exits 2 with the command that adds
914920
`--allow-partial`. With `--allow-partial`, the rest import and each reject is
915921
recorded as `skipped` with its reason. `--dry-run` exits 2 when the real run
916-
would be refused, and 0 otherwise.
922+
would be refused, and 0 otherwise. A refused dry run says so after the checks,
923+
with the advice and commands the real run would give: the `--allow-partial`
924+
command when any user passes, and the earlier run's undo (or continue) when it
925+
created some of the rejected users. In `--json` the preview carries
926+
`refused: true`, with `advice` (the real refusal's sentence) and `steps` (its
927+
commands, each a `command` and `description`).
917928

918929
```
919930
Checks
@@ -936,7 +947,8 @@ Or change the instance instead
936947
Each fix names its instance with `--instance`, so it changes the instance the
937948
import targets, whatever the key's source. A key from `--secret-key` or
938949
`CLERK_SECRET_KEY` names no app, so its fix reads `--app APP_ID`: fill in the
939-
app that owns the instance. When Clerk could not name the
950+
app that owns the instance. The checks say so beneath the fixes, and `--json`
951+
adds `checks.placeholders` (`{ "APP_ID": … }`). When Clerk could not name the
940952
instance (a `key_…` fallback ID), the fix points at the Dashboard instead: in
941953
`--json` it carries `url` in place of `command`.
942954

@@ -958,15 +970,28 @@ users already signed up on the source platform.
958970
Only the first verified email and phone go on `POST /v1/users`. Every
959971
additional verified identifier, and every unverified one, is attached
960972
afterwards with its own request, ahead of any create still queued, and backs
961-
off on a `429` like the create. A refusal there is logged and the user still
962-
counts as imported — a duplicate secondary email should not undo an otherwise
963-
successful user. An attach with no answer stays `pending` for a re-run.
973+
off on a `429` like the create. A refusal there does not fail the user — a
974+
duplicate secondary email should not undo an otherwise successful user — but
975+
it is a warning: the summary lists the user under "Imported without an email or
976+
phone Clerk refused", by kind and Clerk's reason (`result.warnings` in
977+
`--json`), and the user's line records it under `refused`, which `runs <id>`
978+
reports too. An attach with no answer stays `pending` for a re-run, and so
979+
does one whose error says nothing about the identifier: a `401`, a `403` other
980+
than `unsupported_country_code` or `not_allowed_access` (the instance's
981+
allowlist or blocklist), or a `429` that outlasts the retries. Those note why
982+
on the line, and `runs <id>` lists them. An attach Clerk refuses as taken
983+
(`form_identifier_exists`) looks the user up with `GET /v1/users/{user_id}`
984+
first: when the user already holds it, an earlier attach with no answer landed,
985+
and it counts as attached. When the lookup gets no answer, the attach stays
986+
`pending`.
964987

965988
The first phone gets the same treatment when Clerk refuses it — a country the
966989
instance does not support, or a number that is not E.164 — and the user has an
967990
email: the create is retried without the phone. The user counts as imported,
968991
and the summary lists them under "Imported without their phone", by Clerk's
969-
reason (`result.warnings` in `--json`).
992+
reason (`result.warnings` in `--json`). The user's line records the reason
993+
under `phoneRefused`, which `runs <id>` reports too. The retry's `creating`
994+
line carries it as well, so a continue that adopts the user keeps it.
970995

971996
With [reserved](#verified-vs-unverified-identifiers) phones on the create, the
972997
refused one may be any of them, so the verified phone is retried alone first.
@@ -1001,7 +1026,8 @@ that predates the field leaves a development instance at
10011026

10021027
Users that do exceed the limit come back in the error breakdown as
10031028
`You have reached your limit of N users`, annotated with what a development
1004-
instance can do about it. The first refusal stops the import: the users it
1029+
instance can do about it (`result.notes` in `--json`, which carries every such
1030+
note the summary prints). The first refusal stops the import: the users it
10051031
never sent are counted under "Not sent" (`result.notSent` in `--json`, and
10061032
`counts.notSent` in `run.json`), written when the run finishes. A run stopped
10071033
with Ctrl-C never finishes, so it can't count them, and `runs <id>` offers its
@@ -1027,7 +1053,11 @@ clerk migrate runs --json
10271053
It prints the runs folder first. The listing shows each run's ID, date, kind,
10281054
status, target, file and counts. `runs <id>` adds the error breakdown and the
10291055
users that failed, were skipped, or are `creating` with no answer to confirm the
1030-
create (`unconfirmed` in `--json`), with the path to the full record, then what
1056+
create (`unconfirmed` in `--json`). It also warns about users imported without
1057+
their phone, or an email or phone Clerk refused (`warnings` and
1058+
`importedWithout` in `--json`), unless the run was undone in full, and lists
1059+
users with identifiers left to attach and why (`leftToAttach`).
1060+
Then it prints the path to the full record, and what
10311061
to run next: for an interrupted or partial import, the import that continues it,
10321062
with the flags it ran with, then the undo. The continue is left out when
10331063
importing the file again might not reach this run: a newer run of the same file
@@ -1397,12 +1427,15 @@ refused; pass `--new-run` to change modes. A user a run adopts keeps the mode
13971427
its first create recorded (`reserved` on its `creating` line). When Clerk
13981428
refused a phone and the retry went out without the reserved phones, that line
13991429
lists them as `pending`, and a continue that adopts the user attaches them,
1400-
unverified (one Clerk refuses again fails with a note).
1430+
unverified. One Clerk refuses again is a warning, recorded under `refused`. A
1431+
lone reserved phone the retry already counts as refused is under `phoneRefused`
1432+
instead, and isn't sent again.
14011433

14021434
A reserved identifier is checked for duplicates like a verified one, since it
14031435
goes on the user's create. If an instance user, or an earlier record in the
14041436
file that passes the checks, already holds it, the whole user is rejected.
1405-
Without reserving, only that address fails to attach, with a note. If that
1437+
Without reserving, only that address fails to attach: the user is imported
1438+
with a warning, and the address is recorded under `refused`. If that
14061439
rejects users you want, import without reserving.
14071440

14081441
### Firebase hash parameters
@@ -1547,6 +1580,7 @@ A run that finishes part-way exits nonzero without throwing, and carries a code
15471580
| `POST` | `/v1/users` | `migrate import` — creates each user |
15481581
| `POST` | `/v1/email_addresses` | `migrate import` — attaches additional emails |
15491582
| `POST` | `/v1/phone_numbers` | `migrate import` — attaches additional phones |
1583+
| `GET` | `/v1/users/{user_id}` | `migrate import` — an attach refused as taken checks whether the user already holds it |
15501584
| `GET` | `/v1/users?limit=&offset=&order_by=+created_at` | `migrate export clerk` — pages the whole instance, oldest first, 500 at a time |
15511585
| `GET` | `/v1/users?limit=&starting_after=&order_by=+created_at` | `migrate export clerk` — pages the whole instance, oldest first, 500 at a time, each page from the last user of the one before |
15521586
| `GET` | `/v1/users/count` | `migrate import` — headroom against the instance's user limit |

‎packages/cli-core/src/commands/migrate/export/clerk.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -280,17 +280,17 @@ export async function exportClerk(options: ExportClerkOptions): Promise<void> {
280280
});
281281

282282
await withGutter("Exporting users from Clerk", async () => {
283-
const identity = await fetchInstanceIdentity(source.secretKey);
284-
// A key picks the instance on its own and ignores --instance, so the two
285-
// must agree, as on import: otherwise this reads the wrong user pool under
286-
// the requested instance's name.
287283
const keyFrom = options.secretKey
288284
? KEY_SOURCE.secretKeyFlag
289285
: !options.app && process.env.CLERK_SECRET_KEY
290286
? KEY_SOURCE.envVar
291287
: undefined;
292-
if (keyFrom) assertInstanceFlagMatches({ instance: options.instance }, keyFrom, identity);
293288
const keySource = keyFrom ?? source.target;
289+
const identity = await fetchInstanceIdentity(source.secretKey, keySource);
290+
// A key picks the instance on its own and ignores --instance, so the two
291+
// must agree, as on import: otherwise this reads the wrong user pool under
292+
// the requested instance's name.
293+
if (keyFrom) assertInstanceFlagMatches({ instance: options.instance }, keyFrom, identity);
294294
const target = {
295295
platform: "clerk",
296296
env: identity.env,

0 commit comments

Comments
 (0)