Skip to content

[feat](auth) Support MySQL-compatible dual password (RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD) - #66115

Open
raghav-reglobe wants to merge 3 commits into
apache:masterfrom
raghav-reglobe:dual-password-upstream
Open

raghav-reglobe wants to merge 3 commits into
apache:masterfrom
raghav-reglobe:dual-password-upstream

Conversation

@raghav-reglobe

@raghav-reglobe raghav-reglobe commented Jul 27, 2026 •

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue: #66113

ALTER USER ... IDENTIFIED BY is a hard cutover — the old password stops working the instant the statement commits, so every client embedding that account's password must be updated simultaneously or fail to authenticate. This makes online credential rotation impossible without a maintenance window.

This adds MySQL 8.0-compatible dual password support so a password can be rotated while the previous one keeps working, on both ALTER USER and SET PASSWORD:

ALTER USER u IDENTIFIED BY 'new' RETAIN CURRENT PASSWORD;   -- old password stays valid
ALTER USER u IDENTIFIED BY 'new';                           -- secondary unchanged
ALTER USER u DISCARD OLD PASSWORD;                          -- evict the retained one
SET PASSWORD = PASSWORD('new') RETAIN CURRENT PASSWORD;

Doris already mirrors MySQL account management (PASSWORD_HISTORY, PASSWORD_EXPIRE, FAILED_LOGIN_ATTEMPTS, PASSWORD_LOCK_TIME); this fills the remaining gap — PASSWORD_EXPIRE can force a rotation, but there was no way to comply with it online.

Why this matters now: analytical databases are increasingly accessed by a large and growing population of machine identities — AI agents, MCP servers exposing SQL to LLMs, BI tools, autonomous pipelines — each holding a long-lived credential. Rotating a shared credential across such a fleet cannot be a synchronized hard cutover; an overlap window is the only way to do it online, and a stable principal (vs the alternating-account workaround) keeps fe.audit.log attribution clean.

Semantics (mirrors the MySQL 8.0 manual)

Rule MySQL manual
RETAIN CURRENT PASSWORD saves the current primary as the secondary; both authenticate "the current password ... is retained as a secondary password"
Exactly one secondary per user; the next RETAIN replaces it dual = primary + one secondary
A password change without RETAIN leaves an existing secondary unchanged "the secondary password remains unchanged"
An empty new password empties the secondary too, even with RETAIN "the secondary password becomes empty as well"
RETAIN on an account with an empty primary fails "the statement fails"
DISCARD OLD PASSWORD (ALTER USER only, as in MySQL) drops the secondary if one exists; cannot combine with a password change "discards the secondary password, if one exists"
A change of authentication method — binding the account to an authentication integration, unbinding it, or moving it between integrations — discards the secondary; RETAIN is refused across such a transition "If the authentication plugin is changed, the secondary password is discarded"
The dual password clauses are privileged even on one's own account "The APPLICATION_PASSWORD_ADMIN privilege is required to use the RETAIN CURRENT PASSWORD or DISCARD OLD PASSWORD clause ... that apply to your own account"

Privileges (revised per review — thanks @CalvinKirs)

The first cut left SET PASSWORD ... RETAIN CURRENT PASSWORD unprivileged on one's own account, like plain self-service SET PASSWORD. That is a hole: anyone who briefly holds an account's password could set their own, then RETAIN the real one back — leaving their password parked in the secondary slot as a persistent credential that nothing surfaces. MySQL requires APPLICATION_PASSWORD_ADMIN for the clauses even on one's own account for exactly this reason.

Doris has no dynamic-privilege framework, so the clauses now require the GRANT tier (the same tier that already gates ALTER USER on other accounts):

  • plain self-service SET PASSWORD stays unprivileged — unchanged behavior;
  • SET PASSWORD ... RETAIN CURRENT PASSWORD requires global GRANT, including on one's own account;
  • the ALTER USER forms were already GRANT-gated by AlterUserInfo.validate().

A finer-grained APPLICATION_PASSWORD_ADMIN-style privilege — so a low-privilege service account can be delegated the ability to rotate itself with an overlap window without holding GRANT — is left as a follow-up — filed as #66198 (it needs a new Privilege bit and its own compatibility story, better reviewed separately).

Design

  • Storage: Password gains a nullable secondary slot (GSON field pwd2). Absent in images/journals written before this change; ignored by binaries without it — tolerant in both directions, so no FeMetaVersion bump.
  • Authentication: primary is tried first, then the secondary, against one snapshot of the Password object, in one shared helper behind both existing check paths (checkPasswordInternal and checkPasswordInternalForUser), so every protocol lane (MySQL scramble, cleartext/LDAP fallback, Arrow Flight, HTTP, stream load) behaves identically. Failed-login policy fires only when both miss.
  • Atomic swap: a password change builds the full Password (new primary + carried secondary) and swaps it in as a single reference assignment — there is no window where the new primary is visible without the retained secondary, which is exactly the window a zero-downtime rotation must not have.
  • Domain accounts: every copy of a domain account's credentials carries both password slots (Password.copyOf) — the resolved-IP users the domain resolver materializes on its periodic pass and the immediate refresh of those entries right after a password change, an ACCOUNT_LOCK/UNLOCK or a REQUIRE change (UserManager.refreshAuthenticationBinding) — so the retained password authenticates on a domain account at once, with no resolver tick in between.
  • Authentication bindings: a binding transition (into, out of or between authentication integrations) retires the retained secondary instead of carrying it. The bound-user guard refuses native logins only while the account is bound, so a password retained before the external-authentication period would otherwise come back to life beside the new one once the account returns to native authentication. RETAIN CURRENT PASSWORD is refused into and out of a binding (the current password of a bound account is a generated placeholder). The decision is derived from the account's state plus the journal's binding, so a follower replays the same outcome.
  • Replay: the retain flag rides PrivInfo on the existing OP_SET_PASSWORD. DISCARD OLD PASSWORD journals an OP_ALTER_USER entry whose carrier op is SET_PASSWORD_POLICY with every option UNSET, plus a discardOldPasswd marker on AlterUserOperationLog. A pre-feature binary drops the unknown field and replays a policy update that changes nothing (every UNSET branch returns early and nothing is re-journaled; the one side effect is a default PasswordPolicy row for a user that had none, the same row its next login check creates), while a current binary reads the marker and drops the secondary. A new AlterUserOpType name is not journaled because an older FE deserializes an unknown enum name as null and fails replay, and an OP_SET_PASSWORD carrier would append the primary to the password history and refresh the password creation time (both found in review).
  • Observability: the Password column of SHOW ALL GRANTS / SHOW GRANTS FOR / SHOW PROC '/auth' reads Yes (dual) while an account holds a retained secondary, so an administrator can list every account whose previous password still authenticates (the auditing half of the privilege concern above). Authenticating with the retained secondary increments a new FE metric doris_fe_secondary_password_auth_total (plus a DEBUG line naming the account), so operators can confirm every consumer has converged on the new password before the next rotation evicts the old one. The metric fires only after account lock/expiration policy passes, so a rejected login never counts as a successful secondary authentication.
  • Grammar: RETAIN/CURRENT/PASSWORD tokens already existed; DISCARD and OLD are added as nonReserved keywords (still usable as identifiers).

Deliberate differences from MySQL (called out for review)

  • Privileges: no dynamic-privilege framework in Doris, so GRANT tier instead of a dedicated APPLICATION_PASSWORD_ADMIN (see above; finer-grained privilege left as a follow-up).
  • No per-user authentication plugins in Doris, so MySQL's "changing the authentication plugin discards the secondary" rule has no analog.

Out of scope (follow-ups)

Rolling upgrade / downgrade

Rolling upgrade: a binary without the change ignores pwd2 and the flags, so it keeps single-password behavior. Until every FE runs this version, a not-yet-upgraded FE authenticates against the primary only: a login with the retained (old) password is rejected there, and it applies a DISCARD entry as a no-op. Run all FEs on this version before first using the clauses.

Downgrade does not break replay or image load, but it is not lossless once the clauses have been used: an older binary drops pwd2 when it loads an image and replays retainPasswd as a plain set-password, so every retained secondary is silently gone after a downgrade and does not come back on re-upgrade. Only the primary passwords survive. Re-issue RETAIN CURRENT PASSWORD after re-upgrading if an overlap window is still needed.

Tests

DualPasswordTest: retain / evict / discard chain; secondary preserved on a plain change; empty-new empties the secondary; retain-on-empty-primary fails; replay (retain + discard via PrivInfo, legacy-journal compatibility, old-binary interpretation of a discard entry); GSON forward/backward compatibility; parser (ALTER USER + SET PASSWORD clauses parse, retain-without-password rejected, DISCARD/OLD still valid as identifiers); executed parsed commands under a ConnectContext — SET PASSWORD ... RETAIN requires GRANT (rejected without, works with, journaled PrivInfo carries the flag, FORWARD_WITH_SYNC redirect), plain self SET PASSWORD stays unprivileged, ALTER USER ... RETAIN end-to-end, DISCARD journals as OP_SET_PASSWORD and never OP_ALTER_USER; account lock rejects both slots and rejected attempts don't increment the secondary-auth metric; domain-resolver refresh carries the retained secondary across a rotation, and a domain account authenticates both slots immediately after RETAIN, ACCOUNT_UNLOCK and REQUIRE (no periodic refresh in between); binding lifecycle — bind then unbind leaves the pre-binding passwords dead, RETAIN refused into and out of a binding, and the bind/unbind journals replayed from a retained credential. The Password column of SHOW GRANTS is asserted across retain / plain change / discard in both DualPasswordTest and test_dual_password; the suite's expiration step runs on a 16-second interval with 10 s + 8 s sleeps.

Behavior change

ALTER USER gains optional RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD clauses; SET PASSWORD gains an optional RETAIN CURRENT PASSWORD clause (requires GRANT, including on one's own account). No behavior change for statements that don't use them.

Release note

Support MySQL-compatible dual password (RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD) on ALTER USER and SET PASSWORD for zero-downtime credential rotation, and show which accounts hold a retained secondary in the Password column of SHOW GRANTS (Yes (dual)).

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@raghav-reglobe
raghav-reglobe force-pushed the dual-password-upstream branch from 41b5c8d to 39c951f Compare July 27, 2026 14:43
@morrySnow

Copy link
Copy Markdown
Contributor

run buildall

@morrySnow

Copy link
Copy Markdown
Contributor

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 2nd, 2026 1:27 AM.
Workflow run: https://github.com/apache/doris/actions/runs/30322634219

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29384 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 39c951f4ee873f4152977ba1bbdfa8805d850eb9, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17655	4137	4131	4131
q2	2050	318	207	207
q3	10363	1387	815	815
q4	4680	469	339	339
q5	7567	853	560	560
q6	179	166	139	139
q7	747	814	597	597
q8	9889	1534	1571	1534
q9	6031	4396	4295	4295
q10	6796	1722	1483	1483
q11	516	349	322	322
q12	751	579	461	461
q13	18114	3315	2680	2680
q14	265	257	240	240
q15	q16	768	773	698	698
q17	985	949	1087	949
q18	7064	5664	5487	5487
q19	1229	1241	1137	1137
q20	791	668	612	612
q21	5695	2618	2410	2410
q22	425	350	288	288
Total cold run time: 102560 ms
Total hot run time: 29384 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4406	4362	4362	4362
q2	280	315	208	208
q3	4568	4893	4455	4455
q4	2074	2149	1373	1373
q5	4379	4262	4248	4248
q6	225	181	127	127
q7	1748	1930	1764	1764
q8	2485	2154	2089	2089
q9	7790	7828	7734	7734
q10	4869	4619	4177	4177
q11	591	401	381	381
q12	738	757	544	544
q13	3477	3592	2939	2939
q14	314	298	283	283
q15	q16	710	727	672	672
q17	1354	1322	1311	1311
q18	8014	7363	6871	6871
q19	1153	1087	1056	1056
q20	2215	2199	1920	1920
q21	5214	4503	4405	4405
q22	494	473	401	401
Total cold run time: 57098 ms
Total hot run time: 51320 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 176523 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 39c951f4ee873f4152977ba1bbdfa8805d850eb9, data reload: false

query5	4331	623	474	474
query6	470	241	208	208
query7	4848	569	336	336
query8	335	187	174	174
query9	8782	4095	4047	4047
query10	478	363	301	301
query11	5944	2333	2102	2102
query12	166	104	101	101
query13	1267	600	425	425
query14	6257	5205	4865	4865
query14_1	4235	4228	4245	4228
query15	228	204	182	182
query16	1039	480	460	460
query17	1131	739	582	582
query18	2537	487	357	357
query19	226	191	147	147
query20	114	111	104	104
query21	237	163	137	137
query22	13579	13651	13312	13312
query23	17280	16475	16124	16124
query23_1	16182	16202	16264	16202
query24	7585	1765	1308	1308
query24_1	1307	1300	1278	1278
query25	582	462	430	430
query26	1346	368	217	217
query27	2567	625	378	378
query28	4447	1971	1973	1971
query29	1083	604	495	495
query30	360	267	228	228
query31	1114	1087	983	983
query32	109	64	64	64
query33	528	329	258	258
query34	1148	1177	653	653
query35	771	785	670	670
query36	1040	1021	861	861
query37	155	107	102	102
query38	1890	1707	1660	1660
query39	880	899	857	857
query39_1	848	824	843	824
query40	282	169	137	137
query41	66	64	62	62
query42	92	91	90	90
query43	315	319	273	273
query44	1411	770	739	739
query45	197	186	174	174
query46	1067	1186	753	753
query47	2114	2160	2051	2051
query48	385	411	295	295
query49	574	418	309	309
query50	1065	431	329	329
query51	10803	10729	10669	10669
query52	88	89	74	74
query53	261	285	204	204
query54	291	222	210	210
query55	76	70	64	64
query56	304	319	287	287
query57	1320	1290	1228	1228
query58	293	261	273	261
query59	1601	1638	1440	1440
query60	309	273	250	250
query61	144	139	146	139
query62	549	502	436	436
query63	237	197	197	197
query64	2827	1003	858	858
query65	4765	4620	4605	4605
query66	1811	494	371	371
query67	29211	29205	28910	28910
query68	2946	1445	897	897
query69	413	312	270	270
query70	910	787	812	787
query71	363	331	315	315
query72	3043	2655	2307	2307
query73	796	768	413	413
query74	5058	4896	4673	4673
query75	2543	2488	2131	2131
query76	2323	1177	786	786
query77	342	377	275	275
query78	11730	11956	11248	11248
query79	1496	1138	717	717
query80	1285	557	460	460
query81	559	330	300	300
query82	635	154	131	131
query83	364	327	294	294
query84	276	159	132	132
query85	957	593	528	528
query86	416	242	230	230
query87	1812	1836	1767	1767
query88	3705	2765	2744	2744
query89	439	388	337	337
query90	1901	200	185	185
query91	198	190	168	168
query92	62	62	56	56
query93	1750	1515	881	881
query94	711	365	315	315
query95	790	519	463	463
query96	1047	781	343	343
query97	2656	2645	2470	2470
query98	218	214	200	200
query99	1085	1109	978	978
Total cold run time: 262962 ms
Total hot run time: 176523 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.04 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 39c951f4ee873f4152977ba1bbdfa8805d850eb9, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.05	0.05
query3	0.26	0.14	0.14
query4	1.60	0.14	0.14
query5	0.25	0.23	0.22
query6	1.22	1.07	1.10
query7	0.04	0.01	0.00
query8	0.06	0.04	0.04
query9	0.38	0.33	0.31
query10	0.55	0.54	0.55
query11	0.19	0.14	0.14
query12	0.18	0.15	0.15
query13	0.47	0.47	0.48
query14	1.00	1.02	1.01
query15	0.62	0.61	0.61
query16	0.33	0.32	0.33
query17	1.12	1.11	1.10
query18	0.22	0.21	0.22
query19	2.07	1.95	2.02
query20	0.01	0.02	0.01
query21	15.41	0.21	0.14
query22	4.96	0.05	0.05
query23	16.12	0.30	0.11
query24	2.93	0.42	0.32
query25	0.11	0.06	0.05
query26	0.75	0.20	0.14
query27	0.05	0.04	0.04
query28	3.47	0.90	0.53
query29	12.52	4.08	3.26
query30	0.28	0.16	0.15
query31	2.77	0.60	0.31
query32	3.22	0.59	0.49
query33	3.25	3.16	3.16
query34	15.66	4.20	3.53
query35	3.51	3.50	3.55
query36	0.55	0.44	0.42
query37	0.08	0.07	0.06
query38	0.05	0.04	0.03
query39	0.04	0.04	0.03
query40	0.18	0.16	0.14
query41	0.08	0.04	0.03
query42	0.04	0.04	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.75 s
Total hot run time: 25.04 s

@shuke987

Copy link
Copy Markdown
Contributor

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again.
Workflow run: https://github.com/apache/doris/actions/runs/30324864892

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 69.05% (58/84) 🎉
Increment coverage report
Complete coverage report

@shuke987

Copy link
Copy Markdown
Contributor

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again.
Workflow run: https://github.com/apache/doris/actions/runs/30329305274

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@shuke987

Copy link
Copy Markdown
Contributor

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again.
Workflow run: https://github.com/apache/doris/actions/runs/30331517582

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@shuke987

Copy link
Copy Markdown
Contributor

/review

@CalvinKirs CalvinKirs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SET PASSWORD ... RETAIN CURRENT PASSWORD currently doesn't check any privilege at all — SetPassVarOp.validate() bails out early on isSelf. I think that's a problem.

Say someone gets hold of an account's password for a minute:

SET PASSWORD = PASSWORD('mine');
SET PASSWORD = PASSWORD('<the real one>') RETAIN CURRENT PASSWORD;

Now mine is sitting in the secondary slot, the real password still works, and nobody notices anything. And since a plain password change keeps the secondary (which is correct, that's MySQL behaviour), rotating the password later won't get rid of it either. DISCARD OLD PASSWORD is the only thing that will, and there's nothing telling you that you need to run it. The only thing in the way is PASSWORD_HISTORY, which is off by default.

MySQL asks for a privilege here even when you're changing your own account — Dual Password Support:

The APPLICATION_PASSWORD_ADMIN privilege is required to use the RETAIN CURRENT PASSWORD or DISCARD OLD PASSWORD clause for ALTER USER and SET PASSWORD statements that apply to your own account. The privilege is required to manipulate your own secondary password because most users require only one password.

It also puts the secondary in mysql.user.User_attributes so you can actually go looking for leftovers.

Fix is small. ALTER USER is already covered, AlterUserInfo.validate() does a global GRANT check at the end. It's just the SET PASSWORD path:

// SetPassVarOp.validate()
if (isSelf && !retainCurrentPassword) {
    return;
}

I wouldn't bother adding a real APPLICATION_PASSWORD_ADMIN privilege bit for this. That means touching PrivBitSet and the upgrade path, and PrivPredicate.GRANT already lines up with MySQL's CREATE USER tier. Can always add the finer-grained one later if someone actually wants to delegate it.

This does kill the "rotate yourself with no admin in the loop" part of the description, yeah. But that's the same thing as the hole — MySQL just doesn't hand it out for free. The "Deliberate differences from MySQL" section probably needs a tweak too: it says MySQL gates these behind APPLICATION_PASSWORD_ADMIN / CREATE USER, then concludes self-service needs no privilege. The manual wants it for your own account as well.

To be fair, the audit log does keep the clause — geneEncryptionSQL() only masks the literal, so you get ALTER USER u IDENTIFIED BY '***' RETAIN CURRENT PASSWORD in fe.audit.log, and SetPassVarOp.toSql() prints RETAIN too. So you can find it afterwards. You just can't ask "who's holding a secondary password right now", which is the bit you'd need to clean up.

carried = oldPassword == null ? null : oldPassword.getSecondaryPassword();
}
user.setPassword(password);
user.getPassword().setSecondaryPassword(carried);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

user.setPassword(password) allocates a fresh Password, and only the next line fills in the secondary. In between there's a window where the account has the new primary and no secondary, so a client still using the old password gets rejected — during exactly the rotation this feature exists to make seamless.

matchUserPassword also calls user.getPassword() twice, and the compare loop in checkPasswordInternal runs after rlock is released, so those two reads can land on different Password objects.

Cheap fix, User already has a setPassword(Password) overload:

Password newPwd = new Password(password);
newPwd.setSecondaryPassword(carried);
user.setPassword(newPwd);

and hoisting Password pwd = user.getPassword() into a local in matchUserPassword makes that a consistent snapshot.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as suggested — the change now builds the full Password (primary + carried secondary) and swaps it via the setPassword(Password) overload, and matchUserPassword reads user.getPassword() once into a local so both slots come from the same generation. Same treatment for the resolver path: createUserWithoutLock gained a Password-taking variant so resolved-IP users are built with both slots before being published.

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again.
Workflow run: https://github.com/apache/doris/actions/runs/30335286892

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@shuke987

Copy link
Copy Markdown
Contributor

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed: REQUEST_CHANGES

I found four new issues and attached inline comments:

  • ordinary domain username/IP authentication drops the retained slot on resolver refresh;
  • the DISCARD journal payload is not readable by an older FE after downgrade;
  • secondary-password success telemetry is emitted before lock/expiry rejection;
  • the tests do not execute parsed commands or validate their redirect/journal path.

I did not duplicate the live self-service SET privilege concern or the existing password-object atomicity thread. No additional review focus was supplied, so I reviewed the whole PR.

Critical-checkpoint conclusions:

  • Goal/correctness and parallel paths: ordinary non-domain native authentication works on the current version; the domain resolver materialization path misses the secondary slot. Parser, command, audit, and native-protocol routes are otherwise wired.
  • Concurrency/lifecycle: the outer Auth lock serializes production authentication and password mutation, so I found no new race; the accepted domain issue is a cross-object resolver lifecycle defect.
  • Persistence/compatibility: current-version mutation, replay, image load, and failover converge. Masters-last forward upgrade is safe only before feature use, while downgrade after a DISCARD journal can terminate an older reader.
  • Tests/results: the new tests cover direct helpers, parsing, current-version Gson, and replay, but not parsed command execution/journal creation, domain refresh, or locked/expired telemetry rejection. Live Compile, FE UT, and P0 checks are successful; check_coverage_fe is failing. I did not run local builds or tests because the controlling review prompt prohibits them.
  • Security/privileges: the threat model was reviewed. The existing live review already covers the self-service SET privilege issue; no distinct new bypass or secret leak was found.
  • Observability/performance: metric registration/export is correct, but its success event is ordered before admission policy. I did not promote the per-login INFO-volume concern without supported-load evidence.
  • Configuration/protocol/data: this change has no new configuration, FE/BE variable, or user-data write path; it changes FE authentication metadata and its journal/image representation.

Review status: complete at head 39c951f4ee873f4152977ba1bbdfa8805d850eb9; changes requested.

// password. NB: journaled via OP_ALTER_USER — all FEs must run a version
// that knows this value before it is first executed (standard rolling
// upgrade order: masters last).
DISCARD_OLD_PASSWORD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not backward-readable through the existing OP_ALTER_USER payload. An older FE deserializes the unknown enum name as null, then its switch (opType) throws; EditLog.loadJournal treats that replay exception as fatal and exits. Thus a downgrade from an image whose journal boundary precedes a DISCARD operation is not safe, despite the PR's stated downgrade compatibility. Please use a representation an old reader can safely ignore, or explicitly gate/forbid downgrade after first use and add an old-reader image+journal test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — DISCARD OLD PASSWORD now journals via OP_SET_PASSWORD/PrivInfo (discard flag, passwd = the unchanged primary), never OP_ALTER_USER. An older FE replays it as a harmless same-password set; the enum value remains for in-memory dispatch only. Test pins that logAlterUser is never called for DISCARD and that a legacy binary's interpretation of the entry is a no-op.

if (comparePassword(user.getPassword().getPassword(), remotePasswd, randomString, remotePasswdStr, plain)) {
return true;
}
if (user.getPassword().hasSecondaryPassword()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ordinary username+remote-IP authentication, this secondary check fails for domain-host accounts. checkPasswordInternal skips the persistent isDomain() entry, while addUserPrivEntriesByResolvedIPs rebuilds the matched IP User from only domainUser.getPassword().getPassword(), creating a fresh Password with no pwd2. After p1 -> p2 RETAIN, the next resolver refresh makes that entry p2-only instead of p2 plus retained p1. Please clone both password slots when materializing resolver-created users and add a domain-account rotation test across a refresh.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — resolver-materialized IP users are built from a full Password carrying both slots; new test rotates a domain account with RETAIN and asserts the retained password survives a resolver refresh.

if (user.getPassword().hasSecondaryPassword()
&& comparePassword(user.getPassword().getSecondaryPassword(),
remotePasswd, randomString, remotePasswdStr, plain)) {
LOG.info("user {} authenticated with retained secondary password", userDescription);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The success telemetry is emitted too early. Both callers run checkAccountLockedAndPasswordExpiration only after this helper returns; for an expired or currently locked account, that check throws and the client receives no session, but this line has already logged “authenticated” and incremented secondary_password_auth_total. Please return which slot matched and emit the log/counter only after account policy succeeds, with locked/expired tests proving rejected attempts do not affect success telemetry.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — matchUserPassword returns the matched slot; the log + secondary_password_auth_total increment happen only after checkAccountLockedAndPasswordExpiration passes. Test locks an account and asserts both slots reject and the counter does not move.

public void testSetPasswordRetainParses() {
// The clause parses on SET PASSWORD; no exception, RETAIN accepted.
NereidsParser parser = new NereidsParser();
parser.parseSingle("SET PASSWORD = PASSWORD('x') RETAIN CURRENT PASSWORD");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This SET parser test discards the returned command, while the behavioral assertion calls Auth.setPassword(..., true) directly. A SET builder regression—or a downstream SET/ALTER validation, redirect-status, or journal-payload regression—can therefore leave the suite green (ALTER parsing checks its flags, but never executes the command). Please execute the parsed SET and ALTER commands under a suitable ConnectContext, assert their redirect status and emitted PrivInfo/AlterUserOperationLog, and verify the resulting login state and negative privilege cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — parsed SET PASSWORD ... RETAIN and ALTER USER commands are executed under a ConnectContext with redirect-status, journaled-PrivInfo, login-state, and negative-privilege assertions.

@raghav-reglobe

Copy link
Copy Markdown
Contributor Author

Thanks @CalvinKirs — agreed on all counts. The "park your own password in the secondary slot" scenario is real, it survives later rotations exactly as you describe, and MySQL's manual is unambiguous that the clauses are privileged even on one's own account. I'd talked myself into "self SET PASSWORD is unprivileged, so RETAIN on it follows" — but RETAIN is not a password change, it's a persistence grant for a second credential, and that deserves its own gate.

Updated the PR (head 1a5136ab05):

  • SetPassVarOp.validate() now falls through to the global GRANT check when RETAIN CURRENT PASSWORD is present, including for one's own account. A plain self SET PASSWORD (no clause) stays unprivileged — unchanged behavior. ALTER USER was already covered by AlterUserInfo.validate() as you noted.
  • Took your inline fix too: the password change now builds the full Password (new primary + carried secondary) and swaps it in as one reference assignment, and matchUserPassword compares against a single snapshot — no window where a client on the old password gets rejected mid-rotation, no torn read across the two slots.
  • The PR description's "Deliberate differences" section is rewritten — it no longer claims self-service needs no privilege, and the semantics table now includes the MySQL privilege rule.
  • Tests: unprivileged self-RETAIN is rejected (and with GRANT it works), executed through the parsed command rather than direct Auth calls.

On the finer-grained privilege: I agree it doesn't belong in this PR, but I do have the concrete delegation case you mentioned — low-privilege service accounts (BI tools and similar machine identities) that should rotate their own credential with an overlap window but must never hold GRANT, since GRANT is account-management power. Filed as #66198: an APPLICATION_PASSWORD_ADMIN-equivalent Privilege bit accepted (alongside GRANT/ADMIN) for own-account clauses only, plus the "who currently holds a secondary password" introspection you pointed out is missing — MySQL surfaces it in mysql.user.User_attributes, and the two belong together. Would you be open to that direction?

While addressing this I also fixed the journal-compat issue the automated review flagged: DISCARD OLD PASSWORD no longer journals as a new AlterUserOpType on OP_ALTER_USER (an older FE fails replay on the unknown enum). It rides OP_SET_PASSWORD via PrivInfo with a discard flag and passwd = the unchanged primary, so a pre-feature binary replays it as a same-password set — a no-op. Downgrade after first use is safe now; the description's upgrade section is updated accordingly.

@raghav-reglobe

Copy link
Copy Markdown
Contributor Author

@CalvinKirs the self-RETAIN privilege gate and the four automated-review items are all in at 1a5136ab05 (replies inline); the changes-requested state is from the earlier head. Could someone re-trigger /review and run buildall? Happy to rebase if you'd prefer a fresher base — it merges clean as-is.

if (!isReplay) {
byte[] currentPrimary = user.getPassword() == null || user.getPassword().getPassword() == null
? new byte[0] : user.getPassword().getPassword();
PrivInfo info = new PrivInfo(userIdent, null, currentPrimary, null, null, false, true);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One remaining detail: on an older FE, replaying DISCARD as SET_PASSWORD preserves the primary password value, but still appends password history and resets expiration. The password-policy state therefore changes.

The rest looks good to me. Please also add regression tests for dual-password rotation and discard. Sorry for the late review!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. You're right: the primary value survived that replay, but a pre-feature setPasswordInternal still runs passwdPolicyManager.updatePassword, which appends the primary to the history again and refreshes the creation time.

Fixed in 1a08a3c (the branch is also rebased onto current master, and the test class moved to JUnit 5 with the rest of the tree). DISCARD now journals an OP_ALTER_USER entry whose carrier op is SET_PASSWORD_POLICY with every option UNSET, plus a discardOldPasswd marker on AlterUserOperationLog:

  • a pre-feature binary drops the unknown field and replays a policy update that changes nothing (ExpirePolicy.update(UNSET), HistoryPolicy.update(null, UNSET) and both FailedLoginPolicy setters return early, and nothing is journaled on replay);
  • a current binary reads the marker and drops the secondary password. The policy is not touched on this side either, since DISCARD is not a password change.

A new AlterUserOpType name is still not journaled (an unknown enum name deserializes as null and replay fails), so this keeps the same readable-by-older-binaries property, now without the side effect. PrivInfo.discardPasswd is gone; retainPasswd stays for RETAIN.

Tests:

  • DualPasswordTest.testDiscardReplayLeavesPasswordPolicyUntouched replays the entry the way a pre-feature binary sees it (marker dropped, same alterUserInternal path) and the way a current one does, asserting the history size and passwordCreateTime are unchanged in both cases, with the old OP_SET_PASSWORD carrier as the contrast that does change them.
  • testDiscardJournalsAsNoOpPolicyEntry pins the journal shape and its GSON round trip.
  • New regression suite account_p0/test_dual_password covers rotation (both statement forms), eviction, a plain change leaving the secondary alone, discard, the empty-password rules, the privilege gate on the clause, the password-history and expiration interplay, SHOW CREATE USER, and DISCARD/OLD staying usable as identifiers. Run green against an FE+BE built from the branch.

@CalvinKirs CalvinKirs left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic looks right to me now. One thing on the description rather than the code: "Downgrade is safe including after the clauses have been used" reads stronger than what actually happens. An older binary drops pwd2 when it loads an image and replays retainPasswd as a plain set, so every retained secondary is silently gone after a downgrade and does not come back on re-upgrade. Worth stating that explicitly, plus that during a rolling upgrade the not-yet-upgraded FEs reject logins with the old password.

Inline comments below; none of them is a correctness bug.

this.secondaryPassword = secondaryPassword;
}

public boolean hasSecondaryPassword() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concern behind the GRANT gate was a credential parked in this slot that nothing surfaces. After this PR an admin still has no way to see which accounts hold a secondary. SHOW ALL GRANTS / SHOW PROC '/auth' already have a Password column driven by User.hasPassword() (Auth.getAuthInfo), so exposing it is a one-liner there, e.g. Yes (dual). I'd rather have that in this PR than wait for #66198, since it is the auditing half of the same concern.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added here: Auth.getAuthInfo now prints Yes (dual) while the account holds a retained secondary (via a new User.hasSecondaryPassword()), so SHOW ALL GRANTS, SHOW GRANTS FOR and SHOW PROC '/auth' all show it. Covered in DualPasswordTest and the regression suite across retain, a plain change and discard. The follow-up in #66198 is now only the delegated privilege.

* retained secondary password. A user without one is a silent no-op
* (MySQL behavior).
*/
public void discardOldPassword(UserIdentity userIdentity, boolean errOnNonExist) throws DdlException {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small things here.

For a domain account (u@['example.com']) this only clears the domain entry; the resolved-IP entries keep their copy until the next DomainResolver tick (10s) rebuilds them, so the old password stays valid for up to that long after DISCARD. Same lag a plain password change has today, so fine, but the "evict early" wording in the comments does not hold for domain accounts. A note here would save the next reader the trace.

Also this mutates the Password in place while setPassword swaps the whole object. It is safe because both run under the Auth write lock and every auth path holds the read lock, but it contradicts the single-snapshot comment on matchUserPassword. Swapping in a new Password(primary) would keep one rule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both taken. discardOldPassword swaps in new Password(primary) instead of clearing the slot, so a password change and a discard follow the same one-snapshot rule, and the javadoc says that a domain account is cleared in its domain entry only, with the resolved-IP entries following at the next resolver refresh (10 s), the same lag a plain password change has. The "evict early" comment in Auth is reworded to match.

if (matchedSlot != MATCH_SECONDARY) {
return;
}
LOG.info("user {} authenticated with retained secondary password", userDescription);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One INFO line per connection that authenticates with the secondary. For the fleet-of-agents case in the description this fires on every reconnect until they converge, which can be a lot of log for information the counter already carries. DEBUG here, or keep INFO and accept the volume; just flagging it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to DEBUG (guarded). The metric carries the count; the line was only ever useful to name the account, and at INFO it would indeed be one line per reconnect for every consumer still on the old password.

* and an OP_SET_PASSWORD carrier would append the primary to the password
* history and refresh the password creation time.
*/
public static AlterUserOperationLog discardOldPassword(UserIdentity userIdent) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the pre-feature replay is a no-op for the policy values (every UNSET branch returns early, HistoryPolicy.update skips a null password). The one side effect left is getOrCreatePolicy inserting a default PasswordPolicy for a user that had none. Harmless, but it means the carrier is not strictly a no-op, so worth a word in the Javadoc for whoever next touches SET_PASSWORD_POLICY replay.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to the javadoc: the one side effect a pre-feature replay keeps is getOrCreatePolicy inserting a default PasswordPolicy for a user that had none, the same row its next login check would create, so the carrier is a no-op for the policy values rather than for the policy table.

auth.setPasswordInternal(user, MysqlPassword.makeScrambledPassword("p5"), null,
true, false, true /* retain */, false);
Assertions.assertTrue(canLogin("rot", "p4"));
auth.replayAlterUser(new AlterUserOperationLog(AlterUserOpType.DISCARD_OLD_PASSWORD,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This constructs an AlterUserOperationLog with op = DISCARD_OLD_PASSWORD, which is the one shape the PR promises never to journal; it only works because replayAlterUser falls through to log.getOp(). Use AlterUserOperationLog.discardOldPassword(user) so the test exercises the entry a follower actually sees (same at the second call below).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, that op is the one shape the PR never journals. Both call sites now replay AlterUserOperationLog.discardOldPassword(user), so the test sees the same entry a follower does.


// 10. password expiration: a retaining change restarts the expiry
// clock (it is a password change); DISCARD does not touch it
sql "ALTER USER '${user}'@'%' PASSWORD_EXPIRE INTERVAL 8 SECOND"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Timing is tight here: an 8-second expiry with 5s + 4s sleeps and about six JDBC connects in between leaves roughly 2s for the e1 check after DISCARD to land under 8s. test_alter_user runs with similar margins so not blocking on it, but doubling both the interval and the sleeps would make this less likely to flake on a loaded runner.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doubled: PASSWORD_EXPIRE INTERVAL 16 SECOND with 10 s + 8 s sleeps, so the check after DISCARD has about 6 s of margin instead of 2.

@raghav-reglobe

Copy link
Copy Markdown
Contributor Author

Thanks for the careful pass. All six are in at 83ca91dac3, and the description now says what a downgrade actually does.

  • The Password column of SHOW ALL GRANTS / SHOW GRANTS FOR / SHOW PROC '/auth' reads Yes (dual) while an account holds a retained secondary. Agreed this belongs here rather than behind [Feature] Fine-grained privilege for dual password clauses (APPLICATION_PASSWORD_ADMIN equivalent) #66198: it is the auditing half of the same concern, and it is a one-liner plus a User.hasSecondaryPassword(). Asserted in DualPasswordTest and in the regression suite across retain, a plain change (the marker stays, since the secondary does) and discard.
  • DISCARD now swaps in a new Password holding the primary alone, so it follows the same one-snapshot rule as a password change. The javadoc carries the domain-account note: the domain entry is cleared at once, the resolved-IP entries follow at the next resolver refresh, the same lag a plain password change has. The "evict early" wording in Auth is reworded to match.
  • The per-connection line moved to DEBUG; the counter carries the count.
  • AlterUserOperationLog.discardOldPassword names the getOrCreatePolicy side effect.
  • The test replays AlterUserOperationLog.discardOldPassword(user) at both call sites, so it exercises the entry a follower actually sees.
  • The expiration step runs on a 16-second interval with 10 s + 8 s sleeps.

On the description: "Downgrade is safe including after the clauses have been used" is gone. The rolling-upgrade / downgrade section now states that a not-yet-upgraded FE rejects logins with the old password and applies a DISCARD as a no-op, and that a downgrade keeps replay and image load working but silently drops every retained secondary, which does not come back on re-upgrade. The Replay bullet was also stale (it still described the earlier PrivInfo discard flag) and now describes the SET_PASSWORD_POLICY carrier.

Gate, built from the branch: FE build and checkstyle clean; DualPasswordTest, PrivInfoTest, AuthTest, PasswordPolicyTest, ShowGrantsCommandTest, CloudAuthTest green (32 tests), and account_p0/test_dual_password green against an FE built from this head.

…ASSWORD / DISCARD OLD PASSWORD)

Add MySQL 8.0-compatible dual password support so a password can be
rotated online without breaking consumers that still hold the previous
one, on both ALTER USER and SET PASSWORD:

    ALTER USER u IDENTIFIED BY 'new' RETAIN CURRENT PASSWORD;
    ALTER USER u IDENTIFIED BY 'new';         -- secondary password unchanged
    ALTER USER u DISCARD OLD PASSWORD;        -- discard the retained password
    SET PASSWORD = PASSWORD('new') RETAIN CURRENT PASSWORD;

Without an overlap window, every password change is a hard cutover: all
clients (BI tools, service accounts, AI/MCP consumers) must switch at the
same instant or fail to authenticate. The common workaround - two
alternating accounts - fragments audit attribution across two usernames.
MySQL solved this with dual passwords (WL#11544).

Semantics mirror the MySQL 8.0 manual:
- RETAIN CURRENT PASSWORD saves the current primary as the secondary
  password; both authenticate. Exactly one secondary per user: the next
  RETAIN replaces it.
- Changing the password WITHOUT the clause leaves an existing secondary
  password unchanged.
- An empty new password empties the secondary as well, even with RETAIN.
- RETAIN on an account with an empty primary fails.
- DISCARD OLD PASSWORD (ALTER USER only, matching MySQL) drops the
  secondary if one exists; combining it with a password change is rejected.

Privileges (revised per review): the dual password clauses are PRIVILEGED
even on one's own account. MySQL requires APPLICATION_PASSWORD_ADMIN for
RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD on one's own account,
precisely because otherwise anyone who briefly holds a password could park
their own in the secondary slot as a persistent, invisible credential.
Doris has no dynamic-privilege framework, so the clauses require the GRANT
tier (the same tier that already gates ALTER USER on other accounts):
- plain self-service SET PASSWORD stays unprivileged (unchanged);
- SET PASSWORD ... RETAIN CURRENT PASSWORD requires GRANT even on self;
- ALTER USER paths were already GRANT-gated by AlterUserInfo.validate().
A finer-grained APPLICATION_PASSWORD_ADMIN-style privilege (so that
low-privilege service accounts can self-rotate with an overlap window
without holding GRANT) is left as a follow-up.

Design:
- Password gains a nullable secondary slot (GSON field "pwd2"): absent
  in images/journals written before this change, ignored by binaries
  without it - tolerant in both directions, no meta version bump.
- Authentication tries the primary, then the secondary, against ONE
  snapshot of the Password object, in one shared helper behind both
  existing check paths (checkPasswordInternal and
  checkPasswordInternalForUser), so every protocol lane (MySQL scramble,
  cleartext/LDAP fallback, Arrow Flight, HTTP, stream load) behaves
  identically. Failed-login policy fires only when both miss.
- Password changes swap in a fully-built Password object as a single
  reference assignment (primary + carried secondary together), so there
  is no window where the new primary is visible without the retained
  secondary - the exact window a zero-downtime rotation must not have.
- The domain resolver clones BOTH password slots when materializing
  resolved-IP users, so a resolver refresh after a rotation with RETAIN
  does not evict the retained password.
- Replay: the retain flag rides PrivInfo on the existing OP_SET_PASSWORD
  journal op. DISCARD OLD PASSWORD also rides OP_SET_PASSWORD (a discard
  flag in PrivInfo, with passwd = the account's unchanged primary), NOT a
  new operation value on OP_ALTER_USER: an older FE deserializes an
  unknown AlterUserOpType as null and fails replay, whereas it replays
  the PrivInfo form as a plain set-password to the value the account
  already holds - a harmless no-op. Downgrade after first use is
  therefore safe. For correct DISCARD semantics across a mixed-version
  cluster, run all FEs on this version before first using the new
  clauses (an old-binary follower applies the discard as a no-op until
  it loads new state).
- Observability: authenticating with the retained password logs an INFO
  line and increments fe metric secondary_password_auth_total, so
  operators can verify all consumers converged on the new password
  before rotating again (MySQL itself offers no such signal). The
  log/metric fire only after account lock/expiration policy passes, so
  a rejected login never counts as a successful secondary auth.
- Grammar: RETAIN/CURRENT/PASSWORD tokens existed; DISCARD and OLD are
  added as nonReserved keywords (still valid as identifiers).

Deliberate differences from MySQL (noted for review):
- Privileges: Doris has no dynamic privileges, so the clauses use the
  GRANT tier instead of a dedicated APPLICATION_PASSWORD_ADMIN (see
  above; finer-grained privilege left as a follow-up).
- No per-user authentication plugins in Doris, so MySQL's
  plugin-change-discards-secondary rule has no analog.

Out of scope: introspection of whether a user currently holds a retained
password (fe.audit.log keeps the RETAIN clause; a per-account indicator
is a natural follow-up alongside the finer-grained privilege).

### What problem does this PR solve?

Related PR: apache#67527, apache#67396

Problem Summary: apache#67527 added seven JUnit 4 `Assert` calls to
`ConfigTest` after apache#67396 migrated the class to JUnit 5, causing
`fe-common:testCompile` to fail with `cannot find symbol: Assert`. Use
the existing JUnit 5 `Assertions` import. Also migrate the scheduler
test introduced by the same PR to JUnit 5 assertions and lifecycle
annotations, consistent with the FE test migration.

### Release note

None

### Check List (For Author)

- Test: `./run-fe-ut.sh --run
org.apache.doris.common.ConfigTest,org.apache.doris.cloud.CacheHotspotManagerSchedulerTest`
completed with BUILD SUCCESS: ConfigTest 10/10 and
CacheHotspotManagerSchedulerTest 4/4 passed, with no failures, errors,
or skipped tests. Checkstyle passed for fe-common and fe-core with zero
violations; `git diff --check` passed.
- Behavior changed: No
- Does this need documentation: No

Rebased onto current master: the password change paths now carry both the retained-secondary flag and the
authentication-integration binding (setPasswordInternal / alterUserInternal take both), the domain resolver
materializes resolved-IP users with both password slots, and a bound account shows "No" in the password column.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Signed-off-by: Raghvendra Singh <[email protected]>
@CalvinKirs

Copy link
Copy Markdown
Member

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes on head 328f500. Two new findings are verified: an authentication-integration round trip revives a retained local credential (P1), and the immediate domain-alias refresh drops the retained password until the periodic resolver repairs it (P2). Both need lifecycle tests.

Reviewed all 16 changed files and their relevant callers, state transitions and replay paths using the repository code-review skill. Flow: SQL SET/ALTER -> builder and command validation -> Auth write lock -> complete Password replacement -> existing journal opcode -> follower replay/image persistence -> primary/secondary matching -> account policy -> success telemetry. No additional user focus was supplied.

Critical checkpoint conclusions:

  • Goal and proof: normal dual-password retain, replacement, plain-change preservation, discard and empty-password behavior are implemented and have unit/regression coverage; the two reported lifecycle cases prevent fully meeting the seamless-rotation goal.
  • Scope and abstraction: changes are focused on credentials, SQL syntax, persistence, visibility and tests; full Password construction and the shared matcher reuse the existing paths.
  • Concurrency: production native checks hold Auth's read lock and mutations/replay hold its write lock. Complete Password references are swapped under the existing lock order; no additional race or deadlock was substantiated. Existing journal work remains inside the same critical section.
  • Lifecycle: authentication-binding changes fail to reset local secondary credentials (P1); domain alias refresh has a separate primary-only copy (P2). No new thread, resource lifetime or C++ static-initialization issue applies.
  • Configuration: no configuration item was added or changed; dynamic propagation is not applicable.
  • Compatibility: optional Gson fields retain old-reader defaults. DISCARD journals the established SET_PASSWORD_POLICY carrier with null password and all options UNSET, avoiding unknown enum replay and history/expiry resets. Old binaries retain single-password behavior; rolling use requires all FEs upgraded, and downgrade loses secondary credentials as documented. Binding-transition behavior still needs the P1 correction.
  • Thrift/protobuf: no gensrc definitions, field IDs or enum values changed.
  • Parallel paths: host-selected and explicit-identity native authentication, plain and scramble checks, SET and ALTER, journal replay and periodic domain materialization were traced. The immediate domain refresh is the missing parallel path in P2.
  • Conditions and errors: GRANT/root checks, RETAIN requiring a password change and nonempty old primary, empty-new clearing, one-operation ALTER validation and IF EXISTS behavior follow the intended dispatch. RETAIN across a bound-to-native transition needs the P1 lifecycle guard.
  • Test coverage: reviewed all new FE unit and regression cases, including executed parsed commands, journal payloads, policy/history, locking, telemetry and visibility. Missing bind/unbind and immediate-domain assertions are attached to the findings. Hash-literal, malformed/root-specific, actual image/failover and expired-secondary combinations are not exhaustively tested by the new suite.
  • Test results: no golden result files changed. Assertions and expected errors were checked against the code; builds and tests were not run under this static-only contract, so runtime success and generated-result provenance are not claimed.
  • Observability: secondary success telemetry is emitted after account policy accepts the login; DEBUG logging is guarded and SHOW GRANTS/proc expose Yes (dual). The new counter follows existing metric registration and publication patterns.
  • Persistence and failover: retain and discard state have matching EditLog write/read/replay routes, and pwd2 is included in serialized UserManager state. DISCARD preserves policy values on both current and legacy carrier replay. No independent new checkpoint/replay defect was found; runtime failover was not exercised.
  • Data mutation and atomicity: credential changes retain the existing locked mutation-and-journal boundary. No user-data transaction, rowset visibility, delete bitmap or BE memory ownership behavior changes; no additional crash/leak issue was substantiated.
  • FE/BE passing: no new cross-process variable or protocol sending path applies.
  • Performance: each account adds at most one retained hash and login attempts at most one additional password comparison; no material unbounded allocation or superlinear path was introduced.
  • Other issues and prior context: all supplied threads were revalidated. Snapshot publication, periodic resolver copying, success-before-policy telemetry, parser execution tests, DISCARD compatibility/history, dual visibility and logging comments are addressed. The documented domain DISCARD lag and previously discussed timing margins are already known. The new P2 concerns a different immediate refresh helper. No existing P0/P1 finding requires carry-forward, and no additional substantiated candidate remains.

Coverage is complete for the assigned static review scopes. Remaining limits are runtime validation and the test combinations identified above; no builds, tests or source changes were performed.

Review converged after 2 round(s).

} else if (retainCurrent) {
carried = oldPassword == null ? null : oldPassword.getPassword();
} else {
carried = oldPassword == null ? null : oldPassword.getSecondaryPassword();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Clear retained credentials when authentication bindings change

With a valid bound integration and a local account initially using p1, execute ALTER USER u IDENTIFIED BY 'p2' RETAIN CURRENT PASSWORD, then ALTER USER u IDENTIFIED WITH AUTHENTICATION INTEGRATION bound AS 'external', then ALTER USER u IDENTIFIED BY 'p3'. Both binding transitions reach this no-retain branch, so p1 survives beside the generated bound password and is carried again when the binding is cleared. The final native account unexpectedly accepts p1 as well as p3; the bound-user guard suppresses it only during the external-auth period. Clear local secondary state when the authentication binding changes, reject RETAIN of a bound placeholder, and cover bind/unbind plus replay starting with a retained credential. This also follows the MySQL dual-password rule that changing authentication plugins discards the secondary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0f8b492: a binding transition (into, out of or between integrations) now discards the retained secondary instead of carrying it (UserManager.setPassword(..., discardSecondary) from Auth.setPasswordInternal, derived from the account's state plus the journal's binding so replay matches), and RETAIN CURRENT PASSWORD is refused into and out of a binding — the current password of a bound account is a generated placeholder. Tests: testBindingTransitionDiscardsRetainedPassword, testRetainIsRefusedAcrossBindingTransitions, testBindingTransitionReplayDiscardsRetainedPassword.

}
}
userManager.setPassword(userIdent, password, errOnNonExist);
userManager.setPassword(userIdent, password, errOnNonExist, retainCurrent);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve the secondary in the immediate domain refresh

For an already resolved domain account, retaining p1 while setting p2 builds both slots here, but the same method then calls refreshAuthenticationBinding at lines 1258-1259. That helper still executes user.setPassword(domain.getPassword().getPassword()) (UserManager.java:523), replacing every resolved-IP entry with a primary-only Password. Clients using p1 are rejected immediately after the successful RETAIN until the next resolver tick; ACCOUNT_UNLOCK and REQUIRE changes can erase the working secondary through the same helper. The earlier domain thread covered the periodic rebuild, which now repairs this separate immediate-copy failure. Copy both slots in refreshAuthenticationBinding too, and test login immediately after RETAIN/unlock/TLS changes without forcing a periodic refresh first; the new domain unit test currently forces that refresh before checking either password.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0f8b492: refreshAuthenticationBinding copies both slots through the new Password.copyOf (the periodic resolver copy uses it too), so the retained password authenticates immediately after RETAIN, ACCOUNT_UNLOCK and REQUIRE changes on a domain account. testDomainRetainAuthenticatesBothSlotsBeforeTheResolverTick asserts the immediate path without a periodic refresh; the existing resolver test keeps covering the periodic one.

…ion; carry both slots in the immediate domain refresh

Two lifecycle gaps found by the automated review after the rebase onto the
authentication-integration bindings:

- An authentication-binding transition (into, out of or between
  integrations) now retires the retained secondary password instead of
  carrying it, following the MySQL rule that a change of authentication
  plugin discards the secondary. The bound-user guard refuses native logins
  only while the account is bound, so a password retained before the
  external-authentication period came back to life beside the new one once
  the account returned to native authentication. RETAIN CURRENT PASSWORD is
  refused across a binding transition: the current password of a bound
  account is a generated placeholder, not a credential to keep. The decision
  is derived from the account's state and the journal's binding, so a
  follower replays the same outcome.
- The immediate refresh of a domain account's resolved-IP entries
  (UserManager.refreshAuthenticationBinding, run right after a password
  change, an ACCOUNT_LOCK/UNLOCK and a REQUIRE change) copied the primary
  alone, so a rotation with RETAIN rejected the retained password on those
  entries until the DomainResolver's next periodic pass rebuilt them. Both
  refresh paths now copy both slots through Password.copyOf.

DualPasswordTest: bind/unbind lifecycle (the pre-binding passwords stay
dead), RETAIN refused into and out of a binding, the bind/unbind journals
replayed from a retained credential, and the domain account authenticating
both slots immediately after RETAIN, ACCOUNT_UNLOCK and REQUIRE without a
periodic refresh.

Signed-off-by: Raghvendra Singh <[email protected]>
Co-Authored-By: Claude Fable 5.1 <[email protected]>
@raghav-reglobe

Copy link
Copy Markdown
Contributor Author

Thanks for kicking off the review, @CalvinKirs. Both findings are real and both come from the rebase onto the authentication-integration bindings; fixed in 0f8b492.

P2 — the immediate domain refresh. This is the window you asked me to close in the very first round (the new primary visible without the carried secondary, "during exactly the rotation this feature exists to make seamless"). The one-snapshot swap closed it on the account itself, and the periodic resolver copy carried both slots since round 3, but the rebase brought in UserManager.refreshAuthenticationBinding, which rebuilds a domain account's resolved-IP entries at once after a password change, an ACCOUNT_LOCK/UNLOCK and a REQUIRE change, and copied the primary alone. Both refresh paths now go through Password.copyOf, so a domain account authenticates both slots immediately after RETAIN. The domain unit test no longer forces the periodic refresh before asserting; a second test checks the immediate path.

P1 — a retained password surviving a binding change. A binding transition now retires the retained secondary instead of carrying it, following the MySQL rule that a change of authentication plugin discards the secondary password. The bound-user guard only refuses native logins while the account is bound, so a password retained before the external-authentication period came back to life once the account returned to native authentication. RETAIN CURRENT PASSWORD is refused across a binding transition in either direction. The decision is derived from the account's state plus the journal's binding, so a follower replays the same outcome; the new tests cover bind/unbind end to end, the RETAIN refusals, and replay.

The regression suite is unchanged: binding and domain lifecycles need an integration and a resolvable domain the suite environment does not have, so these cases live in DualPasswordTest.

If you would rather see this shaped differently, for example the discard inside refreshAuthenticationBinding rather than as a flag on UserManager.setPassword, or the RETAIN-on-bound refusal at analysis time instead of in Auth, say so and I will rework it. The PR body is updated. This needs a /review and a run buildall from your side when you have a moment.

@CalvinKirs

Copy link
Copy Markdown
Member

/review

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41405961491

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@raghav-reglobe

Copy link
Copy Markdown
Contributor Author

Local pipeline review — ✅ PASS

schema: doris-repo-review/v1
status: PASS
pr: apache/doris#66115
commit: 0f8b49234ac73237d2e58473550efa35871c85e7
base: 886cb3ea48d7cf6cd3344edce0b1422ec5700634
reviewed_at: 2026-11-02T21:11+05:30
reviewer: raghav-reglobe
model: claude-fable-5-1
effort: xhigh
findings: {blocker: 0, major: 0, minor: 1, nit: 5}
rounds: 1
converged: true

Notes for maintainers

  • fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserManager.java:510 — the one Minor: DISCARD OLD PASSWORD on a domain account (u@['d']) swaps the domain entry only and does not call refreshAuthenticationBinding, so the resolver-materialised IP entries keep the discarded secondary until the next DomainResolver pass (≤ 10 s on every FE); SET PASSWORD, LOCK/UNLOCK and REQUIRE refresh at once, so the javadoc's "same lag as a plain password change" is not accurate. One-line fix inside the existing write lock plus a unit test; details in the local review documents.
  • Not verified locally: no build, unit or regression run — the review is static over the authoritative diff, the pinned gson 2.10.1 jar and the MySQL 8.0 dual-password reference.

Reviewed locally with the doris-repo-review pipeline. Repository policy may accept this receipt for the matching commit; it is not a human Apache approval.

@CalvinKirs

Copy link
Copy Markdown
Member

run buildall

@CalvinKirs

Copy link
Copy Markdown
Member

The current privilege model is a bit coarse-grained. We can refine it in a follow-up so users can manage their own secondary passwords without needing global GRANT privileges.

@raghav-reglobe

Copy link
Copy Markdown
Contributor Author

@CalvinKirs thanks for the review and the approval. Agreed — today both clauses fall back to the existing ALTER USER / SET PASSWORD FOR gate (global GRANT_PRIV), which keeps the MySQL intent (it gates them behind APPLICATION_PASSWORD_ADMIN even on one's own account, so a briefly known password cannot be parked in the secondary slot) but is coarser than it, since Doris has no finer privilege for "manage my own secondary password". For the follow-up I see two shapes: a dedicated privilege for the clauses (the closest to MySQL), or allowing them on one's own account when the statement proves the current password (MySQL's REPLACE '' / PASSWORD REQUIRE CURRENT). I'll open an issue to settle which once this lands.

### What problem does this PR solve?

Related PR: apache#66115

Problem Summary: The previous CI merge inherited duplicate normalizeStreamRead helpers from master. Merge current master, which includes apache#69809, to use the upstream fix and validate the current integration result.

### Release note

None

### Check List (For Author)

- Test: Checkstyle validation follows this merge; no behavior changes introduced by the integration commit.
- Behavior changed: No additional behavior beyond upstream master.
- Does this need documentation: No
@CalvinKirs

Copy link
Copy Markdown
Member

run buildall

@CalvinKirs

Copy link
Copy Markdown
Member

Local pipeline review — ✅ PASS

schema: doris-repo-review/v1
status: PASS
pr: apache/doris#66115
commit: 51430482b0bde7a2fb1c74726b3b529aac162ae0
base: e125ea858678fd2aea5ebf179359650aa77fabb6
reviewed_at: 2026-11-12T17:22+08:00
reviewer: CalvinKirs
model: gpt-6.1-sol
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 1}
rounds: 1
converged: true

Notes for maintainers

  • fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Password.java:26-28 — One inherited Nit remains: the lifetime comment (also in DualPasswordTest.java:65-67) says an ordinary password change removes the secondary, while the implementation and tests preserve it. No new finding in this review.
  • All 92 FE Maven reactor modules passed local Checkstyle validation across the initial 74 modules and a heap-limited resume of the remaining 18. No unit/regression, live-cluster, old-FE or failover tests were run locally; buildall was requested separately.

Reviewed locally with the doris-repo-review pipeline (a local port of .github/workflows/code-review-runner.yml). This is not a CI status check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants