Repository navigation
[feat](auth) Support MySQL-compatible dual password (RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD) - #66115
[feat](auth) Support MySQL-compatible dual password (RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD)#66115raghav-reglobe wants to merge 3 commits into
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
41b5c8d to
39c951f
Compare
|
run buildall |
|
/review |
|
Codex automated review failed and did not complete. Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 2nd, 2026 1:27 AM. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
TPC-H: Total hot run time: 29384 ms |
TPC-DS: Total hot run time: 176523 ms |
ClickBench: Total hot run time: 25.04 s |
|
/review |
|
Codex automated review failed and did not complete. Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
FE UT Coverage ReportIncrement line coverage |
|
/review |
|
Codex automated review failed and did not complete. Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
|
Codex automated review failed and did not complete. Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
CalvinKirs
left a comment
There was a problem hiding this comment.
SET PASSWORD ... RETAIN CURRENT PASSWORD currently doesn't check any privilege at all — SetPassVarOp.validate() bails out early on isSelf. I think that's a problem.
Say someone gets hold of an account's password for a minute:
SET PASSWORD = PASSWORD('mine');
SET PASSWORD = PASSWORD('<the real one>') RETAIN CURRENT PASSWORD;Now mine is sitting in the secondary slot, the real password still works, and nobody notices anything. And since a plain password change keeps the secondary (which is correct, that's MySQL behaviour), rotating the password later won't get rid of it either. DISCARD OLD PASSWORD is the only thing that will, and there's nothing telling you that you need to run it. The only thing in the way is PASSWORD_HISTORY, which is off by default.
MySQL asks for a privilege here even when you're changing your own account — Dual Password Support:
The
APPLICATION_PASSWORD_ADMINprivilege is required to use theRETAIN CURRENT PASSWORDorDISCARD OLD PASSWORDclause forALTER USERandSET PASSWORDstatements that apply to your own account. The privilege is required to manipulate your own secondary password because most users require only one password.
It also puts the secondary in mysql.user.User_attributes so you can actually go looking for leftovers.
Fix is small. ALTER USER is already covered, AlterUserInfo.validate() does a global GRANT check at the end. It's just the SET PASSWORD path:
// SetPassVarOp.validate()
if (isSelf && !retainCurrentPassword) {
return;
}I wouldn't bother adding a real APPLICATION_PASSWORD_ADMIN privilege bit for this. That means touching PrivBitSet and the upgrade path, and PrivPredicate.GRANT already lines up with MySQL's CREATE USER tier. Can always add the finer-grained one later if someone actually wants to delegate it.
This does kill the "rotate yourself with no admin in the loop" part of the description, yeah. But that's the same thing as the hole — MySQL just doesn't hand it out for free. The "Deliberate differences from MySQL" section probably needs a tweak too: it says MySQL gates these behind APPLICATION_PASSWORD_ADMIN / CREATE USER, then concludes self-service needs no privilege. The manual wants it for your own account as well.
To be fair, the audit log does keep the clause — geneEncryptionSQL() only masks the literal, so you get ALTER USER u IDENTIFIED BY '***' RETAIN CURRENT PASSWORD in fe.audit.log, and SetPassVarOp.toSql() prints RETAIN too. So you can find it afterwards. You just can't ask "who's holding a secondary password right now", which is the bit you'd need to clean up.
| carried = oldPassword == null ? null : oldPassword.getSecondaryPassword(); | ||
| } | ||
| user.setPassword(password); | ||
| user.getPassword().setSecondaryPassword(carried); |
There was a problem hiding this comment.
user.setPassword(password) allocates a fresh Password, and only the next line fills in the secondary. In between there's a window where the account has the new primary and no secondary, so a client still using the old password gets rejected — during exactly the rotation this feature exists to make seamless.
matchUserPassword also calls user.getPassword() twice, and the compare loop in checkPasswordInternal runs after rlock is released, so those two reads can land on different Password objects.
Cheap fix, User already has a setPassword(Password) overload:
Password newPwd = new Password(password);
newPwd.setSecondaryPassword(carried);
user.setPassword(newPwd);and hoisting Password pwd = user.getPassword() into a local in matchUserPassword makes that a consistent snapshot.
There was a problem hiding this comment.
Done as suggested — the change now builds the full Password (primary + carried secondary) and swaps it via the setPassword(Password) overload, and matchUserPassword reads user.getPassword() once into a local so both slots come from the same generation. Same treatment for the resolver path: createUserWithoutLock gained a Password-taking variant so resolved-IP users are built with both slots before being published.
|
Codex automated review failed and did not complete. Error: Your access token could not be refreshed because your refresh token was revoked. Please log out and sign in again. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
There was a problem hiding this comment.
Automated review completed: REQUEST_CHANGES
I found four new issues and attached inline comments:
- ordinary domain username/IP authentication drops the retained slot on resolver refresh;
- the DISCARD journal payload is not readable by an older FE after downgrade;
- secondary-password success telemetry is emitted before lock/expiry rejection;
- the tests do not execute parsed commands or validate their redirect/journal path.
I did not duplicate the live self-service SET privilege concern or the existing password-object atomicity thread. No additional review focus was supplied, so I reviewed the whole PR.
Critical-checkpoint conclusions:
- Goal/correctness and parallel paths: ordinary non-domain native authentication works on the current version; the domain resolver materialization path misses the secondary slot. Parser, command, audit, and native-protocol routes are otherwise wired.
- Concurrency/lifecycle: the outer
Authlock serializes production authentication and password mutation, so I found no new race; the accepted domain issue is a cross-object resolver lifecycle defect. - Persistence/compatibility: current-version mutation, replay, image load, and failover converge. Masters-last forward upgrade is safe only before feature use, while downgrade after a DISCARD journal can terminate an older reader.
- Tests/results: the new tests cover direct helpers, parsing, current-version Gson, and replay, but not parsed command execution/journal creation, domain refresh, or locked/expired telemetry rejection. Live Compile, FE UT, and P0 checks are successful;
check_coverage_feis failing. I did not run local builds or tests because the controlling review prompt prohibits them. - Security/privileges: the threat model was reviewed. The existing live review already covers the self-service SET privilege issue; no distinct new bypass or secret leak was found.
- Observability/performance: metric registration/export is correct, but its success event is ordered before admission policy. I did not promote the per-login INFO-volume concern without supported-load evidence.
- Configuration/protocol/data: this change has no new configuration, FE/BE variable, or user-data write path; it changes FE authentication metadata and its journal/image representation.
Review status: complete at head 39c951f4ee873f4152977ba1bbdfa8805d850eb9; changes requested.
| // password. NB: journaled via OP_ALTER_USER — all FEs must run a version | ||
| // that knows this value before it is first executed (standard rolling | ||
| // upgrade order: masters last). | ||
| DISCARD_OLD_PASSWORD |
There was a problem hiding this comment.
This is not backward-readable through the existing OP_ALTER_USER payload. An older FE deserializes the unknown enum name as null, then its switch (opType) throws; EditLog.loadJournal treats that replay exception as fatal and exits. Thus a downgrade from an image whose journal boundary precedes a DISCARD operation is not safe, despite the PR's stated downgrade compatibility. Please use a representation an old reader can safely ignore, or explicitly gate/forbid downgrade after first use and add an old-reader image+journal test.
There was a problem hiding this comment.
Fixed — DISCARD OLD PASSWORD now journals via OP_SET_PASSWORD/PrivInfo (discard flag, passwd = the unchanged primary), never OP_ALTER_USER. An older FE replays it as a harmless same-password set; the enum value remains for in-memory dispatch only. Test pins that logAlterUser is never called for DISCARD and that a legacy binary's interpretation of the entry is a no-op.
| if (comparePassword(user.getPassword().getPassword(), remotePasswd, randomString, remotePasswdStr, plain)) { | ||
| return true; | ||
| } | ||
| if (user.getPassword().hasSecondaryPassword() |
There was a problem hiding this comment.
For ordinary username+remote-IP authentication, this secondary check fails for domain-host accounts. checkPasswordInternal skips the persistent isDomain() entry, while addUserPrivEntriesByResolvedIPs rebuilds the matched IP User from only domainUser.getPassword().getPassword(), creating a fresh Password with no pwd2. After p1 -> p2 RETAIN, the next resolver refresh makes that entry p2-only instead of p2 plus retained p1. Please clone both password slots when materializing resolver-created users and add a domain-account rotation test across a refresh.
There was a problem hiding this comment.
Fixed — resolver-materialized IP users are built from a full Password carrying both slots; new test rotates a domain account with RETAIN and asserts the retained password survives a resolver refresh.
| if (user.getPassword().hasSecondaryPassword() | ||
| && comparePassword(user.getPassword().getSecondaryPassword(), | ||
| remotePasswd, randomString, remotePasswdStr, plain)) { | ||
| LOG.info("user {} authenticated with retained secondary password", userDescription); |
There was a problem hiding this comment.
The success telemetry is emitted too early. Both callers run checkAccountLockedAndPasswordExpiration only after this helper returns; for an expired or currently locked account, that check throws and the client receives no session, but this line has already logged “authenticated” and incremented secondary_password_auth_total. Please return which slot matched and emit the log/counter only after account policy succeeds, with locked/expired tests proving rejected attempts do not affect success telemetry.
There was a problem hiding this comment.
Fixed — matchUserPassword returns the matched slot; the log + secondary_password_auth_total increment happen only after checkAccountLockedAndPasswordExpiration passes. Test locks an account and asserts both slots reject and the counter does not move.
| public void testSetPasswordRetainParses() { | ||
| // The clause parses on SET PASSWORD; no exception, RETAIN accepted. | ||
| NereidsParser parser = new NereidsParser(); | ||
| parser.parseSingle("SET PASSWORD = PASSWORD('x') RETAIN CURRENT PASSWORD"); |
There was a problem hiding this comment.
This SET parser test discards the returned command, while the behavioral assertion calls Auth.setPassword(..., true) directly. A SET builder regression—or a downstream SET/ALTER validation, redirect-status, or journal-payload regression—can therefore leave the suite green (ALTER parsing checks its flags, but never executes the command). Please execute the parsed SET and ALTER commands under a suitable ConnectContext, assert their redirect status and emitted PrivInfo/AlterUserOperationLog, and verify the resulting login state and negative privilege cases.
There was a problem hiding this comment.
Done — parsed SET PASSWORD ... RETAIN and ALTER USER commands are executed under a ConnectContext with redirect-status, journaled-PrivInfo, login-state, and negative-privilege assertions.
39c951f to
1a5136a
Compare
|
Thanks @CalvinKirs — agreed on all counts. The "park your own password in the secondary slot" scenario is real, it survives later rotations exactly as you describe, and MySQL's manual is unambiguous that the clauses are privileged even on one's own account. I'd talked myself into "self SET PASSWORD is unprivileged, so RETAIN on it follows" — but RETAIN is not a password change, it's a persistence grant for a second credential, and that deserves its own gate. Updated the PR (head
On the finer-grained privilege: I agree it doesn't belong in this PR, but I do have the concrete delegation case you mentioned — low-privilege service accounts (BI tools and similar machine identities) that should rotate their own credential with an overlap window but must never hold GRANT, since GRANT is account-management power. Filed as #66198: an While addressing this I also fixed the journal-compat issue the automated review flagged: |
|
@CalvinKirs the self- |
| if (!isReplay) { | ||
| byte[] currentPrimary = user.getPassword() == null || user.getPassword().getPassword() == null | ||
| ? new byte[0] : user.getPassword().getPassword(); | ||
| PrivInfo info = new PrivInfo(userIdent, null, currentPrimary, null, null, false, true); |
There was a problem hiding this comment.
One remaining detail: on an older FE, replaying DISCARD as SET_PASSWORD preserves the primary password value, but still appends password history and resets expiration. The password-policy state therefore changes.
The rest looks good to me. Please also add regression tests for dual-password rotation and discard. Sorry for the late review!
There was a problem hiding this comment.
Good catch, thanks. You're right: the primary value survived that replay, but a pre-feature setPasswordInternal still runs passwdPolicyManager.updatePassword, which appends the primary to the history again and refreshes the creation time.
Fixed in 1a08a3c (the branch is also rebased onto current master, and the test class moved to JUnit 5 with the rest of the tree). DISCARD now journals an OP_ALTER_USER entry whose carrier op is SET_PASSWORD_POLICY with every option UNSET, plus a discardOldPasswd marker on AlterUserOperationLog:
- a pre-feature binary drops the unknown field and replays a policy update that changes nothing (
ExpirePolicy.update(UNSET),HistoryPolicy.update(null, UNSET)and bothFailedLoginPolicysetters return early, and nothing is journaled on replay); - a current binary reads the marker and drops the secondary password. The policy is not touched on this side either, since DISCARD is not a password change.
A new AlterUserOpType name is still not journaled (an unknown enum name deserializes as null and replay fails), so this keeps the same readable-by-older-binaries property, now without the side effect. PrivInfo.discardPasswd is gone; retainPasswd stays for RETAIN.
Tests:
DualPasswordTest.testDiscardReplayLeavesPasswordPolicyUntouchedreplays the entry the way a pre-feature binary sees it (marker dropped, samealterUserInternalpath) and the way a current one does, asserting the history size andpasswordCreateTimeare unchanged in both cases, with the oldOP_SET_PASSWORDcarrier as the contrast that does change them.testDiscardJournalsAsNoOpPolicyEntrypins the journal shape and its GSON round trip.- New regression suite
account_p0/test_dual_passwordcovers rotation (both statement forms), eviction, a plain change leaving the secondary alone, discard, the empty-password rules, the privilege gate on the clause, the password-history and expiration interplay,SHOW CREATE USER, andDISCARD/OLDstaying usable as identifiers. Run green against an FE+BE built from the branch.
1a5136a to
b1f1193
Compare
There was a problem hiding this comment.
The logic looks right to me now. One thing on the description rather than the code: "Downgrade is safe including after the clauses have been used" reads stronger than what actually happens. An older binary drops pwd2 when it loads an image and replays retainPasswd as a plain set, so every retained secondary is silently gone after a downgrade and does not come back on re-upgrade. Worth stating that explicitly, plus that during a rolling upgrade the not-yet-upgraded FEs reject logins with the old password.
Inline comments below; none of them is a correctness bug.
| this.secondaryPassword = secondaryPassword; | ||
| } | ||
|
|
||
| public boolean hasSecondaryPassword() { |
There was a problem hiding this comment.
The concern behind the GRANT gate was a credential parked in this slot that nothing surfaces. After this PR an admin still has no way to see which accounts hold a secondary. SHOW ALL GRANTS / SHOW PROC '/auth' already have a Password column driven by User.hasPassword() (Auth.getAuthInfo), so exposing it is a one-liner there, e.g. Yes (dual). I'd rather have that in this PR than wait for #66198, since it is the auditing half of the same concern.
There was a problem hiding this comment.
Added here: Auth.getAuthInfo now prints Yes (dual) while the account holds a retained secondary (via a new User.hasSecondaryPassword()), so SHOW ALL GRANTS, SHOW GRANTS FOR and SHOW PROC '/auth' all show it. Covered in DualPasswordTest and the regression suite across retain, a plain change and discard. The follow-up in #66198 is now only the delegated privilege.
| * retained secondary password. A user without one is a silent no-op | ||
| * (MySQL behavior). | ||
| */ | ||
| public void discardOldPassword(UserIdentity userIdentity, boolean errOnNonExist) throws DdlException { |
There was a problem hiding this comment.
Two small things here.
For a domain account (u@['example.com']) this only clears the domain entry; the resolved-IP entries keep their copy until the next DomainResolver tick (10s) rebuilds them, so the old password stays valid for up to that long after DISCARD. Same lag a plain password change has today, so fine, but the "evict early" wording in the comments does not hold for domain accounts. A note here would save the next reader the trace.
Also this mutates the Password in place while setPassword swaps the whole object. It is safe because both run under the Auth write lock and every auth path holds the read lock, but it contradicts the single-snapshot comment on matchUserPassword. Swapping in a new Password(primary) would keep one rule.
There was a problem hiding this comment.
Both taken. discardOldPassword swaps in new Password(primary) instead of clearing the slot, so a password change and a discard follow the same one-snapshot rule, and the javadoc says that a domain account is cleared in its domain entry only, with the resolved-IP entries following at the next resolver refresh (10 s), the same lag a plain password change has. The "evict early" comment in Auth is reworded to match.
| if (matchedSlot != MATCH_SECONDARY) { | ||
| return; | ||
| } | ||
| LOG.info("user {} authenticated with retained secondary password", userDescription); |
There was a problem hiding this comment.
One INFO line per connection that authenticates with the secondary. For the fleet-of-agents case in the description this fires on every reconnect until they converge, which can be a lot of log for information the counter already carries. DEBUG here, or keep INFO and accept the volume; just flagging it.
There was a problem hiding this comment.
Moved to DEBUG (guarded). The metric carries the count; the line was only ever useful to name the account, and at INFO it would indeed be one line per reconnect for every consumer still on the old password.
| * and an OP_SET_PASSWORD carrier would append the primary to the password | ||
| * history and refresh the password creation time. | ||
| */ | ||
| public static AlterUserOperationLog discardOldPassword(UserIdentity userIdent) { |
There was a problem hiding this comment.
Confirmed the pre-feature replay is a no-op for the policy values (every UNSET branch returns early, HistoryPolicy.update skips a null password). The one side effect left is getOrCreatePolicy inserting a default PasswordPolicy for a user that had none. Harmless, but it means the carrier is not strictly a no-op, so worth a word in the Javadoc for whoever next touches SET_PASSWORD_POLICY replay.
There was a problem hiding this comment.
Added to the javadoc: the one side effect a pre-feature replay keeps is getOrCreatePolicy inserting a default PasswordPolicy for a user that had none, the same row its next login check would create, so the carrier is a no-op for the policy values rather than for the policy table.
| auth.setPasswordInternal(user, MysqlPassword.makeScrambledPassword("p5"), null, | ||
| true, false, true /* retain */, false); | ||
| Assertions.assertTrue(canLogin("rot", "p4")); | ||
| auth.replayAlterUser(new AlterUserOperationLog(AlterUserOpType.DISCARD_OLD_PASSWORD, |
There was a problem hiding this comment.
This constructs an AlterUserOperationLog with op = DISCARD_OLD_PASSWORD, which is the one shape the PR promises never to journal; it only works because replayAlterUser falls through to log.getOp(). Use AlterUserOperationLog.discardOldPassword(user) so the test exercises the entry a follower actually sees (same at the second call below).
There was a problem hiding this comment.
Right, that op is the one shape the PR never journals. Both call sites now replay AlterUserOperationLog.discardOldPassword(user), so the test sees the same entry a follower does.
|
|
||
| // 10. password expiration: a retaining change restarts the expiry | ||
| // clock (it is a password change); DISCARD does not touch it | ||
| sql "ALTER USER '${user}'@'%' PASSWORD_EXPIRE INTERVAL 8 SECOND" |
There was a problem hiding this comment.
Timing is tight here: an 8-second expiry with 5s + 4s sleeps and about six JDBC connects in between leaves roughly 2s for the e1 check after DISCARD to land under 8s. test_alter_user runs with similar margins so not blocking on it, but doubling both the interval and the sleeps would make this less likely to flake on a loaded runner.
There was a problem hiding this comment.
Doubled: PASSWORD_EXPIRE INTERVAL 16 SECOND with 10 s + 8 s sleeps, so the check after DISCARD has about 6 s of margin instead of 2.
|
Thanks for the careful pass. All six are in at
On the description: "Downgrade is safe including after the clauses have been used" is gone. The rolling-upgrade / downgrade section now states that a not-yet-upgraded FE rejects logins with the old password and applies a Gate, built from the branch: FE build and checkstyle clean; |
…ASSWORD / DISCARD OLD PASSWORD)
Add MySQL 8.0-compatible dual password support so a password can be
rotated online without breaking consumers that still hold the previous
one, on both ALTER USER and SET PASSWORD:
ALTER USER u IDENTIFIED BY 'new' RETAIN CURRENT PASSWORD;
ALTER USER u IDENTIFIED BY 'new'; -- secondary password unchanged
ALTER USER u DISCARD OLD PASSWORD; -- discard the retained password
SET PASSWORD = PASSWORD('new') RETAIN CURRENT PASSWORD;
Without an overlap window, every password change is a hard cutover: all
clients (BI tools, service accounts, AI/MCP consumers) must switch at the
same instant or fail to authenticate. The common workaround - two
alternating accounts - fragments audit attribution across two usernames.
MySQL solved this with dual passwords (WL#11544).
Semantics mirror the MySQL 8.0 manual:
- RETAIN CURRENT PASSWORD saves the current primary as the secondary
password; both authenticate. Exactly one secondary per user: the next
RETAIN replaces it.
- Changing the password WITHOUT the clause leaves an existing secondary
password unchanged.
- An empty new password empties the secondary as well, even with RETAIN.
- RETAIN on an account with an empty primary fails.
- DISCARD OLD PASSWORD (ALTER USER only, matching MySQL) drops the
secondary if one exists; combining it with a password change is rejected.
Privileges (revised per review): the dual password clauses are PRIVILEGED
even on one's own account. MySQL requires APPLICATION_PASSWORD_ADMIN for
RETAIN CURRENT PASSWORD / DISCARD OLD PASSWORD on one's own account,
precisely because otherwise anyone who briefly holds a password could park
their own in the secondary slot as a persistent, invisible credential.
Doris has no dynamic-privilege framework, so the clauses require the GRANT
tier (the same tier that already gates ALTER USER on other accounts):
- plain self-service SET PASSWORD stays unprivileged (unchanged);
- SET PASSWORD ... RETAIN CURRENT PASSWORD requires GRANT even on self;
- ALTER USER paths were already GRANT-gated by AlterUserInfo.validate().
A finer-grained APPLICATION_PASSWORD_ADMIN-style privilege (so that
low-privilege service accounts can self-rotate with an overlap window
without holding GRANT) is left as a follow-up.
Design:
- Password gains a nullable secondary slot (GSON field "pwd2"): absent
in images/journals written before this change, ignored by binaries
without it - tolerant in both directions, no meta version bump.
- Authentication tries the primary, then the secondary, against ONE
snapshot of the Password object, in one shared helper behind both
existing check paths (checkPasswordInternal and
checkPasswordInternalForUser), so every protocol lane (MySQL scramble,
cleartext/LDAP fallback, Arrow Flight, HTTP, stream load) behaves
identically. Failed-login policy fires only when both miss.
- Password changes swap in a fully-built Password object as a single
reference assignment (primary + carried secondary together), so there
is no window where the new primary is visible without the retained
secondary - the exact window a zero-downtime rotation must not have.
- The domain resolver clones BOTH password slots when materializing
resolved-IP users, so a resolver refresh after a rotation with RETAIN
does not evict the retained password.
- Replay: the retain flag rides PrivInfo on the existing OP_SET_PASSWORD
journal op. DISCARD OLD PASSWORD also rides OP_SET_PASSWORD (a discard
flag in PrivInfo, with passwd = the account's unchanged primary), NOT a
new operation value on OP_ALTER_USER: an older FE deserializes an
unknown AlterUserOpType as null and fails replay, whereas it replays
the PrivInfo form as a plain set-password to the value the account
already holds - a harmless no-op. Downgrade after first use is
therefore safe. For correct DISCARD semantics across a mixed-version
cluster, run all FEs on this version before first using the new
clauses (an old-binary follower applies the discard as a no-op until
it loads new state).
- Observability: authenticating with the retained password logs an INFO
line and increments fe metric secondary_password_auth_total, so
operators can verify all consumers converged on the new password
before rotating again (MySQL itself offers no such signal). The
log/metric fire only after account lock/expiration policy passes, so
a rejected login never counts as a successful secondary auth.
- Grammar: RETAIN/CURRENT/PASSWORD tokens existed; DISCARD and OLD are
added as nonReserved keywords (still valid as identifiers).
Deliberate differences from MySQL (noted for review):
- Privileges: Doris has no dynamic privileges, so the clauses use the
GRANT tier instead of a dedicated APPLICATION_PASSWORD_ADMIN (see
above; finer-grained privilege left as a follow-up).
- No per-user authentication plugins in Doris, so MySQL's
plugin-change-discards-secondary rule has no analog.
Out of scope: introspection of whether a user currently holds a retained
password (fe.audit.log keeps the RETAIN clause; a per-account indicator
is a natural follow-up alongside the finer-grained privilege).
### What problem does this PR solve?
Related PR: apache#67527, apache#67396
Problem Summary: apache#67527 added seven JUnit 4 `Assert` calls to
`ConfigTest` after apache#67396 migrated the class to JUnit 5, causing
`fe-common:testCompile` to fail with `cannot find symbol: Assert`. Use
the existing JUnit 5 `Assertions` import. Also migrate the scheduler
test introduced by the same PR to JUnit 5 assertions and lifecycle
annotations, consistent with the FE test migration.
### Release note
None
### Check List (For Author)
- Test: `./run-fe-ut.sh --run
org.apache.doris.common.ConfigTest,org.apache.doris.cloud.CacheHotspotManagerSchedulerTest`
completed with BUILD SUCCESS: ConfigTest 10/10 and
CacheHotspotManagerSchedulerTest 4/4 passed, with no failures, errors,
or skipped tests. Checkstyle passed for fe-common and fe-core with zero
violations; `git diff --check` passed.
- Behavior changed: No
- Does this need documentation: No
Rebased onto current master: the password change paths now carry both the retained-secondary flag and the
authentication-integration binding (setPasswordInternal / alterUserInternal take both), the domain resolver
materializes resolved-IP users with both password slots, and a bound account shows "No" in the password column.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Signed-off-by: Raghvendra Singh <[email protected]>
83ca91d to
328f500
Compare
|
/review |
There was a problem hiding this comment.
Request changes on head 328f500. Two new findings are verified: an authentication-integration round trip revives a retained local credential (P1), and the immediate domain-alias refresh drops the retained password until the periodic resolver repairs it (P2). Both need lifecycle tests.
Reviewed all 16 changed files and their relevant callers, state transitions and replay paths using the repository code-review skill. Flow: SQL SET/ALTER -> builder and command validation -> Auth write lock -> complete Password replacement -> existing journal opcode -> follower replay/image persistence -> primary/secondary matching -> account policy -> success telemetry. No additional user focus was supplied.
Critical checkpoint conclusions:
- Goal and proof: normal dual-password retain, replacement, plain-change preservation, discard and empty-password behavior are implemented and have unit/regression coverage; the two reported lifecycle cases prevent fully meeting the seamless-rotation goal.
- Scope and abstraction: changes are focused on credentials, SQL syntax, persistence, visibility and tests; full Password construction and the shared matcher reuse the existing paths.
- Concurrency: production native checks hold Auth's read lock and mutations/replay hold its write lock. Complete Password references are swapped under the existing lock order; no additional race or deadlock was substantiated. Existing journal work remains inside the same critical section.
- Lifecycle: authentication-binding changes fail to reset local secondary credentials (P1); domain alias refresh has a separate primary-only copy (P2). No new thread, resource lifetime or C++ static-initialization issue applies.
- Configuration: no configuration item was added or changed; dynamic propagation is not applicable.
- Compatibility: optional Gson fields retain old-reader defaults. DISCARD journals the established SET_PASSWORD_POLICY carrier with null password and all options UNSET, avoiding unknown enum replay and history/expiry resets. Old binaries retain single-password behavior; rolling use requires all FEs upgraded, and downgrade loses secondary credentials as documented. Binding-transition behavior still needs the P1 correction.
- Thrift/protobuf: no gensrc definitions, field IDs or enum values changed.
- Parallel paths: host-selected and explicit-identity native authentication, plain and scramble checks, SET and ALTER, journal replay and periodic domain materialization were traced. The immediate domain refresh is the missing parallel path in P2.
- Conditions and errors: GRANT/root checks, RETAIN requiring a password change and nonempty old primary, empty-new clearing, one-operation ALTER validation and IF EXISTS behavior follow the intended dispatch. RETAIN across a bound-to-native transition needs the P1 lifecycle guard.
- Test coverage: reviewed all new FE unit and regression cases, including executed parsed commands, journal payloads, policy/history, locking, telemetry and visibility. Missing bind/unbind and immediate-domain assertions are attached to the findings. Hash-literal, malformed/root-specific, actual image/failover and expired-secondary combinations are not exhaustively tested by the new suite.
- Test results: no golden result files changed. Assertions and expected errors were checked against the code; builds and tests were not run under this static-only contract, so runtime success and generated-result provenance are not claimed.
- Observability: secondary success telemetry is emitted after account policy accepts the login; DEBUG logging is guarded and SHOW GRANTS/proc expose Yes (dual). The new counter follows existing metric registration and publication patterns.
- Persistence and failover: retain and discard state have matching EditLog write/read/replay routes, and pwd2 is included in serialized UserManager state. DISCARD preserves policy values on both current and legacy carrier replay. No independent new checkpoint/replay defect was found; runtime failover was not exercised.
- Data mutation and atomicity: credential changes retain the existing locked mutation-and-journal boundary. No user-data transaction, rowset visibility, delete bitmap or BE memory ownership behavior changes; no additional crash/leak issue was substantiated.
- FE/BE passing: no new cross-process variable or protocol sending path applies.
- Performance: each account adds at most one retained hash and login attempts at most one additional password comparison; no material unbounded allocation or superlinear path was introduced.
- Other issues and prior context: all supplied threads were revalidated. Snapshot publication, periodic resolver copying, success-before-policy telemetry, parser execution tests, DISCARD compatibility/history, dual visibility and logging comments are addressed. The documented domain DISCARD lag and previously discussed timing margins are already known. The new P2 concerns a different immediate refresh helper. No existing P0/P1 finding requires carry-forward, and no additional substantiated candidate remains.
Coverage is complete for the assigned static review scopes. Remaining limits are runtime validation and the test combinations identified above; no builds, tests or source changes were performed.
Review converged after 2 round(s).
| } else if (retainCurrent) { | ||
| carried = oldPassword == null ? null : oldPassword.getPassword(); | ||
| } else { | ||
| carried = oldPassword == null ? null : oldPassword.getSecondaryPassword(); |
There was a problem hiding this comment.
[P1] Clear retained credentials when authentication bindings change
With a valid bound integration and a local account initially using p1, execute ALTER USER u IDENTIFIED BY 'p2' RETAIN CURRENT PASSWORD, then ALTER USER u IDENTIFIED WITH AUTHENTICATION INTEGRATION bound AS 'external', then ALTER USER u IDENTIFIED BY 'p3'. Both binding transitions reach this no-retain branch, so p1 survives beside the generated bound password and is carried again when the binding is cleared. The final native account unexpectedly accepts p1 as well as p3; the bound-user guard suppresses it only during the external-auth period. Clear local secondary state when the authentication binding changes, reject RETAIN of a bound placeholder, and cover bind/unbind plus replay starting with a retained credential. This also follows the MySQL dual-password rule that changing authentication plugins discards the secondary.
There was a problem hiding this comment.
Fixed in 0f8b492: a binding transition (into, out of or between integrations) now discards the retained secondary instead of carrying it (UserManager.setPassword(..., discardSecondary) from Auth.setPasswordInternal, derived from the account's state plus the journal's binding so replay matches), and RETAIN CURRENT PASSWORD is refused into and out of a binding — the current password of a bound account is a generated placeholder. Tests: testBindingTransitionDiscardsRetainedPassword, testRetainIsRefusedAcrossBindingTransitions, testBindingTransitionReplayDiscardsRetainedPassword.
| } | ||
| } | ||
| userManager.setPassword(userIdent, password, errOnNonExist); | ||
| userManager.setPassword(userIdent, password, errOnNonExist, retainCurrent); |
There was a problem hiding this comment.
[P2] Preserve the secondary in the immediate domain refresh
For an already resolved domain account, retaining p1 while setting p2 builds both slots here, but the same method then calls refreshAuthenticationBinding at lines 1258-1259. That helper still executes user.setPassword(domain.getPassword().getPassword()) (UserManager.java:523), replacing every resolved-IP entry with a primary-only Password. Clients using p1 are rejected immediately after the successful RETAIN until the next resolver tick; ACCOUNT_UNLOCK and REQUIRE changes can erase the working secondary through the same helper. The earlier domain thread covered the periodic rebuild, which now repairs this separate immediate-copy failure. Copy both slots in refreshAuthenticationBinding too, and test login immediately after RETAIN/unlock/TLS changes without forcing a periodic refresh first; the new domain unit test currently forces that refresh before checking either password.
There was a problem hiding this comment.
Fixed in 0f8b492: refreshAuthenticationBinding copies both slots through the new Password.copyOf (the periodic resolver copy uses it too), so the retained password authenticates immediately after RETAIN, ACCOUNT_UNLOCK and REQUIRE changes on a domain account. testDomainRetainAuthenticatesBothSlotsBeforeTheResolverTick asserts the immediate path without a periodic refresh; the existing resolver test keeps covering the periodic one.
…ion; carry both slots in the immediate domain refresh Two lifecycle gaps found by the automated review after the rebase onto the authentication-integration bindings: - An authentication-binding transition (into, out of or between integrations) now retires the retained secondary password instead of carrying it, following the MySQL rule that a change of authentication plugin discards the secondary. The bound-user guard refuses native logins only while the account is bound, so a password retained before the external-authentication period came back to life beside the new one once the account returned to native authentication. RETAIN CURRENT PASSWORD is refused across a binding transition: the current password of a bound account is a generated placeholder, not a credential to keep. The decision is derived from the account's state and the journal's binding, so a follower replays the same outcome. - The immediate refresh of a domain account's resolved-IP entries (UserManager.refreshAuthenticationBinding, run right after a password change, an ACCOUNT_LOCK/UNLOCK and a REQUIRE change) copied the primary alone, so a rotation with RETAIN rejected the retained password on those entries until the DomainResolver's next periodic pass rebuilt them. Both refresh paths now copy both slots through Password.copyOf. DualPasswordTest: bind/unbind lifecycle (the pre-binding passwords stay dead), RETAIN refused into and out of a binding, the bind/unbind journals replayed from a retained credential, and the domain account authenticating both slots immediately after RETAIN, ACCOUNT_UNLOCK and REQUIRE without a periodic refresh. Signed-off-by: Raghvendra Singh <[email protected]> Co-Authored-By: Claude Fable 5.1 <[email protected]>
|
Thanks for kicking off the review, @CalvinKirs. Both findings are real and both come from the rebase onto the authentication-integration bindings; fixed in 0f8b492. P2 — the immediate domain refresh. This is the window you asked me to close in the very first round (the new primary visible without the carried secondary, "during exactly the rotation this feature exists to make seamless"). The one-snapshot swap closed it on the account itself, and the periodic resolver copy carried both slots since round 3, but the rebase brought in P1 — a retained password surviving a binding change. A binding transition now retires the retained secondary instead of carrying it, following the MySQL rule that a change of authentication plugin discards the secondary password. The bound-user guard only refuses native logins while the account is bound, so a password retained before the external-authentication period came back to life once the account returned to native authentication. RETAIN CURRENT PASSWORD is refused across a binding transition in either direction. The decision is derived from the account's state plus the journal's binding, so a follower replays the same outcome; the new tests cover bind/unbind end to end, the RETAIN refusals, and replay. The regression suite is unchanged: binding and domain lifecycles need an integration and a resolvable domain the suite environment does not have, so these cases live in DualPasswordTest. If you would rather see this shaped differently, for example the discard inside |
|
/review |
|
Codex automated review failed and did not complete. Error: Selected model is at capacity. Please try a different model. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#66115
commit: 0f8b49234ac73237d2e58473550efa35871c85e7
base: 886cb3ea48d7cf6cd3344edce0b1422ec5700634
reviewed_at: 2026-11-02T21:11+05:30
reviewer: raghav-reglobe
model: claude-fable-5-1
effort: xhigh
findings: {blocker: 0, major: 0, minor: 1, nit: 5}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
|
run buildall |
|
The current privilege model is a bit coarse-grained. We can refine it in a follow-up so users can manage their own secondary passwords without needing global GRANT privileges. |
|
@CalvinKirs thanks for the review and the approval. Agreed — today both clauses fall back to the existing ALTER USER / SET PASSWORD FOR gate (global GRANT_PRIV), which keeps the MySQL intent (it gates them behind APPLICATION_PASSWORD_ADMIN even on one's own account, so a briefly known password cannot be parked in the secondary slot) but is coarser than it, since Doris has no finer privilege for "manage my own secondary password". For the follow-up I see two shapes: a dedicated privilege for the clauses (the closest to MySQL), or allowing them on one's own account when the statement proves the current password (MySQL's REPLACE '' / PASSWORD REQUIRE CURRENT). I'll open an issue to settle which once this lands. |
### What problem does this PR solve? Related PR: apache#66115 Problem Summary: The previous CI merge inherited duplicate normalizeStreamRead helpers from master. Merge current master, which includes apache#69809, to use the upstream fix and validate the current integration result. ### Release note None ### Check List (For Author) - Test: Checkstyle validation follows this merge; no behavior changes introduced by the integration commit. - Behavior changed: No additional behavior beyond upstream master. - Does this need documentation: No
|
run buildall |
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#66115
commit: 51430482b0bde7a2fb1c74726b3b529aac162ae0
base: e125ea858678fd2aea5ebf179359650aa77fabb6
reviewed_at: 2026-11-12T17:22+08:00
reviewer: CalvinKirs
model: gpt-6.1-sol
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 1}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
What problem does this PR solve?
Issue: #66113
ALTER USER ... IDENTIFIED BYis a hard cutover — the old password stops working the instant the statement commits, so every client embedding that account's password must be updated simultaneously or fail to authenticate. This makes online credential rotation impossible without a maintenance window.This adds MySQL 8.0-compatible dual password support so a password can be rotated while the previous one keeps working, on both
ALTER USERandSET PASSWORD:Doris already mirrors MySQL account management (
PASSWORD_HISTORY,PASSWORD_EXPIRE,FAILED_LOGIN_ATTEMPTS,PASSWORD_LOCK_TIME); this fills the remaining gap —PASSWORD_EXPIREcan force a rotation, but there was no way to comply with it online.Why this matters now: analytical databases are increasingly accessed by a large and growing population of machine identities — AI agents, MCP servers exposing SQL to LLMs, BI tools, autonomous pipelines — each holding a long-lived credential. Rotating a shared credential across such a fleet cannot be a synchronized hard cutover; an overlap window is the only way to do it online, and a stable principal (vs the alternating-account workaround) keeps
fe.audit.logattribution clean.Semantics (mirrors the MySQL 8.0 manual)
RETAIN CURRENT PASSWORDsaves the current primary as the secondary; both authenticateRETAINreplaces itRETAINleaves an existing secondary unchangedRETAINRETAINon an account with an empty primary failsDISCARD OLD PASSWORD(ALTER USER only, as in MySQL) drops the secondary if one exists; cannot combine with a password changeRETAINis refused across such a transitionAPPLICATION_PASSWORD_ADMINprivilege is required to use theRETAIN CURRENT PASSWORDorDISCARD OLD PASSWORDclause ... that apply to your own account"Privileges (revised per review — thanks @CalvinKirs)
The first cut left
SET PASSWORD ... RETAIN CURRENT PASSWORDunprivileged on one's own account, like plain self-serviceSET PASSWORD. That is a hole: anyone who briefly holds an account's password could set their own, thenRETAINthe real one back — leaving their password parked in the secondary slot as a persistent credential that nothing surfaces. MySQL requiresAPPLICATION_PASSWORD_ADMINfor the clauses even on one's own account for exactly this reason.Doris has no dynamic-privilege framework, so the clauses now require the GRANT tier (the same tier that already gates
ALTER USERon other accounts):SET PASSWORDstays unprivileged — unchanged behavior;SET PASSWORD ... RETAIN CURRENT PASSWORDrequires global GRANT, including on one's own account;ALTER USERforms were already GRANT-gated byAlterUserInfo.validate().A finer-grained
APPLICATION_PASSWORD_ADMIN-style privilege — so a low-privilege service account can be delegated the ability to rotate itself with an overlap window without holding GRANT — is left as a follow-up — filed as #66198 (it needs a newPrivilegebit and its own compatibility story, better reviewed separately).Design
Passwordgains a nullable secondary slot (GSON fieldpwd2). Absent in images/journals written before this change; ignored by binaries without it — tolerant in both directions, so noFeMetaVersionbump.Passwordobject, in one shared helper behind both existing check paths (checkPasswordInternalandcheckPasswordInternalForUser), so every protocol lane (MySQL scramble, cleartext/LDAP fallback, Arrow Flight, HTTP, stream load) behaves identically. Failed-login policy fires only when both miss.Password(new primary + carried secondary) and swaps it in as a single reference assignment — there is no window where the new primary is visible without the retained secondary, which is exactly the window a zero-downtime rotation must not have.Password.copyOf) — the resolved-IP users the domain resolver materializes on its periodic pass and the immediate refresh of those entries right after a password change, anACCOUNT_LOCK/UNLOCKor aREQUIREchange (UserManager.refreshAuthenticationBinding) — so the retained password authenticates on a domain account at once, with no resolver tick in between.RETAIN CURRENT PASSWORDis refused into and out of a binding (the current password of a bound account is a generated placeholder). The decision is derived from the account's state plus the journal's binding, so a follower replays the same outcome.PrivInfoon the existingOP_SET_PASSWORD.DISCARD OLD PASSWORDjournals anOP_ALTER_USERentry whose carrier op isSET_PASSWORD_POLICYwith every option UNSET, plus adiscardOldPasswdmarker onAlterUserOperationLog. A pre-feature binary drops the unknown field and replays a policy update that changes nothing (every UNSET branch returns early and nothing is re-journaled; the one side effect is a defaultPasswordPolicyrow for a user that had none, the same row its next login check creates), while a current binary reads the marker and drops the secondary. A newAlterUserOpTypename is not journaled because an older FE deserializes an unknown enum name asnulland fails replay, and anOP_SET_PASSWORDcarrier would append the primary to the password history and refresh the password creation time (both found in review).Passwordcolumn ofSHOW ALL GRANTS/SHOW GRANTS FOR/SHOW PROC '/auth'readsYes (dual)while an account holds a retained secondary, so an administrator can list every account whose previous password still authenticates (the auditing half of the privilege concern above). Authenticating with the retained secondary increments a new FE metricdoris_fe_secondary_password_auth_total(plus a DEBUG line naming the account), so operators can confirm every consumer has converged on the new password before the next rotation evicts the old one. The metric fires only after account lock/expiration policy passes, so a rejected login never counts as a successful secondary authentication.RETAIN/CURRENT/PASSWORDtokens already existed;DISCARDandOLDare added as nonReserved keywords (still usable as identifiers).Deliberate differences from MySQL (called out for review)
APPLICATION_PASSWORD_ADMIN(see above; finer-grained privilege left as a follow-up).Out of scope (follow-ups)
APPLICATION_PASSWORD_ADMIN-equivalent privilege (delegated self-rotation without GRANT) — [Feature] Fine-grained privilege for dual password clauses (APPLICATION_PASSWORD_ADMIN equivalent) #66198.Rolling upgrade / downgrade
Rolling upgrade: a binary without the change ignores
pwd2and the flags, so it keeps single-password behavior. Until every FE runs this version, a not-yet-upgraded FE authenticates against the primary only: a login with the retained (old) password is rejected there, and it applies aDISCARDentry as a no-op. Run all FEs on this version before first using the clauses.Downgrade does not break replay or image load, but it is not lossless once the clauses have been used: an older binary drops
pwd2when it loads an image and replaysretainPasswdas a plain set-password, so every retained secondary is silently gone after a downgrade and does not come back on re-upgrade. Only the primary passwords survive. Re-issueRETAIN CURRENT PASSWORDafter re-upgrading if an overlap window is still needed.Tests
DualPasswordTest: retain / evict / discard chain; secondary preserved on a plain change; empty-new empties the secondary; retain-on-empty-primary fails; replay (retain + discard viaPrivInfo, legacy-journal compatibility, old-binary interpretation of a discard entry); GSON forward/backward compatibility; parser (ALTER USER + SET PASSWORD clauses parse, retain-without-password rejected,DISCARD/OLDstill valid as identifiers); executed parsed commands under aConnectContext—SET PASSWORD ... RETAINrequires GRANT (rejected without, works with, journaledPrivInfocarries the flag,FORWARD_WITH_SYNCredirect), plain selfSET PASSWORDstays unprivileged,ALTER USER ... RETAINend-to-end,DISCARDjournals asOP_SET_PASSWORDand neverOP_ALTER_USER; account lock rejects both slots and rejected attempts don't increment the secondary-auth metric; domain-resolver refresh carries the retained secondary across a rotation, and a domain account authenticates both slots immediately afterRETAIN,ACCOUNT_UNLOCKandREQUIRE(no periodic refresh in between); binding lifecycle — bind then unbind leaves the pre-binding passwords dead,RETAINrefused into and out of a binding, and the bind/unbind journals replayed from a retained credential. ThePasswordcolumn ofSHOW GRANTSis asserted across retain / plain change / discard in bothDualPasswordTestandtest_dual_password; the suite's expiration step runs on a 16-second interval with 10 s + 8 s sleeps.Behavior change
ALTER USERgains optionalRETAIN CURRENT PASSWORD/DISCARD OLD PASSWORDclauses;SET PASSWORDgains an optionalRETAIN CURRENT PASSWORDclause (requires GRANT, including on one's own account). No behavior change for statements that don't use them.Release note
Support MySQL-compatible dual password (
RETAIN CURRENT PASSWORD/DISCARD OLD PASSWORD) onALTER USERandSET PASSWORDfor zero-downtime credential rotation, and show which accounts hold a retained secondary in thePasswordcolumn ofSHOW GRANTS(Yes (dual)).