Skip to content

HBASE-28952: Add coprocessor hook to authorize user based on client SSL certificate chain - #6447

Merged
stoty merged 9 commits into
apache:masterfrom
anmolnar:HBASE-28952
Jan 17, 2025
Merged

stoty merged 9 commits into
apache:masterfrom
anmolnar:HBASE-28952

Conversation

@anmolnar

@anmolnar anmolnar commented Nov 6, 2024 •

Copy link
Copy Markdown
Contributor

Added new coproc hook to ServerRpcConnection right after authorization is done. Support both Master and RS client connections. Coprocs are allowed to throw AccessDeniedException if connection should not be established. Targeted primarly for X509 cert based authentication.

Example co-proc usage PR is available to see it in action: anmolnar#2

cc @bbeaudreault @apurtell

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache9

Apache9 commented Nov 7, 2024

Copy link
Copy Markdown
Contributor

Maybe we could introduce something like a RpcObserver? So we do not need to do the instance of MasterObserver and RegionServerObserver.

@anmolnar

anmolnar commented Nov 7, 2024

Copy link
Copy Markdown
Contributor Author

Maybe we could introduce something like a RpcObserver? So we do not need to do the instance of MasterObserver and RegionServerObserver.

Makes sense to me. Let me give it a try.

@anmolnar

anmolnar commented Nov 7, 2024

Copy link
Copy Markdown
Contributor Author

@Apache9 Does that need the entire new co-proc infra with: RpcCoprocessor, RpcCoprocessorEnvironment, RpcObserver, RpcCoprocessorHost, etc.?

@Apache9

Apache9 commented Nov 8, 2024

Copy link
Copy Markdown
Contributor

I guess we need to introduce all these things.

You can have a try first, if we think introducing so many new classes does not make sense, we could discuss again.

Thanks.

@anmolnar
anmolnar force-pushed the HBASE-28952 branch 2 times, most recently from 368411f to 81b70ae Compare November 8, 2024 21:53
@anmolnar

anmolnar commented Nov 8, 2024

Copy link
Copy Markdown
Contributor Author

@Apache9 Check it out.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

CoprocessorHost.RPC_COPROCESSOR_CONF_KEY)
) {
LOG.info("Update the RPC coprocessor(s) because the configuration has changed");
initializeCoprocessorHost(newConf);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a Configuration object enough for your usage? I see the RpcCoprossesorEnvironment is also empty?

@anmolnar anmolnar Nov 11, 2024 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly for this particular hook we don't need anything else. Configuration object is needed to instantiate Observers, the rest is passed in method arguments.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

}
this.user = this.rpcServer.userProvider.create(this.ugi);
this.rpcServer.getRpcCoprocessorHost().postAuthorizeConnection(
this.user != null ? this.user.getName() : null, this.clientCertificateChain);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we send the user in directly and let the implementer decide if they want the name or something else?

@anmolnar anmolnar Nov 18, 2024 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense too to me. I'll make this change too.

But:
Isn't that a problem to introduce too many dependencies required to pull in by coprocessors?
User is in hbase-common, ConnectionHeader is in hbase-protocol-shaded, the implementor has to reference all these to compile the co-proc, but since they're generic hbase projects, it might not be a problem at all. I'm not sure.

@anmolnar anmolnar Nov 18, 2024 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In MasterObserver only the username is passed as String to coprocessor.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@anmolnar

Copy link
Copy Markdown
Contributor Author

@Apache9 what do you think of the current state? see anything outstanding?

@anmolnar

Copy link
Copy Markdown
Contributor Author

ping @Apache9 @bbeaudreault

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@bbeaudreault bbeaudreault left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an expert in the coprocessor interface, but I don't have any issues here. Looks pretty straightforward.

In terms of user vs username, is there any way for someone to get the user statically if they wanted? If this were a downstream coprocessor, they could use RpcServer.getCurrentCall() but not sure if that's available at this point.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 41s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 💚 mvninstall 2m 46s master passed
+1 💚 compile 2m 39s master passed
+1 💚 checkstyle 0m 35s master passed
+1 💚 spotbugs 1m 26s master passed
+1 💚 spotless 0m 42s branch has no errors when running spotless:check.
-0 ⚠️ patch 0m 49s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 38s the patch passed
+1 💚 compile 2m 37s the patch passed
+1 💚 javac 2m 37s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 33s the patch passed
+1 💚 spotbugs 1m 31s the patch passed
+1 💚 hadoopcheck 9m 35s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚 spotless 0m 40s patch has no errors when running spotless:check.
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
32m 40s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6447/8/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #6447
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 5c69561358aa 5.4.0-200-generic #220-Ubuntu SMP Fri Sep 27 13:19:16 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd5f204
Default Java Eclipse Adoptium-17.0.11+9
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6447/8/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 38s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 42s master passed
+1 💚 compile 1m 11s master passed
+1 💚 javadoc 0m 35s master passed
+1 💚 shadedjars 6m 25s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ patch 6m 34s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 44s the patch passed
+1 💚 compile 1m 17s the patch passed
+1 💚 javac 1m 17s the patch passed
+1 💚 javadoc 0m 42s the patch passed
+1 💚 shadedjars 7m 18s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 218m 20s hbase-server in the patch passed.
248m 8s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6447/8/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6447
Optional Tests javac javadoc unit compile shadedjars
uname Linux 6c39e8ab15b5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd5f204
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6447/8/testReport/
Max. process+thread count 4853 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6447/8/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@anmolnar

Copy link
Copy Markdown
Contributor Author

Thanks @bbeaudreault . Build is now green. I'm waiting for your approval @Apache9 .

@anmolnar
anmolnar requested a review from Apache9 December 19, 2024 16:16
@anmolnar

Copy link
Copy Markdown
Contributor Author

Ping @apurtell

@anmolnar

Copy link
Copy Markdown
Contributor Author

Another - final - nudge to @apurtell and @Apache9 before submitting this.

@stoty
stoty merged commit d477bf1 into apache:master Jan 17, 2025
stoty pushed a commit that referenced this pull request Jan 17, 2025
…L certificate chain (#6447)

Signed-off-by: Bryan Beaudreault <[email protected]>
(cherry picked from commit d477bf1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants