ech: implement inner hello extension compression#6
Closed
cpu wants to merge 15 commits into
Closed
Conversation
The SVCB/HTTPS record handling in hickory-dns 0.24 was stripping the TLS encoded list prefix from the `ECHConfigList` that is serialized into DNS records. This meant our previous `ech.rs` connect test was subtly wrong: it would only ever deserialize a single `EchConfig` from what it found in DNS. This commit updates Rustls to: 1. Use the new `EchConfigListBytes` type from pki-types to represent what it gets from DNS. Soon we will have more API surface expecting this type. 2. Use a hickory-dns release with some upstream fixes that ensure we get the correct wire-encoding of the `ECHConfigList`. 3. Update the ech connect tests unit tests to assert all of the ECH configs that may be found are the correct version.
For ECH GREASE it's convenient to be able to generate throw-away HPKE key-pairs. Similarly, down the road we may want to support generating server-side ECH configurations, which will require key generation. This commit updates the `Hpke` trait to add a `generate_key_pair()` fn for this purpose. Both the aws-lc-rs and provider-example HPKE trait impls are updated to add this functionality. Along the way I also copied the smoke test and fips unit tests from the aws-lc-rs HPKE provider to the provider-example HPKE provider so that there's coverage of key generation for both. We won't use key generation in the rustls-provider-tests crate since the RFC test vectors come with private key material to use.
This commit introduces the first piece of ECH support to the public API: configuring a client connection with an optional ECH config. Since ECH requires TLS 1.3 we offer a configuration entry-point `with_ech(...)` on the `ConfigBuilder<ClientConfig, WantsVersions>` state, and fix the supported protocol versions to only TLS 1.3. Handling configurations in a way that will be forward compatible and adhering to the draft spec requires some extra care. Notably we need to be able to treat ECH configs in an ECH config list with an unsupported version as opaque blobs. This requires splitting our config representation into an enum with two variants: one for a recognized ECH config and one for an unsupported one. Similarly we need to process optional extensions in ECH config contents to ensure that: 1. There MUST NOT be duplicate extensions. 2. We MUST parse and check for unsupported mandatory extensions. An example TLS client that fetches ECH configurations using DNS-over-HTTPS and configures Rustls to use ECH is added to the examples crate. The HTTPS-over-DNS lookup in that example requires adding a new dep on hickory-dns. We use the recently added aws-lc-rs backed HPKE implementations default selection of HPKE suites for the ECH config. As of this commit the ECH configuration is not used (except as a TODO to avoid a `dead_code` warning). Subsequent commits will introduce actual ECH offering and protocol support.
In order to support ECH we need to be prepared for `emit_client_hello_for_retry` to return an `Error` where it was otherwise infallible - this can occur (for e.g.) if the HPKE provider we use for ECH encryption fails. This commit changes `emit_client_hello_for_retry` to return `NextStateOrError` instead of `NextState` in preparation for that.
Previously we separately iterated the `input.hello.sent_extensions` to track our sent extensions before constructing a `ClientHelloPayload` that contained them. The `ClientHelloPayload` was constructed in-line for the `HandshakeMessagePayload` towards the end of `emit_client_hello_for_retry`. To support ECH we will want to do some additional work with the `ClientHelloPayload`, and so this commit does some minor rearranging to facilitate this.
This commit continues the implementation of client-side ECH support. We now act on the provided ECH configuration when available to produce an appropriate outer/inner client hello. Adding this support requires introduction of a new `EchState` struct for maintaining the required handshake state specific to ECH. We use this in combination with new types for representing the ECH extensions to produce the outer and inner client hello messages. We do not yet properly handle confirmation of ECH acceptance, resulting in decrypt errors when offering ECH. Subsequent commits will update hello retry request and server hello handling to fix this.
This commit continues the implementation of client-side ECH by processing the result of offering ECH in the non-HRR case. This involves processing the received server hello, attempting to derive and match a shared secret indicating ECH acceptance, and forwarding on our understanding of ECH status (not offered, offered and rejected, or offered and accepted) to later steps of the handshake. If we arrive at the end of the handshake and our ECH offer wasn't accepted, then we emit an appropriate alert and return an error potentially containing ECH configs to retry with from the server's response. If our offer _was_ accepted, we change out the handshake transcript, sent extensions and client hello as if the inner client hello was used as the outer client hello. TLS handshaking continues as normal from that point. This level of support is sufficient for the ech-client example to successfully use ECH with the defo.ie test server. Importantly we do **not** yet handle the case where we offer ECH and the server sends a hello retry request. This support will follow in a subsequent commit.
This commit continues the implementation of client-side ECH by supporting the case where we offer ECH and the server replies with a hello retry request (HRR). In this case we detect ECH acceptance in a slightly different manner. If ECH was accepted we update the separate ECH transcript using the received HRR and proceed to offer ECH again in our retried hello. After this point ECH acceptance is handled as normal. This completes the primary functionality of client-side ECH. The remaining work involves GREASE ECH and extension compression.
In the situation where an ECH config is not available some clients may wish to send a "GREASE" ECH extension as an anti-ossification mechanism (see RFC 8701[0] for more information on the general concept of GREASE). To support this we update the configuration to accept an optional ECH mode instead of an optional ECH config. The ECH mode has a variant for enabling ECH that holds an ECH config, and a separate variant that holds only what's required to emit a GREASE ECH extension. We don't automatically fall back to GREASE if ECH configs are provided but none are compatible. If desired this should be handled by the caller. Similarly we don't default to sending GREASE, it is opt-in. [0]: https://www.rfc-editor.org/rfc/rfc8701
* Use docopt to make it feasible to provide args/flags * Add flags for choosing CA cert, DNS-over-HTTPS server, etc * Add config for performing placeholder "GREASE" ECH for hosts without ECH configs, doing so by default for hosts without ECH configs, and forcing it if desired * Add config flag for specifying a ECH config from disk. * Add flag for specifying how many requests to do, to test resumption. * Asserts on the expected ECH status after handshaking.
This matches the rustls crate's current default provider choice.
A `ClientConfig` can only be considered FIPS compatible when using ECH if the configured HPKE suite is FIPS compatible. This commit updates `ClientConfig::fips()` to add that consideration. Similarly, it may be helpful to know if a given client connection was created from a FIPS compatible `ClientConfig`. To do this we update the client config data to stash the `ClientConfig::fips()` result when creating a new client connection. A new `ClientConnection::fips()` accessor returns that stashed value.
The bogo shim requires some updates to support new command line flags. Additionally in order to be able to assert some details in errors (e.g. that an ECH required err contained expected retry configs) we have to pipe the `Options` struct deeper into the client/server processing logic. Since `*ring*` has no HPKE provider, it can't do ECH, which means we need an unfortunate number of cfg guards on imports/logic in the shim related to HPKE/ECH. C'est la vie. We can revisit if `*ring*` gains static ECDH or HPKE. Beyond these changes, it's worth discussing the ignored tests. They're either not applicable, or need upstream bogo fixes: "TLS-ECH-Server*": We ignore all the TLS-ECH-Server tests. We haven't implemented server side ECH yet "TLS-ECH-Client-ExpectECHOuterExtensions" "TLS-ECH-Client-CompressSupportedVersions": These rely on extension compression between inner/outer hellos. NYI. "TLS-ECH-Client-SelectECHConfig" "TLS-ECH-Client-NoSupportedConfigs" These are meant to test unsupported configs are handled correctly: we happen to support the HPKE ciphersuites that make them "unsupported". There's a fix for this upstream we can take later. "TLS-ECH-Client-SkipInvalidPublicName*": Our name validation allows underscores in names. We also don't fallback to GREASE when there are no valid ECH configs. "TLS-ECH-Client-NoSupportedConfigs-GREASE": We don't fallback to GREASE for no ECH configs. "TLS-ECH-Client-Reject-ResumeInnerSession-TLS13": This test expects an unexpected extension error in the resumption connection, but this only happens if the outer hello didn't include GREASE PSK. BoringSSL's impl doesn't. Ours does. As a result we produce `:ECH_REJECTED:` instead of :UNEXPECTED_EXTENSION:` and have to ignore this test. "TLS-ECH-Client-TLS12-RejectRetryConfigs" "TLS-ECH-Client-Reject-NoClientCertificate-TLS12" "TLS-ECH-Client-Reject-TLS12" "TLS-ECH-Client-Reject-ResumeInnerSession-TLS12" "TLS-ECH-GREASE-Client-TLS12-RejectRetryConfigs" "TLS-ECH-Client-Reject-EarlyDataRejected-TLS12" "TLS-ECH-Client-Reject-NoClientCertificate-TLS12-Async" We never offer/support TLS 1.2 when using ECH. There's no fallback to plaintext or GREASE for a server that won't support TLS 1.3
This avoids breaking our MSRV task due to the later versions of hickory-dns (née trust-dns) that use workspace inheritance. A workspace-level patch must be resolved even for crates that don't require the patched dep, and resolving the patch for 0.22+ requires an MSRV of 1.64+ This can be reverted if there's a finalized 0.25 with the SVCB fixes, or a pre-release, made available.
This commit extends the existing client-side ECH support to enable the "SHOULD" recommendation of compressing extensions that are identical between the outer client hello and the inner client hello. This optimization reduces the overhead of ECH, shrinking the total client hello size. Achieving this is a bit tricky (which is why it was done separately at the end!). To-be-compressed extensions must be in a contiguous block in the inner hello. When we encode that contiguous block for the encoded inner hello, we replace the block with a marker extension listing all of the extension types that were replaced. In the outer hello the compressed extensions must be present in the same relative order (but don't need to be contiguous). With this feature implemented we can also activate a couple of bogo tests that were previously excluded.
Owner
Author
I will sort this out (and open a proper PR against the main repo) once 1718 lands. |
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note: opened as a draft pending rustls#1718 being merged first
This commit extends the existing client-side ECH support to enable the "SHOULD" recommendation of compressing extensions that are identical between the outer client hello and the inner client hello. This optimization reduces the overhead of ECH, shrinking the total client hello size.
Achieving this is a bit tricky (which is why it was done separately at the end!). To-be-compressed extensions must be in a contiguous block in the inner hello. When we encode that contiguous block for the encoded inner hello, we replace the block with a marker extension listing all of the extension types that were replaced. In the outer hello the compressed extensions must be present in the same relative order (but don't need to be contiguous).
With this feature implemented we can also activate a couple of bogo tests that were previously excluded.