Skip to content

ech: implement inner hello extension compression#6

Closed
cpu wants to merge 15 commits into
cpu-client-echfrom
cpu-client-ech-compress-exts
Closed

ech: implement inner hello extension compression#6
cpu wants to merge 15 commits into
cpu-client-echfrom
cpu-client-ech-compress-exts

Conversation

@cpu

@cpu cpu commented Jun 10, 2024

Copy link
Copy Markdown
Owner

Note: opened as a draft pending rustls#1718 being merged first

This commit extends the existing client-side ECH support to enable the "SHOULD" recommendation of compressing extensions that are identical between the outer client hello and the inner client hello. This optimization reduces the overhead of ECH, shrinking the total client hello size.

Achieving this is a bit tricky (which is why it was done separately at the end!). To-be-compressed extensions must be in a contiguous block in the inner hello. When we encode that contiguous block for the encoded inner hello, we replace the block with a marker extension listing all of the extension types that were replaced. In the outer hello the compressed extensions must be present in the same relative order (but don't need to be contiguous).

With this feature implemented we can also activate a couple of bogo tests that were previously excluded.

cpu added 15 commits June 10, 2024 09:28
The SVCB/HTTPS record handling in hickory-dns 0.24 was stripping the TLS
encoded list prefix from the `ECHConfigList` that is serialized into DNS
records. This meant our previous `ech.rs` connect test was subtly wrong:
it would only ever deserialize a single `EchConfig` from what it found
in DNS.

This commit updates Rustls to:

1. Use the new `EchConfigListBytes` type from pki-types to represent
   what it gets from DNS. Soon we will have more API surface expecting
   this type.

2. Use a hickory-dns release with some upstream fixes that ensure we get
   the correct wire-encoding of the `ECHConfigList`.

3. Update the ech connect tests unit tests to assert all of the ECH
   configs that may be found are the correct version.
For ECH GREASE it's convenient to be able to generate throw-away HPKE
key-pairs. Similarly, down the road we may want to support generating
server-side ECH configurations, which will require key generation.

This commit updates the `Hpke` trait to add a `generate_key_pair()` fn
for this purpose. Both the aws-lc-rs and provider-example HPKE trait
impls are updated to add this functionality.

Along the way I also copied the smoke test and fips unit tests from the
aws-lc-rs HPKE provider to the provider-example HPKE provider so that
there's coverage of key generation for both. We won't use key generation
in the rustls-provider-tests crate since the RFC test vectors come with
private key material to use.
This commit introduces the first piece of ECH support to the public API:
configuring a client connection with an optional ECH config.

Since ECH requires TLS 1.3 we offer a configuration entry-point
`with_ech(...)` on the `ConfigBuilder<ClientConfig,
WantsVersions>` state, and fix the supported protocol versions to only
TLS 1.3.

Handling configurations in a way that will be forward compatible and
adhering to the draft spec requires some extra care. Notably we need to
be able to treat ECH configs in an ECH config list with an unsupported
version as opaque blobs. This requires splitting our config
representation into an enum with two variants: one for a recognized ECH
config and one for an unsupported one.

Similarly we need to process optional extensions in ECH config contents
to ensure that:

1. There MUST NOT be duplicate extensions.
2. We MUST parse and check for unsupported mandatory extensions.

An example TLS client that fetches ECH configurations using
DNS-over-HTTPS and configures Rustls to use ECH is added to the
examples crate. The HTTPS-over-DNS lookup in that example requires
adding a new dep on hickory-dns. We use the recently added aws-lc-rs
backed HPKE implementations default selection of HPKE suites for the ECH
config.

As of this commit the ECH configuration is not used (except as a TODO to
avoid a `dead_code` warning). Subsequent commits will introduce actual
ECH offering and protocol support.
In order to support ECH we need to be prepared for
`emit_client_hello_for_retry` to return an `Error` where it was
otherwise infallible - this can occur (for e.g.) if the HPKE provider we
use for ECH encryption fails.

This commit changes `emit_client_hello_for_retry` to return
`NextStateOrError` instead of `NextState` in preparation for that.
Previously we separately iterated the `input.hello.sent_extensions` to
track our sent extensions before constructing a `ClientHelloPayload`
that contained them. The `ClientHelloPayload` was constructed in-line
for the `HandshakeMessagePayload` towards the end of
`emit_client_hello_for_retry`.

To support ECH we will want to do some additional work with the
`ClientHelloPayload`, and so this commit does some minor rearranging to
facilitate this.
This commit continues the implementation of client-side ECH support. We
now act on the provided ECH configuration when available to produce an
appropriate outer/inner client hello.

Adding this support requires introduction of a new `EchState`
struct for maintaining the required handshake state specific to ECH. We
use this in combination with new types for representing the ECH
extensions to produce the outer and inner client hello messages.

We do not yet properly handle confirmation of ECH acceptance, resulting
in decrypt errors when offering ECH. Subsequent commits will update
hello retry request and server hello handling to fix this.
This commit continues the implementation of client-side ECH by
processing the result of offering ECH in the non-HRR case. This involves
processing the received server hello, attempting to derive and match
a shared secret indicating ECH acceptance, and forwarding on our
understanding of ECH status (not offered, offered and rejected, or
offered and accepted) to later steps of the handshake.

If we arrive at the end of the handshake and our ECH offer wasn't
accepted, then we emit an appropriate alert and return an error
potentially containing ECH configs to retry with from the server's
response.

If our offer _was_ accepted, we change out the handshake transcript,
sent extensions and client hello as if the inner client hello was used
as the outer client hello. TLS handshaking continues as normal from that
point.

This level of support is sufficient for the ech-client example to
successfully use ECH with the defo.ie test server.

Importantly we do **not** yet handle the case where we offer ECH and the
server sends a hello retry request. This support will follow in
a subsequent commit.
This commit continues the implementation of client-side ECH by
supporting the case where we offer ECH and the server replies with
a hello retry request (HRR).

In this case we detect ECH acceptance in a slightly different manner. If
ECH was accepted we update the separate ECH transcript using the
received HRR and proceed to offer ECH again in our retried hello. After
this point ECH acceptance is handled as normal.

This completes the primary functionality of client-side ECH. The
remaining work involves GREASE ECH and extension compression.
In the situation where an ECH config is not available some clients may
wish to send a "GREASE" ECH extension as an anti-ossification mechanism
(see RFC 8701[0] for more information on the general concept of GREASE).

To support this we update the configuration to accept an optional ECH
mode instead of an optional ECH config. The ECH mode has a variant for
enabling ECH that holds an ECH config, and a separate variant that holds
only what's required to emit a GREASE ECH extension.

We don't automatically fall back to GREASE if ECH configs are provided
but none are compatible. If desired this should be handled by the
caller. Similarly we don't default to sending GREASE, it is opt-in.

[0]: https://www.rfc-editor.org/rfc/rfc8701
* Use docopt to make it feasible to provide args/flags
* Add flags for choosing CA cert, DNS-over-HTTPS server, etc
* Add config for performing placeholder "GREASE" ECH for hosts without
  ECH configs, doing so by default for hosts without ECH configs, and
  forcing it if desired
* Add config flag for specifying a ECH config from disk.
* Add flag for specifying how many requests to do, to test resumption.
* Asserts on the expected ECH status after handshaking.
This matches the rustls crate's current default provider choice.
A `ClientConfig` can only be considered FIPS compatible when using ECH
if the configured HPKE suite is FIPS compatible. This commit updates
`ClientConfig::fips()` to add that consideration.

Similarly, it may be helpful to know if a given client connection was
created from a FIPS compatible `ClientConfig`. To do this we update the
client config data to stash the `ClientConfig::fips()` result when
creating a new client connection. A new `ClientConnection::fips()`
accessor returns that stashed value.
The bogo shim requires some updates to support new command line
flags. Additionally in order to be able to assert some details in errors
(e.g. that an ECH required err contained expected retry configs) we have
to pipe the `Options` struct deeper into the client/server processing
logic.

Since `*ring*` has no HPKE provider, it can't do ECH, which means we
need an unfortunate number of cfg guards on imports/logic in the shim
related to HPKE/ECH. C'est la vie. We can revisit if `*ring*` gains
static ECDH or HPKE.

Beyond these changes, it's worth discussing the ignored tests. They're
either not applicable, or need upstream bogo fixes:

"TLS-ECH-Server*":
  We ignore all the TLS-ECH-Server tests. We haven't implemented server
  side ECH yet

"TLS-ECH-Client-ExpectECHOuterExtensions"
"TLS-ECH-Client-CompressSupportedVersions":
  These rely on extension compression between inner/outer hellos. NYI.

"TLS-ECH-Client-SelectECHConfig"
"TLS-ECH-Client-NoSupportedConfigs"
  These are meant to test unsupported configs are handled correctly: we
  happen to support the HPKE ciphersuites that make them "unsupported".
  There's a fix for this upstream we can take later.

"TLS-ECH-Client-SkipInvalidPublicName*":
  Our name validation allows underscores in names. We also don't
  fallback to GREASE when there are no valid ECH configs.

"TLS-ECH-Client-NoSupportedConfigs-GREASE":
  We don't fallback to GREASE for no ECH configs.

"TLS-ECH-Client-Reject-ResumeInnerSession-TLS13":
  This test expects an unexpected extension error in the resumption
  connection, but this only happens if the outer hello didn't include
  GREASE PSK. BoringSSL's impl doesn't. Ours does. As a result we
  produce `:ECH_REJECTED:` instead of :UNEXPECTED_EXTENSION:` and have
  to ignore this test.

"TLS-ECH-Client-TLS12-RejectRetryConfigs"
"TLS-ECH-Client-Reject-NoClientCertificate-TLS12"
"TLS-ECH-Client-Reject-TLS12"
"TLS-ECH-Client-Reject-ResumeInnerSession-TLS12"
"TLS-ECH-GREASE-Client-TLS12-RejectRetryConfigs"
"TLS-ECH-Client-Reject-EarlyDataRejected-TLS12"
"TLS-ECH-Client-Reject-NoClientCertificate-TLS12-Async"
  We never offer/support TLS 1.2 when using ECH. There's no fallback to
  plaintext or GREASE for a server that won't support TLS 1.3
This avoids breaking our MSRV task due to the later versions of
hickory-dns (née trust-dns) that use workspace inheritance.
A workspace-level patch must be resolved even for crates that don't
require the patched dep, and resolving the patch for 0.22+ requires
an MSRV of 1.64+

This can be reverted if there's a finalized 0.25 with the SVCB fixes, or
a pre-release, made available.
This commit extends the existing client-side ECH support to enable the
"SHOULD" recommendation of compressing extensions that are identical
between the outer client hello and the inner client hello. This
optimization reduces the overhead of ECH, shrinking the total client
hello size.

Achieving this is a bit tricky (which is why it was done separately at
the end!). To-be-compressed extensions must be in a contiguous block in
the inner hello. When we encode that contiguous block for the encoded
inner hello, we replace the block with a marker extension listing
all of the extension types that were replaced. In the outer hello the
compressed extensions must be present in the same relative order (but
don't need to be contiguous).

With this feature implemented we can also activate a couple of bogo
tests that were previously excluded.
@cpu

cpu commented Jun 11, 2024

Copy link
Copy Markdown
Owner Author

This branch has conflicts that must be resolved

I will sort this out (and open a proper PR against the main repo) once 1718 lands.

@cpu cpu closed this Jun 12, 2024
@cpu

cpu commented Jun 12, 2024

Copy link
Copy Markdown
Owner Author

rustls#2001

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant