TLSv1.3: Refactor ClientHello processing so that extensions get parsed earlier#1807
TLSv1.3: Refactor ClientHello processing so that extensions get parsed earlier#1807mattcaswell wants to merge 15 commits intoopenssl:masterfrom
Conversation
ssl/ssl_locl.h
Outdated
There was a problem hiding this comment.
i don't see it in the style, but we conventionally have a struct name here.
ssl/ssl_sess.c
Outdated
There was a problem hiding this comment.
don't need a length check on the incoming session_id?
There was a problem hiding this comment.
No, we did it already when we first parsed it.
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
remove the else after return's
There was a problem hiding this comment.
actually, can't you just do "return e1->type - e2->type" ?
There was a problem hiding this comment.
return e1->type - e2->type isn't feasible. type is an unsigned int...
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
blank line after declarations.
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
I think "collect" is better than "parse_raw" or perhaps "parse_all" instead of "parse_raw"
There was a problem hiding this comment.
Renamed to tls_collect_extensions()
ssl/statem/statem_srvr.c
Outdated
There was a problem hiding this comment.
indent to be under the opening parent?
There was a problem hiding this comment.
The line is too long to do that...you'd have to put the == 0 on the next line. I thought it was more readable this way.
There was a problem hiding this comment.
That's not what @richsalz meant, he meant this:
+ if (s->ctx->app_verify_cookie_cb(s, clienthello.dtls_cookie,
+ clienthello.dtls_cookie_len) == 0) {and I know you disagree with that style...
There was a problem hiding this comment.
no, i meant line it up under the s,
ssl/statem/statem_srvr.c
Outdated
ssl/statem/statem_srvr.c
Outdated
There was a problem hiding this comment.
why parens around ciphers?
There was a problem hiding this comment.
It was there in the original code...but fixed.
ssl/t1_lib.c
Outdated
There was a problem hiding this comment.
you should introduce a local variable to factor out all the hello->pre_proc_exts[loop] expressions inside this loop.
There was a problem hiding this comment.
and why isn't this a big switch statement?
There was a problem hiding this comment.
and make the loop count down:
for (loop = hello->num_extensions; loop; --loop) {
There was a problem hiding this comment.
That suggested for loop is just wrong, you would need to start at num_extensions - 1 or use loop - 1 everywhere inside the for loop. Note that starting from num_extensions - 1 will give you a problem because loop is a size_t. I also don't see why this should loop in the reverse order.
There was a problem hiding this comment.
no, the loop is right. and if you have a local pointer variable like I also said, then you never need to refer to "loop" inside the body of the loop.
There was a problem hiding this comment.
I used a local pointer variable - that makes a lot of sense. I haven't changed the loop to count down...I don't see why it makes any difference?
There was a problem hiding this comment.
It's a micro optimization. On a machine instruction level, comparing with zero is a bit faster than comparing with another value. For a loop with a large span, it actually makes a difference. In this case, the difference is probably deeply marginal.
ssl/t1_lib.c
Outdated
There was a problem hiding this comment.
use a local pointer and make the loop count down:
for (loop = numexts; loop; --loop, ++ext)
if (exts->type == type)
return ext;
There was a problem hiding this comment.
Same as above. Why do you want count down? Is this some premature optimization thing?
There was a problem hiding this comment.
it's just a habit of mine. sure, if you want to call it premature, fine. the key point is have a local pointer variable and always use that.
There was a problem hiding this comment.
Not sure I see the point here? There are only two uses of loop here anyway...it doesn't seem to simplify things at all? Similarly I'm not sure why counting down is better.
There was a problem hiding this comment.
In assembler it's usually faster to compare to 0 than to compare to some other number. In real applications you usually can't measure the difference.
There was a problem hiding this comment.
And I think the compiler is actually smart enough to turn that into a pointer itself. And it might also do some other optimizations depending on in which direction you go. But I really don't see the point.
You could for intance also write it as:
for (ext = exts + numexts; ext != exts; ext--)
if (ext->type == type)
return ext;
There was a problem hiding this comment.
and also, notice that there is actually no extra variable here. look at the code i wrote.
There was a problem hiding this comment.
You mean the one that always looks at the same first entry?
There was a problem hiding this comment.
Can we just agree to disagree on this?
There was a problem hiding this comment.
well it was a comment :) but yeah, ext++ in the end of the for loop.
i'm advocating for a particular style, but i am not going to minus-one over it.
ssl/ssl_sess.c
Outdated
There was a problem hiding this comment.
You forgot to update the documentation above.
ssl/t1_lib.c
Outdated
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
This looks like it's a largely based on tls1_check_duplicate_extensions(). Can we get rid of that function and use this new one for both the client and the server?
There was a problem hiding this comment.
That was actually my thinking, but we can't do it just yet without also refactoring the ServerHello extension parsing too. That was more than I wanted to do in this PR at this stage. I've added a TODO item to remind us to come back to that and delete tls1_check_duplicate_extensions() later.
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
Can we maybe get rid of this client_version variable?
There was a problem hiding this comment.
Hmmmm...the hello structure is currently transient and disappears once we've finished the ClientHello processing. We need client_version later though, so I don't think we can delete it just yet.
|
Pushed new commits to address feedback so far. There's also a couple of commits to fix a memory leak, and to fix a problem in compression method parsing - both issues picked up by Travis. |
|
make update was failing...added a new commit to try and fix it. |
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
Please add some short documentation.
ssl/t1_lib.c
Outdated
ssl/statem/statem_lib.c
Outdated
There was a problem hiding this comment.
You get the extensions now, not just the types.
ssl/statem/statem_srvr.c
Outdated
There was a problem hiding this comment.
I'm wondering if we still have a test that sends a v2 client hello (with TLS 1.0 for instance, like 0.9.8 used to do).
There was a problem hiding this comment.
Yes we do. See 70-test_sslrecords.t
ssl/t1_lib.c
Outdated
There was a problem hiding this comment.
This documentation is outdated.
|
Looks good except for those minor points above. |
|
New commit added to address latest comments from @kroeckx |
ssl/t1_lib.c
Outdated
There was a problem hiding this comment.
I guess that is NULL when not found?
There was a problem hiding this comment.
Yes...failure means not found...do you want me to change it to say that?
cefa5d2 to
8d42da3
Compare
|
Updated commit pushed. |
|
Any more comments on this? |
ssl/statem/statem_srvr.c
Outdated
There was a problem hiding this comment.
blank line after the return.
ssl/statem/statem_srvr.c
Outdated
8d42da3 to
99df412
Compare
| /* | ||
| * Do SSL/TLS version negotiation if applicable. For DTLS we just check | ||
| * versions are potentially compatible. Version negotiation comes later. | ||
| */ |
There was a problem hiding this comment.
Per the discussion in #1771 should we consider moving SNI processing up here, or at least before ssl_get_prev_session()?
I had started on a WIP but got interrupted; there are some gnarly compatibility issues with respect to how currently the servername_callback can inspect the session (and the tlsext_hostname therein). I am not sure how well those issues can be resolved.
There was a problem hiding this comment.
Maybe, but I've not been following #1771, so not as part of this PR.
|
I probably wont have time until the weekend to look at anything. |
|
tls_decrypt_ticket is a pretty icky function - magic number return codes which make little sense in how they are set out - and they should be #defines for each of the return values given they are being used in a switch statement ... I know that is existing code - but it stands out as needing addressing when reading through the changes. |
|
Yeah....I know...but I'd really rather stay focused and keep that out of scope. |
In the case of an SSLv2 compat ClientHello we weren't setting up the compression methods correctly, which could lead to uninit reads or crashes.
We should be freeing up the raw extension data after we've finished with it.
Based on review feedback received.
Add a blank line, take one away - due to feedback received during review.
99df412 to
45f63e2
Compare
|
Rebased to fix the various merge conflicts due to all the changes in the master branch. |
|
Ping @kroeckx |
ssl/ssl_locl.h
Outdated
|
|
||
| typedef struct { | ||
| unsigned int isv2; | ||
| unsigned int version; |
There was a problem hiding this comment.
I'm wondering if we should call this legacy_version
ssl/statem/statem_srvr.c
Outdated
| /* | ||
| * Handle an SSLv2 backwards compatible ClientHello | ||
| * Note, this is only for SSLv3+ using the backward compatible format. | ||
| * Real SSLv2 is not supported, and is rejected above. |
There was a problem hiding this comment.
This comment is at least outdated.
ssl/statem/statem_srvr.c
Outdated
| if (!PACKET_get_sub_packet(pkt, &clienthello.ciphersuites, | ||
| ciphersuite_len) | ||
| || !PACKET_get_sub_packet(pkt, &session_id, | ||
| clienthello.session_id_len) |
There was a problem hiding this comment.
Is there a reason this gets saved in a local PACKET and is not directly written to the clienthello?
There was a problem hiding this comment.
No, I guess not. I'm not sure it makes a lot of difference, but I changed it anyway.
ssl/statem/statem_srvr.c
Outdated
| al = SSL_AD_DECODE_ERROR; | ||
| goto f_err; | ||
| } | ||
| clienthello.session_id_len = session_id_len; |
There was a problem hiding this comment.
Setting the length here just seems wrong without first actually writing the session_id in the buffer.
ssl/statem/statem_srvr.c
Outdated
| memset(s->s3->client_random, 0, SSL3_RANDOM_SIZE); | ||
| challenge_len = challenge_len > sizeof(clienthello.random) | ||
| ? sizeof(clienthello.random) : challenge_len; | ||
| memset(clienthello.random, 0, sizeof(clienthello.random)); |
There was a problem hiding this comment.
I guess I'm going to give just the opposite comment than @richsalz, but I don't like the sizeof() in this case. The SSL3_RANDOM_SIZE here is because that's what the limit is for this protocol. But clienthello is not specific to this protocol.
There was a problem hiding this comment.
Actually reading the SSLv3 RFC about the SSLv2 client hello, we're probably doing this wrong anyway, but I wouldn't bother fixing it.
There was a problem hiding this comment.
hadn't thought of that, fine with either way (although perhaps turning kurt's comment into a C comment is worthwhile)
There was a problem hiding this comment.
It's also unlikely that this size will change, since it's really a fixed size.
There was a problem hiding this comment.
Good point. I changed it back and added a comment.
ssl/t1_lib.c
Outdated
| * 10.8..10.8.3 (which don't work). | ||
| */ | ||
| static void ssl_check_for_safari(SSL *s, const PACKET *pkt) | ||
| static void ssl_check_for_safari(SSL *s, CLIENTHELLO_MSG *hello) |
|
Except for those minor comments, it looks good. |
For consistency with the TLSv1.3 spec.
Don't use a sub-packet, just load it.
…dom) The size if fixed by the protocol and won't change even if sizeof(clienthello.random) does.
The name and type of the argument to ssl_check_for_safari() has changed.
|
New commits pushed addressing all the latest feedback. |
ssl/t1_lib.c
Outdated
| * 1 on success | ||
| * 0 on error | ||
| */ | ||
| int tls_check_client_ems_support(SSL *s, CLIENTHELLO_MSG *hello) |
There was a problem hiding this comment.
Any reason this isn't a "const CLIENTHELLO_MSG *hello" here an at other places?
There was a problem hiding this comment.
I made it const. There were a small number of other places which could also be changed so I did those too. Technically there are other places where the CLIENTHELLO_MSG structure itself is not changed by a function, but something pointed to by it is. I didn't change those because I think that is just too confusing.
| /* |cookie| will only be initialized for DTLS. */ | ||
| PACKET session_id, cipher_suites, compression, extensions, cookie; | ||
| int is_v2_record; | ||
| PACKET session_id, compression, extensions, cookie; |
There was a problem hiding this comment.
I think session_id is an unused variable now?
There was a problem hiding this comment.
Yeah, it's still used.
|
Looks good. |
There were a few places where they could be declared const so this commit does that.
|
New commit added. Ping @kroeckx. |
|
@richsalz can you reconfirm your plus one? If so I'll put you both down as reviewers. |
|
yes, +1 |
|
Merged. Thanks for the review!! |
The existing server side ClientHello extension parsing is a bit of a mess. Historically the way we have parsed the ClientHello is to read each field in turn and then process it in some way immediately. Extensions are at the end so they get parsed and processed last.
That approach became problematic for session tickets, because we needed to know about them earlier in the ClientHello processing, i.e. when we are dealing with the sessionid field. So then we had to introduce the misleadingly named function
tls_check_serverhello_tlsext_early()...which is actually for doing early processing of the ClientHello extensions!!!! This function peeked ahead at the ClientHello extensions to get the session ticket data. Later when we added EMS we had a similar problem, sotls_check_serverhello_tlsext_early()was extended to also deal with that.For TLSv1.3 we're going to have more extensions like this. For example the supported_versions extension is for doing version negotiation. This is one of the first things we need to know about (i.e. even earlier than the processing that
tls_check_serverhello_tlsext_early()currently does)...so maybe we need atls_check_clienthello_tlsext_really_early():-)This doesn't seem to be the right way to go about things, so this PR refactors the whole ClientHello processing to split it into two phases. First we parse the ClientHello, where we just basically collect the data into an easily queryable format. Then subsequently we do the processing. For extensions, actually what we do is parse the extensions block up front and create an array of all the extension types mapped to the raw extension data. We can then process the extension data as and when we need it.
This PR will form the basis for my next one, which is to implement the supported_versions extension.