Skip to content

Instead of global data store it in an OPENSSL_CTX#8857

Closed
mattcaswell wants to merge 11 commits intoopenssl:masterfrom
mattcaswell:fips-evp-new
Closed

Instead of global data store it in an OPENSSL_CTX#8857
mattcaswell wants to merge 11 commits intoopenssl:masterfrom
mattcaswell:fips-evp-new

Conversation

@mattcaswell
Copy link
Member

This is built on top of the commits in #8728 (Make the EVP API available from within the FIPS module) and addresses the current issues in that PR. Not sure whether to replace that PR with this one or keep it separate.

The new stuff is in the last two commits. Essentially all global data in the core and evp code is moved into the OPENSSL_CTX instead. This was always the intention anyway (see #8760).

A new function "openssl_ctx_run_once" is introduced which is similar to RUN_ONCE except initialisation code is run once per OPENSSL_CTX. We also add a new function "openssl_ctx_onfree" which is similar to OPENSSL_atexit() except it runs when the OPENSSL_CTX is freed.

The ex_data global data is a bit of a special case and needs some separate handling since we can't store that in the normal way in the OPENSSL_CTX (because ex_data is used in the implementation of OPENSSL_CTX).

Having made all these changes it is then possible to share all the same initialisation and de-initialisation code between the FIPS provider and the default provider. We don't need to treat FIPS any differently. Everything is initialised "on demand" as it is normally.

WIP because of the dependency on #8728 and because there are various updates required to the internal documentation that I haven't looked at yet.

@paulidale
Copy link
Contributor

Just wondering (having not thought this properly through): would it be reasonable to register the "openssl_ctx_run_once" calls like the "openssl_ctx_onfree" ones and make them automatic?

@mattcaswell
Copy link
Member Author

Just wondering (having not thought this properly through): would it be reasonable to register the "openssl_ctx_run_once" calls like the "openssl_ctx_onfree" ones and make them automatic?

Where would you register them? As a hard coded list perhaps? There are pros and cons:

pro: you don't have to check all the time whether you've done the initialisation yet
con: initialisation is no longer "on demand" so you might end up allocating lots of resources that you don't end up using

@mattcaswell
Copy link
Member Author

fixup up pushed to address "make update" issues.

@mattcaswell
Copy link
Member Author

Another fixup to address a mem leak.

@paulidale
Copy link
Contributor

con: initialisation is no longer "on demand" so you might end up allocating lots of resources that you don't end up using

Even if registered before time (a fixed list is a good place), they'd only be called on context creation. Does that necesarily imply a cascade of other initialisers being called independent of what gets used by the applications??

@levitte
Copy link
Member

levitte commented May 2, 2019

Question: did you have to put ex_data class data in the global context for some reason, or did you just take "everything" in #8760 more literally than I meant it?

My thoughts, that I may or may not have shared, is that the ex_data class data would remain global (note: global to the module that includes the ex_data object file)

@levitte
Copy link
Member

levitte commented May 2, 2019

Not sure whether to replace that PR with this one or keep it separate.

I think this should be an entirely separate PR. There's nothing in this PR that should be inherently dependant on #8728, and having the two mixed in one PR makes it harder to review this one.

@mattcaswell
Copy link
Member Author

Question: did you have to put ex_data class data in the global context for some reason, or did you just take "everything" in #8760 more literally than I meant it?

Yes. This solves the problem of the ex_data_lock. By tying it to the lifetime of the OPENSSL_CTX we avoid all the issues we were having with freeing it on the provider side. I see no reason to treat that global data differently to any other global data. In fact its kind of weird not to. This makes things conceptually much simpler.

@levitte
Copy link
Member

levitte commented May 2, 2019

This makes things conceptually much simpler.

I disagree, especially on a conceptual level. See it like this, the OSSL_EX_DATA_GLOBAL is class data (not class instance data), and with the changes made here, you have basically made it so that you define the class data in terms of instances of that class. Conceptually speaking, that's not simple, not one bit.

@mattcaswell
Copy link
Member Author

I disagree, especially on a conceptual level. See it like this, the OSSL_EX_DATA_GLOBAL is class data (not class instance data), and with the changes made here, you have basically made it so that you define the class data in terms of instances of that class. Conceptually speaking, that's not simple, not one bit.

I think you are overcomplicating it. Think of OPENSSL_CTX as a scope. The OSSL_EX_DATA_GLOBAL is class data for all classes within that scope. The OPENSSL_CTX class of objects is slightly different to other classes in that there is only ever one instance within a given scope. I don't think this is a problem.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will people be very upset of we make CRYPTO_EX_DATA opaque?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're supposed to be trying not to break stuff wherever possible. I think this could be a case where we could break things so I think it is best to avoid it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okie...

Various core and property related code files used global data. We should
store all of that in an OPENSSL_CTX instead.
@mattcaswell
Copy link
Member Author

I updated this to remove the dependency on #8728.

@levitte
Copy link
Member

levitte commented May 2, 2019

I updated this to remove the dependency on #8728.

Thank you!

@levitte
Copy link
Member

levitte commented May 2, 2019

The reason you're getting build errors on Windows is the new inclusion of "internal/cryptlib.h" in include/internal/property.h, which is included by test/property_test.c. The simple solution is this:

: ; git diff
diff --git a/test/build.info b/test/build.info
index ded3bd770a..2800c71d3e 100644
--- a/test/build.info
+++ b/test/build.info
@@ -503,7 +503,7 @@ IF[{- !$disabled{tests} -}]
     DEPEND[wpackettest]=../libcrypto ../libssl.a libtestutil.a
 
     SOURCE[property_test]=property_test.c
-    INCLUDE[property_test]=../include ../apps/include
+    INCLUDE[property_test]=.. ../include ../apps/include
     DEPEND[property_test]=../libcrypto.a libtestutil.a
 
     SOURCE[ctype_internal_test]=ctype_internal_test.c

@levitte
Copy link
Member

levitte commented May 2, 2019

These manuals need an update:

  • doc/internal/man3/ossl_method_construct.pod
  • doc/internal/man3/OSSL_METHOD_STORE.pod
  • doc/internal/man3/openssl_ctx_get_data.pod

@mattcaswell mattcaswell changed the title WIP: Instead of global data store it in an OPENSSL_CTX Instead of global data store it in an OPENSSL_CTX May 2, 2019
@mattcaswell
Copy link
Member Author

Just wondering (having not thought this properly through): would it be reasonable to register the "openssl_ctx_run_once" calls like the "openssl_ctx_onfree" ones and make them automatic?

After a bit of "Doh!" moment I realised that we could do this with all of the openssl_ctx_init_index calls - and suddenly you don't need some of the run_once calls at all because a number of them were only there to initialise the index. Furthermore, I realised I wasn't using OPENSSL_CTX quite right and I could move even more initialisation code into the OPENSSL_CTX_METHOD "new" functions. This significantly simplifies things. Most code can just set up an OPENSSL_CTX_METHOD and it all just initialises itself when needed and frees itself when the OPENSSL_CTX_METHOD is freed. No need to call openssl_ctx_run_once or openssl_ctx_onfree at all.

There is still one spot where openssl_ctx_run_once is required. There are no places where openssl_ctx_onfree is now needed, although I have left the implementation there since I suspect it will be needed as we roll this out to more places in the code.

New commits pushed addressing the feedback and making the changes described above. Taking this out of WIP.

@mattcaswell
Copy link
Member Author

The simple solution is this

Thanks for the hint. Should be sorted now. Also I updated the internal manuals as requested.

@mattcaswell
Copy link
Member Author

Updated to address @levitte's feedback.

@levitte
Copy link
Member

levitte commented May 2, 2019

So, what about #8857 (comment)?

@levitte
Copy link
Member

levitte commented May 2, 2019

(other than that, I'm not reasonably happy with this PR)

@mattcaswell
Copy link
Member Author

So, what about #8857 (comment)?

Responded to

@levitte levitte added approval: done This pull request has the required number of approvals branch: master Applies to master branch labels May 2, 2019
@mattcaswell
Copy link
Member Author

Pusehd! Thanks.

@mattcaswell mattcaswell closed this May 2, 2019
levitte pushed a commit that referenced this pull request May 2, 2019
levitte pushed a commit that referenced this pull request May 2, 2019
Various core and property related code files used global data. We should
store all of that in an OPENSSL_CTX instead.

Reviewed-by: Richard Levitte <[email protected]>
(Merged from #8857)
levitte pushed a commit that referenced this pull request May 2, 2019
We should be seeking to move the OPENSSL_init_crypto and OPENSSL_cleanup
processing into OPENSSL_CTX instead.

Reviewed-by: Richard Levitte <[email protected]>
(Merged from #8857)
levitte pushed a commit that referenced this pull request May 2, 2019
@t8m
Copy link
Member

t8m commented May 3, 2019

(other than that, I'm not reasonably happy with this PR)

I suppose you meant now instead of not :)

@levitte
Copy link
Member

levitte commented May 3, 2019

(other than that, I'm not reasonably happy with this PR)

I suppose you meant now instead of not :)

Uhmmmm... yes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: done This pull request has the required number of approvals branch: master Applies to master branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants