Using higher-level wrapper of OpenSSL SHA1 hash function in civetweb by guitargeek · Pull Request #10380 · root-project/root

guitargeek · 2022-04-11T16:18:56Z

This is to get rid of deprecation warninigs when building on Ubuntu
22.04.

Should cause no backwards compatibility poblems, as the functions that
are used now are around at least since OpenSSL 1.0.2:
https://www.openssl.org/docs/man1.0.2/man3/EVP_DigestInit_ex.html

This patch was already applied to upstream civet:
civetweb/civetweb#1072

phsft-bot · 2022-04-11T16:19:03Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2022-04-11T16:52:34Z

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

[2022-04-11T16:52:04.680Z] c1xx : fatal error C1060: compiler is out of heap space [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\lib\CodeGen\LLVMCodeGen.vcxproj]
[2022-04-11T16:52:04.680Z] C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\include\clang/AST/DeclObjC.h(1699,1): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\Expr.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:04.680Z] C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\memory(3194,1): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\Decl.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:05.553Z] C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\sal.h(1224,1): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\NestedNameSpecifier.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:05.553Z] C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\include\llvm/ADT/FoldingSet.h(225,1): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\DeclObjC.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:05.553Z] C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\include\llvm/ADT/STLExtras.h(161,22): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\MicrosoftCXXABI.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:05.553Z] C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\vector(467,88): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\Mangle.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:05.553Z] C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\include\clang/AST/Expr.h(2773,1): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\AST\ExternalASTSource.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\AST\obj.clangAST.vcxproj]
[2022-04-11T16:52:05.553Z] c1xx : fatal error C1060: compiler is out of heap space [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Parse\obj.clangParse.vcxproj]
[2022-04-11T16:52:06.004Z] C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\include\llvm/ADT/APFloat.h(1100,1): fatal error C1060: compiler is out of heap space (compiling source file C:\build\workspace\root-pullrequests-build\root\interpreter\llvm\src\tools\clang\lib\Frontend\DependencyFile.cpp) [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Frontend\obj.clangFrontend.vcxproj]

And 313 more

guitargeek · 2022-04-12T10:27:52Z

Closing this PR because it has been superseeded by PR in upstream civetweb:
civetweb/civetweb#1072

linev · 2022-04-21T08:46:19Z

Jonas,
we can merge this PR and do not wait for civetweb 1.16 release

phsft-bot · 2022-05-05T11:57:48Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

This is to get rid of deprecation warninigs when building on Ubuntu 22.04. Should cause no backwards compatibility poblems, as the functions that are used now are around at least since OpenSSL 1.0.2: https://www.openssl.org/docs/man1.0.2/man3/EVP_DigestInit_ex.html This patch was already applied to upstream civet: civetweb/civetweb#1072

phsft-bot · 2022-05-05T12:00:58Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2022-05-05T12:57:04Z

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

[2022-05-05T12:44:07.950Z] C:\build\workspace\root-pullrequests-build\build\bin\libRHTTP.dll : fatal error LNK1120: 2 unresolved externals [C:\build\workspace\root-pullrequests-build\build\net\http\RHTTP.vcxproj]

linev · 2022-05-05T12:59:07Z

@guitargeek Something wrong with Windows build?

Axel-Naumann · 2022-05-20T13:40:07Z

@phsft-bot build just on windows10/cxx14

phsft-bot · 2022-05-20T13:40:14Z

Starting build on windows10/cxx14
How to customize builds

phsft-bot · 2022-05-20T14:16:39Z

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

[2022-05-20T14:11:32.535Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Basic\obj.clangBasic.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Driver\obj.clangDriver.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\lib\Object\LLVMObject.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\lib\LTO\LLVMLTO.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\lib\Object\LLVMObject.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Basic\obj.clangBasic.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Driver\obj.clangDriver.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\lib\LTO\LLVMLTO.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\lib\Object\LLVMObject.vcxproj]
[2022-05-20T14:11:44.332Z] c1xx : fatal error C1356: unable to find mspdbcore.dll [C:\build\workspace\root-pullrequests-build\build\interpreter\llvm\src\tools\clang\lib\Driver\obj.clangDriver.vcxproj]

And 163 more

Axel-Naumann · 2022-05-20T14:36:57Z

@phsft-bot build just on windows10/cxx14

phsft-bot · 2022-05-20T14:37:03Z

Starting build on windows10/cxx14
How to customize builds

phsft-bot · 2022-05-20T15:40:50Z

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

[2022-05-20T15:28:35.977Z] C:\build\workspace\root-pullrequests-build\build\bin\libRHTTP.dll : fatal error LNK1120: 2 unresolved externals [C:\build\workspace\root-pullrequests-build\build\net\http\RHTTP.vcxproj]

linev · 2022-05-21T11:01:20Z

@bellenot
Failure is:

civetweb.obj : error LNK2019: unresolved external symbol _EVP_Digest referenced in function _send_websocket_handshake [C:\build\workspace\root-pullrequests-build\build\net\http\RHTTP.vcxproj]
civetweb.obj : error LNK2019: unresolved external symbol _EVP_get_digestbyname referenced in function _send_websocket_handshake [C:\build\workspace\root-pullrequests-build\build\net\http\RHTTP.vcxproj]
C:\build\workspace\root-pullrequests-build\build\bin\libRHTTP.dll : fatal error LNK1120: 2 unresolved externals [C:\build\workspace\root-pullrequests-build\build\net\http\RHTTP.vcxproj]

Some extra ssl library is necessary for windows?

bellenot · 2022-05-23T06:42:52Z

@linev well, I don't know, We don't enable OpenSSL in ROOT on Windows. How does it works now? What changed between the previous version and this one?

linev · 2022-05-23T07:14:09Z

But then this PR will not work on Windows.
Moreover, next civetweb version will require OpenSSL - which is not very good.

Old code was using SHA1_Init, SHA1_Update, SHA1_Final functions.
For the case when SSL not available - inline version sha.inl was included.

Now it is not provided and therefore original SSL functions should be used.
Are there reasons why SSL is not used on Windows?

bellenot · 2022-05-23T07:23:48Z

IIRC, once OpenSSL is enabled, it forces to use https only and fails opening standard http

linev · 2022-05-23T07:29:36Z

IIRC, once OpenSSL is enabled, it forces to use https only and fails opening standard http

No, Linux version always compiled with SSL.
Plain http can be used as well.

bellenot · 2022-05-23T07:32:18Z

I know there is a good reason, I'll try to find it back

bellenot · 2022-05-23T07:41:12Z

OK, so -Dbuiltin_ssl=ON doesn't work on Windows...

linev · 2022-05-23T07:49:17Z

OK, so -Dbuiltin_ssl=ON doesn't work on Windows...

But is there any SSL library provided by Windows itself?

bellenot · 2022-05-23T07:51:59Z

OK, so -Dbuiltin_ssl=ON doesn't work on Windows...

But is there any SSL library provided by Windows itself?

Yes, but it's not compatible with OpenSSL. See for example:
https://docs.microsoft.com/en-us/windows/win32/winsock/winsock-secure-socket-extensions?redirectedfrom=MSDN
https://docs.microsoft.com/en-us/windows/win32/winsock/using-secure-socket-extensions

bellenot · 2022-05-23T07:55:03Z

Note that some OpenSSL binaries are available for Windows, but I'll have to try them

linev · 2022-05-23T07:56:57Z

@guitargeek
Maybe one should revert changes in civetweb, while at that place SSL is not used - it just calculates some special checksum for websocket.

Keep old code for Windows and older SSL versions

phsft-bot · 2022-06-29T10:50:06Z

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14
How to customize builds

linev · 2022-06-29T10:51:02Z

I add fallback solution for older OpenSSL or just for systems without OpenSSL - like Windows.
Let see if it will compile.

phsft-bot · 2022-06-29T11:17:44Z

Build failed on ROOT-performance-centos8-multicore/cxx17.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.runtutorials.tutorial_roofit_rf408_RDataFrameToRooFit

linev · 2022-06-29T14:50:40Z

@bellenot @guitargeek

Seems to be now it can be compiled on Windows. I just enable new feature only when OpenSSL 3.0 is available.
Same kind of PR I submit to civetweb: civetweb/civetweb#1080

guitargeek self-assigned this Apr 11, 2022

guitargeek requested a review from linev as a code owner April 11, 2022 16:18

guitargeek closed this Apr 12, 2022

guitargeek reopened this May 5, 2022

linev approved these changes May 5, 2022

View reviewed changes

guitargeek force-pushed the civetweb_1 branch from 8a4286e to 6f46b89 Compare May 5, 2022 12:00

guitargeek linked an issue May 5, 2022 that may be closed by this pull request

Update civetweb to 1.16 once it's released #10403

Closed

[civetweb] use EVP_Digest only for OPENSSL 3.0

39e12ee

Keep old code for Windows and older SSL versions

linev self-requested a review June 29, 2022 10:55

linev added the affects:master label Jun 29, 2022

linev removed a link to an issue Jun 29, 2022

Update civetweb to 1.16 once it's released #10403

Closed

linev approved these changes Jun 29, 2022

View reviewed changes

linev merged commit 08c41c6 into root-project:master Jun 29, 2022

linev mentioned this pull request Jun 29, 2022

[civetweb] use proper methods with openssl 3.0 [6-26] #10859

Merged

guitargeek deleted the civetweb_1 branch June 30, 2022 11:28

guitargeek linked an issue Jun 30, 2022 that may be closed by this pull request

Update civetweb to 1.16 once it's released #10403

Closed

guitargeek mentioned this pull request Jun 30, 2022

Update civetweb to 1.16 once it's released #10403

Closed

Conversation

guitargeek commented Apr 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phsft-bot commented Apr 11, 2022

Uh oh!

phsft-bot commented Apr 11, 2022

Errors:

Uh oh!

guitargeek commented Apr 12, 2022

Uh oh!

linev commented Apr 21, 2022

Uh oh!

phsft-bot commented May 5, 2022

Uh oh!

phsft-bot commented May 5, 2022

Uh oh!

phsft-bot commented May 5, 2022

Errors:

Uh oh!

linev commented May 5, 2022

Uh oh!

Axel-Naumann commented May 20, 2022

Uh oh!

phsft-bot commented May 20, 2022

Uh oh!

phsft-bot commented May 20, 2022

Errors:

Uh oh!

Axel-Naumann commented May 20, 2022

Uh oh!

phsft-bot commented May 20, 2022

Uh oh!

phsft-bot commented May 20, 2022

Errors:

Uh oh!

linev commented May 21, 2022

Uh oh!

bellenot commented May 23, 2022

Uh oh!

linev commented May 23, 2022

Uh oh!

bellenot commented May 23, 2022

Uh oh!

linev commented May 23, 2022

Uh oh!

bellenot commented May 23, 2022

Uh oh!

bellenot commented May 23, 2022

Uh oh!

linev commented May 23, 2022

Uh oh!

bellenot commented May 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bellenot commented May 23, 2022

Uh oh!

linev commented May 23, 2022

Uh oh!

phsft-bot commented Jun 29, 2022

Uh oh!

linev commented Jun 29, 2022

Uh oh!

phsft-bot commented Jun 29, 2022

Failing tests:

Uh oh!

linev commented Jun 29, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

guitargeek commented Apr 11, 2022 •

edited

Loading

bellenot commented May 23, 2022 •

edited

Loading