[CI, enhancement] use -fvisibility=hidden in Linux Make builds#3080
[CI, enhancement] use -fvisibility=hidden in Linux Make builds#3080icfaust merged 165 commits intouxlfoundation:mainfrom
Conversation
|
/intelci: run |
|
/intelci: run |
|
more advanced CI run showing no problems (ignore infrastructure issues, tests still ran through properly) http://intel-ci.intel.com/effdbfb8-1661-f1b1-9595-a4bf010d0e2d |
|
/intelci: run |
|
Performance benchmarks are available upon request. |
| @@ -129,6 +129,7 @@ y := $(notdir $(filter $(_OS)/%,lnx/so win/dll mac/dylib)) | |||
| -Q := $(if $(OS_is_win),$(if $(COMPILER_is_vc),-,-Q),-) | |||
| -cxx17 := $(if $(COMPILER_is_vc),/std:c++17,$(-Q)std=c++17) | |||
| -fPIC := $(if $(OS_is_win),,-fPIC) | |||
| -visibility := $(if $(OS_is_win),,-fvisibility=hidden) | |||
There was a problem hiding this comment.
What happens if you build it with clang on windows?
There was a problem hiding this comment.
Good question, the default windows macro is used adding __declspec(dllexport) (daal_defines.h and dal/common.hpp).
There was a problem hiding this comment.
But would that still have an effect if not passing the right flags to clang on windows?
There was a problem hiding this comment.
https://www.reddit.com/r/cpp_questions/comments/v1e1t4/question_about_win64_and_other_windows_defines/?rdt=50042 Its set by default by the clang compiler on windows systems (as far as I have found), didn't realize minGW did it too...
There was a problem hiding this comment.
@icfaust I mean: the makefile here is not setting the flag -fvisibility=hidden on windows.
There was a problem hiding this comment.
Also rest assured, we test with both Visual Studio C++ and Intel DPC++ Windows builds, the latter being a clang-derived compiler. If there were issues, there would be failures in intelci associated with it.
There was a problem hiding this comment.
@icfaust I'm checking the windows .dll files produced with ICX by the CI as it is right now, without the flag -fvisibility=hidden. I'm not 100% sure but it looks like it is exporting all symbols.
For example, if I check it with winetools like this:
winedump -C -j export onedal.3.dllThen I see symbols like these:
_onedal_parallel_reduce_int32ptr_int64_simple
?allocate@homogen_table_builder@v1@detail@dal@oneapi@@QEAA?A?<auto>@@_J0@Z
void __cdecl oneapi::dal::detail::threader_for_blocked_size<class `void __cdecl oneapi::dal::backend::copy_convert<struct oneapi::dal::backend::cpu_dispatch_avx512,double,unsigned int>(class oneapi::dal::detail::v1::host_policy const & __ptr64,double const * __ptr64,__int64,unsigned int * __ptr64,__int64,__int64)'::`1'::<lambda_1> >(unsigned __int64,unsigned __int64,class `void __cdecl oneapi::dal::backend::copy_convert<struct oneapi::dal::backend::cpu_dispatch_avx512,double,unsigned int>(class oneapi::dal::detail::v1::host_policy const & __ptr64,double const * __ptr64,__int64,unsigned int * __ptr64,unsigned int * __ptr64,unsigned int * __ptr64)'::`1'::<lambda_1> const & __ptr64)
If I understand it correctly, the logic for windows builds with ICX would not change with this PR, which means they'd still be exported.
There was a problem hiding this comment.
That's right, the topic of the PR is focused on getting visibility correct for Linux make only, in fact Linux bazel builds are still full visibility. So far my research online has only matched that the default visibility for windows is hidden, maybe a follow up would be to check this circumstance and if necessary create a follow-up ticket/PR for windows.
There was a problem hiding this comment.
That's right, the topic of the PR is focused on getting visibility correct for Linux make only, in fact Linux bazel builds are still full visibility. So far my research online has only matched that the default visibility for windows is hidden, maybe a follow up would be to check this circumstance and if necessary create a follow-up ticket/PR for windows.
As far as I'm aware, that depends on the compiler, linker, arguments, and presence of other files like .def. MSVC for example defaults to hiding everything, but it appears ICX doesn't, or at least is not doing so in the CI builds.
But since the Makefile here is not meant to work with every possible compiler, I guess it should be enough to pass -fvisibility=hidden on windows for the compilers that support such flag (mingw, clang, icx).
There was a problem hiding this comment.
From some further investigations:
- The flag
-fvisibilityis not supported by ICX on windows. - It is not exporting all of the symbols in windows either, or at least not when it comes to ICX builds from the Makefile.
- But it is exporting more than needed - for example, it has
ONEDAL_EXPORTapplied to the internal functions used for threading:
| #define DAAL_EXPORT __declspec(dllexport) | ||
| #else | ||
| #define DAAL_EXPORT | ||
| #define DAAL_EXPORT __attribute__((visibility("default"))) |
There was a problem hiding this comment.
Since we're using C++17, how about [[gnu::visibility("default")]] instead?
|
@icfaust Overall question: why does the export need to be added to extra places like |
|
Another good question, all daal algorithms are linked to daal4py using the parser scripts (https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/generator/gen_daal4py.py), and we aren't sure who is using daal algorithms directly without scikit-learn-intelex. In essence 2 reasons: 1) to make sure that scikit-learn-intelex can load daal4py which requires all algorithms in the daal ABI and 2) for possible users of the DAAL ABI which aren't reflected in our common uses cases and possibly testing. Luckily in these circumstances, tests will fail if I didn't do it (including obscure algorithms like BACON, stump, etc.) And adaboost, for example is exposed still in daal4py, but not in sklearnex: https://github.com/uxlfoundation/scikit-learn-intelex/blob/a4105452b7e0dae7f5ca279f347aeddd740b013a/daal4py/sklearn/ensemble/AdaBoostClassifier.py#L40 . I have to say, it made this PR more difficult... |
Vika-F
left a comment
There was a problem hiding this comment.
This is a huge improvement towards backward compatibility testing. Thank you for pushing it forward!
I only have a couple of comments.
Regarding the macros in *_dispatcher.cpp files, I am not sure that there is a good way to generalize those, because there are many differences related to inputs and parameters from file to file. But maybe you will have some ideas to try. It is Ok, if not.
| namespace association_rules | ||
| { | ||
| namespace interface1 | ||
| { | ||
| template <> | ||
| DAAL_EXPORT Batch<DAAL_FPTYPE, association_rules::apriori>::Batch() | ||
| { | ||
| initialize(); | ||
| } | ||
|
|
||
| using BatchType = Batch<DAAL_FPTYPE, association_rules::apriori>; | ||
| template <> | ||
| DAAL_EXPORT BatchType::Batch(const BatchType & other) : input(other.input), parameter(other.parameter) | ||
| { | ||
| initialize(); | ||
| } | ||
| } // namespace interface1 | ||
| } // namespace association_rules |
There was a problem hiding this comment.
This code block looks very similar in many *_fpt_dispatcher.cpp files.
It would be good to create a macro to reduce the amount of code in .cpp files, __DAAL_INSTANTIATE_BATCH_CONSTRUCTORS(...).
Something like this:
#define __DAAL_INSTANTIATE_BATCH_CONSTRUCTORS(BatchClass, DAAL_FPTYPE, method) \
using BatchType = BatchClass<DAAL_FPTYPE, method>; \
template <> \
DAAL_EXPORT BatchType::Batch() \
{ \
initialize(); \
} \
template <> \
DAAL_EXPORT BatchType::Batch(const BatchType & other) : \
input(other.input), parameter(other.parameter) \
{ \
initialize(); \
}And use it like this:
__DAAL_INSTANTIATE_BATCH_CONSTRUCTORS(association_rules::Batch, DAAL_FPTYPE, association_rules::apriori)There was a problem hiding this comment.
I totally agree with you, and thought about this, we could do the standard constructor that way. Unfortunately the copy constructor is different for the various algorithms because of differences on what is in the member initializer list (which was/is very annoying). This is because the contents of initialize() is not standardized and sometimes handled in the initializer. We could go back and redefine all of the initialize() functions to be standard, but I thought it was a bit out of scope of the PR. Let me know what you think @Vika-F .
There was a problem hiding this comment.
I just saw your general comment, it looks like you came to the same conclusion. Should I template the standard constructor? Let me know. I would like to create a follow up task to see if we can make all initialize calls similar. @Vika-F
There was a problem hiding this comment.
Yes, I am Ok to have this as a not too urgent follow up task.
| // due to the very low-level dependence on implicit instantiation for leaf_node_info, | ||
| // this full expansion of the node_info class methods were required in order to comply | ||
| // with the various compilers and ONEDAL_EXPORT, which respond differently depending on | ||
| // the compiler. This verbose solution is unfortunately necessary, as not to break the | ||
| // API or ABI and still work with all supported compilers. |
There was a problem hiding this comment.
Thank you for adding the clarification.
Those are the part of DAAL API. Here is an example that uses that API: Some algorithms like AdaBoost are implemented only on DAAL side and do not have oneDAL API at all. They cannot be just removed from the library. Though I do not fully understand why Windows build was not failing without those exports. |
…undation#3080) * Update build.sh * Update ci.yml * Update ci.yml * Update ci.yml * Update build.sh * Update test.sh * Update ci.yml * Update test.sh * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update makefile * Update build.sh * Update ci.yml * Update makefile * Update build.sh * Update makefile * Update makefile * Update makefile * Update INSTALL.md * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * add shell script * missing whitespace * spelling mistake * set file permissions * remove unwanted changes * Update abi_check.sh * Update ci.yml * add ignore file * fix CI and shell * swap back * close bracket * spy into variables * fix if statement * change spy improve regex * add suggestions * Update ci.yml * Update ci.yml * Update daal_defines.h * Update makefile * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update common.hpp * Update makefile * Update makefile * Update makefile * Update ci.yml * Update adaboost_predict_dense_default_batch_fpt_cpu.cpp * Update adaboost_predict_dense_default_batch_fpt_cpu.cpp * Update adaboost_train_dense_default_batch_fpt_cpu.cpp * Update adaboost_predict_dense_default_batch_fpt_cpu.cpp * Update adaboost_train_dense_default_batch_fpt_cpu.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update compute_types.hpp * fix issues with partial, still leaves decision_tree nodes * remove ONEDAL_EXPORT * return change * further return * fix another implicit instantiation * fix node_info instantiation * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * fix stump too * attempt to define some missing things * move namespaces * Update algorithm_container_base_batch.h * Update algorithm_container_base_batch.h * Update ci.yml * Update uniform_dense_default_batch_fpt_dispatcher.cpp * Update normal_dense_default_batch_fpt_dispatcher.cpp * Update bernoulli_dense_default_batch_fpt_dispatcher.cpp * Update bernoulli_dense_default_batch_fpt_dispatcher.cpp * fix engines * intermediate save * formatting * small fix * next batch * missing newline * fix philox * full covariance fixes * first attempt at linear * Update covariance_distributed.h * attempt to fix windows * fixes for windows and elastic net * fix windows issues again * further windows fixes * svm + svd * further windows fixes * more fixes * kernel function solution * fix implicit_als * linear regression changes * low_order_moments * interim save point * further updates * Update sgd_dense_default_batch_fpt_dispatcher.cpp * pca + fixes * passing first examples * final work * attempt to see if this influences missing symbols * Update coordinate_descent_dense_default_batch_fpt_dispatcher.cpp * fix DAAL_EXPORT * clang fixes part 1 * another round of hell * further fixes for clang * fixes on fixes * further fixes for clang * stopping point * fix mistakes * through linear regression * low order moments * naivebayes * DistributedInput will need to be verified throughout the codebase * passing on clang * return SVM change * working oneapi_c examples * additional fixes for dpc tests * pass editor check * remove python3 * last update? * Update common.hpp * updates * Update common.cpp * Update cpu_info_impl.hpp * Update infer_parameters.hpp * Update common.hpp * Update algorithm_container_base_batch.h * Update pca_online.h * conform to daal ABI * conform to daal ABI * begin fixing spmd * fix spmd exports * make fixes for various compiler warnings * try to solve more spmd/ccl issues * try to fix communicator * Update chunked_array_impl.cpp * add spmd related fixes (continued * fix for spmd * Update ci.yml * Update ci.yml * latest fixes * latest fixes * futher fies * further spmd fixes * fix GNU build * attempt again * remove other PR from this one * remove other PR from this one * Update common.hpp
* Add correlation distance algorithm Signed-off-by: North Iii <[email protected]> * Update with correlation algorithm implementation Signed-off-by: North Iii <[email protected]> * Add clang-format Signed-off-by: richard.north.iii <[email protected]> * Update correlation distance unit test Signed-off-by: North Iii <[email protected]> * Update correlation_distance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update CPU/GPU correlation distance algorithms Signed-off-by: richard.north.iii <[email protected]> * Update cordistance_full_impl.i with clang-format Signed-off-by: richard.north.iii <[email protected]> * Update algo/correlation_distance/BUILD Signed-off-by: richard.north.iii <[email protected]> * Update cordistance_lp_impl.i * Update cordistance_up_impl.i * Update correlation distance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update clang-format for daal cordistance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update cordistance algo Signed-off-by: richard.north.iii <[email protected]> * Update clang-format for cordistance algo Signed-off-by: richard.north.iii <[email protected]> * Update clang-format Signed-off-by: richard.north.iii <[email protected]> * Update cordistance_dense_default_batch_fpt_cpu.cpp * Update correlation distance algo Signed-off-by: richard.north.iii <[email protected]> * Update examples/oneapi Signed-off-by: richard.north.iii <[email protected]> * Update cordistance_full_impl.i Signed-off-by: richard.north.iii <[email protected]> * Add docs for correlation_distance Signed-off-by: richard.north.iii <[email protected]> * Update clang-format for cordistance_full_impl.i Signed-off-by: richard.north.iii <[email protected]> * Add API doc for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Fix docbuild for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Fix docbuild for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Add cosine distance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update BUILD * Reverted file changes for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Clean up correlation distance changes Signed-off-by: richard.north.iii <[email protected]> * Update cosine distance algorithm Signed-off-by: richard.north.iii <[email protected]> * clang-format updates Signed-off-by: richard.north.iii <[email protected]> * update clang format Signed-off-by: richard.north.iii <[email protected]> * Update examples Signed-off-by: richard.north.iii <[email protected]> * Update index.rst Signed-off-by: richard.north.iii <[email protected]> * Update doc error Signed-off-by: richard.north.iii <[email protected]> * Update cosine distance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update clang-format error Signed-off-by: richard.north.iii <[email protected]> * DOC: Replace oneAPI logo with UXL (#3109) * replace oneAPI logo with UXL * correct path * Rolled back bazel to 7.4.1 (#3115) * Rolled back bazel to 7.4.1 * Moved back moddule name to Module.bazel. * Rolled for bazel downgrade. * Reverted changes in MODULE.bazel. * Rolled back all of the bazel changes. * Rolled back changes for bazel 8.0.1 * Changed back the substitution version number. * Add information about ittnotify to INSTALL.md (#3118) * Reduce memory requirements of sparse K-means test (#3117) The test that tests SYCL implementation of sparse K-means on the large number of rows was modified: - It was switched off on CPUs; - On all GPUs except Intel(R) Data Center GPU Max series, the number of rows in the testing data set was reduced 100 times to prevent out of memory crashes. * chore(deps): update dependency editorconfig-checker/editorconfig-checker to v3.2.1 (#3121) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [CI, Enhancement] Refine github actions dpcpp linux make build using AVX512 and debug symbols (#3102) * first try * remove false comment * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update makefile * Update makefile * Update makefile * [CI, enhancement] track oneTBB using renovatebot (#3077) * Update renovate.json * Update tbb.sh * Update tbb.bat * Update renovate.json * Update renovate.json * DOC: Remove broken instructions for building with conda environments on windows (#3125) * don't suggest unavailable dos2unix * remove whole broken instructions for conda+windows * Update correlation distance unit test Signed-off-by: North Iii <[email protected]> * Add clang-format Signed-off-by: richard.north.iii <[email protected]> * Update correlation_distance algorithm Signed-off-by: richard.north.iii <[email protected]> * [CI, enhancement] use -fvisibility=hidden in Linux Make builds (#3080) * Update build.sh * Update ci.yml * Update ci.yml * Update ci.yml * Update build.sh * Update test.sh * Update ci.yml * Update test.sh * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update makefile * Update build.sh * Update ci.yml * Update makefile * Update build.sh * Update makefile * Update makefile * Update makefile * Update INSTALL.md * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * add shell script * missing whitespace * spelling mistake * set file permissions * remove unwanted changes * Update abi_check.sh * Update ci.yml * add ignore file * fix CI and shell * swap back * close bracket * spy into variables * fix if statement * change spy improve regex * add suggestions * Update ci.yml * Update ci.yml * Update daal_defines.h * Update makefile * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update common.hpp * Update makefile * Update makefile * Update makefile * Update ci.yml * Update adaboost_predict_dense_default_batch_fpt_cpu.cpp * Update adaboost_predict_dense_default_batch_fpt_cpu.cpp * Update adaboost_train_dense_default_batch_fpt_cpu.cpp * Update adaboost_predict_dense_default_batch_fpt_cpu.cpp * Update adaboost_train_dense_default_batch_fpt_cpu.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update compute_types.hpp * fix issues with partial, still leaves decision_tree nodes * remove ONEDAL_EXPORT * return change * further return * fix another implicit instantiation * fix node_info instantiation * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * Update adaboost_train_dense_default_batch_fpt_dispatcher.cpp * fix stump too * attempt to define some missing things * move namespaces * Update algorithm_container_base_batch.h * Update algorithm_container_base_batch.h * Update ci.yml * Update uniform_dense_default_batch_fpt_dispatcher.cpp * Update normal_dense_default_batch_fpt_dispatcher.cpp * Update bernoulli_dense_default_batch_fpt_dispatcher.cpp * Update bernoulli_dense_default_batch_fpt_dispatcher.cpp * fix engines * intermediate save * formatting * small fix * next batch * missing newline * fix philox * full covariance fixes * first attempt at linear * Update covariance_distributed.h * attempt to fix windows * fixes for windows and elastic net * fix windows issues again * further windows fixes * svm + svd * further windows fixes * more fixes * kernel function solution * fix implicit_als * linear regression changes * low_order_moments * interim save point * further updates * Update sgd_dense_default_batch_fpt_dispatcher.cpp * pca + fixes * passing first examples * final work * attempt to see if this influences missing symbols * Update coordinate_descent_dense_default_batch_fpt_dispatcher.cpp * fix DAAL_EXPORT * clang fixes part 1 * another round of hell * further fixes for clang * fixes on fixes * further fixes for clang * stopping point * fix mistakes * through linear regression * low order moments * naivebayes * DistributedInput will need to be verified throughout the codebase * passing on clang * return SVM change * working oneapi_c examples * additional fixes for dpc tests * pass editor check * remove python3 * last update? * Update common.hpp * updates * Update common.cpp * Update cpu_info_impl.hpp * Update infer_parameters.hpp * Update common.hpp * Update algorithm_container_base_batch.h * Update pca_online.h * conform to daal ABI * conform to daal ABI * begin fixing spmd * fix spmd exports * make fixes for various compiler warnings * try to solve more spmd/ccl issues * try to fix communicator * Update chunked_array_impl.cpp * add spmd related fixes (continued * fix for spmd * Update ci.yml * Update ci.yml * latest fixes * latest fixes * futher fies * further spmd fixes * fix GNU build * attempt again * remove other PR from this one * remove other PR from this one * Update common.hpp * Upgrade to Bazel 8.1.1 (#3120) * Bazel 8.0.1 changes. * Fixed file name. * fixed llvm-ar name. * updated llvm-ar path. * Use llvm-ar tool if found. * Updated to 8.1.1 * Updated format. * Added lines to license. * Added comment. * chore(deps): update actions/upload-artifact action to v4.6.2 (#3132) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency platforms to v0.0.11 (#3133) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency rules_cc to v0.1.1 (#3134) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update naming and dropping Intel specifics (#3116) * Fix outlier detection assignment operators (rule of three violation, Coverity) (#3122) * Update outlierdetection_bacon.cpp * Update outlierdetection_multivariate.cpp * Update outlier_detection_univariate.cpp * Update outlier_detection_bacon_types.h * Update outlier_detection_multivariate_types.h * Update outlier_detection_univariate_types.h * Update outlier_detection_univariate.cpp * Update outlierdetection_multivariate.cpp * Regression algorithm Coverity fixes (rule-of-three violation) (#3127) * Update regression_training_types.h * Update regression_predict_types.h * Update regression_training_input.cpp * Update regression_prediction_batch.cpp * chore(deps): update dependency uxlfoundation/onetbb to v2022 (#3130) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * feature(rf optimizations): enabling oneDPL and sort primitive refactoring (#3046) * [bug] fix intermittent failures in LinuxMakeDPCPP(AVX512) github action (#3135) * Update makefile.ver * Update makefile.ver * Update makefile.ver * Update makefile.ver * Update makefile.ver * Update makefile * Update makefile.ver * Update makefile.ver * Update makefile.ver * Update makefile.ver * Update makefile (#3136) * [CI, enhancement] enforce ABI checking of linux DPCPP build (#3112) * first try * remove false comment * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * renew * change permissions * Update abi_check.sh * Update makefile * Update abi_check.sh * Update ci.yml * Update .abignore * Update ci.yml * Update ci.yml * Update ci.yml * Update .ci/scripts/abi_check.sh Co-authored-by: Alexander Andreev <[email protected]> --------- Co-authored-by: Alexander Andreev <[email protected]> * remove redundant path (managed by conda) (#3137) * Update abi_check.sh (#3140) * Update CPU/GPU correlation distance algorithms Signed-off-by: richard.north.iii <[email protected]> * Fix rule of three violations in Linear Regression (#3143) Fix rule of three violation in DAAL Linear Regression algorithm and related classes by adding (or marking =delete) the missing copy constructors, copy assignment operators and destructors. * Updated globs for bazel 8.0.0 (#3045) * Removed the noincompatible_disallow_empty_glob flag. * Updated globs. * Allow empty of ccl library. * Removed the empty test. * chore(deps): update dependency uxlfoundation/onetbb to v2022.1.0 (#3145) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [CI, enhancement] set compute runtime GPU settings to improve GPU runner stability (#3146) * Update ci.yml * Update ci.yml * [docs] add missing documentation of gcov support from #3010 (#3147) * Update makefile * Update makefile * Update INSTALL.md * Update INSTALL.md * Update INSTALL.md * Update INSTALL.md * Update makefile * Update INSTALL.md * [fix] add copy assignment operator to quantiles (coverity fix) (#3144) * Update quantiles_types.h * Update quantiles.cpp * [Enhancement] Add float datatype support to rocAucScore (#3074) * Update roc_auc_score.cpp * Update roc_auc_score.cpp * Update roc_auc_score.cpp * Update roc_auc_score.cpp * Update roc_auc_score.h * Update roc_auc_score.cpp * Update roc_auc_score.cpp * clang-formatting and comment * Squashed commit of the following: commit 35e0b62 Author: Faust, Ian <[email protected]> Date: Mon Mar 10 14:56:52 2025 +0100 renew commit 1c9299c Author: Ian Faust <[email protected]> Date: Mon Mar 10 07:24:21 2025 +0100 Update ci.yml commit c6a7b4a Author: Ian Faust <[email protected]> Date: Mon Mar 10 06:44:43 2025 +0100 Update ci.yml commit 18dac73 Author: Ian Faust <[email protected]> Date: Thu Mar 6 01:28:24 2025 +0100 Update ci.yml commit d84c663 Author: Ian Faust <[email protected]> Date: Thu Mar 6 00:53:40 2025 +0100 Update ci.yml commit 91a523d Author: Ian Faust <[email protected]> Date: Thu Mar 6 00:05:02 2025 +0100 Update ci.yml commit a1048df Author: Ian Faust <[email protected]> Date: Wed Mar 5 23:54:30 2025 +0100 Update ci.yml commit b6a84b8 Author: Ian Faust <[email protected]> Date: Wed Mar 5 23:18:26 2025 +0100 Update ci.yml commit d496c2f Author: Ian Faust <[email protected]> Date: Wed Mar 5 22:18:02 2025 +0100 Update ci.yml commit 272fdfe Author: Ian Faust <[email protected]> Date: Wed Mar 5 19:15:04 2025 +0100 Update ci.yml commit 05ee213 Author: Ian Faust <[email protected]> Date: Wed Mar 5 18:35:28 2025 +0100 Update ci.yml commit fd8bf73 Author: Faust, Ian <[email protected]> Date: Wed Mar 5 18:17:24 2025 +0100 remove false comment commit 71c19df Author: Faust, Ian <[email protected]> Date: Wed Mar 5 18:10:59 2025 +0100 first try * chmod * Update abi_check.sh * Update makefile * Update roc_auc_score.cpp * Update roc_auc_score.h * Update roc_auc_score.h * Update roc_auc_score.cpp * add ABI fix * Update abi_check.sh * [CI] Update openBLAS to v0.3.29 (#3076) * Update openblas.sh * Update renovate.json * Remove a blank line in modules (#3150) * [enhancement] Add ability to integrate sanitizers in oneDAL Make builds (#3148) * add first attempt * add second attempt * add lopts * add INSTALL.md change * Update makefile * Update INSTALL.md * Update makefile * Update makefile * Update makefile * Update INSTALL.md * Update makefile * change how its used to open up all sanitizers * change wording * change docs again * additional documenation * additional documenation * another english change * add -fno-omit-frame-pointer * Update INSTALL.md Co-authored-by: david-cortes-intel <[email protected]> * Update makefile Co-authored-by: david-cortes-intel <[email protected]> --------- Co-authored-by: david-cortes-intel <[email protected]> * [bugfix] make makefile.ver work with POSIX shells (#3149) * Update makefile.ver * Update makefile.ver * Update CMake version in examples to 3.5 (#3156) * Remove dispatching to SSE2 code path on non-intel x86 CPUs (#3154) * Remove dispatching to sse2 code path on non-intel x86 CPUs * Update minimal cmake version in daal examples to 3.5 * Update minimal cmake version in onedal examples to 3.5 * Fix decision tree coverity hits (#3124) * Update node_info_impl.hpp * Update common.hpp * Fixing new Coverity hits on KNN (#3152) * minor fix * minor fix * FIX: Fix potential non-definitiness of BFGS approximation (#3158) * fix potential non-definitiness on BFGS approximation * correct in more places * DOC: Remove suggestion to use to daal4py in readme (v2) (#3162) * DOC: Remove reference to daal4py * remove link to daal4py examples * add instructions for DPL on conda (#3163) * [CI] Update Windows BaseKit, Windows OpenCL Runtime installs to 2025.1 (#3164) * [bugfix] make shared-libasan default option for ```REQSAN=address``` (#3160) * Update makefile * fix on local testing * ci: add initial AArch64 CI (#3100) * ci: add initial AArch64 CI - Builds openBLAS and tbb. - Builds oneDAL. - Builds examples with static and dynamic linkage. - Executes on GitHub-Hosted Arm runners. - Executes on Arm-Hosted GitHub runners. Signed-off-by: Hamza Butt <[email protected]> * Updates to conform to other CI infrastructure * place oneDAL in the main workspace directory * Update ci-aarch64.yml * revert change * Update apt.sh * Update ci-aarch64.yml * Update ci-aarch64.yml * Update ci-aarch64.yml * Update ci-aarch64.yml * Update ci-aarch64.yml * Update apt.sh * Update apt.sh * Update ci-aarch64.yml * attempt to simplify and restor to OpenBLAS build * fix mistake in label * remove ninja * attempt to fix issues * force timeout on examples due to enable_thread_pinning * try with tbb * try again * abuse the runner to get possible tbb versions * try again * get keys * check intel-tbb * remove checks for tbb --------- Signed-off-by: Hamza Butt <[email protected]> Co-authored-by: Ian Faust <[email protected]> * chore(deps): update dependency windows to v2025 (#3165) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update lukka/get-cmake action to v4 (#3166) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * parallelize doc build by default (#3169) * chore(deps): update lukka/get-cmake action to v4.0.1 (#3167) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * DOC: Fix broken link to logreg builder (#3168) * fix broken link * fix formatting * Updated OpenCL dependencies for MKL (#3157) * chore(dev): multiple versions of OpenCL lib for bazel * Work around for opencl issues. --------- Co-authored-by: homksei <[email protected]> * chore(deps): update dependency bazel to v8.2.0 (#3172) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update third-party-program files (#3171) * fix: remove dead code (#3174) * fix: static cast uniqueDepth to int to avoid ull underflow (#3173) * fix: static cast uniqueDepth to int to avoid ull underflow * cast uniqueDepth to int and check overflow * extend to uniqueDepthPartialWeights * fixup * chore(deps): update dependency packaging to v25 (#3179) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency bazel to v8.2.1 (#3178) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Use aligned loads and stores where possible in DAAL memory management (#3159) * Revert "fix: static cast uniqueDepth to int to avoid ull underflow (#3173)" (#3181) This reverts commit c17ea8b. * update version to 2025.6.0 (#3177) * [CI] swap uxl self-hosted runners to github hosted (#3176) * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * replace cpath (#3183) * fix(docs): Install dependencies fails over debian-installed packages (#3186) * empty commit to trigger doc build * set -euo pipefail * modification in docs to trigger CI * use venv * apt-get update first * revert change to docs/Makefile, only used to trigger docbuild * feature: onedal verbose profiler (#3155) * Revert "[CI] swap uxl self-hosted runners to github hosted (#3176)" (#3188) This reverts commit dace9fb. * cplus_include_path lnx hotfix (#3189) * remove non-applicable footer, update copyright, fix ignored options (#3184) * Update cordistance_full_impl.i with clang-format Signed-off-by: richard.north.iii <[email protected]> * Update algo/correlation_distance/BUILD Signed-off-by: richard.north.iii <[email protected]> * Use parallel reduce threading primitive in covariance algorithm (#3126) Ads new daal::Reducer interface class that defines the API that have to be implemented in the algorithms to allow the use of reduction primitives based on tbb::parallel_reduce and tbb::parallel_deterministic_reduce. Two new threading primitives were added: threader_reduce implements parallel reduction using dynamic work balancing, static_threader_reduce implements parallel reduction using static work balancing and deterministic reduction. Dense covariance algorithm in oneDAL was modified to use new static_threader_reduce primitive instead of static_threader_for + single thread reduction as it was done previously. tls_data_t structure previously used as a thread local storage for partial results in Covariance algorithm was replaced with CovarianceReduser class which implements the interface of new daal::Reducer to perform parallel reduction. * Fix for copy_without_assign Coverity hits. (#3119) * Fixed the Copy without assign (COPY_WITHOUT_ASSIGN) coverity hit. * chore(deps): update slackapi/slack-github-action action to v2.1.0 (#3193) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency editorconfig-checker/editorconfig-checker to v3.3.0 (#3194) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update actions/setup-python action (#3113) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency snowballstemmer to v3 (#3198) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * suggest CPLUS_INCLUDE_PATH instead of CPATH (#3206) * DOC: Correct installation instructions (#3205) * update installation instructions * Update docs/source/get-started/prerequisites.rst Co-authored-by: Victoriya Fedotova <[email protected]> * Update docs/source/installation.rst Co-authored-by: Victoriya Fedotova <[email protected]> * Update docs/source/installation.rst Co-authored-by: Victoriya Fedotova <[email protected]> * Update docs/source/installation.rst Co-authored-by: Victoriya Fedotova <[email protected]> * use substitutions for library name --------- Co-authored-by: Victoriya Fedotova <[email protected]> * fix misrendered substitution (#3208) * Fix issues with Logistic Regression and Newton-CG (#3196) * Add debug outputs * Add debug outputs * Fix issues in logloss functior, backtracking algorithm, add tests, remove debug outputs * Clang-format * Minor * Update tolerance, lighten the condition for early-stop in line-search * Improve thread safety in dynamic symbols loading on Windows (#3195) Locally declared static variables are used to get rid of unnecessary checks which might be leading to data races previously. * enh: replace unix2dos with sed (#3210) * [enhancement] Compiler hints for arm (#3141) '#pragma omp simd' is added in place of '#pragma GCC ivdep' compiler hint for GNU in aarch64. This compiler hint boosts DBSCAN, Linear Regression and Ridge Regression by ~10%. --------- Co-authored-by: Alexander Andreev <[email protected]> Co-authored-by: Victoriya Fedotova <[email protected]> * DOC: Add instructions for executing examples with ASAN (#3207) * add instructions for executing examples with ASAN * Update INSTALL.md * Add grain_size hyperparameter into Covariance and PCA algorithms (#3197) Following changes were made: - API for setting and getting the grain_size efficiency parameter was added into batch Covariance algorithm - API for setting and getting macro_block and grain_size efficiency parameters was added into PCA algorithm with cov method * [CI, enhancement] Switch an arm github actions runner to clang build (#3213) * Update ci-aarch64.yml * Update ci-aarch64.yml * Update ci-aarch64.yml * Update ci-aarch64.yml * fix: use github action instead of gh-pages (#3212) * fix: use github action instead of gh-pages * fix: fix permissions * fix: move permissin and use eaxt commit id for git actions --------- Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com> * use white logo in dark mode (#3214) * Update cordistance_lp_impl.i * Update cordistance_up_impl.i * Update correlation distance algorithm Signed-off-by: richard.north.iii <[email protected]> * enh: adding mkl_core deps (#3209) * update version to 2025.7.0 (#3218) * fix: coverity fixes for table and backend functions (#3190) * Fix hyperparameterIdCount in Covariance (#3221) * Update clang-format for daal cordistance algorithm Signed-off-by: richard.north.iii <[email protected]> * fix: nightly win issue with profiler(main) (#3222) * Update cordistance algo Signed-off-by: richard.north.iii <[email protected]> * Update clang-format for cordistance algo Signed-off-by: richard.north.iii <[email protected]> * Update clang-format Signed-off-by: richard.north.iii <[email protected]> * Update cordistance_dense_default_batch_fpt_cpu.cpp * Don't need DisableScratchPages with newer version of gpu driver. (#3185) * chore(deps): update lukka/get-cmake action to v4.0.2 (#3215) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Downgrade nightly build to support 22.04 sklearnex testing (#3052) * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * Update nightly-build.yml * [CI] Update apt Linux MKL and DPCPP 2025.1, TBB 2022.1 (#3225) * Update apt.sh * Update apt.sh * Update .abignore * Update correlation distance algo Signed-off-by: richard.north.iii <[email protected]> * Update examples/oneapi Signed-off-by: richard.north.iii <[email protected]> * DOC: Mention oneAPI examples in the build instructions (#3231) * mention oneAPI examples in addition to DAAL * mention DPC examples too * clarify that examples run on sycl devices * Fix: xcsrmv computation wrong arguments (#3236) * print correct version of linter being used (#3233) * Update cordistance_full_impl.i Signed-off-by: richard.north.iii <[email protected]> * Add docs for correlation_distance Signed-off-by: richard.north.iii <[email protected]> * Update clang-format for cordistance_full_impl.i Signed-off-by: richard.north.iii <[email protected]> * Add API doc for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Fix docbuild for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Fix docbuild for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Add cosine distance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update BUILD * Reverted file changes for correlation distance Signed-off-by: richard.north.iii <[email protected]> * Clean up correlation distance changes Signed-off-by: richard.north.iii <[email protected]> * Update cosine distance algorithm Signed-off-by: richard.north.iii <[email protected]> * clang-format updates Signed-off-by: richard.north.iii <[email protected]> * DOC: Better description of what oneDAL does (#3245) * better description * Update README.md Co-authored-by: Victoriya Fedotova <[email protected]> * Update README.md Co-authored-by: Victoriya Fedotova <[email protected]> --------- Co-authored-by: Victoriya Fedotova <[email protected]> * update links (#3244) * Fix architecture typos (#3249) * [CI] prevent some CI jobs from running on forks by default (#3247) * Update ci-aarch64.yml * Update docker-validation-ci.yml * Updated version of CODEOWNERS (#3250) * First iteration of codeowners * Update .github/CODEOWNERS Co-authored-by: ethanglaser <[email protected]> * Update .github/CODEOWNERS Co-authored-by: Anatoly Volkov <[email protected]> --------- Co-authored-by: ethanglaser <[email protected]> Co-authored-by: Anatoly Volkov <[email protected]> * chore(deps): update ubuntu:24.04 docker digest to b59d215 (#3239) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update ossf/scorecard-action action to v2.4.2 (#3238) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency platforms to v1 (#3229) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [CI] remove arm cross compile azure pipelines runners (#3241) * remove arm cross compile azp runners * Delete .ci/env/arm-clang-crosscompile-toolchain.cmake * Delete .ci/env/arm-gnu-crosscompile-toolchain.cmake * chore(deps): update dependency requests to v2.32.4 [security] (#3251) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * remove note about MKL FKP (#3252) * [CI, enhancement] add windows sklearnex azure pipelines template (#3216) * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * Update ci.yml * ci: add `cve-bin-tool` scan to nightly build (#3204) * ci: add `cve-bin-tool` scan to nightly build * Update .github/workflows/nightly-build.yml Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Nikolay Petrov <[email protected]> Co-authored-by: Copilot <[email protected]> * chore(deps): update lukka/get-cmake action to v4.0.3 (#3259) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * reset pointer (#3257) * update clang format Signed-off-by: richard.north.iii <[email protected]> * Update examples Signed-off-by: richard.north.iii <[email protected]> * Update index.rst Signed-off-by: richard.north.iii <[email protected]> * chore(deps): update dependency urllib3 to v2.5.0 [security] (#3265) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [bugfix] separate stability issues of ```intel/cve-bin-tool-action``` from nightly-build (#3267) * Create nightly-test.yml * Update nightly-test.yml * Update nightly-build.yml * Update nightly-test.yml * Update nightly-test.yml * Update nightly-test.yml * Update nightly-build.yml * Fix Coverity hits in headers (#3192) Following changes were made to fix Coverity hits: - daal::services::internal::Buffer and related APIs were deprecated, as it was only useful in DAAL SYCL APIs that were removed in version 2025.0; - Rule of 3 was fixed in daal::services::Atomic, daal::services::ErrorCollection and daal::services::HostAppIface; - Potential self assignment issue was fixed in SharedPtr. * chore(deps): update doc packages (#3240) * chore(deps): update doc packages * try downgrading sphinx-book-theme * revise sphinx_prompt in extensions --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: ethanglaser <[email protected]> Co-authored-by: ethanglaser <[email protected]> * chore(deps): update dependency bazel to v8.3.0 (#3268) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * ENH: Non-batched route for high-dimensional covariance (#3230) * MAINT: Reuse common routines in covariance (#3227) * reuse common routines * linter * fix sizes for memcpy, restore aligned pragmas * fix * revert missing change * chore(deps): update dependency lxml to v6 (#3272) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * CI jobs for OpenRNG-OpenBLAS-Arm build (#2972) * Added ci jobs for openrng openblas arm build * Modified the build script to enable cross compile * Fixed static link build with openrng backend * Extended exclude list for openrng build Signed-off-by: Dhanus M Lal <[email protected]> * excluded more examples Signed-off-by: Dhanus M Lal <[email protected]> * revert: externals/service_openrng.h Changes made to service_openrng.h by commit ed04659 breaks the build because the header service_stat_rng_ref.h is not included and the macros present in this file are not compatible with the openrng interface. Signed-off-by: Dhanus M Lal <[email protected]> * CI: moved openrng jobs to github workflows Signed-off-by: Dhanus M Lal <[email protected]> * CI testing: always pass daal/cpp tests Signed-off-by: Dhanus M Lal <[email protected]> * CI arm64-openrng-openblas: set oneapi/cpp examples to always pass Signed-off-by: Dhanus M Lal <[email protected]> * typo: changed job name; added comments Signed-off-by: Dhanus M Lal <[email protected]> * refactor: address review comments Got rid of unnecessary attributes in class BaseRNG. Signed-off-by: Dhanus M Lal <[email protected]> * documentation: address review comments Signed-off-by: Dhanus M Lal <[email protected]> --------- Signed-off-by: Dhanus M Lal <[email protected]> Co-authored-by: Nikolay Petrov <[email protected]> * [bug] fix shared library AddressSanitizer integration for the gnu compiler (#3256) * Update icx.mkl.32e.mk * Update vc.mkl.32e.mk * Update gnu.mk * Update dpcpp.mk * Update clang.mk * Update makefile * chore(deps): update lukka/get-cmake action to v4.0.3 (#3273) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * adding local_trees mode in RF (#3139) * add bazel support for daal examples (#3235) * chore(deps): update dependency bazel to v8.3.1 (#3277) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency uxlfoundation/onetbb to v2022.2.0 (#3276) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Alexander Andreev <[email protected]> * Add correlation distance algorithm (#3131) * Adds correlation distance algorithm, examples, docs, and unit tests Signed-off-by: North Iii <[email protected]> Co-authored-by: Victoriya Fedotova <[email protected]> * Update doc error Signed-off-by: richard.north.iii <[email protected]> * Update cosine distance algorithm Signed-off-by: richard.north.iii <[email protected]> * Update clang-format error Signed-off-by: richard.north.iii <[email protected]> * add instructions for clang (#3279) * chore(deps): update dependency bazel_skylib to v1.8.0 (#3278) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency rules_cc to v0.1.2 (#3263) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update bazel dependency handling for Catch2 and fmt (#3280) * FIX: Initialize pointer to null in topology struct (#3281) * set init pointer to null * disambiguate references to global object and instantiated object * Update ci.yml (#3284) * update version to 2025.8.0 (#3282) * remove bazel http_archive rule (#3286) * [ENH] Read Support for CSR tables in oneapi (#3270) * Added SVE intrinsics for postGemmPart function (#3271) * Added SVE intrinsics for postGemm function * Removed SVE implementation of float exponential due to accuracy issues * Format code using clang-format --------- Co-authored-by: shubham.chaudhari <[email protected]> * Update index.rst --------- Signed-off-by: North Iii <[email protected]> Signed-off-by: richard.north.iii <[email protected]> Signed-off-by: richard.north.iii <[email protected]> Signed-off-by: richard.north.iii <[email protected]> Signed-off-by: Hamza Butt <[email protected]> Signed-off-by: Dhanus M Lal <[email protected]> Co-authored-by: richard.north.iii <[email protected]> Co-authored-by: richard.north.iii <[email protected]> Co-authored-by: richard.north.iii <[email protected]> Co-authored-by: david-cortes-intel <[email protected]> Co-authored-by: kmcgrie <[email protected]> Co-authored-by: Victoriya Fedotova <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Ian Faust <[email protected]> Co-authored-by: Nikolay Petrov <[email protected]> Co-authored-by: Aleksandr Solovev <[email protected]> Co-authored-by: Alexander Andreev <[email protected]> Co-authored-by: RakshithGB <[email protected]> Co-authored-by: Maria Petrova <[email protected]> Co-authored-by: Hamza <[email protected]> Co-authored-by: homksei <[email protected]> Co-authored-by: Andreas Huber <[email protected]> Co-authored-by: Kamil Jackiewicz <[email protected]> Co-authored-by: Victoriya Fedotova <[email protected]> Co-authored-by: Anatoly Volkov <[email protected]> Co-authored-by: yuejiaointel <[email protected]> Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: ethanglaser <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: ethanglaser <[email protected]> Co-authored-by: Dhanus M Lal <[email protected]> Co-authored-by: Khalil <[email protected]> Co-authored-by: shubhamsvc <[email protected]> Co-authored-by: shubham.chaudhari <[email protected]>
Description
This PR uses
__attribute__ ((visibility ("default")))https://gcc.gnu.org/wiki/Visibility to make the oneDAL dynamic libraries in Linux follow Windows in having default visibility of symbols as hidden. This will make the Linux ABI explicit, and make possible improvements in use of the oneDAL library as described in the linked article. In general, most problems are related to implicit Instantiation, which doesn't respect the added attribute for visibility.NOTE: Building with Bazel still maintains standard linux visibility, the change in visibility only occurs in Make builds.
The changes focus in several overarching themes:
DAAL Implicit Instantiation of the AlgorithmDispatchContainer is removed by moving all Batch, Online and Distributed object constructors from the class definition to individual definitions in the *_dispatch.cpp files. This follows convention set by some algorithms: KNN, stump, etc. Follow up work is recommended to solve public exposition and requirement of an internal namespace in SVM which violates convention.
DAAL Linear model interpolation cannot follow the convention in 1) due to certain re-use of the AlgorithmDispatchContainer, instead the
initializemethod is moved to *_dispatch.cpp for Linear Regression, Logistic Regression, Lasso, Ridge, and ElasticNet. A note is made on this change in the codebase, as it is a special case of a reuse of another algorithms AlgorithmDispatchContainer.DAAL RNG engine instantion of the object is moved from a general
.cppfile to just thecreatefunction, and the other instantiation of the contructors are done following the methodology laid out in 1)DAAL An outlier file in Expectation Maximization is removed (which is the implementation of 1) in the wrong file and follows now convention set out in 1) using
*_dispatcher.cpp.DAAL Macro definition is changed for AlgorithmDispatchContainer to guarantee that it is exported for various architectures for Batch computations.
General changes to the makefile and DAAL_EXPORT/ ONEDAL_EXPORT are made to generalize and activate the default hidden visibility. Subtle logic changes are made
oneDAL ONEDAL_EXPORT is added in several places where the logic differs between windows use of
__declspec(dllexport)and__attribute__ ((visibility ("default"))), which varies by compiler and OS to make it valid for all circumstances.oneDAL some implicit instantiations are fixed. (See incremental algorithms)
oneDAL ONEDAL_EXPORT for spmd is properly addressed which should make it possible to use with windows OS (not that this is recommended)
Note: this PR has been additionally tested against a main sklearnex build to verify conformance.
This reduces the dynamic table length in Linux by 10x in DAAL, and x5 overall.
PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.
You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).
Checklist to comply with before moving PR from draft:
PR completeness and readability
Testing
Performance