biovault_bfloat16

A bfloat16 implementation for projects of BioVault (Biomedical Visual Analytics Unit LUMC - TU Delft)

Originally based upon dnnl::impl::bfloat16_t from the Deep Neural Network Library (DNNL) of Intel Corporation. Updated to version 1.7 of their library, which has been renamed to oneAPI Deep Neural Network Library (oneDNN).

• https://github.com/oneapi-src/oneDNN/blob/v1.7/src/cpu/bfloat16.cpp
• https://github.com/oneapi-src/oneDNN/blob/v1.7/src/common/bfloat16.hpp

Other consulted implementations: tensorflow::bfloat16 and Eigen::bfloat16

References:

• Intel®, BFLOAT16 – Hardware Numerics Definition", White Paper, November 2018, Revision 1.0 Document Number: 338302-001US https://software.intel.com/en-us/download/bfloat16-hardware-numerics-definition

• Wikipedia bfloat16 floating-point format (The consulted revision was last edited at 20:39, 18 May 2020)

• John D. Cook, 15 November 2018, Comparing bfloat16 range and precision to other 16-bit numbers

• Shibo Wang, Pankaj Kanwar, August 23, 2019 BFloat16: The secret to high performance on Cloud TPUs

• Intel®, March 2020, Architecture Instruction Set Extensions and Future Features, Programming Reference

Reported related issues:

• oneDNN pull request #646, Feb 3, 2020: Avoid undefined behavior (UB) bfloat16_t by removing type-pun via union (Closed, fixed by Eugene Chereshnev (@echeresh), commit ff67087, Feb 4, 2020: all: introduce and use utils::bit_cast() for safe type punning)

• oneDNN pull request #649, Feb 10, 2020: Prevent constructing signaling NaNs and denormals (subnormal floats) by bfloat16_t (Closed with unmerged commit)

• oneDNN pull request #780, Jul 17, 2020: bfloat16: add operator+=(a) overload for float as argument type (Merged: common: bfloat16: replace operator+=(bfloat16_t) by operator+=(float), and gtests: internals: add bfloat16_t tests, committed by Evarist M Fomenko (@emfomenk))

• oneDNN pull request #790, Jul 25, 2020: all: bfloat16: initialize variables by T{} instead of integer constant 0 (Closed with unmerged commits)

• oneDNN pull request #801, Aug 10, 2020: bfloat16: faster conversion and assignment from integer types (Merged by Evarist M Fomenko (@emfomenk): commit 073e346

• TensorFlow issue #36514, Feb 6, 2020: bfloat16 does not flush denormals (subnormal floats) to zero (Closed, resolved by @tensorflower-gardener, commit b04c4e0, Mar 20, 2020: Flush denormals to +/- 0 when converting float to bfloat16)

• TensorFlow pull request #41070, Jul 4, 2020: Avoid undefined behavior by union type punning in round_to_bfloat16 (Closed with unmerged commits)

• Eigen merge request !163, Jul 11, 2020: Allow implicit conversion from bfloat16 to float and double (Merged: commit 4ab32e2d)

• Eigen merge request !164, Jul 14, 2020: Avoid undefined behavior by union type punning in float_to_bfloat16_rtne (Merged: commit b11f817b)

• Eigen merge request !166, Jul 14, 2020: Faster conversion from integer types to bfloat16 (Merged: commit 0e1a33a4)

• Eigen issue #1954, Jul 24, 2020: Disagreement Eigen::bfloat16 and Intel on conversion from NaN to bfloat16

• Visual C++ problem 903305, Feb 2, 2020: Signaling NaN (float, double) becomes quiet NaN when returned from function (both x86 and x64)