ENH: Add an "opt-in" feature in NumPy for CPU features, NPY_ENABLE_CPU_FEATURES

Proposed new feature or change:

This is a "mirror image" feature of the NPY_DISABLE_CPU_FEATURES SIMD flag, which allows users to opt out of certain CPU features.

This new feature would add a second environment variable, NPY_ENABLE_CPU_FEATURES, which functions in the opposite way: NumPy would default to the baseline features it is compiled with, and then it will check any dispatch options listed in NPY_ENABLE_CPU_FEATURES and, if they are detected, enable them. This creates an "opt-in" deviation from baseline.

The motivation for this comes from wanting to enable a homogeneous set of CPU features in NumPy across a heterogeneous set of computer architectures; e.g., some machines support AVX512 and some do not. Setting NPY_DISABLE_CPU_FEATURES can work, but gets painful to set many different times in many different configurations to address the hardware variety. Instead, setting NPY_ENABLE_CPU_FEATURES once and identically for every machine would dramatically reduce overhead.

For the problem of having both NPY_DISABLE_* and NPY_ENABLE_* set: it could simply throw an error. Otherwise, if NPY_DISABLE_* is set, all the listed dispatch features become opt-out; if NPY_ENABLE_* is set, all the listed dispatch features become opt-in.

@seiko2plus

This makes sense to me I think, it could help with debugging/robustness and perhaps building distributable binaries in environments where the CPU/platform config is not so easy to control. @seiko2plus would you agree?

To ensure clarity, the environment variable NPY_DISABLE_CPU_FEATURES, it only disables the dispatched features during runtime without the need to rebuild, this process is inexpensive because the process of detecting and disabling CPU features occurs once during the importing of the NumPy module.

When do I need to disable the dispatched CPU features during runtime?

Accuracy or performance tests
Reservations about certain dispatched features. for example, some users may prefer to disable AVX512 features due to frequency scaling issues caused by heavy fp instructions in earlier CPUs(excludes Xeon phi). see :
*. https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/
*.https://lemire.me/blog/2018/09/04/per-core-frequency-scaling-and-avx-512-an-experiment/
Tracing bugs, or as a workaround.

The motivation for this comes from wanting to enable a homogeneous set of CPU features in NumPy across a heterogeneous set of computer architectures; e.g., some machines support AVX512 and some do not.

Do you want to say that you're looking for a way to disable AVX512 even if it doesn't support by the running machine?

Have you tried just to suppress RuntimeWarning?

import os, warnings
os.environ['NPY_DISABLE_CPU_FEATURES'] = 'XXXX'
warnings.simplefilter("ignore", RuntimeWarning)
import numpy as np
warnings.resetwarnings()

Setting NPY_DISABLE_CPU_FEATURES can work, but gets painful to set many different times in many different configurations to address the hardware variety.

But NPY_ENABLE_CPU_FEATURES isn't gonna change the situation here that much, what if you trying to only enable a CPU feature that isn't supported by the running machine? you will still receive RuntimeWarning!

Instead, setting NPY_ENABLE_CPU_FEATURES once and identically for every machine would dramatically reduce overhead.

I'm sorry I couldn't understand this part, could you explain more what you mean?

Accuracy or performance tests

That is far and away the major motivator in this feature request. There's also some interest in bit-for-bit reproducibility, but since there's explicitly no such guarantee in NumPy, that's a discussion that's out of scope. Instead, including this new feature provides an easy way to achieve bit-for-bit reproducibility in NumPy across heterogeneous architectures without needing the explicit guarantee, just by using the baseline CPU features.

Do you want to say that you're looking for a way to disable AVX512 even if it doesn't support by the running machine?

No, just that, with sufficient hardware heterogeneity, it's easier to specify which features should be enabled than disabled, especially when it comes to designing and reproducibly disseminating accuracy and performance tests.

But NPY_ENABLE_CPU_FEATURES isn't gonna change the situation here that much, what if you trying to only enable a CPU feature that isn't supported by the running machine? you will still receive RuntimeWarning!

True! But in this case, we're trying to build up a common set of CPU features starting from baselines (i.e., "opt in"), rather than by iteratively disabling dispatch features machine-by-machine to achieve the same common set (i.e., "opt out"; the current NPY_DISABLE_* approach).

The warning (or error, or however NumPy responds to a mis-configuration) isn't the issue; rather, it's a symptom: the issue is the up-front config time that's needed just to get a common set of CPU features set across a wide range of machine hardware.

This feature is generally accepted, but I fear that it may be misused. For example, if the AVX features are enabled without the non-baseline SSE features e.g. sse41, it will affect the performance of some kernels that only dispatch SSE. see

numpy/numpy/core/src/umath/loops_unary_fp.dispatch.c.src

Lines 1 to 14 in 65eb581

    
           /*@targets 
        
            ** $maxopt baseline 
        
            ** sse2 sse41 
        
            ** vsx2 
        
            ** neon asimd 
        
            ** vx vxe 
        
            **/ 
        
           /** 
        
            * Force use SSE only on x86, even if AVX2 or AVX512F are enabled 
        
            * through the baseline, since scatter(AVX512F) and gather very costly 
        
            * to handle non-contiguous memory access comparing with SSE for 
        
            * such small operations that this file covers. 
        
           */ 
        
           #define NPY_SIMD_FORCE_128

There was no performance gain by enabling AVX2 or AVX512F for rint, floor, ceil, trunc, sqrt, absolute, square, reciprocal actually the other way around we lost performance on non-contiguous memory access.

For example, if the AVX features are enabled without the non-baseline SSE features e.g. sse41

Not sure if it was the intent to allow such a thing like enabling AVX but not SSE - but that should be forbidden I think. A baseline should be "all instruction sets up to a certain level". So if you say AVX2 is the baseline, it should include AVX and SSE*.

This seems like specifics that would, regardless, benefit from being documented somewhere.

Navigation Menu

Repository navigation

ENH: Add an "opt-in" feature in NumPy for CPU features, `NPY_ENABLE_CPU_FEATURES` #22058

Proposed new feature or change:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	/*@targets
	** $maxopt baseline
	** sse2 sse41
	** vsx2
	** neon asimd
	** vx vxe
	**/
	/**
	* Force use SSE only on x86, even if AVX2 or AVX512F are enabled
	* through the baseline, since scatter(AVX512F) and gather very costly
	* to handle non-contiguous memory access comparing with SSE for
	* such small operations that this file covers.
	*/
	#define NPY_SIMD_FORCE_128

ENH: Add an "opt-in" feature in NumPy for CPU features, NPY_ENABLE_CPU_FEATURES #22058

Description

Proposed new feature or change:

Activity

rgommers commented on Jul 29, 2022

seiko2plus commented on Aug 18, 2022

magsol commented on Sep 6, 2022

seiko2plus commented on Sep 19, 2022

rgommers commented on Sep 20, 2022

magsol commented on Sep 22, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

ENH: Add an "opt-in" feature in NumPy for CPU features, `NPY_ENABLE_CPU_FEATURES` #22058