Skip to content

ENH: build fallback lapack_lite with 64-bit integers on 64-bit platforms#15218

Merged
charris merged 5 commits intonumpy:masterfrom
pv:lapack-lite-64
Jan 2, 2020
Merged

ENH: build fallback lapack_lite with 64-bit integers on 64-bit platforms#15218
charris merged 5 commits intonumpy:masterfrom
pv:lapack-lite-64

Conversation

@pv
Copy link
Member

@pv pv commented Jan 1, 2020

Build the lapack fallback library (used when no LAPACK installed) with 64-bit integer size when building on a 64-bit platform.

Add an indicator flag in numpy.linalg.lapack_lite showing whether it was built with 64-bit integers, and use that in the tests (instead of checking whether numpy was linked with external 64-bit BLAS).

Closes: gh-5906

Note that the 64-bit mode is turned on unconditionally, regardless of the NPY_USE_BLAS_ILP64 setting. I think this is generally OK --- this stuff is fully internal to Numpy, and the change is not visible outside, and the environment flag is anyway only used as a way for the user to control which BLAS library Numpy links against.

NB. the one s_rnge is unused function (error reporting for f2c bounds checking, which is turned off here. Just corrected its prototype here to match f2c.c.).

@charris charris added 01 - Enhancement 09 - Backport-Candidate PRs tagged should be backported labels Jan 1, 2020
@charris charris added this to the 1.18.1 release. milestone Jan 1, 2020
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Jan 1, 2020
@charris charris removed this from the 1.18.1 release. milestone Jan 1, 2020
@charris
Copy link
Member

charris commented Jan 1, 2020

Hmm, I originally marked this for backport but changed my mind as it might lead to confusion going forward to change this in 1.18.1. It is probably a bit riskier as well. @pv If you would like to see this in 1.18 let me know. It would be good if the version used was available in np.__config__.show().

Make numpy.__config__.show() to print information about the fallback
lapack_lite library when it is used.
@pv pv force-pushed the lapack-lite-64 branch from dc4d14f to df5ba1f Compare January 1, 2020 20:35
@pv
Copy link
Member Author

pv commented Jan 1, 2020

I agree this doesn't need backport. Added the information to numpy.__config__.

typedef struct { real r, i; } complex;
typedef struct { doublereal r, i; } doublecomplex;
typedef int logical;
typedef CBLAS_INT logical;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little surprised this is the case, but I assume it comes down to the definitions of functions from cblas?

Copy link
Member Author

@pv pv Jan 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

npy_cblas.h does not contain functions that have logical arguments, and numpy.linalg does not use such functions either. (LAPACK does however contain such functions, but the point is probably moot.)

I think there is no standard for size of logical in Fortran. For gfortran, -fdefault-integer-8 implies also LOGICAL is 8 bytes, and I think that's also true for ifort.

The "canonical" C API LAPACKE defines lapack_logical as lapack_int, also in its ILP64 configuration, https://github.com/Reference-LAPACK/lapack/blob/f24797e40e44c6d98ea7fa4e36bd1ec6c02c95e5/LAPACKE/include/lapacke_config.h#L46-L56 so I think precedent says it should be like that also here (even though it generally does not matter as the ABI is not exposed).

extern integer s_rdfe(cilist *);
extern integer s_rdue(cilist *);
extern integer s_rnge(char *, integer, char *, integer);
extern int s_rnge(char *, int, char *, int);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea what defines this function? Perhaps the definition should change. (f2c.c?)

Copy link
Member Author

@pv pv Jan 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The definition is like this in f2c.c (i.e. int instead of integer), and the function is unused.

@charris
Copy link
Member

charris commented Jan 1, 2020

A release note seems appropriate for this.

@charris charris merged commit e803e1f into numpy:master Jan 2, 2020
@charris
Copy link
Member

charris commented Jan 2, 2020

Thanks Pauli.

@pv
Copy link
Member Author

pv commented Jan 2, 2020

I realize unfortunately only now the situation is here the same as with openblas64 without symbol suffix, so corner-case stuff like LD_PRELOAD=/usr/lib64/libopenblas.so python3 -mpytest --pyarg numpy will segfault in this configuration due to the name clashes. I'll do follow-up that renames the symbols too...

Numpy could also consider using symbol hiding on Linux (which I think would also fix this), along e.g. the lines Scipy has been using for many years now:
https://github.com/scipy/scipy/blob/47625f4d5c5dccfe1e678509af64aecf56869717/setup.py#L228-L269

Probably needs finding out first if there's some common software that uses symbols directly from the *.so files instead of getting them via the array API pointer arrays.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lapack_lite should use 64-bit integer indices

3 participants