Link ATLAS/MKL to an installed Numpy

Question

TL;DR how to link ATLAS/MKL to existing Numpy without rebuilding.

I have used Numpy to calculate with the large matrix and I found that it is very slow because Numpy only use 1 core to do calculation. After doing a lot of search I figure that my Numpy does not link to some optimized library like ATLAS/MKL. Here is my config of numpy:

>>>import numpy as np
>>>np.__config__.show()
blas_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
lapack_info:
    libraries = ['lapack']
    library_dirs = ['/usr/lib']
    language = f77
atlas_threads_info:
    NOT AVAILABLE
blas_opt_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

For this reason, I want to link ATLAS/MKL to Numpy. However, my Numpy is installed from PIP so I don't want to install manually because I want to use the latest version. I have done some search but they are only for building from scratch. For this reason, my question are:

• Are there any way to link ATLAS/MKL to Numpy without rebuilding again?
• I have found that the config info is saved in _config_.py in the installed folder of Numpy. So will modifying it solve my problem? If yes, would you please show me how?

Answer 1

Assuming you're running some flavour of linux, here's one way you could do it:

1. Find out what BLAS library numpy is currently linked against using ldd.

   • For versions of numpy older than v1.10:

     $ ldd /<path_to_site-packages>/numpy/core/_dotblas.so
     

     For example, if I install numpy via apt-get, it links to

     ...
     libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fed81de8000)
     ...
     

     If _dotblas.so doesn't exist, this probably means that numpy failed to detect any BLAS libraries when it was originally installed, in which case it simply doesn't build any of the BLAS-dependent components. This often happens if you install numpy using pip without manually specifying a BLAS library (see below). I'm afraid you'll have no option but to rebuild numpy if you want to link against an external BLAS library.

   ---

   • For numpy v1.10 and newer:

     _dotblas.so has been removed from recent versions of numpy, but you should be able to check the dependencies of multiarray.so instead:

     $ ldd /<path_to_site-packages>/numpy/core/multiarray.so
     

2. Install ATLAS/MKL/OpenBLAS if you haven't already. By the way, I would definitely recommend OpenBLAS over ATLAS - take a look at this answer (although the benchmarking data is now probably a bit out of date).

3. Use update-alternatives to create a symlink to the new BLAS library of your choice. For example, if you installed libopenblas.so into /opt/OpenBLAS/lib, you would do:

   $ sudo update-alternatives --install /usr/lib/libblas.so.3 \
                                        libblas.so.3 \
                                        /opt/OpenBLAS/lib/libopenblas.so \
                                        50
   

   You can have multiple symlinks configured for a single target library, allowing you to manually switch between multiple installed BLAS libraries.

   For example, when I call $ sudo update-alternatives --config libblas.so.3, I can choose between one of 3 libraries:

     Selection    Path                                    Priority   Status
   ------------------------------------------------------------
     0            /opt/OpenBLAS/lib/libopenblas.so         40        auto mode
     1            /opt/OpenBLAS/lib/libopenblas.so         40        manual mode
     2            /usr/lib/atlas-base/atlas/libblas.so.3   35        manual mode
   * 3            /usr/lib/libblas/libblas.so.3            10        manual mode
   

If you really want the "newest" version of numpy, you could also take a look at my answer on compiling numpy from source with OpenBLAS integration.

Installing numpy with BLAS support using pip

As @tndoan mentioned in the comments, it's possible to make pip respect a particular configuration for numpy by placing a config file in ~/.numpy-site.cfg - see this answer for more details.

My personal preference is to configure and build numpy by hand. It's not particularly difficult, and it gives you better control over numpy's configuration.

Answer 2

The answer depends on how NumPy was built initially. If it was built against BLAS and LAPACK, then at least there is no way to force numpy.dot to use ATLAS/MKL later without rebuilding. Other functions do not use numpy.dot and you can use update-alternatives to change the targets of the symlinks libblas.so.3 and liblapack.so.3. This is because numpy.dot requires ATLAS styled CBLAS, or OpenBLAS/MKL, but not the BLAS/CBLAS and LAPACK from netlib.

I'm using openSUSE and I've installed the standard cblas-devel from netlib. However it seems just impossible to force NumPy to use the shipped cblas/cblas-devel. That is, if you built NumPy against netlib BLAS/LAPACK/CBLAS (as the official package), then _dotblas.so(which provides the BLAS version of numpy.dot) cannot be built (before 1.10), or multiarray.so(1.10 and later) does not link to libblas.so.3 at all. See the issue on github: https://github.com/numpy/numpy/issues/1265 and the cited Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464784. Maybe someone can dive into the source code to make a patch...Anyway, it's just one function that is affected (numpy.dot) and you can always rebuild the whole NumPy easily using the faster OpenBLAS now, so probably no big deal after all.

Conclusion: You can link to ATLAS/MKL/OpenBLAS later without rebuilding, but numpy.dot will still be extremely slow if NumPy was not built against ATLAS/MKL/OpenBLAS initially (because numpy.dot simply didn't use any BLAS in the first place and you can do nothing about that once the compiling was done).

Update: Actually you can force numpy to build _dotblas.so. I've made a patch for numpy-1.9.2:

diff -Npru numpy-1.9.2.orig/numpy/core/setup.py numpy-1.9.2/numpy/core/setup.py
--- numpy-1.9.2.orig/numpy/core/setup.py        2015-02-01 11:38:25.000000000 -0500
+++ numpy-1.9.2/numpy/core/setup.py     2016-03-28 01:31:12.948885383 -0400
@@ -953,8 +953,8 @@ def configuration(parent_package='',top_
     #blas_info = {}
     def get_dotblas_sources(ext, build_dir):
         if blas_info:
-            if ('NO_ATLAS_INFO', 1) in blas_info.get('define_macros', []):
-                return None # dotblas needs ATLAS, Fortran compiled blas will not be sufficient.
+            #if ('NO_ATLAS_INFO', 1) in blas_info.get('define_macros', []):
+            #    return None # dotblas needs ATLAS, Fortran compiled blas will not be sufficient.
             return ext.depends[:3]
         return None # no extension module will be built

Now that _dotblas.so is linked to libblas.so.3, you can use update-alternatives to test the difference.