Contents

Installing Psi4 from source

Navigation:

Installing Psi4 from source

Steps:

(1) Configuration commands:

cmake -H. -B/home/alston/Psi4/git/master/build-pepito \
        -DCMAKE_INSTALL_PATH="/home/alston/psi4conda/master" \
        -DCMAKE_INSTALL_PREFIX="/home/alston/psi4conda/master/install" \
        -DCMAKE_C_COMPILER=gcc \
        -DCMAKE_Fortran_COMPILER=gfortran \
        -DPYTHON_EXECUTABLE="/usr/bin/python2.7" \
        -DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
        -DPYTHON_INCLUDE_DIR="/usr/include/python2.7/numpy" \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON

On spasrv12 I used:

do-configure-icc-spasrv12.bash

export CMAKE_ROOT=/home/alston/tmp/cmake-3.4.0-Linux-x86_64/
export MATH_ROOT=/opt/ajm/
export MKL_ROOT=/opt/ajm/intel/current/mkl/lib/intel64/

PYTHON_VER=2.7

#       -DBLAS_TYPE="/opt/ajm/lib/libopenblas.so" \
#       -DLAPACK_TYPE="/opt/ajm/lib/libopenblas.so" \

$CMAKE_ROOT/bin/cmake -H. -B/home/alston/Psi4/git/master/build-spasrv12-2-icc \
        -DCMAKE_INSTALL_PATH="/opt/ajm/Psi4/1.2-2" \
        -DCMAKE_INSTALL_PREFIX="/opt/ajm/Psi4/1.2-2/install-icc" \
        -DCMAKE_C_COMPILER=icc \
        -DCMAKE_Fortran_COMPILER=ifort \
        -DPYTHON_EXECUTABLE="/usr/bin/python"${PYTHON_VER} \
        -DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
        -DPYTHON_INCLUDE_DIR="/usr/include/python"${PYTHON_VER}"/numpy" \
        -DBLAS_TYPE=OPENBLAS \
        -DLAPACK_TYPE=OPENBLAS \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON

and for gcc:

do-configure-gcc-spasrv12.bash

export CMAKE_ROOT=/home/alston/tmp/cmake-3.4.0-Linux-x86_64/
export MATH_ROOT=/opt/ajm/
export MKL_ROOT=/opt/ajm/intel/current/mkl/lib/intel64/

PYTHON_VER=2.7

#       -DBLAS_TYPE="/opt/ajm/lib/libopenblas.so" \
#       -DLAPACK_TYPE="/opt/ajm/lib/libopenblas.so" \

$CMAKE_ROOT/bin/cmake -H. -B/home/alston/Psi4/git/master/build-spasrv12-2-gcc \
        -DCMAKE_INSTALL_PATH="/opt/ajm/Psi4/1.2-2" \
        -DCMAKE_INSTALL_PREFIX="/opt/ajm/Psi4/1.2-2/install-gcc" \
        -DCMAKE_C_COMPILER=gcc \
        -DCMAKE_Fortran_COMPILER=gfortran \
        -DPYTHON_EXECUTABLE="/usr/bin/python"${PYTHON_VER} \
        -DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
        -DPYTHON_INCLUDE_DIR="/usr/include/python"${PYTHON_VER}"/numpy" \
        -DBLAS_TYPE=OPENBLAS \
        -DLAPACK_TYPE=OPENBLAS \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON

(2) Make the code:

make-code

make -j`getconf _NPROCESSORS_ONLN`

Go to the build directory (build-pepito) and issue this command.

(3) Test the installation:

run-short-tests

ctest -L smoke -j`getconf _NPROCESSORS_ONLN` -V

This runs a few short tests.

Psi4 1.2

Pepito: Ubuntu 16.04

Configuration commands:

_do-configure.bash

cmake -H. -B/home/alston/Psi4/git/master/build-pepito \
        -DCMAKE_INSTALL_PATH="/home/alston/psi4conda/master" \
        -DCMAKE_INSTALL_PREFIX="/home/alston/psi4conda/master/install" \
        -DCMAKE_C_COMPILER=gcc \
        -DCMAKE_Fortran_COMPILER=gfortran \
        -DPYTHON_EXECUTABLE="/usr/bin/python2.7" \
        -DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
        -DPYTHON_INCLUDE_DIR="/usr/include/python2.7/numpy" \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON

GCC

This compilation works. I had to do the following before it compiled and ran the tests correctly:

Installed libopenblas
Fixed a bug in the NumPy installation. See below for a detailed description.
Python needed module MPMath Installed using apt-get for both versions of Python.

Intel compilers

I needed more libraries from Intel. These were defined using the commands (in my .bashrc):

  # Intel compilers:
  INTEL=/opt/intel/current
  export PATH=${PATH}:${INTEL}/bin/
  export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${INTEL}/mkl/lib/intel64/:${INTEL}/lib/intel64
/:

Spasrv12

ICC

Problems here that seem to be related to the age of the OS on this machine. System cmake does not work as it is too old (see GCC notes below). I installed cmake 3.4.0 and this does proceed with the configuration, but only if I use ICC 2017. And even there it fails with these errors:

icc-2017

-- Math lib search order is MKL;OPENBLAS;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- You can select a specific type by defining for instance -D BLAS_TYPE=ATLAS or -D LAPACK_TYPE=ACML
-- or by redefining MATH_LIB_SEARCH_ORDER
-- Found BLAS: ATLAS (/usr/lib/atlas-base/libf77blas.so;/usr/lib/atlas-base/libcblas.so;/usr/lib/atlas-base/libatlas.so)
-- Found LAPACK: ATLAS (/usr/lib/atlas-base/libatlas.so;/usr/lib/atlas-base/atlas/liblapack.so)
-- Found LAPACK: /usr/lib/atlas-base/libatlas.so;...
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Failed
-- Performing Test HAS_CPP11_FLAG
-- Performing Test HAS_CPP11_FLAG - Failed
CMake Error at /opt/ajm/Psi4/current/share/cmake/pybind11/pybind11Tools.cmake:29 (message):
  Unsupported compiler -- pybind11 requires C++11 support!
Call Stack (most recent call first):
  /opt/ajm/Psi4/current/share/cmake/pybind11/pybind11Config.cmake:107 (select_cxx_standard)
  external/upstream/pybind11/CMakeLists.txt:1 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeOutput.log".
See also "/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeError.log".

With ICC 2018 the first test of the compiler fails:

icc-2018

$ ./do-configure-icc 
-- The C compiler identification is Intel 18.0.1.20171018
-- The CXX compiler identification is GNU 5.2.0
-- Check for working C compiler: /opt/ajm/intel/current/bin/icc
-- Check for working C compiler: /opt/ajm/intel/current/bin/icc -- broken
CMake Error at /home/alston/tmp/cmake-3.4.0-Linux-x86_64/share/cmake-3.4/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler "/opt/ajm/intel/current/bin/icc" is not able to compile a
  simple test program.

  It fails with the following output:

   Change Dir: /home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp

  

  Run Build Command:"/usr/bin/make" "cmTC_5fc5d/fast"

  /usr/bin/make -f CMakeFiles/cmTC_5fc5d.dir/build.make
  CMakeFiles/cmTC_5fc5d.dir/build

  make[1]: Entering directory
  `/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp'

  Building C object CMakeFiles/cmTC_5fc5d.dir/testCCompiler.c.o

  /opt/ajm/intel/current/bin/icc -o
  CMakeFiles/cmTC_5fc5d.dir/testCCompiler.c.o -c
  /home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp/testCCompiler.c


  Linking C executable cmTC_5fc5d

  /home/alston/tmp/cmake-3.4.0-Linux-x86_64/bin/cmake -E cmake_link_script
  CMakeFiles/cmTC_5fc5d.dir/link.txt --verbose=1

  /opt/ajm/intel/current/bin/icc CMakeFiles/cmTC_5fc5d.dir/testCCompiler.c.o
  -o cmTC_5fc5d -rdynamic

  ld: cannot find -limf

  ld: cannot find -lsvml

  ld: cannot find -lirng

  ld: cannot find -lipgo

  ld: cannot find -ldecimal

  ld: cannot find -lirc

  ld: cannot find -lsvml

  make[1]: '''* [cmTC_5fc5d] Error 1

  make[1]: Leaving directory
  `/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp'

  make: '''* [cmTC_5fc5d/fast] Error 2

  

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:5 (project)


-- Configuring incomplete, errors occurred!

What to do?

GCC

do-configure-gcc

cmake -H. -B/home/alston/Psi4/git/master/build-spasrv12 \
        -DCMAKE_INSTALL_PATH="/opt/ajm/Psi4/1.2" \
        -DCMAKE_INSTALL_PREFIX="/opt/ajm/Psi4/1.2/install-gcc" \
        -DCMAKE_C_COMPILER=gcc \
        -DCMAKE_Fortran_COMPILER=gfortran \
        -DPYTHON_EXECUTABLE="/usr/bin/python2.7" \
        -DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
        -DPYTHON_INCLUDE_DIR="/usr/include/python2.7/numpy" \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON

no MPMath for Python
cmake too old. Needs version 3.3 or newer. Installed 3.4 under HOME/tmp. Got binaries from https://cmake.org/files/v3.4/ This worked.
No MPMath package for Python. Used instructions on this site https://gist.github.com/saurabhshri/46e4069164b87a708b39d947e4527298 to install package locally (for myself) without root. More details below.
Failure of some jobs which appear to need plugins:

382: Traceback (most recent call last):
382:   File "/home/alston/Psi4/git/master/tests/json/gradient/input.py", line 2, in <module>
382:     import psi4
382:   File "/home/alston/Psi4/git/master/build-spasrv12/stage/opt/ajm/Psi4/1.2/install/lib/psi4/__init__.py", line 81, in <module>
382:     from .driver import endorsed_plugins
382:   File "/home/alston/Psi4/git/master/build-spasrv12/stage/opt/ajm/Psi4/1.2/install/lib/psi4/driver/endorsed_plugins.py", line 32, in <module>
382:     import v2rdm_casscf
382:   File "/opt/ajm/Psi4/current/lib/python2.7/site-packages/v2rdm_casscf/__init__.py", line 39, in <module>
382:     psi4.core.plugin_load(sofile)
382: RuntimeError: 
382: Fatal Error: Plugins are not supported on your platform.
382: 
382: Error occurred in file: /home/alston/Psi4/git/master/psi4/src/psi4/libplugin/load_plugin.cc on line: 99
382: The most recent 5 function calls were:
382: 
382: psi::PsiException::PsiException(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char const*, int)
382: psi::plugin_load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)
382: py_psi_plugin_load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
382: void pybind11::cpp_function::initialize<int (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::scope, pybind11::sibling, char [88]>(int (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), int (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, char const (&) [88])::{lambda(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle)#3}::_FUN(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle)
382: pybind11::cpp_function::dispatcher(_object*, _object*, _object*)
382: 
382: Exit Status: infile ( 1 ); autotest ( None ); sowreap ( None ); overall ( 1 )
 1/13 Test #382: json-gradient ....................'''*Failed    2.01 sec

SAPT(DFT) example works correctly. So, some success!

Run-time on spasrv12

Important

Although I installed the code under /opt/ajm/Psi4/1.2/, the installation was not complete as the binary needed the use of libraries in the build directory. And if the build directory name was changed, then the binary would not run!

Is there a way of performing a full installation, that is, including all libraries?

Speed: GCC

The GCC binary is much slower than the ICC-compiled Psi4 1.1 (from the Psi4 site). Differences seem to vary by module, but the floating-point intensive SAPT modules are worst hit.

See runs in /home/alston/molecules/pyridine/scans/dimers_from_Model3_noS2/Min2-S1/aTZMb3-MC+/psi4 for outputs.

How to install Python packages without ROOT

Instructions from this site: https://gist.github.com/saurabhshri/46e4069164b87a708b39d947e4527298

Summary:

wget https://bootstrap.pypa.io/get-pip.py && python get-pip.py --user
cd .local/bin
./pip install <package_name> --user

For Python3.5 or another version:

wget https://bootstrap.pypa.io/get-pip.py && python3 get-pip.py --user
cd .local/bin
./pip3.5 install <package_name> --user

Description:

Why?

Many users when are given server access, do not have root (or sudo) privileges and can not simply do sudo apt-get install python-pip . Here's an easy way you can install and use pip without root (or sudo) access in a local directory. Note : This works without easy_install too.

How?

Download pip from an online repository : wget https://bootstrap.pypa.io/get-pip.py Install the downloaded package into a local directory : python get-pip.py --user This will install pip to your local directory (.local/bin). Now you may navigate to this directory (cd .local/bin) and then use pip or better set your $PATH'' variable this directory to use pip anywhere : ''PATH=$PATH:~/.local/bin followed by source ~/.bashrc to apply the changes. And that's it. You may now install all python modules you require via pip in a local directory without root.

Bug in NumPy installation

On PEPITO (Ubuntu 16.04) I have installed NumPy for both python2.7 and python3.5. It is possible to import NumPy with a stand-alone Python code such as

test-numpy.py

# NumPy import
try:
    import numpy as np
    msg = """
      Success
    """
except:
    msg = """
    No NumPy. Failure.
    """
    
print(msg)

This code works using both versions of Python. But when called from Psi4 in the code $PSI_HOME/lib/psi4/driver/dependency_check.py NumPy //sometimes// cannot be imported. I don't know why this was not always the case. For some jobs it worked and some of the Psi4 examples ran. But others failed with this error:

7: Traceback (most recent call last):
7:   File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/bin/psi4", line 159, in <module>
7:     import psi4
7:   File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/lib//psi4/__init__.py", line 81, in <module>
7:     from .driver import endorsed_plugins
7:   File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/lib//psi4/driver/__init__.py", line 30, in <module>
7:     from . import dependency_check
7:   File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/lib//psi4/driver/dependency_check.py", line 56, in <module>
7:     raise ImportError(msg)
7: ImportError:
7:     NumPy is a runtime requirement for Psi4. Please install NumPy to proceed.
7:
7:     NumPy installation with a package manager can be accomplished by the following lines:
7:        1. conda install numpy
7:        1. sudo yum install numpy
7:        1. sudo apt-get install python-numpy
7:        1. brew install numpy
7:

The jobs that failed were always the same ones (sapt1 was one). I removed the test in dependency_check.py and then got an error (which I have not saved) associated with multiarrary from NumPy and the library libblas.so.3 containing undefined labels to gotoblas.

The offending libraries were libblas.so.3 and liblapack.so.3, both in /usr/lib, but these were links to libraries in /etc/alternatives/.

Using nm on this library there was indeed a reference to gotoblas:

$ nm -D libblas.so.3 | grep goto
                 U gotoblas

I installed libopenblas.so hoping that it might contain the missing labels, but this did not appear to solve the problem. So I finally did the following:

Move libblas.so.3 to libblas.so.3_ajm. Same for liblapack.so.3
ln -s libopenblas.so libblas.so.3
ln -s libopenblas.so liblapack.so.3

This removed the errors associated with NumPy and Psi4 worked.

There are a couple of mysteries:

Why can NumPy be imported outside of Psi4?
Why do some Psi4 jobs work?
How did my NumPy installation come to include such a weird error?

AJMPublic/camcasp/psi4-installation