|
Size: 16457
Comment:
|
Size: 16631
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from AJMGrpOnly/camcasp/psi4-installation ## page was renamed from ajm/camcasp/psi4-installation |
|
| Line 114: | Line 116: |
| [[attachment:do-configure.bash]] | [[attachment:_do-configure.bash]] |
| Line 153: | Line 155: |
[[attachment:icc-2017]] |
|
| Line 178: | Line 182: |
[[attachment:icc-2018]] |
Contents
Navigation:
Installing Psi4 from source
Steps:
(1) Configuration commands:
cmake -H. -B/home/alston/Psi4/git/master/build-pepito \
-DCMAKE_INSTALL_PATH="/home/alston/psi4conda/master" \
-DCMAKE_INSTALL_PREFIX="/home/alston/psi4conda/master/install" \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_Fortran_COMPILER=gfortran \
-DPYTHON_EXECUTABLE="/usr/bin/python2.7" \
-DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
-DPYTHON_INCLUDE_DIR="/usr/include/python2.7/numpy" \
-DMAX_AM_ERI=6 \
-DENABLE_gdma=ON \
-DBUILD_SHARED_LIBS=ONOn spasrv12 I used:
do-configure-icc-spasrv12.bash
export CMAKE_ROOT=/home/alston/tmp/cmake-3.4.0-Linux-x86_64/
export MATH_ROOT=/opt/ajm/
export MKL_ROOT=/opt/ajm/intel/current/mkl/lib/intel64/
PYTHON_VER=2.7
# -DBLAS_TYPE="/opt/ajm/lib/libopenblas.so" \
# -DLAPACK_TYPE="/opt/ajm/lib/libopenblas.so" \
$CMAKE_ROOT/bin/cmake -H. -B/home/alston/Psi4/git/master/build-spasrv12-2-icc \
-DCMAKE_INSTALL_PATH="/opt/ajm/Psi4/1.2-2" \
-DCMAKE_INSTALL_PREFIX="/opt/ajm/Psi4/1.2-2/install-icc" \
-DCMAKE_C_COMPILER=icc \
-DCMAKE_Fortran_COMPILER=ifort \
-DPYTHON_EXECUTABLE="/usr/bin/python"${PYTHON_VER} \
-DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
-DPYTHON_INCLUDE_DIR="/usr/include/python"${PYTHON_VER}"/numpy" \
-DBLAS_TYPE=OPENBLAS \
-DLAPACK_TYPE=OPENBLAS \
-DMAX_AM_ERI=6 \
-DENABLE_gdma=ON \
-DBUILD_SHARED_LIBS=ONand for gcc:
do-configure-gcc-spasrv12.bash
export CMAKE_ROOT=/home/alston/tmp/cmake-3.4.0-Linux-x86_64/
export MATH_ROOT=/opt/ajm/
export MKL_ROOT=/opt/ajm/intel/current/mkl/lib/intel64/
PYTHON_VER=2.7
# -DBLAS_TYPE="/opt/ajm/lib/libopenblas.so" \
# -DLAPACK_TYPE="/opt/ajm/lib/libopenblas.so" \
$CMAKE_ROOT/bin/cmake -H. -B/home/alston/Psi4/git/master/build-spasrv12-2-gcc \
-DCMAKE_INSTALL_PATH="/opt/ajm/Psi4/1.2-2" \
-DCMAKE_INSTALL_PREFIX="/opt/ajm/Psi4/1.2-2/install-gcc" \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_Fortran_COMPILER=gfortran \
-DPYTHON_EXECUTABLE="/usr/bin/python"${PYTHON_VER} \
-DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
-DPYTHON_INCLUDE_DIR="/usr/include/python"${PYTHON_VER}"/numpy" \
-DBLAS_TYPE=OPENBLAS \
-DLAPACK_TYPE=OPENBLAS \
-DMAX_AM_ERI=6 \
-DENABLE_gdma=ON \
-DBUILD_SHARED_LIBS=ON(2) Make the code:
make -j`getconf _NPROCESSORS_ONLN`
Go to the build directory (build-pepito) and issue this command.
(3) Test the installation:
ctest -L smoke -j`getconf _NPROCESSORS_ONLN` -V
This runs a few short tests.
Psi4 1.2
Pepito: Ubuntu 16.04
Configuration commands:
cmake -H. -B/home/alston/Psi4/git/master/build-pepito \
-DCMAKE_INSTALL_PATH="/home/alston/psi4conda/master" \
-DCMAKE_INSTALL_PREFIX="/home/alston/psi4conda/master/install" \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_Fortran_COMPILER=gfortran \
-DPYTHON_EXECUTABLE="/usr/bin/python2.7" \
-DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
-DPYTHON_INCLUDE_DIR="/usr/include/python2.7/numpy" \
-DMAX_AM_ERI=6 \
-DENABLE_gdma=ON \
-DBUILD_SHARED_LIBS=ON
GCC
This compilation works. I had to do the following before it compiled and ran the tests correctly:
Installed libopenblas
Fixed a bug in the NumPy installation. See below for a detailed description.
Python needed module MPMath Installed using apt-get for both versions of Python.
Intel compilers
I needed more libraries from Intel. These were defined using the commands (in my .bashrc):
# Intel compilers:
INTEL=/opt/intel/current
export PATH=${PATH}:${INTEL}/bin/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${INTEL}/mkl/lib/intel64/:${INTEL}/lib/intel64
/:
Spasrv12
ICC
Problems here that seem to be related to the age of the OS on this machine. System cmake does not work as it is too old (see GCC notes below). I installed cmake 3.4.0 and this does proceed with the configuration, but only if I use ICC 2017. And even there it fails with these errors:
-- Math lib search order is MKL;OPENBLAS;ESSL;ATLAS;ACML;SYSTEM_NATIVE -- You can select a specific type by defining for instance -D BLAS_TYPE=ATLAS or -D LAPACK_TYPE=ACML -- or by redefining MATH_LIB_SEARCH_ORDER -- Found BLAS: ATLAS (/usr/lib/atlas-base/libf77blas.so;/usr/lib/atlas-base/libcblas.so;/usr/lib/atlas-base/libatlas.so) -- Found LAPACK: ATLAS (/usr/lib/atlas-base/libatlas.so;/usr/lib/atlas-base/atlas/liblapack.so) -- Found LAPACK: /usr/lib/atlas-base/libatlas.so;... -- Performing Test HAS_CPP14_FLAG -- Performing Test HAS_CPP14_FLAG - Failed -- Performing Test HAS_CPP11_FLAG -- Performing Test HAS_CPP11_FLAG - Failed CMake Error at /opt/ajm/Psi4/current/share/cmake/pybind11/pybind11Tools.cmake:29 (message): Unsupported compiler -- pybind11 requires C++11 support! Call Stack (most recent call first): /opt/ajm/Psi4/current/share/cmake/pybind11/pybind11Config.cmake:107 (select_cxx_standard) external/upstream/pybind11/CMakeLists.txt:1 (find_package) -- Configuring incomplete, errors occurred! See also "/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeOutput.log". See also "/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeError.log".
With ICC 2018 the first test of the compiler fails:
$ ./do-configure-icc -- The C compiler identification is Intel 18.0.1.20171018 -- The CXX compiler identification is GNU 5.2.0 -- Check for working C compiler: /opt/ajm/intel/current/bin/icc -- Check for working C compiler: /opt/ajm/intel/current/bin/icc -- broken CMake Error at /home/alston/tmp/cmake-3.4.0-Linux-x86_64/share/cmake-3.4/Modules/CMakeTestCCompiler.cmake:61 (message): The C compiler "/opt/ajm/intel/current/bin/icc" is not able to compile a simple test program. It fails with the following output: Change Dir: /home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp Run Build Command:"/usr/bin/make" "cmTC_5fc5d/fast" /usr/bin/make -f CMakeFiles/cmTC_5fc5d.dir/build.make CMakeFiles/cmTC_5fc5d.dir/build make[1]: Entering directory `/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_5fc5d.dir/testCCompiler.c.o /opt/ajm/intel/current/bin/icc -o CMakeFiles/cmTC_5fc5d.dir/testCCompiler.c.o -c /home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp/testCCompiler.c Linking C executable cmTC_5fc5d /home/alston/tmp/cmake-3.4.0-Linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/cmTC_5fc5d.dir/link.txt --verbose=1 /opt/ajm/intel/current/bin/icc CMakeFiles/cmTC_5fc5d.dir/testCCompiler.c.o -o cmTC_5fc5d -rdynamic ld: cannot find -limf ld: cannot find -lsvml ld: cannot find -lirng ld: cannot find -lipgo ld: cannot find -ldecimal ld: cannot find -lirc ld: cannot find -lsvml make[1]: '''* [cmTC_5fc5d] Error 1 make[1]: Leaving directory `/home/alston/Psi4/git/master/build-spasrv12-icc/CMakeFiles/CMakeTmp' make: '''* [cmTC_5fc5d/fast] Error 2 CMake will not be able to correctly generate this project. Call Stack (most recent call first): CMakeLists.txt:5 (project) -- Configuring incomplete, errors occurred!
What to do?
GCC
cmake -H. -B/home/alston/Psi4/git/master/build-spasrv12 \
-DCMAKE_INSTALL_PATH="/opt/ajm/Psi4/1.2" \
-DCMAKE_INSTALL_PREFIX="/opt/ajm/Psi4/1.2/install-gcc" \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_Fortran_COMPILER=gfortran \
-DPYTHON_EXECUTABLE="/usr/bin/python2.7" \
-DPYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/" \
-DPYTHON_INCLUDE_DIR="/usr/include/python2.7/numpy" \
-DMAX_AM_ERI=6 \
-DENABLE_gdma=ON \
-DBUILD_SHARED_LIBS=ON- no MPMath for Python
cmake too old. Needs version 3.3 or newer. Installed 3.4 under HOME/tmp. Got binaries from https://cmake.org/files/v3.4/ This worked.
No MPMath package for Python. Used instructions on this site https://gist.github.com/saurabhshri/46e4069164b87a708b39d947e4527298 to install package locally (for myself) without root. More details below.
- Failure of some jobs which appear to need plugins:
382: Traceback (most recent call last):
382: File "/home/alston/Psi4/git/master/tests/json/gradient/input.py", line 2, in <module>
382: import psi4
382: File "/home/alston/Psi4/git/master/build-spasrv12/stage/opt/ajm/Psi4/1.2/install/lib/psi4/__init__.py", line 81, in <module>
382: from .driver import endorsed_plugins
382: File "/home/alston/Psi4/git/master/build-spasrv12/stage/opt/ajm/Psi4/1.2/install/lib/psi4/driver/endorsed_plugins.py", line 32, in <module>
382: import v2rdm_casscf
382: File "/opt/ajm/Psi4/current/lib/python2.7/site-packages/v2rdm_casscf/__init__.py", line 39, in <module>
382: psi4.core.plugin_load(sofile)
382: RuntimeError:
382: Fatal Error: Plugins are not supported on your platform.
382:
382: Error occurred in file: /home/alston/Psi4/git/master/psi4/src/psi4/libplugin/load_plugin.cc on line: 99
382: The most recent 5 function calls were:
382:
382: psi::PsiException::PsiException(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char const*, int)
382: psi::plugin_load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)
382: py_psi_plugin_load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
382: void pybind11::cpp_function::initialize<int (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::scope, pybind11::sibling, char [88]>(int (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), int (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, char const (&) [88])::{lambda(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle)#3}::_FUN(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle)
382: pybind11::cpp_function::dispatcher(_object*, _object*, _object*)
382:
382: Exit Status: infile ( 1 ); autotest ( None ); sowreap ( None ); overall ( 1 )
1/13 Test #382: json-gradient ....................'''*Failed 2.01 sec- SAPT(DFT) example works correctly. So, some success!
Run-time on spasrv12
Important
Although I installed the code under /opt/ajm/Psi4/1.2/, the installation was not complete as the binary needed the use of libraries in the build directory. And if the build directory name was changed, then the binary would not run!
Is there a way of performing a full installation, that is, including all libraries?
Speed: GCC
The GCC binary is much slower than the ICC-compiled Psi4 1.1 (from the Psi4 site). Differences seem to vary by module, but the floating-point intensive SAPT modules are worst hit.
See runs in /home/alston/molecules/pyridine/scans/dimers_from_Model3_noS2/Min2-S1/aTZMb3-MC+/psi4 for outputs.
How to install Python packages without ROOT
Instructions from this site: https://gist.github.com/saurabhshri/46e4069164b87a708b39d947e4527298
Summary:
wget https://bootstrap.pypa.io/get-pip.py && python get-pip.py --user cd .local/bin ./pip install <package_name> --user
For Python3.5 or another version:
wget https://bootstrap.pypa.io/get-pip.py && python3 get-pip.py --user cd .local/bin ./pip3.5 install <package_name> --user
Description:
Why?
Many users when are given server access, do not have root (or sudo) privileges and can not simply do sudo apt-get install python-pip . Here's an easy way you can install and use pip without root (or sudo) access in a local directory. Note : This works without easy_install too.
How?
Download pip from an online repository : wget https://bootstrap.pypa.io/get-pip.py Install the downloaded package into a local directory : python get-pip.py --user This will install pip to your local directory (.local/bin). Now you may navigate to this directory (cd .local/bin) and then use pip or better set your $PATH'' variable this directory to use pip anywhere : ''PATH=$PATH:~/.local/bin followed by source ~/.bashrc to apply the changes. And that's it. You may now install all python modules you require via pip in a local directory without root.
Bug in NumPy installation
On PEPITO (Ubuntu 16.04) I have installed NumPy for both python2.7 and python3.5. It is possible to import NumPy with a stand-alone Python code such as
# NumPy import
try:
import numpy as np
msg = """
Success
"""
except:
msg = """
No NumPy. Failure.
"""
print(msg)This code works using both versions of Python. But when called from Psi4 in the code $PSI_HOME/lib/psi4/driver/dependency_check.py NumPy //sometimes// cannot be imported. I don't know why this was not always the case. For some jobs it worked and some of the Psi4 examples ran. But others failed with this error:
7: Traceback (most recent call last): 7: File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/bin/psi4", line 159, in <module> 7: import psi4 7: File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/lib//psi4/__init__.py", line 81, in <module> 7: from .driver import endorsed_plugins 7: File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/lib//psi4/driver/__init__.py", line 30, in <module> 7: from . import dependency_check 7: File "/home/alston/Psi4/git/master/build-pepito/stage/home/alston/psi4conda/master/install/lib//psi4/driver/dependency_check.py", line 56, in <module> 7: raise ImportError(msg) 7: ImportError: 7: NumPy is a runtime requirement for Psi4. Please install NumPy to proceed. 7: 7: NumPy installation with a package manager can be accomplished by the following lines: 7: 1. conda install numpy 7: 1. sudo yum install numpy 7: 1. sudo apt-get install python-numpy 7: 1. brew install numpy 7:
The jobs that failed were always the same ones (sapt1 was one). I removed the test in dependency_check.py and then got an error (which I have not saved) associated with multiarrary from NumPy and the library libblas.so.3 containing undefined labels to gotoblas.
The offending libraries were libblas.so.3 and liblapack.so.3, both in /usr/lib, but these were links to libraries in /etc/alternatives/.
Using nm on this library there was indeed a reference to gotoblas:
$ nm -D libblas.so.3 | grep goto
U gotoblasI installed libopenblas.so hoping that it might contain the missing labels, but this did not appear to solve the problem. So I finally did the following:
- Move libblas.so.3 to libblas.so.3_ajm. Same for liblapack.so.3
- ln -s libopenblas.so libblas.so.3
- ln -s libopenblas.so liblapack.so.3
This removed the errors associated with NumPy and Psi4 worked.
There are a couple of mysteries: