|
BlueGeneInstalling GROMACS 4.5.xautotools, as described for 4.0.x can be used, but do not use --enable-fortran. In all cases you may find it necessary to disable dlopen and/or XML support. Alternatively cmake can be used (requires cmake >=2.8.4, read the cmake installation guide for general advice). To build GROMACS 4.5.6 (when it is released) for use on the back end, use: ccmake .. \ -DFFTW3F_INCLUDE_DIR:PATH=/your/path/to/fftw3/include \ -DFFTW3F_LIBRARIES:FILEPATH=/your/path/to/fftw3/lib/libfftw3f.a \ -DCMAKE_TOOLCHAIN_FILE=../cmake/Toolchain-BlueGeneP.cmake \ -DBUILD_SHARED_LIBS=OFF \ -DGMX_XML=OFF \ -DCMAKE_C_FLAGS="-O3 -qarch=450d -qtune=450" \ -DCMAKE_INSTALL_PREFIX=/path/to/install/location The warning about unused CMAKE_TOOLCHAIN_FILE at the end is harmless. You can suppress it on the ccmake command line with --no-warn-unused-cli. For GROMACS 4.5 to 4.5.5 for use on the back end, use
ccmake .. \ -DFFTW3F_INCLUDE_DIR:PATH=/path/to/fftw3/include \ -DFFTW3F_LIBRARIES:FILEPATH=/path/to/fftw3/lib/libfftw3f.a \ -DCMAKE_SYSTEM_NAME=BlueGeneP-static \ -DBUILD_SHARED_LIBS=OFF \ -DGMX_XML=OFF \ -DCMAKE_C_COMPILER=mpixlc_r \ -DGMX_ACCELERATION=BlueGene \ -DCMAKE_C_FLAGS="-O3 -qarch=450d -qtune=450" \ -DCMAKE_INSTALL_PREFIX=/path/to/install/location For BlueGene/L, the standard CMake toolchain has been packaged with GROMACS, so you can use ccmake .. \ -D CMAKE_TOOLCHAIN_FILE=../cmake/Toolchain-BlueGeneL-xlc.cmake \ -D FFTW3F_INCLUDE_DIR:PATH=/your/path/to/fftw3/include \ -D FFTW3F_LIBRARIES:FILEPATH=/your/path/to/fftw3/lib/libfftw3f.a Installing and Configuring Gromacs 4.0.x on IBM BlueGene\P and BlueGene\L1) Getting the package.Download the latest CVS source tree using
Go to the Alternatively, download the latest 4.0.x release package, untar it and change to the gromacs-4.0 directory. 2) Prerequisitesi) The MPI Library in driver R1V2M0 on BlueGene/P has a known defect reported as PMR 48603,033,724 for which a specific fix must be requested from IBM. Without the fix the R1V2M0 driver will generate a segmentation fault early in the program startup inside MPI_Scatterv(). Later drivers should have the fix included. ii) You will need the FFTW 3.1.X package or higher configured and installed for both the BlueGene system itself and the frontend node. Note that as for non-BlueGene GROMACS installations, you do need to match the precision with which FFTW and GROMACS have been configured, single vs double. Double precision will be much slower on at least BlueGene/L, but might be wanted for some applications. Alternatively, Stephan Kral's version of FFTWGEL-2.1.5 can also be used for improved FFT performance over FFTW 3.1 in many cases. mabraham notes that this seems unlikely to show gains, since Kral's website claims the SIMD-vectorizing code generator has been incorporated in FFTW, and FFTW's website claims they have such a generator. With 3) Configuring and Building the packagei) You first need to configure/build GROMACS for the BlueGene frontend system. a) Inside the ../configure --prefix=<path-to-your-gromacs-install-dir> \ --enable-ppc-sqrt \ --disable-ppc-altivec \ --enable-fortran \ --with-fft=fftw3 \ --without-x \ CC="xlc_r -q64" CFLAGS="-O3 -qarch=auto -qtune=auto" \ CXX="xlC_r -q64" CXXFLAGS="-O3 -qarch=auto -qtune=auto" \ CPPFLAGS="-I/home/admin/ibm021/include -I/home/admin/ibm021/fftw-3.1.2/include" \ F77="xlf_r -q64" FFLAGS="-O3 -qnoprefetch -qarch=auto -qtune=auto" \ LDFLAGS="-L/home/admin/ibm021/lib -L/home/admin/ibm021/fftw-3.1.2/lib" \ LIBS="-lmass" You may need to adjust the include paths for C-headers and libaries to your system installation in the CPPFLAGS and LDFLAGS variables. Note, that you need a 64-bit compiled version of FFTW-3.1 or higher for LINUX on Power. c) Type d) Type ii) You now need to configure/build the mdrun binary for the actual BlueGene system. a) Inside the b) Execute the configure using the following recommended set of options ../configure --prefix=<your-gromacs-install-dir> \ --host=ppc \ --build=ppc64 \ --enable-ppc-sqrt=1 \ --enable-bluegene \ --enable-fortran \ --enable-mpi \ --with-fft=fftw3 \ --without-x \ --program-suffix=_mpi_bg \ CC="mpixlc_r" CFLAGS="-O3 -qarch=450d -qtune=450" \ MPICC="mpixlc_r" \ CXX="mpixlC_r" CXXFLAGS="-O3 -qarch=450d -qtune=450" \ CPPFLAGS="-I/home/admin/ibm021/bg/include -I/home/admin/ibm021/bg/fftw-3.1.2-openmp/include" \ F77="mpixlf_r" FFLAGS="-O3 -qarch=auto -qtune=auto" \ LDFLAGS="-L/home/admin/ibm021/bg/lib -L/home/admin/ibm021/bg/fftw-3.1.2-openmp/lib" \ LIBS="-lmass" Again, you may need to adjust the include paths for C-headers and libaries to your system installation. On a BlueGene/L type system you may not have the mabraham finds that on BlueGene/L the following works well to compile for the compute nodes: #!/bin/bash FFTW=/bgl/local/pkg/fftw/3.1.2-single MPI=/bgl/BlueLight/ppcfloor/bglsys export CC="blrts_xlc" export MPICC=$CC export CC_FOR_BUILD="xlc" export F77="blrts_xlf" BG_INCLUDE="-I$FFTW/include -I$MPI/include" GENERICFLAGS="-O3 -qbgl -qarch=440d -qcache=level=1:type=i:size=32:line=32:assoc=64:cost=8 \ -qcache=level=1:type=d:size=32:line=32:assoc=64:cost=8 \ -qcache=level=2:type=c:size=4096:line=128:assoc=8:cost=40 -qnoautoconfig -qfloat=norngchk -qhot $BG_INCLUDE" export CFLAGS="$GENERICFLAGS" export FFLAGS="$GENERICFLAGS -qspillsize=4096" MPILIBS="-L$MPI/lib -lmpich.rts -lmsglayer.rts -ldevices.rts -lrts.rts -ldevices.rts -lrts.rts" FFTLIBS="-L$FFTW/lib" export LIBS="-L$XLCPATH/lib $FFTLIBS -lmassv -lxlopt -lnss_files -lnss_dns -lresolv -lc -lnss_files \ -lnss_dns -lresolv $MPILIBS" ../configure --prefix $HOME/progs --without-x --enable-ppc-sqrt --config-cache --program-suffix=_mpi_bg \ --with-fft=fftw3 --enable-bluegene --host=powerpc --build=ppc64 --enable-mpi --enable-fortran make mdrun make install-mdrun Again, one will need to tweak directories for FFTW libraries, etc. There may well be value in using Additional hints on compiler options: You may also want to try to configure the compiler options as You may also want to build with c) Type d) Type Installing Gromacs 3.xAfter ./mknb -software_invsqrt >>> Gromacs nonbonded kernel generator (-h for help) >>> Generating single precision functions in C. >>> Using Gromacs software version of 1/sqrt(x). Error: Cannot open nb_kernel010_c.c for writing. A work-around was reported by Fiona Reid in November 2007 on the GROMACS mailing list. Manually running on the back-end the Then in the final link stage there may be problems with undefined symbols beginning ./configure LDFLAGS="-lnss_files -lnss_dns -lresolv -lc -lnss_files -lnss_dns -lresolv" (in addition to your normal |