Pereiti prie turinio

Translation status

This page has not yet been translated into Lithuanian, or its translation is out of date. The current English version is shown as a fallback.

Building for A64FX

FrontISTR has implementations tuned for computers (such as "Fugaku") equipped with A64FX. This page explains the build procedure for incorporating this implementation using the Fujitsu compiler, and the execution method.

The tuned implementation is incorporated only when FX64 is specified for the ARCH option of cmake. For the content of the incorporated implementation, see CMake Build Procedure.

Prerequisites

Environment Setup

Add the path of the Fujitsu compiler to PATH, and set the command names of the MPI compilers to environment variables. The subsequent procedures use these environment variables.

LANGDIR=/path/to/fujitsu_compiler
export PATH=${LANGDIR}/bin:$PATH
export LD_LIBRARY_PATH=${LANGDIR}/lib64:$LD_LIBRARY_PATH
export MPICC=mpifcc
export MPICXX=mpiFCC
export MPIFC=mpifrt

In the case of cross-compilation, append px to the end of the command name.

export MPICC=mpifccpx
export MPICXX=mpiFCCpx
export MPIFC=mpifrtpx

Building METIS

tar xzvf metis-5.1.0.tar.gz
cd metis-5.1.0
make config prefix=/path/to/metis cc=${MPICC} openmp=1
make -j 12 install
cd ..

Building REVOCAP_Refiner

Extract the archive, set MakefileConfig.in to the following content, and build.

tar xzvf REVOCAP_Refiner-1.1.04.tar.gz
cd REVOCAP_Refiner-1.1.04

ARCH = a64fx
CC = ${MPICC}
CFLAGS = -Kcmodel=large -Nlst=t -Kocl -Kfast -Kstrict_aliasing -Krestp=all -Kzfill
CXX = ${MPICXX}
CXXFLAGS = -Kcmodel=large -Nlst=t -Kocl -Kfast -Kstrict_aliasing -Krestp=all -Kzfill
F90 = ${MPIFC}
FFLAGS = -Kcmodel=large -Nlst=t -Kocl -Kfast -Kstrict_aliasing -Krestp=all -Kzfill
AR = ar
ARFLAGS = rsv
LD = ${MPIFC}
LDFLAGS = $(FFLAGS) --linkfortran
LIBPATH =
LIBS = -lm
EXTOBJS =
RM = rm -f
DOXYGEN = doxygen
TAR = tar

make -j 12 Refiner
cd ..

Building FrontISTR

For obtaining the source code, see CMake Build Procedure. Set the paths of the dependency libraries and the installation destination, and run cmake.

export METISDIR=/path/to/metis
export REFINERDIR=/path/to/REVOCAP_Refiner-1.1.04
export FISTRDIR=/path/to/frontistr

cd FrontISTR
mkdir build
cd build
cmake \
  -DARCH=FX64 \
  -DCMAKE_BUILD_TYPE=RELEASE \
  -DCMAKE_INSTALL_PREFIX=${FISTRDIR} \
  -DCMAKE_C_COMPILER=${MPICC} \
  -DCMAKE_CXX_COMPILER=${MPICXX} \
  -DCMAKE_Fortran_COMPILER=${MPIFC} \
  -DCMAKE_C_FLAGS="-Kcmodel=large -Nlst=t -Kocl -Kfast -Kzfill -Koptmsg=2 -V" \
  -DCMAKE_CXX_FLAGS="-Kcmodel=large -Nlst=t -Kocl -Kfast -Kzfill -Koptmsg=2 -V" \
  -DCMAKE_Fortran_FLAGS="-Kcmodel=large -Nlst=t -Kocl -Kfast -Kzfill -Koptmsg=2 -V" \
  -DCMAKE_Fortran_MODDIR_FLAG=-M \
  -DCMAKE_EXE_LINKER_FLAGS="--linkfortran -Kopenmp -Kparallel" \
  -DOpenMP_C_FLAGS="-Kopenmp -Kparallel" \
  -DOpenMP_CXX_FLAGS="-Kopenmp -Kparallel" \
  -DOpenMP_Fortran_FLAGS="-Kopenmp -Kparallel" \
  -DBLAS_LIBRARIES="-SSL2BLAMP" \
  -DLAPACK_LIBRARIES="-SSL2BLAMP" \
  -DSCALAPACK_LIBRARIES="-SCALAPACK" \
  -DWITH_TOOLS=ON -DWITH_MPI=ON -DWITH_OPENMP=ON \
  -DWITH_METIS=ON \
  -DMETIS_INCLUDE_PATH=${METISDIR}/include \
  -DMETIS_LIBRARIES=${METISDIR}/lib/libmetis.a \
  -DWITH_REFINER=ON \
  -DREFINER_INCLUDE_PATH=${REFINERDIR}/Refiner \
  -DREFINER_LIBRARIES=${REFINERDIR}/lib/a64fx/libRcapRefiner.a \
  ..
make -j 12 install

The key points of the specifications are as follows.

Specification Description
-DARCH=FX64 Incorporates the implementation tuned for A64FX. If not specified, it is built with only the standard implementation
CMAKE_*_FLAGS Optimization options of the Fujitsu compiler (-Kfast, -Kzfill, -Kocl), specification of the memory model (-Kcmodel=large), options for outputting translation information (-Nlst=t, -Koptmsg=2, -V)
OpenMP_*_FLAGS, CMAKE_EXE_LINKER_FLAGS Specification of OpenMP and automatic parallelization with the Fujitsu compiler (-Kopenmp -Kparallel)
BLAS_LIBRARIES, LAPACK_LIBRARIES, SCALAPACK_LIBRARIES Specification for linking the Fujitsu math libraries (the thread-parallel version of SSL II, ScaLAPACK)
CMAKE_Fortran_MODDIR_FLAG=-M Option for specifying the output destination of module files with the Fujitsu compiler

When -DARCH=FX64 takes effect, cmake prints Architecture-tuned build is ENABLED and a list of the implementations to be incorporated.

Execution

The execution procedure is the same as in other environments. The following is an example of executing a tutorial included in the source code with 4 processes after domain partitioning.

cd FrontISTR/tutorial/02_elastic_hinge_parallel
mpiexec -n 1 ${FISTRDIR}/bin/hecmw_part1
mpiexec -n 4 ${FISTRDIR}/bin/fistr1 -t 1

In a build of ARCH=FX64, the iterative solver uses the CSR-format implementation by default. To compute with the standard implementation, specify MATRIXFORMAT=BSR for !SOLVER in the Analysis Control Data.

For the general execution procedure, see the Execution Guide.

See Also