H5 Surface Source Read and MPI

User Support
1

Hello,

has anyone used a .h5 Surface Source Read with a MPI build of OpenMC before?

I have an .h5 SSR that works in OpenMC-0.13.3-dev built w/o MPI.
But when built with MPI support, OpenMC crashes on running with the SSR.
The MPI build works and runs w/o an SSR (SDEF).
The MPI build also runs to completion when reading in an MCPL SSR file.

I am running the MPI build locally.

Cheers,
Perry

Crash log:

Reading settings XML file...
 Reading source file from ../ijk.h5...
Abort(873042700) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Info_dup: Invalid argument, error stack:
PMPI_Info_dup(161): MPI_Info_dup(MPI_INFO_NULL, newinfo=0x7ffe6cead7b8) failed
PMPI_Info_dup(119): Null MPI_Info
HDF5: infinite loop closing library
      L,T_top,P,P,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD

My make command:
cmake -DCMAKE_INSTALL_PREFIX=xyz -DCMAKE_PREFIX_PATH=abc -DOPENMC_USE_MCPL=on -DHDF5_PREFER_PARALLEL=on -DOPENMC_USE_MPI=on

Using Intel One API ( IntelLLVM 2023.0.0):

-- The C compiler identification is IntelLLVM 2023.0.0
-- The CXX compiler identification is IntelLLVM 2023.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/intel/oneapi/compiler/2023.0.0/linux/bin/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/oneapi/compiler/2023.0.0/linux/bin/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /opt/intel/oneapi/mpi/2021.8.0/lib/libmpifort.so (found version "3.1")
-- Found MPI_CXX: /opt/intel/oneapi/mpi/2021.8.0/lib/libmpicxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- Found PNG: /usr/lib/x86_64-linux-gnu/libpng.so (found version "1.6.37")
-- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/openmpi/libhdf5.so;/usr/lib/x86_64-linux-gnu/libcrypto.so;/usr/lib/x86_64-linux-gnu/libcurl.so;/usr/lib/x86_64-linux-gnu/libsz.so;/usr/lib/x86_64-linux-gnu/libz.so;/usr/lib/libdl.a;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.10.7") found components: C HL
-- Using parallel HDF5

Warning at end of make:
/usr/bin/ld: warning: libmpi.so.40, needed by /usr/lib/x86_64-linux-gnu/hdf5/openmpi/libhdf5.so, may conflict with libmpi.so.12

2

Hello,

I have resolved the issue. It was the HDF5 library.

I have since compiled OpenMC MPI on a cluster with:

  • /opt/ohpc/pub/libs/intel/impi/hdf5/1.10.6/lib/libhdf5.so (likely a parallel version distributed with a past Intel compiler release (2019?))
  • And on my own computer with hdf5 1.10.9 compiled with 2023 oneAPI with MPI on

With both of these OpenMC can run in MPI and read the .h5 source file.

I guess it didn’t like my default installation of HDF5 (/usr/lib/x86_64-linux-gnu/hdf5/openmpi/libhdf5.so)
or perhaps that conflict warning at the end of make

Perry

1 Like