Problem creating state.point files with MPI

User Support
1

Hi guys,

I compiled OpenMC following the detailed guide on the website, and MPI (OpenMPI) works well. When running a case with just one computer, it works very well. However, when running with more computers, the simulation gets stuck creating the state.point file indefinitely at the end of the simulation. Does anyone know if it is possible to run MPI (OpenMPI) with more than one computer? And how can I resolve this problem?

2

Hi, I have the same problem!!

I’m using ArchLinux and build OpenMC using OpenMPI in both computers with:

cmake .. \
        -DCMAKE_BUILD_TYPE=Release \
        -DOPENMC_USE_MPI=ON \
        -DHDF5_PREFER_PARALLEL=ON \
        -DCMAKE_INSTALL_PREFIX=/opt/openmc
make -j 16
make install
ln -s /opt/openmc/bin/openmc /usr/bin

I put the *.XML input files generate by OpenMC Python API in both computers and run just 1 node per computer:

mpiexec --host nuclear206:1,nuclear205:1 openmc

And results in a simulation stucked on Creating state point statepoint.100.h5... :

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.14.0
        Git SHA1 | fa2330103de61a864c958d1a7250f11e5dd91468
       Date/Time | 2024-03-09 21:50:36
   MPI Processes | 2
  OpenMP Threads | 2

 Reading settings XML file...
 Reading cross sections XML file...
 Reading materials XML file...
 Reading geometry XML file...
 Reading U234 from /opt/nuclear-data/endfb-viii.0-hdf5/neutron/U234.h5
ETC------------------------------------
 Minimum neutron data temperature: 250 K
 Maximum neutron data temperature: 2500 K
 Preparing distributed cell instances...
 Reading plot XML file...
 Writing summary.h5 file...
 Maximum neutron transport energy: 20000000 eV for H1
 Initializing source particles...

 ====================>     K EIGENVALUE SIMULATION     <====================

  Bat./Gen.      k            Average k
  =========   ========   ====================
        1/1    0.83603
        2/1    0.97081
ETC------------------------------------
       99/1    0.83910    0.83729 +/- 0.00311
      100/1    0.88466    0.83782 +/- 0.00312
 Creating state point statepoint.100.h5...

And got stuck here.

But if I use just one computer

mpiexec --host nuclear215:8 openmc

… it’s ok:

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.14.0
        Git SHA1 | fa2330103de61a864c958d1a7250f11e5dd91468
       Date/Time | 2024-03-09 21:56:30
   MPI Processes | 8
  OpenMP Threads | 2
ETC------------------------------------
       99/1    0.83910    0.83729 +/- 0.00311
      100/1    0.88466    0.83782 +/- 0.00312
 Creating state point statepoint.100.h5...

 =======================>     TIMING STATISTICS     <=======================

 Total time for initialization     = 2.9245e+00 seconds
   Reading cross sections          = 2.8973e+00 seconds
 Total time in simulation          = 8.3410e-01 seconds
   Time in transport only          = 6.9938e-01 seconds
   Time in inactive batches        = 1.0086e-01 seconds
   Time in active batches          = 7.3324e-01 seconds
   Time synchronizing fission bank = 1.2698e-01 seconds
     Sampling source sites         = 1.3046e-03 seconds
     SEND/RECV source sites        = 4.5505e-04 seconds
   Time accumulating tallies       = 4.1398e-04 seconds
   Time writing statepoints        = 3.0814e-03 seconds
 Total time for finalization       = 1.9480e-06 seconds
 Total time elapsed                = 3.7773e+00 seconds
 Calculation Rate (inactive)       = 99146.1 particles/second
 Calculation Rate (active)         = 122742 particles/second

 ============================>     RESULTS     <============================

 k-effective (Collision)     = 0.84048 +/- 0.00283
 k-effective (Track-length)  = 0.83782 +/- 0.00312
 k-effective (Absorption)    = 0.84819 +/- 0.00359
 Combined k-effective        = 0.84252 +/- 0.00250
 Leakage Fraction            = 0.02019 +/- 0.00053