Recently I am using OpenMC for depletion on the UMich supercomputer Greatlakes. I installed OpenMC from source for MPI parallelization, which worked perfectly for steady-state transport simulations. When I learned to perform depletion calculations using the PythonAPI, I do not know how to instruct OpenMC to run in MPI parallelization. Since the transport simulation is much more time-consuming than depletion part, probably we can somehow let OpenMC execute in MPI mode while doing depletion serially or in OpenMP parallelization as a remedy for now. Can we do that?
If you have a python script that run openmc with depletion, and you have compatible parallel hdf5 + h5py + mpi4py packages, then your script should be runnable using mpiexec python script.py.
This will run the transport in parallel, and also deplete a chunk of burnable materials on each process through MPI. We also use the python multiprocessing module to help each process deplete faster. If you wish to deplete in serial on each process, set the variable openmc.deplete.pool.USE_MULTIPROCESSING to be False before any calls to <integrator>.integrate()
I just wanted to point out, for future reference, that I encountered the same problem as Andrew.
This was on the Merlin cluster at PSI Local HPC Cluster: PSI Merlin6 | AWI | Paul Scherrer Institut (PSI), and it was exactly in the same way that Andrew described, except that it didn’t hang every time. It randomly hung or finished, but most of the times it hung.
In the end openmc.deplete.pool.USE_MULTIPROCESSING=False solved my problem. Now it finishes every time.
Thanks for the discussion, this thread was very helpful.