OpenMC Outputs with High Performance Computing

User Support
1

Hello community,

I am running OpenMC in parallel with High Performance Computing (HPC). I am having some problem with running it, however. The script won’t run or produce outputs when I run it without pre-existing output files in the directory. The HPC is also not outputting any errors so i am unsure why this is happening. However, if I include the output file: ‘statepoint.100.h5’, ‘summary.h5’, and ‘tallies.out’ from a locally run sim then there are no problems. However, when this is done, there are no new files created and the existing output files: ‘statepoint.100.h5’, ‘summary.h5’, and ‘tallies.out’ are not overwritten to replace the content with the new results. Please see my running script and python script below:

HPC Running Script:

#!/bin/sh
#PBS -lselect=01:ncpus=20:mem=18gb
#PBS -lwalltime=7:45:00

## MODULES TO LOAD @ RUNTIME:
module load anaconda3/personal
source activate myenv

## RUN COMMAND
python3 /rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/SNRE_3D.py

Python Script:

# OpenMC simulation parameters

print(openmc.config)
print(openmc.__version__)

# source = openmc.IndependentSource()
# source.space = openmc.stats.Point(xyz=(0.0, 0.0, RH_coreBottomBoundary/2))

settings = openmc.Settings()
settings.run_mode = 'eigenvalue'

source = openmc.Source()
source.space = openmc.stats.Box([-RH_pressureVesselRadius, -RH_pressureVesselRadius, RH_pressureVesselBottomBoundary],[RH_pressureVesselRadius, RH_pressureVesselRadius, 0.0], only_fissionable=True)

settings.source = source
settings.particles = 5000
settings.generations_per_batch = 20
settings.batches = 100
settings.inactive = 20
settings.export_to_xml('/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/xmlFiles/')

tallies = openmc.Tallies()

FE_fuelThermalTallyi = []
FE_fuelFastTallyi = []
for i in range(0, 60):
        FE_fuelThermalTallyi.append(openmc.Tally(tally_id=(2*i)+1))
        FE_fuelThermalTallyi[i].filters = [openmc.EnergyFilter([0,0.625]), openmc.UniverseFilter(301+i)]
        FE_fuelThermalTallyi[i].scores = ['absorption', 'fission']
        tallies.append(FE_fuelThermalTallyi[i])

        FE_fuelFastTallyi.append(openmc.Tally(tally_id=(2*i)+2))
        FE_fuelFastTallyi[i].filters = [openmc.EnergyFilter([0.625,19.6403e6]), openmc.UniverseFilter(301+i)]
        FE_fuelFastTallyi[i].scores = ['absorption', 'fission']
        tallies.append(FE_fuelFastTallyi[i])

tallies.export_to_xml('/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/xmlFiles/')

openmc.run(threads=2, path_input='/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/xmlFiles')

with openmc.StatePoint('statepoint.{}.h5'.format(settings.batches)) as sp:
    keff = sp.keff
    k_generation = sp.k_generation

    with open('/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/txtFiles/keff{}.txt'.format(int(CD_drumAngularPos)), 'w') as f:
        f.write("{}\n".format(keff))

    with open('/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/txtFiles/k_generation{}.txt'.format(int(CD_drumAngularPos)), 'w') as f:
        for value in k_generation:
2

@jimmyvdw Did you compile OpenMC using MPI? If so, you should provide the mpi_args argument when calling openmc.run().

3

Unfortunately, I know very little about running in parallel and high powered computing. I have installed OpenMC with MPI enabled. I included the following into my openmc.run() command:

mpi_args=['mpiexec', '-n', '8']

And got the following error:

Traceback (most recent call last):
  File "/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/SNRE_3D.py", line 987, in <module>
    openmc.run(threads=2, path_input='/rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/xmlFiles', mpi_args=['mpiexec', '-n', '8'])
  File "/rds/general/user/jv20/home/anaconda3/envs/myenv/lib/python3.9/site-packages/openmc/executor.py", line 314, in run
    _run(args, output, cwd)
  File "/rds/general/user/jv20/home/anaconda3/envs/myenv/lib/python3.9/site-packages/openmc/executor.py", line 97, in _run
    p = subprocess.Popen(args, cwd=cwd, stdout=subprocess.PIPE,
  File "/rds/general/user/jv20/home/anaconda3/envs/myenv/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/rds/general/user/jv20/home/anaconda3/envs/myenv/lib/python3.9/subprocess.py", line 1837, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mpiexec'
4

You’ll have to consult with your HPC system administrator on the proper way to utilize MPI on your system as the command is not always mpiexec (for example, systems using SLURM use srun instead of mpiexec).

5

Thank you for clarifying, the code is now successfully running with MPI. However, still not outputting any files and still won’t run without the statepoint file made from a locally run sim.

This is the output from the sim ran on the HPC, it has been shortened for ease:

{}
0.13.3

# mpiexec: MPI Program startup


# mpiexec: Running in job 8385877.pbs at Wed 25 Oct 09:38:50 BST 2023
# mpiexec: Fabric configuration:
# mpiexec: node class CX3
# mpiexec: libfabric provider verbs
# mpiexec: libfabric inferface eth0
# mpiexec: MPI-IO configuration on / gpfs
# mpiexec: full path to program is /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/openmc
# mpiexec: program arguments are: -s 2 /rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/xmlFiles
# mpiexec: 20 ranks allocated via PBS select, but -n argument reduces that to 8
# mpiexec: 1 OpenMP threads / rank allocated by PBS select
# mpiexec: 20 ranks per node
# mpiexec: There are 256 cores/node. 20 will be used for this job
# mpiexec: Job has shared use of the allocated nodes. Disabling process-pinning
# mpiexec: Node is shared. Disabling process pinning
# mpiexec: machinefile configured as:
cx3-12-2.cx3.hpc.ic.ac.uk:20
#
# mpiexec: Checking all nodes are ONLINE using ping:
# mpiexec: All nodes appear ONLINE
# mpiexec: Checking all nodes are ONLINE using ssh:
# cx3-12-2.cx3.hpc.ic.ac.uk : # mpiexec: Dynamic linking for /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/openmc:
	linux-vdso.so.1 (0x00007ffcac7c7000)
	/apps/rcs-tmpfile-replacement/rcs-tmpfile-replacement/libc-replaced-tmpnam.so (0x000014d1a9573000)
	/apps/rcs-tmpfile-replacement/rcs-tmpfile-replacement/libc-replaced-tmpfile.so (0x000014d1a9371000)
	/apps/rcs-setxattr-replacement/rcs-setxattr-replacement/libc-setxattr-shim.so (0x000014d1a9995000)
	libopenmc.so => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/libopenmc.so (0x000014d1a90da000)
	libstdc++.so.6 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/libstdc++.so.6 (0x000014d1a97b2000)
	libgcc_s.so.1 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/libgcc_s.so.1 (0x000014d1a90bf000)
	libc.so.6 => /lib64/libc.so.6 (0x000014d1a8ce7000)
	libhdf5.so.200 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./libhdf5.so.200 (0x000014d1a88da000)
	libhdf5_hl.so.200 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./libhdf5_hl.so.200 (0x000014d1a88b6000)
	libpng16.so.16 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./libpng16.so.16 (0x000014d1a8879000)
	libm.so.6 => /lib64/libm.so.6 (0x000014d1a84f7000)
	libgomp.so.1 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./libgomp.so.1 (0x000014d1a83f5000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x000014d1a81d5000)
	librt.so.1 => /lib64/librt.so.1 (0x000014d1a7fcd000)
	/lib64/ld-linux-x86-64.so.2 (0x000014d1a9775000)
	libcrypto.so.3 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././libcrypto.so.3 (0x000014d1a7ac1000)
	libcurl.so.4 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././libcurl.so.4 (0x000014d1a7a16000)
	libsz.so.2 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././libsz.so.2 (0x000014d1a7a09000)
	libz.so.1 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././libz.so.1 (0x000014d1a79ee000)
	libdl.so.2 => /lib64/libdl.so.2 (0x000014d1a77ea000)
	libnghttp2.so.14 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./././libnghttp2.so.14 (0x000014d1a77bd000)
	libssh2.so.1 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./././libssh2.so.1 (0x000014d1a7778000)
	libssl.so.3 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./././libssl.so.3 (0x000014d1a76d6000)
	libgssapi_krb5.so.2 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./././libgssapi_krb5.so.2 (0x000014d1a7683000)
	libzstd.so.1 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/./././libzstd.so.1 (0x000014d1a7573000)
	libkrb5.so.3 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././././libkrb5.so.3 (0x000014d1a749f000)
	libk5crypto.so.3 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././././libk5crypto.so.3 (0x000014d1a7486000)
	libcom_err.so.3 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././././libcom_err.so.3 (0x000014d1a97a2000)
	libkrb5support.so.0 => /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/../lib/././././libkrb5support.so.0 (0x000014d1a7477000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x000014d1a7260000)
# mpiexec: launch started at Wed 25 Oct 09:38:50 BST 2023
# mpiexec: launching program...
# mpiexec: /rds/general/user/jv20/home/anaconda3/envs/myenv/bin/openmc -s 2 /rds/general/user/jv20/home/OpenMC/Simulations/ReactorAssembly/ENDF70/temp/xmlFiles
# mpiexec: PROGRAM OUTPUT FOLLOWS
(cx3-12-2.cx3.hpc.ic.ac.uk:0,1,2,3,4,5,6,7)

                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
 Reading cross sections XML file...
 Reading cross sections XML file...
 Reading cross sections XML file...
 Reading cross sections XML file...
 Reading cross sections XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading settings XML file...
                                %%%%%%%%%%%%%%%
                           %%%%%%%%%%%%%%%%%%%%%%%%
                        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
                                    %%%%%%%%%%%%%%%%%%%%%%%%
                                     %%%%%%%%%%%%%%%%%%%%%%%%
                 ###############      %%%%%%%%%%%%%%%%%%%%%%%%
                ##################     %%%%%%%%%%%%%%%%%%%%%%%
                ###################     %%%%%%%%%%%%%%%%%%%%%%%
                ####################     %%%%%%%%%%%%%%%%%%%%%%
                #####################     %%%%%%%%%%%%%%%%%%%%%
                ######################     %%%%%%%%%%%%%%%%%%%%
                #######################     %%%%%%%%%%%%%%%%%%
                 #######################     %%%%%%%%%%%%%%%%%
                 ######################     %%%%%%%%%%%%%%%%%
                  ####################     %%%%%%%%%%%%%%%%%
                    #################     %%%%%%%%%%%%%%%%%
                     ###############     %%%%%%%%%%%%%%%%
                       ############     %%%%%%%%%%%%%%%
                          ########     %%%%%%%%%%%%%%
                                      %%%%%%%%%%%

                 | The OpenMC Monte Carlo Code
       Copyright | 2011-2023 MIT, UChicago Argonne LLC, and contributors
         License | https://docs.openmc.org/en/latest/license.html
         Version | 0.13.3
        Git SHA1 | 50e39a4e20dc9e0f3d7ccf07333f6a5e6c797c8c
       Date/Time | 2023-10-25 09:38:51
  OpenMP Threads | 2

 Reading cross sections XML file...
 Reading cross sections XML file...
 Reading settings XML file...
 Reading cross sections XML file...
 Reading materials XML file...
 Reading geometry XML file...
 Reading materials XML file...
 Reading geometry XML file...
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading materials XML file...
 Reading materials XML file...
 Reading geometry XML file...
 Reading geometry XML file...
 Reading materials XML file...
 Reading materials XML file...
 Reading geometry XML file...
 Reading geometry XML file...
 Reading materials XML file...
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading geometry XML file...
 Reading materials XML file...
 Reading geometry XML file...
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
 Reading U235 from
 /rds/general/user/jv20/home/Serpent/DataLibraries/ENDFB7-0/h5Files/U235.h5
...
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Minimum neutron data temperature: 294 K
 Maximum neutron data temperature: 294 K
 Reading tallies XML file...
 Reading tallies XML file...
 Reading tallies XML file...
 Reading tallies XML file...
 Reading tallies XML file...
 Reading tallies XML file...
 Reading tallies XML file...
 Reading tallies XML file...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Preparing distributed cell instances...
 Reading plot XML file...
 Writing summary.h5 file...
 Reading plot XML file...
 Writing summary.h5 file...
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
 Reading plot XML file...
 Writing summary.h5 file...
 Reading plot XML file...
 Writing summary.h5 file...
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
 Reading plot XML file...
 Writing summary.h5 file...
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
 Reading plot XML file...
 Writing summary.h5 file...
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
 Reading plot XML file...
 Writing summary.h5 file...
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
 Reading plot XML file...
 Writing summary.h5 file...
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5F.c line 532 in H5Fcreate(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #001: H5VLcallback.c line 3282 in H5VL_file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 3248 in H5VL__file_create(): file create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_file.c line 63 in H5VL__native_file_create(): unable to create file
    major: File accessibility
    minor: Unable to open file
  #004: H5Fint.c line 1898 in H5F_open(): unable to lock the file
    major: File accessibility
    minor: Unable to lock file
  #005: H5FD.c line 1625 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Unable to lock file
  #006: H5FDsec2.c line 1002 in H5FD__sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: Virtual File Layer
    minor: Unable to lock file
 ERROR: Failed to open HDF5 file with mode 'w': summary.h5
 Maximum neutron transport energy: 20000000 eV for U235
 Initializing source particles...

 ====================>     K EIGENVALUE SIMULATION     <====================

  Bat./Gen.      k            Average k
  =========   ========   ====================
        1/1    0.89835
...
      40/20    1.15472    0.94290 +/- 0.01091
 Creating state point statepoint.40.h5...

 =======================>     TIMING STATISTICS     <=======================

 Total time for initialization     = 1.4526e+01 seconds
   Reading cross sections          = 1.4039e+01 seconds
 Total time in simulation          = 3.1175e+01 seconds
   Time in transport only          = 3.1110e+01 seconds
   Time in inactive batches        = 5.3390e+00 seconds
   Time in active batches          = 2.5836e+01 seconds
   Time synchronizing fission bank = 3.4057e-03 seconds
     Sampling source sites         = 2.5620e-03 seconds
     SEND/RECV source sites        = 5.5049e-04 seconds
   Time accumulating tallies       = 3.7461e-03 seconds
   Time writing statepoints        = 5.1165e-02 seconds
 Total time for finalization       = 8.0718e-04 seconds
 Total time elapsed                = 4.5730e+01 seconds
 Calculation Rate (inactive)       = 3746.02 particles/second
 Calculation Rate (active)         = 774.12 particles/second

 ============================>     RESULTS     <============================

 k-effective (Collision)     = 0.94865 +/- 0.00938
 k-effective (Track-length)  = 0.94290 +/- 0.01043
 k-effective (Absorption)    = 0.95592 +/- 0.00743
 Combined k-effective        = 0.95401 +/- 0.00808
 Leakage Fraction            = 0.21910 +/- 0.00339

# mpiexec: finished at Wed 25 Oct 09:39:37 BST 2023
6

If you see the logo multiple times like that, that’s a good indication that the code itself was not compiled with MPI but is being called from an MPI wrapper. You’ll need to recompile the code with support for MPI (cmake -DOPENMC_USE_MPI=ON ...).

7

It was installed in a conda environment and the MPI set up was not part of that installation, how do I turn this on for the conda environment. Also, even when I run it serially on a single node with a single core on the HPC, there are still no output files created.

8

If you wanted to install OpenMC built against MPICH, it would be:

conda install "openmc=*=*mpich*"

or built against OpenMPI would be:

conda install "openmc=*=*openmpi*"

The strange package matching syntax is explained here.

1 Like