Hi all,
I’m currently in the process of running my first few simulations with OpenMC. I have a strange issue: statepoint files ceasing to save, seemingly at random. Simulations seem to start perfectly well using both multiple threads and, when on HPC, multiple cores, using my relatively simple CAD geometry. I’m using a fixed point neutron source, and a thermal neutron spectrum. I’m aiming for high spatial resolution over a large area, so am running with high numbers of neutrons (currently aiming for 10^9 in total but could want to go higher). I’m also using photon transport.
However, sometimes my simulations just stop producing statepoint files. I’ve reduced the number per batch to 1e6 so that I can catch this relatively soon after it happens; the resultant buildup of statepoint files does not create a storage issue. Simulations can be restarted from the last statepoint file fine, but there is no predicting when they’ll stop producing the statepoint files. I use the following code to set the statepoints to be saved: -
settings.batches = 2000
settings.particles = int(1e6)
settings.run_mode = "fixed source"
settings.statepoint = {'batches': range(1,settings.batches+1)}
The problem seems to happen less frequently for smaller batch sizes; a batch size of 1e8 seems to always stop saving files after ~4 batches, whereas a batch size of 1e3 seems to run for all batches reliably. As the statepoint files are the same size regardless of the number of neutrons per batch, I know the batch size isn’t creating a storage issue.
The OpenMC process continues to run once the statepoints have stopped saving, though I can’t say for sure that it continues to compute. Whenever I kill it, it seems to stop at line 104 in executor.py: -
line = p.stdout.readline()
This issue seems to operate the same whether on my laptop (WSL under Windows 11), my desktop (Linux) or my institution’s HPC (Linux).
Please may you advise, is there something I’m missing? Some sort of upper limit on neutrons per simulation or something?
Thanks for any help.