# Discrepancy between total simulation wallclock time, time spent in transport, and depletion time **URL:** \ **Category:** User Support\ **Created:** [July 1, 2026, 9:27am UTC](https://openmc.discourse.group/t/discrepancy-between-total-simulation-wallclock-time-time-spent-in-transport-and-depletion-time/6393 "2026-07-01T09:27:12Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![baltasar](https://avatars.discourse-cdn.com/v4/letter/b/3d9bf3/32.png) [@baltasar](https://openmc.discourse.group/u/baltasar)\ **Post date:** [July 1, 2026, 9:27am UTC](https://openmc.discourse.group/t/discrepancy-between-total-simulation-wallclock-time-time-spent-in-transport-and-depletion-time/6393/1 "2026-07-01T09:27:12Z")
I apologize in advance for the long post. I’m trying to provide as much relevant information as possible. **Model description:** I am modeling a simple MSR, and have two models, A and B. Both models have the same geometry, consisting of concentric spheres/spherical shells of inner moderator, fuel, outer moderator, and a blanket. I am performing transport-coupled depletion simulations for both models, and the only difference between Model A and Model B is the number of material transfer rates. - Model A has a single transfer rate from the fuel to an off-gas system material representing the removal of Xe and Kr. Total number of depletable materials: 4 - Model B has additional transfer rates between the internal (represented in the geometry) fuel, blanket, and moderator and their external (not modeled explicitly) counter parts. In this case the Xe and Kr are removed from the external fuel. Total number of depletable materials: 7 **Simulation/run settings:** For both models I simulated 150,000 particles, 200 batches, 20 of which were inactive. The calculations were performed on a HPC which uses slurm as a workload manager. I installed OpenMC in a conda environment, and when I run **openmc -v** while in the environment I obtain the following output: _OpenMC version 0.15.3_ _Commit hash: 27e38e894697bb32a1dac7848d2618818b6b8daf_ _Copyright (c) 2011-2025 MIT, UChicago Argonne LLC, and contributors_ _MIT/X license at \< [License Agreement — OpenMC Documentation](https://docs.openmc.org/en/latest/license.html) \>_ _Build type: Release_ _Compiler ID: GNU 14.3.0_ _MPI enabled: yes_ _Parallel HDF5 enabled: yes_ _PNG support: yes_ _DAGMC support: yes_ _libMesh support: no_ _MCPL support: no_ _Coverage testing: no_ _Profiling flags: no_ _UWUW support: n_ For both models, the simulation is run using both OpenMP and MPI parallelism on a single node, with 8 MPI processes (tasks) and 16 OpenMP threads per MPI process. Both models are depleted for a single 30 day step using the following integrator: integrator = openmc.deplete.PredictorIntegrator(operator = operator, timesteps = [30], timestep\_units = ‘d’, power = 1e6, solver = ‘cram16’) **The issue:** My question pertains to the run time and CPU utilization of these calculations. - Model A uses 4.7 hours (wallclock time), has a CPU utilization of 91.7%, and memory usage of 48.4 GiB. - Model B uses 7.8 hours (wallclock time), has a CPU utilization of 50.1%, and memory usage of 48.4 GiB. The increased runtime for Model B initially made sense to me since the model has additional materials to deplete, but the significantly lower CPU utilization surprised me. After some reading on other posts ( [Depletion simulation on supercomputer gets stuck between depletion steps. - #21 by pdeng](https://openmc.discourse.group/t/depletion-simulation-on-supercomputer-gets-stuck-between-depletion-steps/606/21) and [Determing real simulation time - #6 by Jarret](https://openmc.discourse.group/t/determing-real-simulation-time/5377/6) ) I assumed that either my runs were getting stuck on the “creating openmc\_simulation\_nX.h5…” step or that I was simply observing a consequence of depletion being performed sequentially. To investigate this further I looked at the runtime information contained in the depletion\_results.h5 and openmc\_simulation.h5 files. Here’s what I found + some quantities I calculated: _Model A:_ - Transport time: 16173.206191880001 seconds - Statepoint time: 0.06152004899999999 seconds - Burn-up time (average): 0.6852113008499146 seconds - Burn-up time \* # depletable materials: 2.740845203399658 seconds - Total (slurm) time: 16920.0 seconds - transport + BU time (all mat): 16175.9470370834 seconds - CPU utilization (%): 91.7 _Model B:_ - Transport time: 13319.936292557 seconds - Statepoint time: 0.058729711999999996 seconds - Burn-up time (average): 1677.14281725032 seconds - Burn-up time \* # depletable materials: 11739.99972075224 seconds - transport + BU time (all mat) 25059.93601330924 seconds - Total (slurm) time: 28080.0 seconds - CPU utilization (%): 50.1 According to these files the program is spending barely anytime writing statepoint files for either model. When I sum the transport time and what I assume is the time performing transmutation calculations there is still a significant amount of time leftover in the total slurm wall-clock time. Almost 800 seconds for Model A and 3000 seconds for Model B. I took another look at the time information in the final statepoint.h5 file for both models and got the following: | | B | C | | --- | --- | --- | | **Key** | **Model A statepoint.h5** | **Model B statepoint.h5** | | SEND-RECV source sites | 0.031202494999999983 | 0.031676670000000004 | | accumulating tallies | 0.5846612360000003 | 0.36081441200000003 | | active batches | 8160.979186167 | 7433.907243442 | | inactive batches | 367.416181962 | 355.833702045 | | reading cross sections | 0.0 | 0.0 | | sampling source sites | 0.160469112 | 0.16418679499999994 | | simulation | 8528.395258101 | 7789.740831161 | | synchronizing fission bank | 564.1339287830001 | 2017.9341377629996 | | total | 8528.949728731 | 7790.286593861 | | total initialization | 0.0 | 0.0 | | transport | 7963.5831385599995 | 5771.347751955998 | | writing statepoints | 0.005471432 | 0.0054026 | From these numbers the “synchronizing fission bank” numbers stand out to me, as they are ‘relatively’ close to the above discrepancy. Then again, I may be imagining a connection where there is none. I don’t know at what point openmc performs the fission bank synchronization, does this occur before the next transport calculation? Would this overlap with the program printing “creating state point …” line? I would guess that the time spent synchronizing fission banks depends on the number of MPI processes and the system architecture on which I’m running the calculation, but is there some way in which this time can be reduced? Or is it simply something I should approach using trial and error to see what job parameters give me the best performance? I’m also a little confused why Model B would have a fission bank synchronization time so much longer than Model A when the actual geometry in which neutrons are tracked is identical for both. ---
**Author:** ![Jarret](https://yyz2.discourse-cdn.com/free1/user_avatar/openmc.discourse.group/jarret/32/4075_2.png) [@Jarret](https://openmc.discourse.group/u/Jarret)\ **Post date:** [July 1, 2026, 3:03pm UTC](https://openmc.discourse.group/t/discrepancy-between-total-simulation-wallclock-time-time-spent-in-transport-and-depletion-time/6393/2 "2026-07-01T15:03:10Z")
Hi Baltasar, I would highly suggest setting up a script to monitor cpu usage and memory usage during the run and export the results to a spreadsheet to evaluate. The Cpu usage varies significantly within a run as you can see from the post you linked from me. Transport requires the use of the CPU, but depletion/burnup uses very little cpu and mostly memory. From your numbers, you see that your burn-up time is very low for model A, but significant in model B. I found from tests on my pc that during transport i used ~90 cpu% and during depletion I used ~5 cpu%, which if you multiply the usage percentage by simulation time of their respective step and then divide by the overall usage time as seen here, (cpu% of transport \* time of tranport + cpu% of depletion\*time of depletion)/(time of transport+ time of depletion) you get the ~91% and ~50% for run A and B respectively. So the real question is why your first test completed depletion so fast while your second did not. You said you have 4 depletable materials in problem A and 7 in problem B, can I ask why the discrepancy if all you are changing between both problems are the transfer rates? This could easily be the reason for what you are seeing. Problems A depletion is extremely fast making me question if it even underwent depletion. Lastly, I would suggest if you are evaluating computer computation for runs always do a small predictor run (like insignificantly small) since the first run on fresh fuel has most of the nuclide population matrices (besides those explicitly created in the materials definition) initially empty until the first run in which it populates each section according to whatever nuclear database you are using. Therefore, the first run in a depletion chain is signficantly faster than the others, and this can cause the actual computational times to vary as it will significantly underestimate the first transport steps computational time (which for the predictor is very significant). ---
**Author:** ![baltasar](https://avatars.discourse-cdn.com/v4/letter/b/3d9bf3/32.png) [@baltasar](https://openmc.discourse.group/u/baltasar)\ **Post date:** [July 1, 2026, 6:40pm UTC](https://openmc.discourse.group/t/discrepancy-between-total-simulation-wallclock-time-time-spent-in-transport-and-depletion-time/6393/3 "2026-07-01T18:40:11Z")
Hi Jarret, Thank you for your reply! Model A and Model B are identical in terms of geometry, i.e. they have the same four layers (inner mod, fuel, outer mod, blanket). The inner moderator and outer moderator are defined using the same material, so these four layers contain three depletable materials. In Model A the only material transfer rate is: **fuel → OGS** where the OGS is not modeled in the geometry but is defined as a depletable material. In Model B meanwhile there are three additional materials, fuel-external, moderator-external, and blanket-external. Model B then contains three additional depletable materials and transfer rates include: **fuel (geometry) → fuel-external** **fuel-external → fuel (geometry)** and corresponding rates for the moderator and blanket. Based on this I guess the burn-up matrix for Model B is a lot larger/more complicated than for Model A, but I don’t know if this difference is enough to account for the difference in burn-up time. When you say “small predictor run” are you referring to the number of particles/generators? I will take another look at setting up a monitoring script and for now accept that the run-time is not unreasonable. ---
**Author:** ![Jarret](https://yyz2.discourse-cdn.com/free1/user_avatar/openmc.discourse.group/jarret/32/4075_2.png) [@Jarret](https://openmc.discourse.group/u/Jarret)\ **Post date:** [July 3, 2026, 7:35pm UTC](https://openmc.discourse.group/t/discrepancy-between-total-simulation-wallclock-time-time-spent-in-transport-and-depletion-time/6393/4 "2026-07-03T19:35:30Z")
Hi Baltasar, I would still expect longer burnup times then you got for Model A. I doubt depletion is even occurring, I would check the flux within this region and the material composition after depletion to see if there was actually a change. Less than a second for 30 days burnup is crazy low no matter the model. Especially when transport time was 16000 seconds. If depletion is not being completed like I expect it could be do to your source definition, a wrong boundary condition preventing neutrons to your depletable materials, or your geometry just has too much shielding between the source and the depletable materials. As for what I said about the predictor run, I was referring to the predictor integrator within the depletion module. When you run the depletion module (any integrator) the first transport steps matrix only contains the materials you explicitly defined in the material.xml. The first depletion step will populate the entire matrix in relation to whatever depletion cross section library you are using. So depending on if you are using the full chain (1000s) or simplified Casl (~300) your matrix for the next transport run will be that size instead of the number of nuclides in your material.xml. This will significantly increase the transport time, usually to multiples higher time. This is not a concern if you are only using depletion module and not doing a series of them or trying to test how optimized they are. But if you are trying to determine which integrator to use for a series of depletions, not doing a depletion run beforehand will underestimate the first transports impact on computational efficiency, and depending on the integrator have a more significant impact on their overall efficiency if that makes sense to you. ---
**Author:** ![baltasar](https://avatars.discourse-cdn.com/v4/letter/b/3d9bf3/32.png) [@baltasar](https://openmc.discourse.group/u/baltasar)\ **Post date:** [July 29, 2026, 7:28am UTC](https://openmc.discourse.group/t/discrepancy-between-total-simulation-wallclock-time-time-spent-in-transport-and-depletion-time/6393/5 "2026-07-29T07:28:19Z")
Hi Jarret, I’m sorry for the late reply. I think what you said about the predictor integrator makes sense to me. I will take another look at the material composition but based on the analysis performed by a colleague of mine there was a measurable change that’s comparable to a reference we have for the model. I have some other model variations of the same core for which I have results, so I will take another look at those as well. Thank you for all the tips!