I’ve made heavy use of the Python API for OpenMC 0.13.3 for running parametric simulations. They involved materials and geometry sweeps. A function built the model which was exported to an XML and then run. I found that the constant loading and unloading of XS was bottlenecking my sweep. 4 seconds for XS IO for 1.5 seconds of actual simulation. The situation was similar on an HPC cluster. I noticed that the search_for_keff feature also does this. Given that users running parametric sweeps wouldn’t expect very high fidelity and thus, the number of particles run would be low, it seems that the XS IO bottleneck would be common for parametric simulations.
I believe that OpenMC is uniquely suited to parametric runs and I’m interested in writing a dedicated parametric modelling module for it. For now, I’m still planning it but I’ve got this:
Use openmc.lib in a dedicated parametric sweep function to only load XS as required like the depletion module
Minimise IO for parametric sweeps, store StatePoint data in memory until the end of the sweep
Refactor the search_for_keff to use the parametric module
Include a class to handle the results of parametric sweeps using StatePoint and Numpy arrays
Parametric depletion??
Parallel model runs??
I’d love for this topic to be thoroughly discussed here and users and devs provide suggestions and inputs. Examples include the method of implementation (Sweep should take in an array OpenMC models vs Sweep should accept an array of parameters for a callable function builder) and functionality to include or exclude from the module.
@paulromano and @gridley thank you for your support with the BC albedo implementation. I wanted to draw some attention to another thing I was working on here. What do you guys think about having a dedicated parametric module in 0.14.0?
I had this idea because I’ve been working with scripts that build models and run them. The bottleneck I had was the constant loading and unloading of the cross section data.
I am aware that you can get into openmc.lib to have fine control over how the simulation is setup and run but I find the documentation sparse and not a lot of great examples.
I believe having a feature that only loads the XS once for all the runs a script wants to do will save a lot of time and improve the overall efficiency a lot.
Ah, gotcha! Yes, that would be nice, but this is exactly the kind of thing the C API is made for!
If anything, maybe what we should do is add an example Notebook showing how to do this through openmc.lib. If the code ends up being that ugly, maybe we should add a dedicated utility to streamline the interface to parameter sweeps.
I’m happy to take this up, I have some research that needs lots of different types of parametric modeling, including different types of materials, therefore different XS loading for any given run as well.
I’ve looked at the documentation for openmc.lib on the Read the Docs but it’s not descriptive. Could you please suggest some code examples or other documentation so that I can understand how openmc.lib is intended to be used and how code that uses it can be debugged.
I may also try to update the openmc.lib documentation once I develop proficiency.
For this example the nuclear data now gets loaded just once instead of 5 times which saves a decent amount of time as the simulation itself is very quick
This example is what I’ve been needing as the openmc.lib documentation is a list of functions without information on how to use them. @gridley and me have been discussing on email on what sort of parameterisation should be supported. For now, we have material compositions, densities and surface parameters.
I have work that requires running a sweep different materials and MF volume ratios to find the optimal MF ratio for the moderators (C, BeO, REE hydrides) in question. For me, I’d like to load all the nuclides for all the materials and then rebuild all the models in the sweep loop.
Overall, I was thinking something like below. You can see there some of the functions/functionality I assumed aren’t present, do in fact exist:
import openmc
# Ensure dependancies are updated and do not conflict with
# openmc.deplete. Simulatenous use of openmc.lib with
# openmc.deplete may cause issues as it uses openmc.lib too.
import openmc.lib
# Import or define model builder function
import model_builder
# Initialise the session in memory, ensure MPI support
with openmc.lib.run_in_memory():
# Have function to strip xml generated from
# openmc.Materials.to_xml() function in companion
# script. Introduce the concept of a materials database
# companion script.
# TODO: This function doesn't exist yet.
NucArr = openmc.lib.get_nuclides_from_xml(matxmlpath)
# Load all nuclides from all materials, once per session
for nuclide in NucArr:
openmc.lib.load_nuclide(nuclide)
# Recreate Materials object for passing to model builder.
# Do it here to minimise IO in the sweep loop.
MatArr = openmc.Materials.from_xml(matxmlpath)
# Parameters must be passed as a dictionary from a python set
# of dictionaries. Dictionaries used to avoid confusion in
# multiparameter sweeps.
for param_dict in param_dict_set:
# Make model from Materials list and parameters.
# Ideally, minimise IO in model_builder.
# TODO: Material search functions don't exist yet. They
# are necessary if you want to look up a material in
# a Materials object to modify and assign to geometry in
# a model.
model : openmc.Model = model_builder(param_dict, MatArr)
# Export to xml for run. TODO: Find way to have model in
# memory and run from there, to minimise IO.
model.export_to_model_xml()
# Ensure MPI and parallel compute work
openmc.run(threads=8)
# Write the results with a dictionary of parameters.
# TODO: This function must handle printing dicts
# and create sensible file names. Maybe create
# a new overload of statepoint_write with 2 arguments?
openmc.lib.statepoint_write(f'statpoint_param_{
"_".join([str(n) for n in param_dict.values()])}')
# Reset openmc Tallies, Materials and Geometry but
# continue to track XS data in memory.
# TODO: This function doesn't exist yet.
openmc.reset_simulation()
# Selectively unload XS, so the main script may do other things
for nuclide in NucArr:
# TODO: This function doesn't exist yet.
openmc.lib.unload_nuclide(nuclide)
# Option to have openmc.lib.unload_all_XS() required too
# TODO: This function doesn't exist yet either.
I can see that the philosophy with openmc.lib is load into memory and then have functions to edit things in memory. I think with materials, when new nuclides aren’t added, it might work. Am I right? I’m not sure how modifying surfaces and then triggering a rewrite of the cell data would work? So, I was thinking of an approach where the model builder is called over and over, to have the most general schema. What do you think?
Also, I think for general parametric simulations like I’ve outlined in my pseudocode, tangling with openmc.lib may not be something general users might want. So making a wrapper around only the openmc.lib functionality that parametrisation requires and including it in the standard Python API as a parameterisation module might be a good idea. Any thoughts?