What could be the possible causes of get_values(value='rel_err') returning NAN only. I am trying to access the relative error of a mesh tally to analyze the impact of weight windows. Tally.mean looks fine. Thanks.
One potential cause might be the number of batches being 1.
The number of batches is 10. Could it be something related to DAGMC geometries? I can run one of your examples without issues: openmc_weight_window_generator/generate_single_ww_and_apply.ipynb at master Ā· fusion-energy/openmc_weight_window_generator Ā· GitHub. However, if I try something similar to a DAGMC geometry, all my rel_err are NAN.
That is interesting, so WW work on a CSG model but not on a DAGMC model.
I can have a go at trying WW with DAGMC tomorrow to reproducing the error.
I bet @pshriwise would be interested in this thread as he is working on both topics.
Thanks for reporting this
Yes, it seems like I canāt get it working with DAGMC. Another thing is that I canāt use generate_wws_magic_method with DAGMC models as well. Only generate_wws seems to run. If @pshriwise is using any variance reduction method with DAGMC, I would love to hear about it.
Hi @fnovais, can you provide some additional information about the tally youāre examining? The filters itās using would probably be the most helpful. Iām guessing a mesh filter, but I donāt want to assume.
Iāve successfully used weight windows with DAGMC geometries before, so Iād be surprised if that were the cause of the problem. Iāve re-run a couple of problems this morning to confirm this on my end. I canāt speak to why the generate_wws_magic_method might not be working for DAGMC models as those are in @Shimwellās project. Are you seeing an error when using those methods? Or are they running successfully but failing to generate effective weight windows?
Thanks,
Patrick
I canāt see anything in the source code that wouldnāt work for DAGMC geometry but then again I donāt have tests for DAGMC geometry so it could certainly be something in there. Also interesting to know the generate_wws method works but the generate_wws_magic_method doesnāt work. Iām wondering if the best option is just to wait for the weight windows to get into the openmc c++ layer and help review that PR.
@pshriwise I am using a mesh filter:
mesh = openmc.RegularMesh()
mesh.dimension = [100, 100, 80]
mesh.lower_left = [-1000, -1000, -413]
mesh.upper_right = [1000, 1000, 413]
# %%
flux_tally = openmc.Tally(name="n_flux_on_mesh")
mesh_n_filter = openmc.MeshFilter(mesh)
flux_tally.filters = [mesh_n_filter]
flux_tally.scores = ["flux"]
I was able to use it yesterday, and I have seen a very small improvement after generating WW (comparing the neutron flux on ParaView). Do you have any suggestions for the most efficient way to reduce variance in complex DAGMC geometries? The issues is that when I use get_values(value='rel_err'), it return all NANs.
I am running on a 360-degree tokamak model with very detailed materials, so simulations are not super quick, which is taking a lot of time for me to keep testing different approaches. As you can see below, I have no particles penetrating deep into the inboard.
@Shimwell is working now. The workaround was to define the model again after the simulation without WW. See below:
model = openmc.Model(geometry, mats, settings)
model.tallies = [flux_tally]
output_file = model.run()
# %%
import openmc_weight_window_generator
model = openmc.Model(geometry, mats, settings, tallies)
model.generate_wws_magic_method(tally=flux_tally, iterations=5, rel_err_tol=0.7)
@fnovais, interesting! Iām glad you figured out a workaround. Were you able to get particles into the inboard regions with that adjustment?
@pshriwise, after 5 iterations particles did go a little far, but not substantially.
It will be really interesting to compare the stop gap python solution with the more efficient c++ weight windows that Patrick has just made a PR for Global Weight Window Generation via CAPI by pshriwise Ā· Pull Request #2359 Ā· openmc-dev/openmc Ā· GitHub
Thanks Patrick you are a star
Hi @fnovais! As we discussed last week, would you be willing to send this model and your XML files our way so we can take a look in more detail?
Thanks!
-Patrick
Hi @pshriwise, I just sent you an email with my files.
Thanks for looking at this!
Hi @fnovais,
I havenāt touched on all your queries yet, but I wanted to give you an update on this. I ran the model through our weight window generation methods and was able to generate plots of the mean and relative error. Iām not sure why you werenāt able to visualize the relative error in your runs. We might need to dig into how you were extracting the data with the Python API to sort that out.
As for the effectiveness of the weight windows, even with a few thousand particles I was able to increase the particle population on the outboard side, but not the inboard side of the device. Iāll look into that more going forward.
Un-normalized flux mean iteration-to-iteration:
Relative error iteration-to-iteration:
Thanks @pshriwise, it looks great! Is the variance reduction method used the one you showed in your presentation?
For the relative error, I used something like this:
sp = openmc.StatePoint('statepoint.50.h5')
flux_tally = sp.get_tally(name='flux_on_mesh')
flux_tally.get_values(value='rel_err')
It works for non-Dagmc models.
I believe Patrick is referring to this method
At this point, Iām pretty convinced that tally results shouldnāt be affected by the type of geometry used in the model (CSG, DAGMC, or a hybrid model). Weight windows should also be applicable regardless of geometry type. It seems there is something else at play, but Iām not sure what exactly.
I pulled down the openmc_weight_window_generator repo and fixed the issue with using DAGMC models (opened a PR for you @shimwell). It was setting a working directory for the openmc.run command that was different than the one containing the dagmc.h5m file. Updating the filename property of the DAGMCUniverse to an absolute path fixes that issue.
The generator then ran successfully and I was able to visualize the resulting relative error in the statepoint data. Looking at the relative error info in a Python interpreter, there are definitely a lot of nan values due to empty values in the tally data. The mesh goes beyond the bounds of the model, so there are certainly places where the mean value is zero, resulting in nan values in the relative error array. Iām seeing those as well. But there are also finite results in there too. Have you checked your relative error array using something like np.isnan(rel_err).all()?
Rel. Err.
Thank you very much. In MCNP, I was always using a cylindrical mesh for the sector model, so I did not realize that the mesh geometry could be the reason. I was always just looking at the few first values, and they are of course nan because they are out of bounds; that was my mistake. Have you also tested the openmc_weight_window_generator by any chance? I will start testing and working with it today. Again, thank you for your time on that.
openmc_weight_window_generator has only been tested a tiny amount by a few users running a few models and reporting issues.
You might notice see the repo doesnāt have a tests folder and that is because this has no automated CI or pytest.
You are most welcome to add tests to the package but it might be more productive to test the c++ implementation now that it has an open PR.
If you do make any tests it might be useful to consider contributing them to the as part of the package or even better part of openmc