Improving Transport Speed On Unstructured Meshes

Good afternoon everyone!

This is not really a new idea, more of a discussion point but I thought I would make a post to see if anyone may have suggestions as to how to improve performance on models with unstructured meshes. I recently did a small study to see the impact of using an unstructured mesh on transport speed, and found that the performance hit is substantial (10x reduction in particles per second with the inclusion of one mesh), which is unfortunate as to resolve results on a mesh more histories are required. Naturally I would expect that the changes in performance are model and mesh dependent.

Here is a brief summary, these are moab meshes.

Effects of Mesh Tallies on Performance

Analog Run With No Tallies

On 20 cores ~10,000 pps (particles per second)

Analog Run With One Unstructured Mesh Tally

Mesh size 16872 tets

On 20 cores ~1,100 pps

Analog Run Two Unstructured Mesh Talleis

Second Mesh size 11887 tets

On 20 cores ~600 pps

Can I just asked if OpenMC was compiled with DAGMC with double down and Embree enabled?

If not than this is one way of speeding up the simulation

Yes, I did compile with double-down and embree, definitely a good idea

Wondering if the mesh element type impacts speed. DAGMC supports tet meshes but LibMesh supports mixed mesh types. Perhaps some hexahedral meshes can represent the shape well but require less elements.

1 Like

I didn’t consider using LibMesh, I didn’t know it supported hex meshes, that’s interesting to know in the future. I’ll leave this up in case it helps someone like me in the future, but what got my problem going fast enough that it is now doable was rebuilding OpenMC as release (I had it as debug :face_with_open_eyes_and_hand_over_mouth:), and increasing the number of MPI processes I was using. Thanks for the help!

@Edgar I have been looking into unstructured mesh tally performance recently and found similar behavior. One thing I discovered is that the k-d tree that we use from MOAB for accelerating spatial searches on the mesh can be optimized by explicitly choosing parameters. See the associated pull request here:

Since that has been merged in OpenMC, there are now better defaults in place but you may want to fiddle with them to see if you can eke out better performance still.

1 Like