Unstructured Mesh Filter: libMesh crashing while writing ExodusII/Nemesis mesh file

User Support
1

Hi OpenMC users and developers,

I’m interested in using unstructured mesh tallies with hex elements.
To get started I successfully ran the unstructured-mesh-part-i.ipynb notebook using mesh_library=moab. However, when changing to mesh_library=libmesh, libMesh crashes while writing the ExodusII/Nemesis file tally_1.100.e for unstructured mesh 1.

I’m running Ubuntu 22.04.4 LTS with a compiled version of OpenMC 0.15.0 with both DAGMC and libMesh. I’ve tried pointing OpenMC to the libMesh from MOOSE, and also building libMesh from scratch. Both cases give the same error. Below is the output from the OpenMC with MOOSE’s libMesh:

(...)
       99/1    0.23078    0.23149 +/- 0.00072
      100/1    0.23166    0.23150 +/- 0.00072
 Creating state point statepoint.100.h5...
 Writing file: tally_1.100.e for unstructured mesh 1
libMesh terminating:
Error creating ExodusII/Nemesis mesh file.
Stack frames: 16
0: libMesh::print_trace(std::ostream&)
1: libMesh::MacroFunctions::report_error(char const*, int, char const*, char const*, std::ostream&)
2: libMesh::ExodusII_IO_Helper::create(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
3: libMesh::ExodusII_IO::write_nodal_data_common(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool)
4: libMesh::ExodusII_IO::write_nodal_data_discontinuous(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<double, std::allocator<double> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)
5: libMesh::ExodusII_IO::write_discontinuous_exodusII(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, libMesh::EquationSystems const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*)
6: openmc::LibMesh::write(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const
7: openmc::write_unstructured_mesh_results()
8: openmc_statepoint_write
9: openmc::finalize_batch()
10: openmc_next_batch
11: openmc_run
12: main
13: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0b2d6b9d90]
14: __libc_start_main
15: openmc(+0xce35) [0x56249e8b9e35]
[0] ../src/mesh/exodusII_io_helper.C, line 2185, compiled Sep 10 2024 at 09:42:37

--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

I’d appreciate any help.
Thanks
Luiz

2

After coming across this thread Error when writing Libmesh exodus file - User Support - OpenMC, reverting to an older version of libMesh (v1.7.1) resolved the issue.

I followed the instructions in the shell scripts available at openmc-dev/openmc/tools/ci in GitHub.

Specifically, gha-install-libmesh.sh

#!/bin/bash
set -ex

# libMESH install
pushd $HOME
mkdir LIBMESH && cd LIBMESH
git clone https://github.com/libmesh/libmesh -b v1.7.1 --recurse-submodules
mkdir build && cd build
export METHODS="opt"

if [[ $MPI == 'y' ]]; then
    ../libmesh/configure --prefix=$HOME/LIBMESH CXX=mpicxx CC=mpicc FC=mpifort F77=mpif77 \
        --enable-exodus --disable-netcdf-4 --disable-eigen --disable-lapack
else
    ../libmesh/configure --prefix=$HOME/LIBMESH --enable-exodus --disable-netcdf-4 --disable-eigen --disable-lapack --disable-mpi
fi
make -j4 install
rm -rf $HOME/LIBMESH/build

popd

and gha-install-dagmc.sh

#!/bin/bash
set -ex

# MOAB Variables
MOAB_BRANCH='Version5.1.0'
MOAB_REPO='https://bitbucket.org/fathomteam/moab/'
MOAB_INSTALL_DIR=$HOME/MOAB/

# DAGMC Variables
DAGMC_BRANCH='develop'
DAGMC_REPO='https://github.com/svalinn/dagmc'
DAGMC_INSTALL_DIR=$HOME/DAGMC/

CURRENT_DIR=$(pwd)

# MOAB Install
cd $HOME
mkdir MOAB && cd MOAB
git clone -b $MOAB_BRANCH $MOAB_REPO
mkdir build && cd build
cmake ../moab -DENABLE_HDF5=ON -DENABLE_NETCDF=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$MOAB_INSTALL_DIR -DENABLE_BLASLAPACK=OFF
make -j && make -j install
rm -rf $HOME/MOAB/moab $HOME/MOAB/build

# DAGMC Install
cd $HOME
mkdir DAGMC && cd DAGMC
git clone -b $DAGMC_BRANCH $DAGMC_REPO
mkdir build && cd build
cmake ../dagmc -DBUILD_TALLY=ON -DCMAKE_INSTALL_PREFIX=$DAGMC_INSTALL_DIR -DBUILD_STATIC_LIBS=OFF -DMOAB_DIR=$MOAB_INSTALL_DIR
make -j install
rm -rf $HOME/DAGMC/dagmc $HOME/DAGMC/build

cd $CURRENT_DIR