"Illegal instruction (core dumped)" when importing openmc.lib on Theta

User Support
1

Hi again,

I am still in the process of getting OpenMC set up on Theta. I’ve been working with installing everything directly on the KNL nodes to minimize any funky errors from cross-compilation.

I am having trouble running a python script that runs openmc depletion multiple times. Whenever I run this script normally using python script.py, I get a Illegal instruction (core dumped) error. I’ve traced the source of this error to be caused by imports from the openmc.lib module within openmc.deplete module, specifically when CDLL(_filename) is called on line 32 in lib/__init__.py.

The strange thing is that wrapping this command in aprun (e.g. aprun -n 1 -N 1 -d 16 -j 4 --cc depth python script.py) fixes the issue with openmc.lib. However, this causes another issue, as I need to make aprun calls within script.py via the subprocess module, and for some reason trying to make aprun calls via subprocess.run when the script is already being run by aprun doesn’t work; subprocess will tell me that aprun is an unrecognized command.

I am wondering if anyone who has experience using shared libraries on Theta would be able to help me troubleshoot this issue with the openmc.lib module.

I am using the following environment variables

module swap PrgEnv-intel PrgEnv-gnu
module load cray-mpich
module load cmake
module load cray-hdf5-parallel
module load conda/2021-09-22
export CRAYPE_LINK_TYPE=dynamic
export CRAY_ADD_RPATH=yes
export PATH=/lus/swift/home/yardasol/projects/openmc/app/bin:$PATH
OPENMC_CROSS_SECTIONS=/lus/swift/home/yardasol/projects/cross-section-libraries/endfb71_h5/cross_sections.xml
export HDF5_USE_FILE_LOCKING=FALSE

I ran the following line to install OpenMC:

HDF5_ROOT=$HDF5_DIR CC=cc CXX=CC cmake -DCMAKE_INSTALL_PREFIX=../app -DOPENMC_USE_MPI=on -DHDF5_PREFER_PARALLEL=on -DCMAKE_BUILD_TYPE=Release -DOPENMC_ENABLE_PROFILE=on ..

Any help would be appreciated!

–Olek

2

Brief update: it appears this issue is related to login vs nonlogin shells on theta. Based on my correspondence with INL support staff, it appears to be the case that aprun requires a login shell in order to work. By default, subprocess.run() launches non-login shells.

1 Like
3

I could not in the end find a solution to this issue. I opted to change machines (one where the login nodes and compute nodes use the same architecture).