# HDF5 errors when using openmc\_init(some\_arbitrary\_comm) **URL:** \ **Category:** User Support\ **Created:** [March 8, 2017, 4:14pm UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259 "2017-03-08T16:14:59Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Ronald\_Rahaman](https://avatars.discourse-cdn.com/v4/letter/r/67e7ee/32.png) [@Ronald\_Rahaman](https://openmc.discourse.group/u/Ronald_Rahaman)\ **Post date:** [March 8, 2017, 4:14pm UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259/1 "2017-03-08T16:14:59Z")
I’m developing a driver that calls openmc\_init() with an arbitrary communicator. When I do so, however, I get some errors from HDF5 that I don’t understand. Here’s a modified version of main.F90 that demonstrates the problematic parts of my driver. It creates a new communicator from a subset of procs of MPI\_COMM\_WORLD. Then it passes that communicator to openmc\_init(). > program main > > use constants > > use finalize, only: openmc\_finalize > > use global > > use initialize, only: openmc\_init > > use message\_passing > > use particle\_restart, only: run\_particle\_restart > > use plot, only: run\_plot > > use simulation, only: run\_simulation > > use volume\_calc, only: run\_volume\_calculations > > use mpi > > implicit none > > integer :: world\_group ! Group belonging to MPI\_0COMM\_WORLD > > integer :: openmc\_comm, openmc\_group ! A new group/comm for running OpenMC > > ! Initialize run – when run with MPI, pass communicator > > #ifdef MPI > > call MPI\_Init(mpi\_err) > > ! Make a new communicator from a subset of procs. For demonstrative purposes, we make a > > ! communicator from just one proc. > > call MPI\_Comm\_group(MPI\_COMM\_WORLD, world\_group, mpi\_err) > > call MPI\_Group\_incl( & > > world\_group, & > > 1, & > > (/ 0 /), & > > openmc\_group, & > > mpi\_err) > > call MPI\_Comm\_create(MPI\_COMM\_WORLD, openmc\_group, openmc\_comm, mpi\_err) > > if (openmc\_comm /= MPI\_COMM\_NULL) then > > call openmc\_init(openmc\_comm) > > endif > > #else > > call openmc\_init() > > #endif > > ! start problem based on mode > > select case (run\_mode) > > case (MODE\_FIXEDSOURCE, MODE\_EIGENVALUE) > > call run\_simulation() > > case (MODE\_PLOTTING) > > call run\_plot() > > case (MODE\_PARTICLE) > > if (master) call run\_particle\_restart() > > case (MODE\_VOLUME) > > call run\_volume\_calculations() > > end select > > ! finalize run > > call openmc\_finalize() > > end program main After I build and run it, however, I get the following errors. These are the same errors I get with the real driver I’m developing. > HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) MPI-process 1: > > #000: H5T.c line 1723 in H5Tclose(): not a datatype > > major: Invalid arguments to routine > > minor: Inappropriate type > > HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) MPI-process 1: > > #000: H5T.c line 1723 in H5Tclose(): not a datatype > > major: Invalid arguments to routine > > minor: Inappropriate type > > Fatal error in PMPI\_Type\_free: Invalid datatype, error stack: > > PMPI\_Type\_free(157): MPI\_Type\_free(datatype\_p=0xce4ba8) failed > > PMPI\_Type\_free(89).: Invalid datatype Any suggestions? ---
**Author:** ![paulromano](https://yyz2.discourse-cdn.com/free1/user_avatar/openmc.discourse.group/paulromano/32/486_2.png) [@paulromano](https://openmc.discourse.group/u/paulromano)\ **Post date:** [March 9, 2017, 5:48am UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259/2 "2017-03-09T05:48:35Z")
Ron, I’m not able to reproduce this error on my system. Can you give some more details? What OS are you using? How did you install HDF5? Does the error happen on openmc\_finalize? (it would seem so based on that error message) Thanks, Paul ---
**Author:** ![Ronald\_Rahaman](https://avatars.discourse-cdn.com/v4/letter/r/67e7ee/32.png) [@Ronald\_Rahaman](https://openmc.discourse.group/u/Ronald_Rahaman)\ **Post date:** [March 10, 2017, 2:42pm UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259/3 "2017-03-10T14:42:40Z")
I’ve attached the stdout/sterr when I configure, make, and run the example OpenMC on my system. I’ve also pushed my working copy to a new branch ([https://github.com/RonRahaman/openmc/tree/hdf5-err](https://github.com/RonRahaman/openmc/tree/hdf5-err)); it doesn’t have all the upstream changes from ‘develop’. I’m using: - Ubuntu 16.04 LTS - HDF5 1.10.0-patch1 I installed HDF5 from source using the following configuration: ./configure --prefix=/home/rahaman/install/hdf5-1.10.0-intel-17/ --enable-build-mode=debug --enable-build-all --enable-fortran --enable-fortran2003 --enable-parallel CC=/homes/rahaman/install/mpich-3.2-intel-17/bin/mpicc FC=/homes/rahaman/install/mpich-3.2-intel-17/bin/mpif90 I assume the error occurs during openmc\_init(), as the title and header are not printed before OpenMC aborts. How would I obtain a backtrace? Also, OpenMC only aborts when I use more than one process (mpiexec -n 2). When I use only one process (mpiexec -n 1), OpenMC runs to completion. [run.out](https://openmc.discourse.group/uploads/short-url/gZPvbEGbZy0RdOidXcEKNGF20wE.out) (541 Bytes) [make.out](https://openmc.discourse.group/uploads/short-url/Fk3mR8wpXNa66eaDewMC6RL6KC.out) (31.2 KB) [cmake.out](https://openmc.discourse.group/uploads/short-url/r0Kd7WFzIpqXjK8gKBijj96lKKb.out) (1.39 KB) ---
**Author:** ![paulromano](https://yyz2.discourse-cdn.com/free1/user_avatar/openmc.discourse.group/paulromano/32/486_2.png) [@paulromano](https://openmc.discourse.group/u/paulromano)\ **Post date:** [March 10, 2017, 3:15pm UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259/4 "2017-03-10T15:15:18Z")
Ok, when I run with two processes I can reproduce the error. So if I understand wrap.F90 correctly, you are creating a communicator with just the first process, but then you are calling openmc\_init from all processes. That subroutine should only be called from the processes that are in openmc\_comm. Does that make sense? ---
**Author:** ![Ronald\_Rahaman](https://avatars.discourse-cdn.com/v4/letter/r/67e7ee/32.png) [@Ronald\_Rahaman](https://openmc.discourse.group/u/Ronald_Rahaman)\ **Post date:** [March 10, 2017, 4:31pm UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259/5 "2017-03-10T16:31:44Z")
I’m fairly certain that I’m only calling openmc\_init from the procs in openmc\_comm. I call openmc\_init after running this setup: > call MPI\_Comm\_group(MPI\_COMM\_WORLD, world\_group, mpi\_err) > call MPI\_Group\_incl( & > world\_group, & > 1, & > (/ 0 /), & > openmc\_group, & > mpi\_err) > call MPI\_Comm\_create(MPI\_COMM\_WORLD, openmc\_group, openmc\_comm, mpi\_err) > > if (openmc\_comm /= MPI\_COMM\_NULL) then > call openmc\_init(openmc\_comm) > endif I’m exploiting the fact that, when MPI\_Comm\_create is called from processes that aren’t in openmc\_group, then openmc\_comm will be returned as MPI\_COMM\_NULL. I’ve confirmed this behavior in the standard (see the description of MPI\_Comm\_create [here](http://mpi-forum.org/docs/mpi-3.1/mpi31-report/node156.htm#Node156)), and I’ve used this behavior successfully in the past. With some print statements, I’ve also confirmed that openmc\_init is only called from processes in open\_comm. ---
**Author:** ![Ronald\_Rahaman](https://avatars.discourse-cdn.com/v4/letter/r/67e7ee/32.png) [@Ronald\_Rahaman](https://openmc.discourse.group/u/Ronald_Rahaman)\ **Post date:** [March 11, 2017, 1:13am UTC](https://openmc.discourse.group/t/hdf5-errors-when-using-openmc-init-some-arbitrary-comm/259/6 "2017-03-11T01:13:39Z")
Paul, it seems like the error comes from openmc\_finalize, as you originally suspected. In my driver, openmc\_finalize is called by processes outside of openmc\_comm, which is problematic. For example, if a process is not in openmc\_comm, it skips openmc\_initialize and the initialization of some HDF5 datatypes. This is as I intended. However, that process will still run openmc\_finalize and attempt to free those datatypes. I believe that causes the error I saw. Furthermore, the process outside openmc\_comm reaches openmc\_finalize while the process in openmc\_comm is still running openmc\_init. So when the former process aborted in openmc\_finalize, I mistakenly guessed that the latter process aborted in openmc\_init. I should be able to fix the problem in my driver pretty easily. Thanks for your help!