Skip to content

Adding support for MPI serial library #696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jayeshkrishna merged 3 commits into from
May 22, 2026
Merged

Adding support for MPI serial library #696

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8bb7729
Skip MPI_Comm_split_type() for mpi serial
jayeshkrishna May 15, 2026
ca95501
Avoid MPI_Type_dup() for mpi serial
jayeshkrishna May 15, 2026
6edbb07
Include MPI header compatible with MPI serial lib
jayeshkrishna May 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/clib/core/pioc.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1547,8 +1547,13 @@ int PIOc_Init_Intracomm_impl(MPI_Comm comp_comm, int num_iotasks, int stride, in

/* Create the node local comm - all procs in comp_comm that are local to
* this compute node (share memory) */
#ifndef MPI_SERIAL
mpierr = MPI_Comm_split_type(ios->comp_comm, MPI_COMM_TYPE_SHARED, 0,
ios->info, &(ios->node_comm));
#else
/* MPI serial library does not support MPI_Comm_split_type() */
Comment on lines +1550 to +1554

@jayeshkrishna jayeshkrishna May 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can skip adding MPI serial support for ADIOS (IMO, we are never going to need that)

mpierr = MPI_Comm_dup(ios->comp_comm, &(ios->node_comm));
#endif
if(mpierr != MPI_SUCCESS){
return check_mpi(ios, NULL, mpierr, __FILE__, __LINE__);
}
Expand Down Expand Up @@ -2472,8 +2477,13 @@ int PIOc_init_async_impl(MPI_Comm world, int num_io_procs, const int *io_proc_li

/* Create the node local comm - all procs in comp_comm that are local to
* this compute node (share memory) */
#ifndef MPI_SERIAL
mpierr = MPI_Comm_split_type(my_iosys->comp_comm, MPI_COMM_TYPE_SHARED, 0,
my_iosys->info, &(my_iosys->node_comm));
#else
/* MPI serial lib does not support MPI_Comm_split_type() */
mpierr = MPI_Comm_dup(my_iosys->comp_comm, &(my_iosys->node_comm));
#endif
if(mpierr != MPI_SUCCESS){
return check_mpi(NULL, NULL, mpierr, __FILE__, __LINE__);
}
Expand Down Expand Up @@ -3072,8 +3082,13 @@ int PIOc_init_intercomm_impl(int component_count, const MPI_Comm peer_comm,

/* Create the node local comm - all procs in comp_comm that are local to
* this compute node (share memory) */
#ifndef MPI_SERIAL
ret = MPI_Comm_split_type(iosys[i]->comp_comm, MPI_COMM_TYPE_SHARED, 0,
iosys[i]->info, &(iosys[i]->node_comm));
#else
/* MPI Serial lib does not support MPI_Comm_split_type() */
ret = MPI_Comm_dup(iosys[i]->comp_comm, &(iosys[i]->node_comm));
#endif
if(ret != MPI_SUCCESS){
return check_mpi(NULL, NULL, ret, __FILE__, __LINE__);
}
Expand Down
11 changes: 10 additions & 1 deletion src/clib/core/progress_engine/spio_async_tcomm.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,17 @@
#include <thread>
#include <chrono>
#include <list>
#include "mpi.h"
#include <vector>
#include "pio_config.h"
#ifdef MPI_SERIAL
extern "C" {
#endif

#include "mpi.h"

#ifdef MPI_SERIAL
}
#endif
#include "spio_async_tpool.hpp"

namespace SPIO_Util{
Expand Down
5 changes: 5 additions & 0 deletions src/clib/core/util/spio_decomp_logger.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,12 @@ namespace SPIO_Util{
int ret = MPI_SUCCESS;

ret = MPI_Comm_dup(ucomm, &comm); assert(ret == MPI_SUCCESS);
#ifndef MPI_SERIAL
ret = MPI_Comm_split_type(comm, MPI_COMM_TYPE_SHARED, 0, MPI_INFO_NULL, &agg_comm); assert(ret == MPI_SUCCESS);
#else
/* MPI serial library does not have support for MPI_Comm_split_type() */
ret = MPI_Comm_dup(comm, &agg_comm); assert(ret == MPI_SUCCESS);
#endif
ret = MPI_Comm_rank(agg_comm, &agg_comm_rank); assert(ret == MPI_SUCCESS);

int color = (agg_comm_rank == 0) ? 0 : MPI_UNDEFINED;
Expand Down
1 change: 0 additions & 1 deletion tests/cunit/test_req_block_wait.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
#include "mpi.h"
#include "pio.h"
#include "pio_internal.h"
#include "pio_minmax.h"
Expand Down
7 changes: 6 additions & 1 deletion tests/cunit/test_spio_rearr_utils_gather.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,12 +221,17 @@ int test_gatherw_contig_block_decomp(MPI_Comm comm, int wrank, int wsz)
recvcounts[i] = 1;
rdispls[i] = i * LOCAL_SZ * sizeof(double);

#ifndef MPI_SERIAL
ret = MPI_Type_dup(sendtype, &(recvtypes[i]));
#else
/* MPI serial does not support MPI_Type_dup() */
ret = MPI_Type_contiguous(LOCAL_SZ, MPI_DOUBLE, &(recvtypes[i]));
#endif
Comment on lines 225 to +229

@jayeshkrishna jayeshkrishna May 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed and rebased branch

if(ret == MPI_SUCCESS){
ret = MPI_Type_commit(&(recvtypes[i]));
}
if(ret != MPI_SUCCESS){
LOG_RANK0(wrank, "ERROR: Unable to create MPI dup of send type to recv doubles\n");
LOG_RANK0(wrank, "ERROR: Unable to create recv type (same as send type) to recv doubles\n");
return PIO_EINTERNAL;
Comment on lines 230 to 235

@jayeshkrishna jayeshkrishna May 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably a bigger change than what can fit in this PR (There are multiple cases in tests, across several tests, where we don't handle freeing of resources on fails). IMO, we can fix this (free resources when tests fail) in a later PR.

}
}
Expand Down
1 change: 0 additions & 1 deletion util/argparser.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
#include "mpi.h"
#include <iostream>
#include <sstream>
#include <string>
Expand Down
Loading