CollectiveCommunication Class Reference

This class is used to transfer several values of different types with a single command. More...

#include <CollectiveCommunication.h>

Inheritance diagram for CollectiveCommunication:

Public Member Functions
virtual void	init (MPI_Comm communicator, int numValues, int key=0) override
	allocate memory for the values to be sent, initialize counters More...
virtual void	finalize () override
	delete memory and MPI_Type More...
void	appendInt (int intValue) override
void	appendUnsLong (unsigned long unsLongValue) override
void	appendFloat (float floatValue) override
void	appendDouble (double doubleValue) override
void	appendLongDouble (long double longDoubleValue) override
MPI_Comm	getTopology () override
void	broadcast (int root=0) override
void	allreduceSum () override
	Performs an all-reduce (sum) More...
void	allreduceCustom (ReduceType type) override
virtual void	allreduceSumAllowPrevious () override
void	scanSum () override
	Performs a scan (sum) More...
virtual size_t	getTotalSize () override
Public Member Functions inherited from CollectiveCommBase
void	init (int numValues)
	allocate memory for the values to be stored, initialize getter-iterator More...
virtual int	getInt () override
virtual unsigned long	getUnsLong () override
virtual float	getFloat () override
virtual double	getDouble () override
virtual long double	getLongDouble () override
Public Member Functions inherited from CollectiveCommBaseInterface
virtual	~CollectiveCommBaseInterface ()
	virtual destructor
Public Member Functions inherited from CollectiveCommunicationInterface
	~CollectiveCommunicationInterface () override=default
	virtual destructor

Protected Member Functions
void	setMPIType ()
	defines a MPI datatype which can be used to transfer a CollectiveCommunication object More...

Static Protected Member Functions
static void	add (valType *invec, valType *inoutvec, int *, MPI_Datatype *dtype)
	method used by MPI to add variables of this type More...
static void	max (valType *invec, valType *inoutvec, int *, MPI_Datatype *dtype)
static void	min (valType *invec, valType *inoutvec, int *, MPI_Datatype *dtype)

Protected Attributes
std::vector< MPI_Datatype >	_types
	Vector of the corresponding MPI types for the values stored in _values.
MPI_Datatype	_agglomeratedType
MPI_Comm	_communicator
	Communicator to be used by the communication commands.
Protected Attributes inherited from CollectiveCommBase
std::vector< valType >	_values
	Vector to store the values which shall be communicated.
std::vector< valType >::const_iterator	_getter
	Iterator to extract the values which were communicated.

Detailed Description

This class is used to transfer several values of different types with a single command.

Author

Nikola Tchipev, Martin Buchholz

At different points in the simulation process, collective communication commands are necessary. One thing that always has to be done is a reduce command to get global macroscopic values from the local ones. But depending on the application, other commands might also be possible (also broadcast commands). The number of values to be transferred from/to each proc is usually very small (about ten or even less). The main costs of those communications are therefore not caused by the amount of data, but by the MPI overhead caused by each call of a communication command. The purpose of this class is to use a single MPI command to transfer several values of possible different types. Currently supported commands are:

• broadcast
• reduce using add as reduce operation

Currently supported datatypes are:

• MPI_INT
• MPI_UNSIGNED_LONG
• MPI_FLOAT
• MPI_DOUBLE
• MPI_LONG_DOUBLE

Further datatypes and reduce operations could be added very easily. A typical usage of this class could look like this:

// create object
CollectiveCommunication collComm;
// Set number of values to be sent
collComm.init(comm, 4);
 
// store values to be sent
collComm.appendInt(5);
collComm.appendInt(8);
collComm.appendDouble(1.3);
collComm.appendUnsLong(9);
 
// execute collective communication
collComm.allreduceSum();
 
// read values (IMPORTANT: same order as for storing)
int i1 = collComm.getInt();
int i2 = collComm.getInt();
double d = collComm.getDouble();
unsigned long l = collComm.getUnsLong();
 
// finalize the communication (important for deleting memory)
collComm.finalize();

Member Function Documentation

add()

static void CollectiveCommunication::add ( valType *  invec, valType *  inoutvec, int *  , MPI_Datatype *  dtype  )

inlinestaticprotected

method used by MPI to add variables of this type

For collective reduce commands, an operation has to be specified which should be used to combine the values from the different processes. As with this class, special datatypes are sent, the build-in MPI operations don't work on those datatypes. Therefore, operations have to be defined which work on those datatypes. MPI allows to create own operations (type MPI_Op) by specifying a function which will be used in the reduce operation. MPI specifies the signature of such functions This methods checks from which basic datatypes the given datatype was constructed and performs an add operation for each of the basic types.

allreduceCustom()

void CollectiveCommunication::allreduceCustom ( ReduceType  type )

inlineoverridevirtual

Performs an allreduce operation with a custom reduce type.

Parameters

type	the type of the operation

Reimplemented from CollectiveCommBase.

allreduceSum()

void CollectiveCommunication::allreduceSum ( )

inlineoverridevirtual

Performs an all-reduce (sum)

Reimplemented from CollectiveCommBase.

allreduceSumAllowPrevious()

virtual void CollectiveCommunication::allreduceSumAllowPrevious ( )

inlineoverridevirtual

Performs an all-reduce (sum), however values of previous iterations are permitted. By allowing values from previous iterations, overlapping communication is possible. One possible use case for this function is the reduction of slowly changing variables, e.g. the temperature.

Implements CollectiveCommunicationInterface.

Reimplemented in CollectiveCommunicationSingleNonBlocking.

appendDouble()

void CollectiveCommunication::appendDouble ( double  doubleValue )

inlineoverridevirtual

Append a double value to the list of values to be stored

Parameters

doubleValue

the value

Reimplemented from CollectiveCommBase.

appendFloat()

void CollectiveCommunication::appendFloat ( float  floatValue )

inlineoverridevirtual

Append a float value to the list of values to be stored

Parameters

floatValue

the value

Reimplemented from CollectiveCommBase.

appendInt()

void CollectiveCommunication::appendInt ( int  intValue )

inlineoverridevirtual

Append an int value to the list of values to be stored

Parameters

intValue

the value

Reimplemented from CollectiveCommBase.

appendLongDouble()

void CollectiveCommunication::appendLongDouble ( long double  longDoubleValue )

inlineoverridevirtual

Append a long double value to the list of values to be stored

Parameters

longDoubleValue

the value

Reimplemented from CollectiveCommBase.

appendUnsLong()

void CollectiveCommunication::appendUnsLong ( unsigned long  unsLongValue )

inlineoverridevirtual

Append a unsigned long value to the list of values to be stored

Parameters

unsLongValue

the value

Reimplemented from CollectiveCommBase.

broadcast()

void CollectiveCommunication::broadcast ( int  = 0 )

inlineoverridevirtual

Performs a broadcast of the values to all processes in the communicator

Parameters

root	of the broadcast

Reimplemented from CollectiveCommBase.

finalize()

virtual void CollectiveCommunication::finalize ( )

inlineoverridevirtual

delete memory and MPI_Type

Returns

the value

Reimplemented from CollectiveCommBase.

Reimplemented in CollectiveCommunicationSingleNonBlocking.

getTopology()

MPI_Comm CollectiveCommunication::getTopology ( )

inlineoverridevirtual

Get the MPI communicator

Returns

MPI communicator

Implements CollectiveCommunicationInterface.

getTotalSize()

virtual size_t CollectiveCommunication::getTotalSize ( )

inlineoverridevirtual

get the size of the entire object allocated memory

Returns

size of the dynamically allocated memory in bytes

Reimplemented from CollectiveCommBase.

init()

virtual void CollectiveCommunication::init ( MPI_Comm  communicator, int  numValues, int  key = 0  )

inlineoverridevirtual

allocate memory for the values to be sent, initialize counters

Parameters

numValues

number of values that shall be communicated

Implements CollectiveCommunicationInterface.

Reimplemented in CollectiveCommunicationSingleNonBlocking.

scanSum()

void CollectiveCommunication::scanSum ( )

inlineoverridevirtual

Performs a scan (sum)

Reimplemented from CollectiveCommBase.

setMPIType()

void CollectiveCommunication::setMPIType ( )

inlineprotected

defines a MPI datatype which can be used to transfer a CollectiveCommunication object

before this method is called, init has to be called and all values to be communicated have to be added with the append<type> methods. Then this method will construct a MPI-datatype which represents all the added values. The datatype is stored in the member variable _valuesType;

Member Data Documentation

_agglomeratedType

MPI_Datatype CollectiveCommunication::_agglomeratedType

protected

MPI_Datatype which will be used in the communication command and which represents all values

---

The documentation for this class was generated from the following file:

• src/parallel/CollectiveCommunication.h