Title: COMP309/509 - Lecture 21 class: middle, center, inverse

COMP309/509 - Parallel and Distributed Computing

Lecture 21 - Massage Passing Interface

By Mitchell Welch

University of New England

Reading

A users guide to MPI (This respurce is a bit dataed, however the examples are still current)
man mpirun
Assignment 4 - comming this afternoon
Project (COMP509)

Summary

Advanced MPI
Intergral Estimation - The Trapaziodal rule

Advanced MPI

We have finished Elementary MPI!
We’ll continue on our tour of MPI looking at Advanced MPI
The Integration examples from the MPI user’s guide
Multiplying Matrices on a Torus in MPI.

Advanced MPI

Our coverage of advanced MPI will touch upon:
- Simple synchronization.
- Complex patterns of communication.
- User defined datatypes.
- Non-blocking communication.
- Communicators and topologies.

Advanced MPI

There is no guarantee about message orderings.
There are wildcards for receiving.
There are no wild cards for sending.
MPI_ANY_SOURCE is the wild card for receiving.
Messages can be distinguished by tags and communicators - more on communicators later.
There is a wild card for tags: MPI_ANY_TAG
There are no wild cards for communicators.

Advanced MPI

#include "mpi.h"
int MPI_Barrier(MPI_Comm comm)

Is a simple analogue to the PVM synchronization primitive.
Blocks until all processes in the communicator have reached this routine.
In other words, it blocks the caller until all group members have called it.
The call returns at any process only after all group members have entered the call.
Easy to use with MPI_COMM_WORLD
If the underlying device cannot do better, a tree-like or combine algorithm is used to broadcast a message to all members of the communicator.

Intergral Estimation - The Trapaziodal rule

Suppose that we want to calculate:

.center[ intergral ]

for some fixed f

In these lectures we will fix f(x) = x² , just to keep things simple.
Remember the integral computes the area under the curve f in the region from a to b

Intergral Estimation - The Trapaziodal rule

One way to do this is to use the infamous Trapezoidal Rule!
Recall from those hot math nights that a trapezoid is quadrilateral having only two sides parallel.
Estimating an Integral - The Trapezoid Rule

Intergral Estimation - The Trapaziodal rule

We partition the interval [a,b] into n equal subintervals of width h , where: h = ( b - a ) / n

.center[ intergral ]

Intergral Estimation - The Trapaziodal rule

.center[ intergral ]

Intergral Estimation - The Trapaziodal rule

We then compute the area $ A$ of each trapeziod:

.center[ intergral ]

Thus:

.center[ intergral ]

Then sum these:

.center[ intergral ]

Intergral Estimation - The Trapaziodal rule

Giving the Trapezoidal Rule:

.center[ intergral ]

Where for 0 < i < n we have x_i = a + (i x h) .


float f(float x){ return x*x; }

float Trap(float a, float b, int n, float h){ 
    float integral, x;
    int i; 
    integral = (f(a) + f(b))/2.0; 
    x = a; 
    for (i = 1; i <= n-1; i++) { 
        x += h; 
        integral += f(x); } 
    return integral * h;
}

Intergral Estimation - The Trapaziodal rule

Prior to parallelizing we do a simple sequential analogue.
We pick an interval and the number of trapezoids n.
From this we compute our h.
We divide the interval into nproc subtasks/subintervals.
On each subtask we use Trap to compute the integral on the subintervals.
The sum is the result

Intergral Estimation - The Trapaziodal rule

#include <stdio.h>
#include "trap.h"
int main(int argc, char** argv) {
  float h, a = 0.0, b = 1.0, integral = 0, local_a, local_b;    
  int i, n = 1000, nproc = 10, local_n = n/nproc;
  h = (b-a)/n;    
  for(i = 0; i < nproc; i++){
    local_a = a + i*local_n*h;
    local_b = a + (i + 1)*local_n*h;
    integral += Trap(local_a, local_b, local_n, h);
  }
  printf("With n = %d trapezoids\n", n);
  printf("The integral from %f to %f is approx %f\n", 
         a, b, integral); 
  return 0;
}

Intergral Estimation - The Trapaziodal rule

We’ll look at five versions in the next few lectures. These are:
Version 0: The simplest version (everything hardwired).
Version 1: Obtains the endpoints and increment from the user.
Version 2: Uses MPI’s collective communication primitives.
Version 3: Uses MPI’s User defined type mechanism.
Version 4: Uses MPI’s pack and unpack primitives.
Version 5: Uses the C sizeof() macro and user defined types.

Intergral Estimation - The Trapaziodal rule

From Peter Pacheco’s An Introduction to Parallel Programming.
Parallel Trapezoid Rule: version 0.
Estimate of the integral from a to b of f(x) using the trapezoid rule and n trapezoids.
Algorithm:
1. Each process calculates its interval of integration.
2. Each process estimates the integral of f(x) over its interval using the trapezoid rule.
3. Each process whose rank is not 0 sends its integral to 0.
4. Process 0 sums the calculations received from the individual processes and prints the result.
Note: in this version f(x) , a , b , and n are all hardwired.

Intergral Estimation - The Trapaziodal rule

Review parallel_trap0.c

Intergral Estimation - The Trapaziodal rule

The first thing we can do to improve this program is eliminate the hardwired endpoints and the number of trapezoids.
Could use argument arrays, but remember they are not standard in MPI.
So the member with rank zero asks the user and then sends them to the others.
We’ll send the info one by one, disambiguated by tags.

Intergral Estimation - The Trapaziodal rule

Review parallel_trap1.c

Intergral Estimation - The Trapaziodal rule

A more efficient way of communicating this sort of information is obtained using the MPI broadcasting facility.
MPI encourages efficient implementation (via trees) of broadcasting.
In a similar fashion, accumulating the results is done via the MPI reduce facility.

Intergral Estimation - The Trapaziodal rule

#include "mpi.h"
int MPI_Bcast(void*         buffer, 
              int           count,  
              MPI_Datatype  datatyp, 
              int           root, 
              MPI_Comm      comm)

N.B. All tasks in the communicatior must call this routine!
Broadcasts a message from the process with rank root to all other processes of the communicator group.
This call acts like a send for the process with rank root.
For all other processes in the group it acts like a receive!

Intergral Estimation - The Trapaziodal rule

buffer is the starting address of the buffer.
count is the number of things in the buffer.
datayp is the data type of the stuff in the buffer.
root is the rank of the person broadcasting.

Intergral Estimation - The Trapaziodal rule

#include "mpi.h"
int MPI_Reduce (void*        sndbuf, 
                void*        rcvbuf, 
                int          count, 
                MPI_Datatype datatyp, 
                MPI_Op       op, 
                int          root, 
                MPI_Comm     comm)

N.B. All tasks in the communicatior must call this routine!
This procedure reduces values on all processes to a single value
This is a more efficient way of collecting results, since it uses tree shaped communication.

Intergral Estimation - The Trapaziodal rule

sndbuf is the send buffer
rcvbuf address of receive buffer
- significant only at root
datatyp is data type of elements of send buffer
op is the reduce operation.
root is the rank of the root process
comm is the communicator

Intergral Estimation - The Trapaziodal rule

It is possible to create you own.
Built in’s include: MPI_MAX MPI_MIN MPI_SUM MPI_PROD MPI_LAND MPI_LOR
For intergration MPI_SUM will be useful.

Intergral Estimation - The Trapaziodal rule

Review parallel_trap2.c

Summary

Advanced MPI
Intergral Estimation - The Trapaziodal rule

class: middle, center, inverse

Questions?

Reading

A users guide to MPI (This respurce is a bit dataed, however the examples are still current)
man mpirun
Assignment 4 - comming this afternoon
Project (COMP509)