Title: COMP309/509 - Lecture 19 class: middle, center, inverse

COMP309/509 - Parallel and Distributed Computing

Lecture 19 - Matrix Multiplication Performance

By Mitchell Welch

University of New England

Reading

The Program Examples from PVM: Parallel Virtual Machine A Users' Guide and Tutorial for Networked Parallel Computing
Assignment 3
Project (COMP509)

Summary

Versions
Performance Runs

Versions

The program evolved in a series of several steps:
Version mm.0.0.c:
- This is the origonal version.
- Downloaded from Oakridge National Laborotory.
Version mm.0.1.c:
- added some primitive timing information and a verbose mode.
Version mm.0.2.c:
- tailored it to single architecture mode.
- enlarged MAXROW to something more reasonable.
- This had no real effect on the times.

Versions

The program continued to evolve:
Version mm.0.3.c:
- Added real argument arrays.
- Used these instead of the initial broadcasting of details.
- Added three file arguments for the input and output, and got rid of the goto.
- Added VERSION prior to making it automatic and put everything in a library

Performance Runs

bourbaki Mon May 11 Torus $ ./mkMatrices ~/A ~/B ~/C 2000
Done writing files
bourbaki Mon May 11 Torus $ ./spmm.1 ~/A ~/B ~/C 2000
Done smoothly after 84 seconds
bourbaki Mon May 11 Torus $ ls -lh ~/A ~/B ~/C
-rw------- 1 comp309 comp309 16M May 11 20:57 /homes/unit/comp309/A
-rw------- 1 comp309 comp309 16M May 11 20:57 /homes/unit/comp309/B
-rw-rw-r-- 1 comp309 comp309 16M May 11 20:58 /homes/unit/comp309/C
bourbaki Mon May 11 Torus $

Performance Runs

bourbaki Mon May 11 Torus $ ./mm.0.4 ~/A ~/B ~/C 2 1000
Task 262243 with:
    m = 2
    blksize =  1000
    ntask  =  4
Spawning 3 tasks
Waiting at barrier
Crossed  barrier
Opening, allocating and initializing matrices
Finished opening, allocating and initializing matrices
Looping: iteration  1 out of a total of 2
Looping: iteration  2 out of a total of 2
Finished computation
Waiting at barrier
Crossed  barrier
Writing C
mm.0.4 2 1000 task 262243 done successfully after 26 seconds.

Performance Runs

bourbaki Mon May 11 Torus $ ./mm.0.4 ~/A ~/B ~/C 4 500
Task 262240 with:
    m = 4
    blksize =  500
    ntask  =  16
Spawning 15 tasks
Waiting at barrier
Crossed  barrier
Opening, allocating and initializing matrices
Finished opening, allocating and initializing matrices
Looping: iteration  1 out of a total of 4
Looping: iteration  2 out of a total of 4
Looping: iteration  3 out of a total of 4
Looping: iteration  4 out of a total of 4
Finished computation
Waiting at barrier
Crossed  barrier
Writing C
mm.0.4 4 500 task 262240 done successfully after 9 seconds.

Performance Runs

bourbaki Mon May 11 Torus $ ./mm.0.4 ~/A ~/B ~/C 8 250
Task 262158 with:
    m = 8
    blksize =  250
    ntask  =  64
Spawning 63 tasks
Waiting at barrier
Crossed  barrier
Opening, allocating and initializing matrices
Finished opening, allocating and initializing matrices
Looping: iteration  1 out of a total of 8
Looping: iteration  2 out of a total of 8
Looping: iteration  3 out of a total of 8
Looping: iteration  4 out of a total of 8
Looping: iteration  5 out of a total of 8
Looping: iteration  6 out of a total of 8
Looping: iteration  7 out of a total of 8
Looping: iteration  8 out of a total of 8
Finished computation
Waiting at barrier
Crossed  barrier
Writing C
mm.0.4 8 250 task 262158 done successfully after 8 seconds.

Performance Runs


bourbaki Mon May 11 Torus $ ./mm.0.4 ~/A ~/B ~/C 16 125
Task 262166 with:
    m = 16
    blksize =  125
    ntask  =  256
Spawning 255 tasks
Waiting at barrier
Crossed  barrier
Opening, allocating and initializing matrices
Finished opening, allocating and initializing matrices
Looping: iteration  1 out of a total of 16
Looping: iteration  2 out of a total of 16
Looping: iteration  3 out of a total of 16
Looping: iteration  4 out of a total of 16
Looping: iteration  5 out of a total of 16
Looping: iteration  6 out of a total of 16
Looping: iteration  7 out of a total of 16
Looping: iteration  8 out of a total of 16
Looping: iteration  9 out of a total of 16
Looping: iteration  10 out of a total of 16
Looping: iteration  11 out of a total of 16
Looping: iteration  12 out of a total of 16
Looping: iteration  13 out of a total of 16
Looping: iteration  14 out of a total of 16
Looping: iteration  15 out of a total of 16
Looping: iteration  16 out of a total of 16
Finished computation
Waiting at barrier
Crossed  barrier
Writing C
mm.0.4 16 125 task 262166 done successfully after 11 seconds.

Performance Runs


bourbaki Mon May 11 Torus $ ./mm.0.4 ~/A ~/B ~/C 20 100
Task 262195 with:
    m = 20
    blksize =  100
    ntask  =  400
Spawning 399 tasks
Waiting at barrier
Crossed  barrier
Opening, allocating and initializing matrices
Finished opening, allocating and initializing matrices
Looping: iteration  1 out of a total of 20
Looping: iteration  2 out of a total of 20
Looping: iteration  3 out of a total of 20
Looping: iteration  4 out of a total of 20
Looping: iteration  5 out of a total of 20
Looping: iteration  6 out of a total of 20
Looping: iteration  7 out of a total of 20
Looping: iteration  8 out of a total of 20
Looping: iteration  9 out of a total of 20
Looping: iteration  10 out of a total of 20
Looping: iteration  11 out of a total of 20
Looping: iteration  12 out of a total of 20
Looping: iteration  13 out of a total of 20
Looping: iteration  14 out of a total of 20
Looping: iteration  15 out of a total of 20
Looping: iteration  16 out of a total of 20
Looping: iteration  17 out of a total of 20
Looping: iteration  18 out of a total of 20
Looping: iteration  19 out of a total of 20
Looping: iteration  20 out of a total of 20
Finished computation
Waiting at barrier
Crossed  barrier
Writing C
mm.0.4 20 100 task 262195 done successfully after 12 seconds.

Performance Runs

Here we never actually create the matrices A, B, and C.
Rather we make their blocks.
For this we use the mkBlocks program.

bourbaki Mon May 11 Torus $ mkBlocks /tmp/A /tmp/B /tmp/C 4 100
Done writing files
bourbaki Mon May 11 Torus $ ls -l /tmp/A*
-rw------- 1 comp309 comp309     0 May 11 21:42 /tmp/A
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.0.0
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.0.1
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.0.2
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.0.3
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.1.0
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.1.1
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.1.2
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.1.3
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.2.0
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.2.1
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.2.2
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.2.3
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.3.0
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.3.1
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.3.2
-rw------- 1 comp309 comp309 40000 May 11 21:42 /tmp/A.3.3


bourbaki.une.edu.au.mmult> 
bourbaki.une.edu.au.mmult> mm.0.5 /tmp/A /tmp/B /tmp/C 4 100
mm.0.5 4 100 task 262181 done successfully after 7 seconds.
bourbaki.une.edu.au.mmult> cmp /tmp/A.2.3 /tmp/C.2.3
bourbaki.une.edu.au.mmult>
bourbaki.une.edu.au.mmult> mkBlocks /tmp/A /tmp/B /tmp/C 4 200
Done writing files
bourbaki.une.edu.au.mmult> mm.0.5 /tmp/A /tmp/B /tmp/C 4 200
mm.0.5 4 200 task 262197 done successfully after 47 seconds.
bourbaki.une.edu.au.mmult>
bourbaki.une.edu.au.mmult> mkBlocks /tmp/A /tmp/B /tmp/C 4 300
Done writing files
bourbaki.une.edu.au.mmult> mm.0.5 /tmp/A /tmp/B /tmp/C 4 300
mm.0.5 4 300 task 262213 done successfully after 163 seconds.
bourbaki.une.edu.au.mmult>
bourbaki.une.edu.au.mmult> mm.0.5 /tmp/A /tmp/B /tmp/C 4 400
mm.0.5 4 400 task 262229 done successfully after 525 seconds.
bourbaki.une.edu.au.mmult> cmp /tmp/A.0.3 /tmp/C.0.3
bourbaki.une.edu.au.mmult>

Performance Runs

But now we hit the snag that tmp is not big enough.

bourbaki.une.edu.au.mmult> mkBlocks /tmp/A /tmp/B /tmp/C 4 500
write failed: Success
write of matrix C: Success
Done writing files
bourbaki.une.edu.au.mmult>

Performance Runs

The mm.0.7.c Program
This version aims to provide more robust handling of errors.
The program begins by the master process spawning a number of pvm tasks.
The master then checks if all children were successfully spawned, and waits to recieve one of two signals.
The master process does not perform any of the matrix calculations.

Performance Runs

It’s job is simply wait to receive either a SIGUSR1 or a SIGUSR2 signal from its slave tasks.
A SIGUSR1 from a child indicates some error has occured (e.g opening a file, receiving a block etc.
A SIGUSR2 from a child indicates that the child has finished, and is exiting.
Upon receiving a SIGUSR1, the master will kill all slave tasks, report and error and shutdown.
Upon receiving a SIGUSR2 from a child, the master will check if any slaves are still running.
If there are no slaves running, the calcualtions are complete, and the program will exit, otherwise it goes back to waiting for signals.

Summary

Versions
Performance Runs

class: middle, center, inverse

Questions?

Reading

The Program Examples from PVM: Parallel Virtual Machine A Users' Guide and Tutorial for Networked Parallel Computing
Assignment 3
- Project (COMP509)