Title: COMP309/509 - Lecture 5 class: middle, center, inverse

COMP309/509 - Parallel and Distributed Computing

Lecture 6 - More Interprocess Communication

By Mitchell Welch

University of New England

Reading

Chapters 14 and 15 from Advanced Programming in the UNIX environment
Assignment 1 Description

Summary

FIFOs
File Locking fcntl
Shared Memory

FIFOs

FIFOs are essentially named pipes that can be used for communication between processes that do not have a common ancestor.
FIFOs exist within the file system and have a pathname just like a file
They allow unrelated processes to exchange data

FIFOs

There are two calls for creating FIFOs

#include <sys/types.h>
#include <sys/stat.h>

int mkfifo(const char *path, mode_t mode);

int mkfifoat(int dirfd, const char *pathname, mode_t mode);

FIFOs

path is the path for the FIFO (if this is an absolute pathname fd is ignored in mkfifoat(..)
mode is the open mode (i.e. read/write).
fd is a file descriptor that represents a directory. the FIFO is created relative to the directory.

FIFOs

A simple example that creates a FIFO and writes a string:

#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>

int main(){
    int fd;
    char * myfifo = "/tmp/myfifo";
    /* create the FIFO (named pipe) */
    mkfifo(myfifo, 0666);
    /* write "Hi" to the FIFO */
    fd = open(myfifo, O_WRONLY);
    write(fd, "Hi", sizeof("COMP309 is cool!"));
    close(fd);
    /* remove the FIFO */
    unlink(myfifo);
    return 0;
}

FIFOs

A simple example that reads from our fifo

#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>
#define MAX_BUF 1024

int main(){
    int fd;
    char * myfifo = "/tmp/myfifo";
    char buf[MAX_BUF];
    /* open, read, and display the message from the FIFO */
    fd = open(myfifo, O_RDONLY);
    read(fd, buf, MAX_BUF);
    printf("Received: %s\n", buf);
    close(fd);
    return 0;
}

FIFOs

FIFOs can be used to create client-server styled process architectures. .center[]

FIFOs

This approach allows the client to send data to the server.
The client can’t read information back through the fifo.
Solution: Establish an additional fifo when a client connects to the server.
This new fifo is only used by the server to send information back to specific clients
The well-known fifo is still used by all clients to send information to the server.
This approach can be achieved by using the clients pid to name the reply FIFO.

FIFOs

Full duplex communication:

.center[ center-aligned image ]

File Locking

Sane concurrency often requires mutual exclusion.
This is the need to prevent other things from touching something while you futz with it.
We can use file locking as a means of mutual exclusion.
There are a variety of file locks in UNIX:
- flock in <sys/file.h>
- lockf in <unistd.h>
- fcntl in <fcntl.h>
We’ll use fcntl. fcntl locks are also called record locks
- a.k.a file-segment locks
- a.k.a file-region locks

File Locking

The file system allows both read (l_type of F_RDLCK) and write (F_WRLCK) locks on file descriptors.
You lock open file descriptors.
The file descriptor must be readable for a read lock.
The file descriptor must be writable for a write lock.
Many processes can have a read lock, only one may have a write lock.

File Locking

Since we desire mutual exclusion we’ll use write locks.
Locks are not inherited via a fork().
Locks are preserved by an exec().
Locks can be seen in /proc/locks.

File Locking

Locks are advisory in the sense that they don’t actually prevent IO operations on the file descriptor.
Linux also has mandatory locks but we will not use them. (POSIX doesn’t require them.)
Locks are lost by unlocking them.
Locks are lost on exiting.

File Locking

Locks are lost on closing the file descriptor.
Locks are also lost if you open and close a file that the file descriptor refers to.
Locks can be quite fine grained and can lock a specific region of a file, though we will not use this.

File Locking

There are three fcntl prototypes:

int fcntl(int fd, int cmd);
int fcntl(int fd, int cmd, long arg);
int fcntl(int fd, int cmd, struct flock *lock);

who said C didn’t have overloading?
We’ll use the third:

int fcntl(int fd,               /* open file descriptor            */
          int cmd,              /* F_GETLK,  F_SETLK  or F_SETLKW  */
          struct flock *lock);  /* a struct containing details etc */

Read man fcntl!

File Locking

The flock struct

 struct flock {
         short l_type;    /* Type of lock: F_RDLCK,F_WRLCK, F_UNLCK */
         short l_whence;  /* How to interpret l_start: SEEK_SET, SEEK_CUR, SEEK_END */
         off_t l_start;   /* Starting offset for lock */
         off_t l_len;     /* Number of bytes to lock */
         pid_t l_pid;     /* PID of process blocking our lock (F_GETLK only) */

     };

File Locking

To lock an open file descriptor fd do the following:
Create a clean struct flock object lock.
- Set the l_type field of lock to F_RDLCK for a read lock.
- Set the l_type field of lock to F_WRLCK for a write lock.
Call fcntl like so:

errcode = fcntl(fd, F_SETLK, &lock);

On success errcode is 0. On error errcode is -1, and errno is set.
This call will block until you have acquired the lock or an error occurs, or it is interrupted by a signal.

File Locking

In our programs we will be locking and unlocking most of the time, so we can write some functions to reuse (headers):

/*filelock.h*/
#include <fcntl.h> /* for struct flock */

typedef struct flock fileLock_t;
typedef fileLock_t *fileLock_p;

/* make a nice clean lock                                     */
fileLock_p makeLock(void);

/* obtains a write lock on the fd, returns the call to fcntl  */
int getLock(fileLock_p lock, int fd);

/* unlocks the write lock on fd,   returns the call to fcntl  */
int unLock(fileLock_p lock, int fd);

/* frees the lock made by a makeLock                          */
void freeLock(fileLock_p lock);

File Locking

#include <stdlib.h>  /* for NULL            */
#include <stdio.h>   /* for fprintf         */
#include <string.h>  /* for memset          */

#include "fileLock.h" /* includes <fcntl.h> */

fileLock_p makeLock(void){
  fileLock_p retval = (fileLock_p)calloc(1, sizeof(fileLock_t));
  if(retval == NULL){
    perror("makeLock: calloc failed!");
  }
  return retval;
}

int getLock(fileLock_p lock, int fd){
  memset(lock, 0, sizeof(fileLock_t));
  lock->l_type = F_WRLCK;
  return fcntl(fd, F_SETLKW, lock);
}
...

File Locking

...
int unLock(fileLock_p lock, int fd){
  lock->l_type = F_UNLCK;
  return fcntl(fd, F_SETLKW, lock);
}

void freeLock(fileLock_p lock){
  free(lock);
}

File Locking

A simple example of file locking in action (using our functions)

#include <stdio.h>      /* for fprintf */
#include <stdlib.h>     /* for exit    */
#include <sys/types.h>  /* for open    */
#include <sys/stat.h>   /* for open    */
#include <fcntl.h>      /* for open    */

#include "fileLock.h"

int main(int argc, char** argv){
  int fd;
  char* file;
  if(argc != 2){
    fprintf(stderr, "Usage: %s <filename>\n", argv[0]);
    exit(EXIT_FAILURE);
  }
  ...

File Locking

  file = argv[1];
  fd = open(file,  O_RDWR, S_IRWXU);
  if(fd < 0){
    perror("File opening failed.");
    exit(EXIT_FAILURE);
  } else {
    int lval;
    fileLock_p lock = makeLock();
    if(lock == NULL){
      perror("Lock creation failed.");
      exit(EXIT_FAILURE);
    }
 ...

File Locking

    fprintf(stderr, "Attempting to lock %s\n", file);
    lval = getLock(lock, fd);
    if(lval == -1){
      perror("Locking failed");
      exit(EXIT_FAILURE);
    }
    fprintf(stderr, "Successfully locked %s (hit any key to continue)\n", file);
    getchar();
    unLock(lock, fd);
    freeLock(lock);
    exit(EXIT_SUCCESS);
  }
}

File Locking

Final points:
- Beware of Deadlock - this occurs when two processes are each waiting for a resource that the other has.
- Remember that a process blocks while waiting for a lock on a file.
- The kernel can detect deadlock and will return an error to the child or the parent (this is implementation specific)
- The kernel will break the deadlock, by giving the resource to one of the processes and returning the error to the other.

Shared Memory

There are two ways in Linux for processes to share memory:
- Allocating a POSIX Shared memory segment using shmget:


#include <sys/ipc.h>
#include <sys/shm.h>

int shmget(key_t key, size_t size, int shmflg);

Mapping a file into memory using mmap

#include <sys/mman.h>

 void  *mmap(void *start, size_t length, int prot , 
             int flags, int fd,  off_t offset);

We’ll use mmap in this unit as its slightly more portable.

Shared Memory

mmap allows us to map a file into a processes' memory.
Operations on the memory are reflected in the files' contents.
Thus processes that map the same file may communicate through it.

 void  *mmap(void *start, size_t length, int prot ,int flags, int fd,  off_t offset);

fd is the (file descriptor) of the file that is to be mapped.
length is the amount to mapped.
offset is where to start in the file.
start is an address where you would like it to be, or 0 if you are not fussy. We will never be fussy.

Shared Memory

We’ll look at prot & flags shortly.
If successful mmap returns the address of the mapped memory
On failure MAP_FAILED is returned, MAP_FAILED is -1.
Memory mapped by mmap is preserved across forks, with the same attributes.

Shared Memory

prot is the desired memory protection of the mapped memory.
prot is PROT_NONE or the bitwise OR of one or more of
- PROT_READ – The memory may be read.
- PROT_WRITE – The memory may be written.
- PROT_EXEC – The memory may be executed.
prot being PROT_NONE means the memory is inaccessible.
prot must not conflict with open mode of the file.

Shared Memory

flags can be a (sensible) combination of:
- MAP_FIXED
- MAP_PRIVATE
- MAP_SHARED
MAP_PRIVATE and MAP_SHARED are incompatible.
MAP_PRIVATE means the mapping is not visible to other processes.
MAP_FIXED means you are fussy about the starting address start.
MAP_SHARED is the mode we will use.

Shared Memory

We will use mmap like so:

startaddr = mmap(0, 
                 size, 
                 PROT_READ | PROT_WRITE,
                 MAP_SHARED,
                 fd, 
                 0);

Where fd is a file we have made especially.
size is how much space we need and will be the size of the file. The return address will be what we are interested in!

Shared Memory

Since the memory will be shared amongst several processes: we will need to enforce mutual exclusion.
Two processes should not access the mapped memory at the same time.
Advisory file locking the file is the obvious choice.
We will use the shared data protocol throughout this unit.

Shared Memory

To each unit of shared data we assign a mutual exclusion device.
To read or write the unit of shared data we:
- Lock: First obtain sole ownership of the mutual exclusion device.
- Examine: Futz with the unit of shared data.
- Unlock: Relinquish sole ownership of the mutual exclusion device.
Simple, yes?

Shared Memory

In the first example we will treat the memory startaddr as a pointer to an int:

int *count;
...
size_t size = sizeof(int);
...
startaddr = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if(start_addr == MAP_FAILED){
  perror("mmap failed");
  exit(EXIT_FAILURE);
}
...
count = (int *)start_addr;

Shared Memory

In the second example we will treat the memory startaddr as a pointer to an struct:

typedef struct shared {
  int    nprocs;
  pid_t  pids[PROC_MAX];
} shared_t;

size_t size = sizeof(shared_t);
shared_t *shared;

startaddr = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if(start_addr == MAP_FAILED){
  perror("mmap failed");
  exit(EXIT_FAILURE);
}
shared = (shared_t *)start_addr;

Shared Memory

Review example01.c
Review exmaple02.c

Summary

FIFOs
File Locking fcntl
Shared Memory

class: middle, center, inverse

Questions?

Reading

Chapters 14 and 15 from Advanced Programming in the UNIX environment
Assignment 1 Description