Summary of process creation and process i/o redirection in unix

Most of this material is covered fairly thoroughly in Haviland et al chapter 5.

Processes

A process is a running instance of a program.

ps ax

Your shell is a process; everything you run is a process.

Process creation

After the OS is initialized, the only way new processes come into being is by the fork() system call, which is a sufficiently fundamental tool as to have no parameters. The only thing a process can do is to split into two like an amoeba, by calling fork().

So, when one process calls fork(), two processes return from fork(). They differ only in their process ID number (every process has a unique process ID number ("pid")), and in the return value from fork().

It's a bit like a Star Trek transporter malfunction where there are suddenly two of a person. They start off identical, but their behaviour quickly diverges as the two of them observe their different situations. In the case of unix processes, they usually do an 'if' or 'switch' on the return value of fork and act differently immediately.

Executing programs

There is a family of library calls collectively referred to as "exec". What they all do is to execute another program by replacing this program with another. The difference between the library calls in the exec family is how the parameters work; they all do the same thing, but you can call different ones depending on the format of the data you are working with.

Example of using execl(): execl.c
Example of using execve(): execve.c

Note that if you call exec (of any flavour), unless it fails, this program is gone. So you usually want something more complicated as in the next section.

The fork/exec/wait idiom

Since execing another program replaces this program with that program, if you want to "spawn" a new program, you first call fork(); then the child process can exec the other program.

Furthermore, you usually then want THIS program to be suspended until the other program is complete. For example, when you type a command to the shell, that program is executed, and only when it terminates do you get the next shell prompt. Even though that program and the shell are separate processes which can run independently.

Fork(), exec(), etc are very general tools, but we often want to use them in this very specific way.

So the basic idiom for running another program like how the shell does is to fork(), then the child process calls exec, and the parent calls wait() to wait for the other program to terminate before proceeding.

In programming you have to be very careful that if failures occur, the child process does not end up returning from functions or falling through 'if's and executing code meant for only the parent to be executing.

Example of the fork/exec/wait idiom: spawn.c

I/O redirection

Every unix/linux process expects to be started with three open files, on file descriptors 0, 1, and 2, being the standard input, standard output, and standard error, respectively.

But sometimes we do "i/o redirection" before execing another program, as you have seen in using the shell.

Let's take redirecting the standard output as an example. Since that other program is going to write to file descriptor 1 for its standard output (because that's what "standard" about standard output -- that it is on file descriptor 1), if we want this to go into a file named "file" instead of to wherever this process's standard output is going, what we need to do is to arrange for the file "file" to be open on file descriptor 1 when we exec that program.

If we are doing the equivalent of the shell command "prog >file", for a program named "prog", we need the redirection to happen after the fork(), because the parent's standard output should not be redirected. (If it's not obvious, anything we do before the fork() will apply to both processes, but anything we do after the fork() applies only to the process we do it in.)

A successful open() always gives us the lowest available file descriptor number. If file descriptors 0, 1, and 2 are open, and we call open() successfully, it will give us file descriptor 3. However, if we close file descriptor 1 before calling open(), so that the open file descriptors are now only 0 and 2, then a successful open() will return 1, because that's the lowest available file descriptor number. So, the file will be open on file descriptor 1.

Example of running "ls" with stdout redirected: spawnredir.c

Example of running "tr" with stdin redirected: spawnredirin.c

dup2()

It's not always possible to arrange file descriptors such that the creation of a new file descriptor happens on the file descriptor number that we want. We will see this when we get to pipes. But first, let's examine the dup2() call, which is a way to renumber file descriptors.

dup2() actually duplicates a file descriptor, rather than renumbering it. We call dup2(oldfd,newfd)

This is like if in the shell we had a "cp" command but no "mv" command. So if we wanted to do "mv file1 file2", we could do "cp file1 file2" and then "rm file1".
Similarly, we can use dup2() to renumber (say) file descriptor 5 to file descriptor 1 by calling dup2(5,1) and then close(5).
Of course our program would usually be using some variable for the oldfd value, rather than a literal 5.

So, this is not necessarily our preferred way to redirect stdout, because it's unnecessary as we can just close(1) before doing the open(), but to illustrate dup2(), here is an example of using dup2() to do stdout redirection: spawndup2.c

pipes

To create a pipe, we use the pipe() system call. This creates TWO file descriptors, for the two ends of the pipe.

If we are trying to set up a pipe between two processes, we need to call pipe() in a common ancestor of the two processes. E.g. if we type "ls|tr" to the shell, it forks() like always, then it calls pipe() in the child, then it forks again for the two processes ls and tr.
The reason that pipe() needs to be called in a common ancestor of the two communicating processes is that the only way we have of identifying the pipe is by the file descriptor.

Full example of setting up a pipeline: pipe-example.c