Terminal Sickness

I have a simple requirement: I want my programme to be able to spawn another programme on the same machine and talk to it using the simplest possible means of communication, that is, via the spawned programme's standard input and output streams.

The driving programme is written in C and the spawned programme could have been written in any programming language (hence the need for using as simple a form of communication as possible). The messages exchanged between the programmes are terminated by newline characters.

This should have been a relatively simple task with code somewhat like the following (error-handling has been omitted for clarity):

  /* File descriptors for pipes connecting driver and inferior
     processes. */
  int drv2inf[2], inf2drv[2];
  
  /* Create communication pipes. */
  pipe (drv2inf); pipe (inf2drv);

  if (fork () == 0) {
    /* In child process - close "other" ends of pipes first. */
    close (drv2inf[1]); close (inf2drv[0]);

    /* Redirect stdin and stdout. */
    close (0); dup (drv2inf[0]); close (drv2inf[0]);
    close (1); dup (inf2drv[1]); close (inf2drv[1]);

    /* Spawn inferior process - should never return. */
    char *args[] = { prog, NULL};
    execv (prog, args);
  } else {
    /* In parent process - close "other" ends of pipes first. */
    close (drv2inf[0]); close (inf2drv[1]);
  }

However, running this with a test programme (written in C) revealed a problem: messages sent by the spawned programme were not reaching the driving programme!

The reason is that stdout is normally "line-buffered" - i.e. output is buffered till either the buffer becomes full or till a newline character is encountered. When stdout is not connected to a terminal however (for example, when output is redirected to a file), it becomes "block-buffered" - i.e. output is buffered till the buffer becomes full. Of course, you can force the output buffer to be flushed out using fflush() at any time. This behaviour of the C runtime is a reasonable tradeoff that seeks to optimise throughput and response under different conditions. In my case the C runtime determined that stdout was not connected to a terminal and therefore should be block-buffered. The messages sent by the spawned process were therefore still sitting in its output buffer while the driving process waited to hear from it.

My knee-jerk reaction was to insert a call to setvbuf() with a mode of _IOLBF just before the call to execv() in order to force a line-buffered mode for the spawned process. Of course, this does not work for the simple reason that the C runtime is re-initialised after the call to execv().

A possible solution is to mandate that the spawned programme force a line-buffered mode for stdout using something like setvbuf(). Another solution is to mandate that the spawned programme always use the equivalent of fflush() to flush its output buffers after every message. However, these just work around the problem rather than solving it. I also do not want to place unnecessary restrictions on spawned programmes.

So it seems the only solution left is to use a pseudo terminal to fool the spawned programme's runtime into thinking that it is attached to a terminal. A glance through the sample code given in my copy of Stevens's "Advanced Programming in the UNIX Environment" reveals that some ugly code is needed to make it work properly. It also makes my programme even less portable than it currently is. I am therefore a bit loath to take this approach.

Does anyone know of a better solution to this problem?

Update (2006-09-16): Thanks to all who responded. Using a pseudo terminal solved my problem. The forkpty() function from libutil provided by the GNU C Library (thank you Scott) conveniently wraps everything I need into a single function. The only problem is that it also redirects stderr to the newly-created pseudo terminal and this robs the spawned programme of a simple method of debugging. To work around this issue, I had to save the stderr stream using a dup() before calling forkpty() and then restore it using a dup2() in the child process.

One of the problems with using a pseudo terminal that initially stumped me was the fact that the spawned programme seemed to send the driver programme all the messages it was sent by the latter in addition to sending messages of its own. It took me a while to realise that this was because of terminal echo. Switching terminal echo off resolved this problem.

Update (2007-10-18): Here is the relevant portion of the programme that finally worked for me:

#include <stdio.h>
#include <stdlib.h>

#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <termios.h>
#include <pty.h>

/* Spawns a process with redirected standard input and output
   streams. ARGV is the set of arguments for the process,
   terminated by a NULL element. The first element of ARGV
   should be the command to invoke the process.
   Returns a file descriptor that can be used to communicate
   with the process. */
int
spawn (char *argv[]) {
  int ret_fd = -1;

  /* Find out if the intended programme really exists and
     is accessible. */
  struct stat stat_buf;
  if (stat (argv[0], &stat_buf) != 0) {
    perror ("ERROR accessing programme");
    return -1;
  }

  /* Save the standard error stream. */
  int saved_stderr = dup (STDERR_FILENO);
  if (saved_stderr < 0) {
    perror ("ERROR saving old STDERR");
    return -1;
  }

  /* Create a pseudo terminal and fork a process attached
     to it. */
  pid_t pid = forkpty (&ret_fd, NULL, NULL, NULL);
  if (pid == 0) {
    /* Inside the child process. */

    /* Ensure that terminal echo is switched off so that we
       do not get back from the spawned process the same
       messages that we have sent it. */
    struct termios orig_termios;
    if (tcgetattr (STDIN_FILENO, &orig_termios) < 0) {
      perror ("ERROR getting current terminal's attributes");
      return -1;
    }

    orig_termios.c_lflag &= ~(ECHO | ECHOE | ECHOK | ECHONL);
    orig_termios.c_oflag &= ~(ONLCR);

    if (tcsetattr (STDIN_FILENO, TCSANOW, &orig_termios) < 0) {
      perror ("ERROR setting current terminal's attributes");
      return -1;
    }


    /* Restore stderr. */
    if (dup2 (saved_stderr, STDERR_FILENO) < 0) {
      perror ("ERROR restoring STDERR");
      return -1;
    }

    /* Now spawn the intended programme. */
    if (execv (argv[0], argv)) {
      /* execv() should not return. */
      perror ("ERROR spawning programme");
      return -1;
    }
  } else if (pid < 0) {
    perror ("ERROR spawning programme");
    return -1;
  } else {
    close (saved_stderr);
  }

  return ret_fd;
}

Tweet