File which responds to isatty(3)

Question

I'm wanting to write my own psuedo-shell and would like to get pretty colors etc. How do I go about tricking a subprocess into thinking that it is in a TTY? I've read about virtual TTY's but haven't found much practical information about how to either create one or how that makes a subprocess think that isatty(stdout) == 1.

Answer 1

What you're looking for are called pseudoterminals, pseudo-ttys or ptys. These exist in master/slave pairs, that behave similarly to socket pairs (the bidirectional version of pipes; what is written to one end can be read on the other). In the controlling process, use posix_openpt to open a master, then ptsname to get the slave's name (probably /dev/pts/X):

int master = posix_openpt(O_RDWR | O_NOCTTY);
grantpt(master);     /* change ownership and permissions */
unlockpt(master);    /* must be called before obtaining slave */
int slave = open(ptsname(master), O_RDWR | O_NOCTTY);

As usual, each function can fail, so add error checking. The slave fd now refers to the slave device. Use dup2(slave, STDOUT_FILENO) in the child process to set standard output to the slave pseudoterminal; similarly for stdin and stderr.

(Note that some Linux manpages incorrectly state that posix_openpt returns char *. Also, don't get confused by the openpty family of functions; these represent an older interface to pseudo-ttys that is deprecated.)