bash: force exec'd process to have unbuffered stdout

Question

I've got a script like:

#!/bin/bash
exec /usr/bin/some_binary > /tmp/my.log 2>&1

Problem is that some_binary sends all of its logging to stdout, and buffering makes it so that I only see output in chunks of a few lines. This is annoying when something gets stuck and I need to see what the last line says.

Is there any way to make stdout unbuffered before I do the exec that will affect some_binary so it has more useful logging?

(The wrapper script is only setting a few environment variables before the exec, so a solution in perl or python would also be feasible.)

Answer 1

You might find that the unbuffer script that comes with expect may help.

Answer 2

GNU coreutils-8.5 also has the stdbuf command to modify I/O stream buffering:

http://www.pixelbeat.org/programming/stdio_buffering/

So, in your example case, simply invoke:

stdbuf -oL /usr/bin/some_binary > /tmp/my.log 2>&1

This will allow text to appear immediately line-by-line (once a line is completed with the end-of-line "\n" character in C). If you really want immediate output, use -o0 instead.

This way could be more desirable if you do not want to introduce dependency to expect via unbuffer command. The unbuffer way, on the other hand, is needed if you have to fool some_binary into thinking that it is facing a real tty standard output.

Answer 3

Some command line programs have an option to modify their stdout stream buffering behaviour. So that's the way to go if the C source is available ...

# two command options ...
man file | less -p '--no-buffer'
man grep | less -p '--line-buffered'

# ... and their respective source code

# from: http://www.opensource.apple.com/source/file/file-6.2.1/file/src/file.c
if(nobuffer)
   (void) fflush(stdout);

# from: http://www.opensource.apple.com/source/grep/grep-28/grep/src/grep.c
if (line_buffered)
   fflush (stdout);

As an alternative to using expect's unbuffer script or modifying the program's source code, you may also try to use script(1) to avoid stdout hiccups caused by a pipe:

See: Trick an application into thinking its stdin is interactive, not a pipe

# Linux
script -c "[executable string]" /dev/null

# FreeBSD, Mac OS X
script -q /dev/null "[executable string]"

Answer 4

I scoured the internets for an answer, and none of this worked for uniq which is too stubborn to buffer everything except for stdbuf:

{piped_command_here} | stdbuf -oL uniq | {more_piped_command_here}

Answer 5

GNU Coreutils-8 includes a program called stdbuf which essentially does the LD_PRELOAD trick. It works on Linux and reportedly works on BSD systems.

Answer 6

An environment variable can set the terminal IO mode to unbuffered.

export NSUnbufferedIO=YES

This will set the terminal unbuffered for both C and Ojective-C terminal output commands.