C: multi-processes stdio append mode

Question

I wrote this code in C:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

void random_seed(){
    struct timeval tim;
    gettimeofday(&tim, NULL);
    double t1=tim.tv_sec+(tim.tv_usec/1000000.0);
    srand (t1);
}

void main(){
    FILE *f;
    int i;
    int size=100;
    char *buf=(char*)malloc(size);

    f = fopen("output.txt", "a");
    setvbuf (f, buf, _IOFBF, size);
    random_seed();

    for(i=0; i<200; i++){
          fprintf(f, "[ xx - %d - 012345678901234567890123456789 - %d]\n", rand()%10, getpid());
          fflush(f);
    }

    fclose(f);
    free(buf);
}

This code opens in append mode a file and attaches 200 times a string. I set the buf of size 100 that can contains the full string. Then I created multi processes running this code by using this bash script:

#!/bin/bash

gcc source.c
rm output.txt

for i in `seq 1 100`;
do
    ./a.out &
done

I expected that in the output the strings are never mixed up, as I read that when opening a file with O_APPEND flag the file offset will be set to the end of the file prior to each write and i'm using a fully buffered stream, but i got the first line of each process is mixed as this:

[ xx - [ xx - 7 - 012345678901234567890123456789 - 22545]

and some lines later

2 - 012345678901234567890123456789 - 22589]

It looks like the write is interrupted for calling the rand function.

So...why appear these lines? Is the only way to prevent this the use file locks...even if i'm using only the append mode?

Thanks in advance!

Answer 1

You will need to implement some form of concurrency control yourself, POSIX makes no guarantees with respect to concurrent writes from multiple processes. You get some guarantees for pipes, but not for regular files written to from different processes.

Quoting POSIX write():

This volume of POSIX.1-2008 does not specify behavior of concurrent writes to a file from multiple processes. Applications should use some form of concurrency control.

(At the end of the Rationale section.)

Answer 2

You open the file in the fully buffered mode. That means that every line of the output first goes into the buffer and when the buffer overflows it gets flushed to the file regardless whether it contains incomplete lines. That causes chunks of output from different processes writing into the same file concurrently to be interleaved.

An easy fix would be to open the file in line buffered mode _IOLBF, so that the buffer gets flushed on each complete line. Just make sure that the buffer size is at least as big as your longest line, otherwise it will end up writing incomplete lines. The buffer is normally flushed with a single write() system call, so that lines from different processes won't interleave each other.

There is no guarantee that write() system call is atomic for different filesystems though, but it normally works as expected because write() normally locks the file descriptor in the kernel with a mutex before proceeding.