Piping log output through a C program for easy log rotation
Question
I'm trying to make it really easy to logrotate some of my apps that log via bash redirection. Basically, I have a C program that reads STDIN into a buffer. It reads this buffer, and whenever it encounters a newline, it will write the output it has gathered to a file.
The difference in this program is that it does not leave the file open. It opens it for appending each time a new line is encountered. This works great with the logrotate utility, but I'm wondering if there's some sort of horrible unforseen issue I'm not accounting for that I'll run into later on.
Is it better just to implement signal handling in this utility and have logrotate send it a SIGHUP? Are there horrible performance penalties to what I'm doing?
So normally where you'd do:
./app >> output.log
With the logger util you do:
./app | ./mylogger output.log
Although I'm too bad in C, I'm not very well versed in its best practices. Any guidance would be greatly appreciated.
Here's the source:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#define BUFSIZE 1024
#define MAX_WRITE_FAILS 3
/**
* outputs the given content to the specified file.
*/
int file_output(char *filename, char *content, size_t content_length)
{
FILE *fp;
fp = fopen(filename, "a");
content[content_length + 1] = '\0';
if(fp == NULL) return errno;
fwrite(content, sizeof(char), content_length, fp);
fclose(fp);
return 0;
}
/**
* Loops over STDIN and whenever it finds a newline, sends the current content
* buffer to the file specified on the command line.
*/
int main(int argc, char *argv[])
{
int i;
char buffer[BUFSIZE];
char *content = malloc(sizeof(char) * BUFSIZE);
size_t content_size = 0;
int content_buf_size = BUFSIZE;
int write_failures = 0;
char *file;
if(argc < 2)
{
fprintf(stderr, "Usage: logger <file>");
exit(1);
}
file = argv[1];
// loop over STDIN
while(fgets(buffer, BUFSIZE, stdin))
{
int output_err;
int buflength = strlen(buffer);
// loop over character for character, searching for newlines and
// appending our buffer to the output content as we go along
for(i = 0; i < buflength; i++)
{
char *old = content;
// check if we have a newline or end of string
if(buffer[i] == '\n' || buffer[i] == '\0' || (i != (buflength - 1) && buffer[i] == '\r' && buffer[i+1] == '\n'))
{
content[content_size] = '\n';
output_err = file_output(file, content, content_size + 1);
if(output_err == 0)
{
// success! reset the content size (ie more or less resets
// the output content string)
content_size = 0;
write_failures = 0;
}
else
{
// write failed, try to keep going. this will preserve our
// newline so that the next newline we encounter will write
// both lines (this AND and the next).
content_size++;
write_failures++;
}
}
if(write_failures >= MAX_WRITE_FAILS)
{
fprintf(stderr, "Failed to write output to file %d times (errno: %d). Quitting.\n", write_failures, output_err);
exit(3);
}
if(buffer[i] != '\n' && buffer[i] != '\r' && buffer[i] != '\0')
{
// copy buffer into content (if it's not a newline/null)
content[content_size] = buffer[i];
content_size++;
}
// check if we're pushing the limits of our content buffer
if(content_size >= content_buf_size - 1)
{
// we need to up the size of our output buffer
content_buf_size += BUFSIZE;
content = (char *)realloc(content, sizeof(char) * content_buf_size);
if(content == NULL)
{
fprintf(stderr, "Failed to reallocate buffer memory.\n");
free(old);
exit(2);
}
}
}
}
return 0;
}
Thanks!
Solution
Since my suggestion in the comments turned out to be what you needed, I am adding it as an answer, with more of an explanation.
When you have a logging application which can not be told to close its logfile (usually via SIGHUP), you can use the 'copytruncate' option in your logrotate.conf.
Here is the description from the man page:
Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one, It can be used when some program can not be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some log- ging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.