Question

When I use the cluster and winston in nodejs ,the log's maxsize does not work.All the log will logging one file although the size beyound the maxsize. See the same problem here: https://github.com/flatiron/winston/issues/275

Was it helpful?

Solution

The obvious solution is: Let only the master log to the file.

var winston = require('winston');
var cluster = require('cluster');

if(cluster.isMaster) {
    cluster.setupMaster({ silent: true }); // Keep cluster from automatically grabbing stdin/out/err
    for(var i = 0; i < 4; i++) {
        cluster.fork();
    }
    winston.add(winston.transport.File, { filename: 'log.txt' });
    cluster.workers.forEach(function(worker, i) {
        worker.process.stdout.on('data', function(chunk) {
            winston.info('worker ' + i + ': ' + chunk);
        });
        worker.process.stderr.on('data', function(chunk) {
            winston.warn('worker ' + i + ': ' + chunk);
        });
    });
} else {
    // Leave winston alone and only log to stdout/err in the workers
}

Then only one process has a hold on the file descriptor, so your log rotation should work as usual.

OTHER TIPS

Maybe you should use DailyFileRotation instead. As of today, I don't see any problem. I use winston every where in master and forked processes, and they share the same file.

Here is what I found from testing. You can see that all logs are evenly distributely before and after the rotation. Also you can clearly see the cut-off in the logs. Also old logs stopped growing after the switch.

joephone@Fengs-MacBook-Pro-2:logs$ ls
total 3184
-rw-r--r--  1 joephone  staff  985111 Aug 24 20:01 info.log.2016-08-25-03-00
-rw-r--r--  1 joephone  staff  641506 Aug 24 20:01 info.log.2016-08-25-03-01
joephone@Fengs-MacBook-Pro-2:logs$ cut -f8 -d"\"" info.log.2016-08-25-03-00 | sort | uniq -c
3299 65151
3355 65152
3307 65153
3345 65154
   1 cpucount:4
   1 worker 65151 starts working.
   1 worker 65152 starts working.
   1 worker 65153 starts working.
   1 worker 65154 starts working.
joephone@Fengs-MacBook-Pro-2:logs$ cut -f8 -d"\"" info.log.2016-08-25-03-01 | sort | uniq -c
2193 65151
2207 65152
2122 65153
2147 65154
joephone@Fengs-MacBook-Pro-2:logs$ tail -3 info.log.2016-08-25-03-00
{"level":"info","message":"65154","timestamp":"2016-08-25T03:00:59.997Z"}
{"level":"info","message":"65153","timestamp":"2016-08-25T03:00:59.997Z"}
{"level":"info","message":"65152","timestamp":"2016-08-25T03:00:59.997Z"}
joephone@Fengs-MacBook-Pro-2:logs$ head -3 info.log.2016-08-25-03-01
{"level":"info","message":"65151","timestamp":"2016-08-25T03:01:00.000Z"}
{"level":"info","message":"65151","timestamp":"2016-08-25T03:01:00.012Z"}
{"level":"info","message":"65152","timestamp":"2016-08-25T03:01:00.006Z"}

For my curiosity, I also tested by putting all code into each forked process. I still don't see any problem.

joephone@Fengs-MacBook-Pro-2:logs$ ls
total 3216
-rw-r--r--  1 joephone  staff  993838 Aug 24 20:14 info.log.2016-08-25-03-13
-rw-r--r--  1 joephone  staff  650312 Aug 24 20:14 info.log.2016-08-25-03-14
joephone@Fengs-MacBook-Pro-2:logs$ cut -f8 -d"\"" info.log.2016-08-25-03-13 | sort | uniq -c
3402 65755
3342 65756
3344 65757
3337 65758
   1 worker 65755 starts working.
   1 worker 65756 starts working.
   1 worker 65757 starts working.
   1 worker 65758 starts working.
joephone@Fengs-MacBook-Pro-2:logs$ cut -f8 -d"\"" info.log.2016-08-25-03-14 | sort | uniq -c
2183 65755
2202 65756
2230 65757
2173 65758
joephone@Fengs-MacBook-Pro-2:logs$ tail -3 info.log.2016-08-25-03-13 
{"level":"info","message":"65757","timestamp":"2016-08-25T03:13:59.986Z"}
{"level":"info","message":"65755","timestamp":"2016-08-25T03:13:59.986Z"}
{"level":"info","message":"65756","timestamp":"2016-08-25T03:13:59.994Z"}
joephone@Fengs-MacBook-Pro-2:logs$ head -3 info.log.2016-08-25-03-14
{"level":"info","message":"65758","timestamp":"2016-08-25T03:14:00.005Z"}
{"level":"info","message":"65757","timestamp":"2016-08-25T03:14:00.005Z"}
{"level":"info","message":"65758","timestamp":"2016-08-25T03:14:00.024Z"}

For my curiosity of curiosity, I even tested it with an error bringing down a forked process and the master bring a new one up. The log is still working.

Actually the two ways may not be different, if you use ps to grep the script you are running you will see the master process and the forked processes. They are simply similar to you running them in multiple terminals I think.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top