Windows event logs to Flume

Question

I've installed a Cloudera Flume node (0.9.4) on my windows 2003 server and it appears to be running. However, I'm stuck as to the next steps to take to send windows server event log data to the master node. My master node is located on a Linux machine. What next steps are needed to connect my Windows flume node to the master node? thanks, Ralph.

Answer 1

I'm baffled as to why this seems to be the only decent "open-source" (if not community-developed) solution, but after a few research efforts over the last several years, I've repeatedly come up with NXLog as the best option for handling Windows event logs in a primarily *nix-based environment.

NXLog has a special input module for this purpose called im_msvistalog. I've been using this with NXLog Community Edition and it works well so far. (FYI, I'm shipping Windows logs directly to Solr.)

I presume that there just aren't that many people using tools of this flavor (i.e., Apache Flume, Solr, Java, typically Linux-based tools) for handling Windows event logs. :-) I'd like to know why if anyone cares to chime in. I guess people with Windows infrastructure they care about will just have something like a centralized Windows Event Viewer that operates as a syslog daemon would in a *nix environment?

If this solution doesn't work for you, you can also try querying the Windows event logs using the Windows Events Command Line Utility. I haven't yet had to resort to that since everything I've needed has been available using that NXLog input module I mentioned above.

Answer 2

You need to connect the Windows Event Log to Flume. I haven't tried this but I suggest you try a tool such as KiwiSyslog to turn Windows Events into Syslog. You then configure Flume with a Syslog source and tell KiwiSyslog to sent the events there.

BTW, Flume 0.9.4 is very old. I suggest you change to a recent Apache Flume as that is where the active support (largely by Cloudera staff) is.