JMeter Output to Event Log

Question

I've just started using JMeter to do some simple sythentic transactions on our production environments and I've added a number of listeners to capture the summary of test runs to an XML file. I'm now looking at getting this data into Splunk for reporting purposes for the rest of the team and was looking for the best way to do this.

I've initially done it by indexing the XML file generated by JMeter in Splunk and this works well and allows us to generate the reports required. However, given this test plan will eventually execute twice a minute hitting 20+ boxes I'd like to ideally skip the XML file (and it's potential size!) and directly get the data into Splunk.

What alternatives are there for this?

Answer 1

Natively you could use CSV instead of XML, see :

• http://jmeter.apache.org/usermanual/listeners.html#defaults

Another option is to write a custom listener using a protocol understood by splunk, see:

• Extending JMeter

Answer 2

And just to add to the advice already presented...

I was hoping somehow you could use log4j or logback, because then you could use SplunkJavaLogging , but it looks like JMeter doesn't let you use a logging framework as your listener output.

So it looks like you'll have to create a custom JMeter listener if you want stream your results directly into Splunk , over raw TCP or HTTP REST.

Fortunately, we already have a Splunk Java SDK that will take care of the transport for you. Heaps of code examples available.

You'll just have to wrap it up in a JMeter Listener Implementation.

Answer 3

Building on "PMD UBIK-INGENIERIE's" answer, how are you actually getting the logs to Splunk?

You best would most definately be to use a scripted input from Splunk's side, this script depending on your OS can include python, Bash, Batch, etc. Please see the following chapter in the Splunk documentation:

http://docs.splunk.com/Documentation/Splunk/latest/Developer/ScriptedInputsIntro

Not knowing the JMeter tool, I would say, one solution would be to have a "wrapper" script run by Splunk which calls the API querying script of your choice. You can then have Splunk monitor the STDOUT of that script, or write it to file in a nice format for Splunk to monitor.

In terms of the deployment, it would be best (where possible) to install a Splunk forwarder (note: Universals don't include python), on the remote hosts and have the forwarder pass the data to the Indexer over a secure TCP connection, this will help prevent against loss of data as Splunk will queue the data in case of a lost connection.

Also, it may be useful to head over to Splunk's official forum @ SplunkBase. Although there aren't any directly matching questions, there are many experienced and helpful Splunkers there (including employees, partners/resellers, and customers). There may even be another user that has indexed the JMeter data as in your scenario.