Splunk Regex for filtering through host names

Question

I am fairly new to splunk. We have three hbase clusters and all of which have multiple zookeeper nodes and region servers listed on them like what i mentioned below.

Cluster 1

test101.blah.com
test102.blah.com
..
test199.blah.com

Cluster2

test201.blah.com
test202.blah.com
..
test299.blah.com

Cluster3

test301.blah.com
..
test399.blah.com

I am trying to filter for logs centered around a particular cluster. so i used a generator to create filtering statements like these host="test101.blah.com" or host=test102.blah.com"..or host="test199.blah.com" like this and build the same queries for the other clusters as well.

I know its inefficient. Is there an efficient way to do this via regex or pattern matching through splunk?

Answer 1

I found an easier alternative to solve this problem without regex but using the splunk feature directly and my rules for the three clusters ended up being this.

host>="test101*" AND host<="test199*" 
host>="test201*" AND host<="test299*"
host>="test301*" AND host<="test399*"

Answer 2

The easiest answer to search by cluster for your example hostnames would be: For Cluster1: host="test1*" For Cluster2: host="test2*" For Cluster3: host="test3*"

If you're doing a lot of this over a large data set you may want to test the efficiency of the comparisons.

You will probably want to be able to do further operations such as stats or timechart by cluster, so I recommend that you create a new field for "cluster" using eval, rex, field extraction in props.conf, or by creating a custom field at index time for fastest searching.

For example, with a "rex", this takes the events from your base search and looks for the "test" pattern in the "host" field, then captures the following digit into a field named "cluster":

 | rex field=host "(?i)test(?P<cluster>[\\d])"

so now you can show stats by cluster:

 | rex field=host "(?i)test(?P<cluster>[\\d])" | stats count by cluster