StatsD and Graphite issues with stats_counts metrics

Question

I'm having issues with the stats_counts metrics which whipser saves.

In my app I'm firing two kind of metrics in the same time: Counter increment + Time. When I compare the specific metrics graphs: stats_counts + "count" of the timer I get the same results for 4 hours approx but when looking on greater period - say 12 hours, I can see great differences.

I've found this post:Tracking metrics using StatsD (via etsy) and Graphite, graphite graph doesn't seem to be graphing all the data But it didn't helped me.

It seems to me that it is a configuration problem, but I can't understand what's the problem.

This is how my configurations files looks:

storage-schemas.conf:

[stats]
priority = 110
pattern = ^stats\..*
retentions = 10s:6h,1m:7d,10m:1y

[stats_counts]
priority = 110
pattern = ^stats_counts\..*
retentions = 10s:6h,1m:7d,10m:1y

storage-aggregation.conf:

[min]
pattern = \.min$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = \.max$
xFilesFactor = 0.1
aggregationMethod = max

[sum]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum

[stats_counts]
pattern = ^stats_counts\.
xFilesFactor = 0
aggregationMethod = sum

[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average

As you can see I have "stats_counts" rule for aggregation but it seems that it isn't working so good, any idea what am I missing? maybe the syntax of the Regex which I wrote there isn't good: ^stats_counts.

Answer 1

The problem was that when I call "stop" on the carbon-cache process , in order to refersh its memory with the newer configuration , it didn't really stop and therefore the configuration was the old one.

So, there is nothing wrong with the configuration described above. Just make sure that your configuration was actually absorbed by the carbon-process - using the whisper-info.py command:

whisper-info.py your_metric_file_name.wsp

Notice that there seems an issue with the carbon-cache stopping: https://answers.launchpad.net/graphite/+question/135957