Graphite - Multi archives level retrieval issue

Question

I've configured my files with the following retentions settings:

retentions = 10s:7d,1m:30d,5m:1y 

And this indeed works, however my problem is that the web-app decides to get the data from the lower archive instead of the higher. For example: Since I have a resolution of 10s per 1 week I expect that if I'm looking on less than 1 week (in my case I was just looking on data for 1 day) then the data will be retrieved from the first level (which is accurate to 10 sec), however the web-app pulls the data from the next level and therefore I'm getting av average data.

I've found this in the documentation:

"When data is retrieved (scoped by a time range), the first archive which can satisfy the entire time period is used. If the time period overlaps an archive boundary, the lower-resolution archive will be used. This allows for a simpler behavior while retrieving data as the data’s resolution is consistent through an entire returned series."

How can I make graphite get the data from the highest resolution archive? My intent is to have high resolution for 1 week (10 seconds) and the next level is 30 days for 1 minute resolution.

Is there a way to achieve this or maybe to re-configure my storage schema?

Answer 1

After further investigation I found the problem.

When I query Whisper directly via the "Fetch" command it indeed got the value from the higher level as expected. When I used the render API it also worked as expected, but when I used the web-app graphs (or the API with SVG format) it did some smoothing to the graph which cause it to loose spikes. I was only able to see those spikes when I did some drill down over the period (for 2 hour resolution).

The solution is to use minXStep property for each graph and set it to 0 (0.1 will also work good enough). If you use the render API just use the minXStep property (default is 1) and if you use the Dashboards area go to "Render Options" -> "X-Axis" -> "Point-width Consolidation Threshold" and set it so some lower value than 1.

For further information see http://graphite.readthedocs.org/en/latest/render_api.html#minxstep

Answer 2

Good to see you solved your problem. I'll post the answer I would have given, since it may be of use to others (even though it sounds like it wouldn't be in your case).

Typically what happens is that you look for, say, an hour today and you get the fine resolution you expected, but if you look for an hour two weeks ago, you get the coarser resolution because that's all Graphite has available. You can also find it picking the coarser resolution when you're looking at date ranges that go over a boundary in the retention policy. In your case, if you look at the last 8 days you get the 1m resolution because even though it has 10s resolution for most of the time, it only has the 1m resolution for the 8th day, and so it uses the 1m resolution for everything.