[HOWTO] View Historical Data after Consolidation (MAX)

[gandalf]

Please find current version at http://docs.cacti.net


Preface
This tutorial is based on cacti, written by Ian Berry, Larry Adams and Tony Roman. Cacti itself is based on rrdtool, written by Tobias Oetiker. This text was written by Reinhard Scheck to help you figuring out, how to configure cacti to display MAXimum values alongside the commonly plotted AVERAGE ones. This will help getting more information out of the usually defined rra's without the need to change anything concerning the existing rrd files.
As always: Use this information at your own risk.

Here we go!
As an attachment to the forum entry you will find a Graph Template that contains all items discussed here. It is a modified Traffic Graph Template. Things discussed here will of course apply to other Graphs as well.

Common view of Traffic Graph
First, I'll show you the modified template viewed with the standard timeframe of one day. It doesn't look very strange, but let me talk about few things:

• You'll notice, that Outbound Traffic is displayed on the negative side. This is often done; there are lots of those graphs on the forum. It is simply done by a CDEF named Turn bytes into bits, make negative (include in the Template below) that works like
  cdef=CURRENT_DATA_SOURCE,8,*,-1,*
  
• You'll see both a deeper green and a deeper blue line that fits exactly to the AREA definitions.
  
• You'll notice a black line that does some TRENDing (there's a nice post on that, I've copied from there) for Inbound and Outbound Traffic
  
• As usual, you'll see Current, Average and Maximum legend entries

Well, you'll notice that my laptop wasn't online the whole day ...
The magic comes, when you look at some historic data.

See the MAXimum Data after Consolidation
There are some post on the forum complaining about the data loss after consolidation (that is: data is automatically “compressed” by rrdtool). But usually, there's not only the AVERAGE rra but also the MAXimum rra defined. And while executing consolidation automatically, not only the AVERAGE values are stored in the rrd but also the MAXimum ones.
Example:

• Values stored initially: 40, 50, 60, 60, 70, 80
  
• Averaged value after consolidation 6 data Points: (40+50+60+60+70+80)/6 = 60
  
• Additionally stored MAXimum value: 80
  
• (if chosen: additionally stored MINimum value: 40)

After consolidation, there is still knowledge about what was MAXimum! And this may be graphed as well, see:

So you do not only notice the Graph Overall Maximum from the legend (that is: 29.40 k for Inbound) but also when it occurred and additionally the whole timeseries for that MAXimum. (Well, whether TRENDing is helpful here may be answered by yourself)
In this case, CONSOLIDATION took place for 6 data points each, so each AVERAGE value displayed here stands for 6 original data points. You will see this if zooming a little deeper:


Viewing Historical Data
Well, of course this works even if choosing historical data (e.g. Monthly)

The minimum resolution now is 2 hours. But the MAXimum values plotted still represent the biggerst of those consolidated values.

Conclusion
When you look at your rrd's, you will notice that often MAXimum is already defined. To display these values, nothing has to be modified at those rrd's. And there is no additional disk space required compared to methods, that keep data without consolidation.
While graphing the MAXimum values along with the AVERAGE ones, you'll be able to discover the strength of the rrdtool principles.

This Howto may be found as pdf at http://www.team-scheck.de/cacti/view-max-values/howto-view-max-values.pdf
I appreciate any feedback to improve this document.

Reinhard Scheck aka lvm

[henryd31]

Hi there...

I tried your template but am running into some issues...
First, after getting it in I see the Trend Negative CDEF value #1 pointed to one of my other CDEFS, that could be easily fixed by repointing it to the Trend Positive....ok, but then when applying the template to an existing graph it adds items such as this:
Item # 1 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): Inbound AREA AVERAGE 00FF00
Item # 2 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): Current: GPRINT LAST
Item # 3 (traffic_in): Average: GPRINT AVERAGE
Item # 4 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): Maximum: GPRINT MAX
Item # 5 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): MAX LINE1 MAX 00BF47
Item # 6 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): 30min moving avg<HR> LINE1 AVERAGE 000000
Item # 7 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_out): Outbound AREA AVERAGE 00BED9
Item # 8 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_out): Current: GPRINT LAST
+ Item # 9 (traffic_out): Average: GPRINT AVERAGE
+ Item # 10 (traffic_out): Maximum: GPRINT MAX
+ Item # 11 (No Task): MAX LINE1 MAX 005199
+ Item # 12 (No Task): 30min moving avg<HR> LINE1 AVERAGE 000000

and then when trying to create a graph I get error in the rpn expression:
ERROR: invalid rpn expression 'c,1800,TREND,8,*'
this is running on 0.8.6g on fc4, any help would be appreciated...

[gandalf]

[edged]

Hi I have tried to use the template file interface_traffic_bitssec_with_maximum and am having trouble getting the graphs to generate (I get an error "ERROR: invalid rpn expression in: ,-1,*". I am using Cacti-0.8.6g and rrdtool-1.2.11. below is the rrd output

/usr/local/rrdtool-1.2.11/bin/rrdtool graph - \
--imgformat=PNG \
--start=1133347829 \
--end=1133434229 \
--title="******* - Traffic - ******
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale \
COMMENT:"From 2005/11/30 10\:50\:29 To 2005/12/01 10\:50\:29\c" \
COMMENT:" \n" \
--vertical-label="bits per second" \
--slope-mode \
DEF:a="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_in:AVERAGE \
DEF:b="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_in:MAX \
DEF:c="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_in:AVERAGE \
DEF:d="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_out:AVERAGE \
DEF:e="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_out:MAX \
DEF:f="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_out:AVERAGE \
CDEF:cdefa=c,8,* \
CDEF:cdefc=a,8,* \
CDEF:cdefd=b,8,* \
CDEF:cdeff=c,1800,TREND,8,* \
CDEF:cdefg=f,8,*,-1,* \
CDEF:cdefh=f,8,* \
CDEF:cdefi=a,8,* \
CDEF:cdefba=e,8,*,-1,* \
CDEF:cdefbb=,-1,* \
AREA:cdefa#00FF00:"Inbound" \
GPRINT:cdefa:LAST:" Current\:%8.2lf%s" \
GPRINT:cdefc:AVERAGE:"Average\:%8.2lf%s" \
GPRINT:cdefd:MAX:"Maximum\:%8.2lf%s" \
LINE1:cdefd#00BF47:"MAX" \
LINE1:cdeff#000000:"30min moving avg\n" \
AREA:cdefg#00BED9:"Outbound" \
GPRINT:cdefh:LAST:"Current\:%8.2lf%s" \
GPRINT:cdefi:AVERAGE:"Average\:%8.2lf%s" \
GPRINT:cdefi:MAX:"Maximum\:%8.2lf%s" \
LINE1:cdefba#005199:"MAX" \
LINE1:cdefbb#000000:"30min moving avg\n"

Can anyone tell me what i've done wrong?

Thanks

[gandalf]

cdefbb should point to the NEGATIVE TREND CDEF. In your code, there's only the multiplication by -1. Look at that CDEF to see, if it needs correction
Reinhard

[bdiez]

Hello edged.

As far as I can see your problem is in the following point:

...
CDEF:cdefba=e,8,*,-1,* \
CDEF:cdefbb=,-1,* \
AREA:cdefa#00FF00:"Inbound" \
...

As you can see this CDEF is not correctly constructed, and a DEF is missed here. It's for this reason that you're having the error.

This CDEF is related to the very last entry in your graph definition:

...
LINE1:cdefba#005199:"MAX" \
LINE1:cdefbb#000000:"30min moving avg\n"

...so dig in this point. First try to delete it and see the results, and then try to add it again with special care of the CDEF selection.

Good luck !!

[CeeS]

[gandalf]

Please find this at RRAs definition (select Data Sources to see RRAs below. Click RRAs and find those definitions. Select them and find, which consolidation functions are use (multi-select))
But be aware of the fact, that this affect ALL FURTHER graph generation only. This will not change existing rrd files.
Reinhard

[CeeS]