[HOWTO] View Historical Data after Consolidation (MAX)

If you figure out how to do something interesting/cool in Cacti and want to share it with the community, please post your experience here.

Moderators: Moderators, Developers

Post Reply
Author
Message
User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

[HOWTO] View Historical Data after Consolidation (MAX)

#1 Post by gandalf » Thu Oct 13, 2005 1:10 pm

Please find current version at http://docs.cacti.net


Preface
This tutorial is based on cacti, written by Ian Berry, Larry Adams and Tony Roman. Cacti itself is based on rrdtool, written by Tobias Oetiker. This text was written by Reinhard Scheck to help you figuring out, how to configure cacti to display MAXimum values alongside the commonly plotted AVERAGE ones. This will help getting more information out of the usually defined rra's without the need to change anything concerning the existing rrd files.
As always: Use this information at your own risk.

Here we go!
As an attachment to the forum entry you will find a Graph Template that contains all items discussed here. It is a modified Traffic Graph Template. Things discussed here will of course apply to other Graphs as well.

Common view of Traffic Graph
First, I'll show you the modified template viewed with the standard timeframe of one day. It doesn't look very strange, but let me talk about few things:
  • You'll notice, that Outbound Traffic is displayed on the negative side. This is often done; there are lots of those graphs on the forum. It is simply done by a CDEF named Turn bytes into bits, make negative (include in the Template below) that works like
    cdef=CURRENT_DATA_SOURCE,8,*,-1,*
  • You'll see both a deeper green and a deeper blue line that fits exactly to the AREA definitions.
  • You'll notice a black line that does some TRENDing (there's a nice post on that, I've copied from there) for Inbound and Outbound Traffic
  • As usual, you'll see Current, Average and Maximum legend entries
Image
Well, you'll notice that my laptop wasn't online the whole day ...
The magic comes, when you look at some historic data.

See the MAXimum Data after Consolidation
There are some post on the forum complaining about the data loss after consolidation (that is: data is automatically “compressed” by rrdtool). But usually, there's not only the AVERAGE rra but also the MAXimum rra defined. And while executing consolidation automatically, not only the AVERAGE values are stored in the rrd but also the MAXimum ones.
Example:
  • Values stored initially: 40, 50, 60, 60, 70, 80
  • Averaged value after consolidation 6 data Points: (40+50+60+60+70+80)/6 = 60
  • Additionally stored MAXimum value: 80
  • (if chosen: additionally stored MINimum value: 40)
After consolidation, there is still knowledge about what was MAXimum! And this may be graphed as well, see:
Image
So you do not only notice the Graph Overall Maximum from the legend (that is: 29.40 k for Inbound) but also when it occurred and additionally the whole timeseries for that MAXimum. (Well, whether TRENDing is helpful here may be answered by yourself)
In this case, CONSOLIDATION took place for 6 data points each, so each AVERAGE value displayed here stands for 6 original data points. You will see this if zooming a little deeper:
Image

Viewing Historical Data
Well, of course this works even if choosing historical data (e.g. Monthly)
Image
The minimum resolution now is 2 hours. But the MAXimum values plotted still represent the biggerst of those consolidated values.

Conclusion
When you look at your rrd's, you will notice that often MAXimum is already defined. To display these values, nothing has to be modified at those rrd's. And there is no additional disk space required compared to methods, that keep data without consolidation.
While graphing the MAXimum values along with the AVERAGE ones, you'll be able to discover the strength of the rrdtool principles.

This Howto may be found as pdf at http://www.team-scheck.de/cacti/view-ma ... values.pdf
I appreciate any feedback to improve this document.

Reinhard Scheck aka lvm
Attachments
cacti_graph_template_interface_traffic_bitssec_with_maximum.xml
Graph Template as discussed here
You'll need cacti 0.8.6g at minimum
(19.48 KiB) Downloaded 1224 times
Last edited by gandalf on Wed Jun 18, 2008 1:32 pm, edited 1 time in total.

henryd31
Posts: 30
Joined: Wed Apr 13, 2005 3:11 pm

#2 Post by henryd31 » Wed Nov 23, 2005 11:48 am

Hi there...

I tried your template but am running into some issues...
First, after getting it in I see the Trend Negative CDEF value #1 pointed to one of my other CDEFS, that could be easily fixed by repointing it to the Trend Positive....ok, but then when applying the template to an existing graph it adds items such as this:
Item # 1 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): Inbound AREA AVERAGE 00FF00
Item # 2 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): Current: GPRINT LAST
Item # 3 (traffic_in): Average: GPRINT AVERAGE
Item # 4 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): Maximum: GPRINT MAX
Item # 5 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): MAX LINE1 MAX 00BF47
Item # 6 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_in): 30min moving avg<HR> LINE1 AVERAGE 000000
Item # 7 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_out): Outbound AREA AVERAGE 00BED9
Item # 8 (|host_description| - Traffic - |query_ifIP| - |query_ifName| - traffic_out): Current: GPRINT LAST
+ Item # 9 (traffic_out): Average: GPRINT AVERAGE
+ Item # 10 (traffic_out): Maximum: GPRINT MAX
+ Item # 11 (No Task): MAX LINE1 MAX 005199
+ Item # 12 (No Task): 30min moving avg<HR> LINE1 AVERAGE 000000

and then when trying to create a graph I get error in the rpn expression:
ERROR: invalid rpn expression 'c,1800,TREND,8,*'
this is running on 0.8.6g on fc4, any help would be appreciated...

User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

#3 Post by gandalf » Thu Nov 24, 2005 2:28 pm

henryd31 wrote:and then when trying to create a graph I get error in the rpn expression:
ERROR: invalid rpn expression 'c,1800,TREND,8,*'
this is running on 0.8.6g on fc4, any help would be appreciated...
Sorry for that. I'm using rrdtool 1.2.11 that supports the TREND function. To get rid of this, simply delete the appropriate Graph Item
Reinhard

edged
Posts: 3
Joined: Thu Dec 01, 2005 5:37 am

CACTI newbie has template problem!

#4 Post by edged » Thu Dec 01, 2005 5:51 am

Hi I have tried to use the template file interface_traffic_bitssec_with_maximum and am having trouble getting the graphs to generate (I get an error "ERROR: invalid rpn expression in: ,-1,*". I am using Cacti-0.8.6g and rrdtool-1.2.11. below is the rrd output

/usr/local/rrdtool-1.2.11/bin/rrdtool graph - \
--imgformat=PNG \
--start=1133347829 \
--end=1133434229 \
--title="******* - Traffic - ******
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale \
COMMENT:"From 2005/11/30 10\:50\:29 To 2005/12/01 10\:50\:29\c" \
COMMENT:" \n" \
--vertical-label="bits per second" \
--slope-mode \
DEF:a="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_in:AVERAGE \
DEF:b="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_in:MAX \
DEF:c="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_in:AVERAGE \
DEF:d="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_out:AVERAGE \
DEF:e="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_out:MAX \
DEF:f="/var/www/html/projects/cacti/rra/*******_traffic_in_20.rrd":traffic_out:AVERAGE \
CDEF:cdefa=c,8,* \
CDEF:cdefc=a,8,* \
CDEF:cdefd=b,8,* \
CDEF:cdeff=c,1800,TREND,8,* \
CDEF:cdefg=f,8,*,-1,* \
CDEF:cdefh=f,8,* \
CDEF:cdefi=a,8,* \
CDEF:cdefba=e,8,*,-1,* \
CDEF:cdefbb=,-1,* \
AREA:cdefa#00FF00:"Inbound" \
GPRINT:cdefa:LAST:" Current\:%8.2lf%s" \
GPRINT:cdefc:AVERAGE:"Average\:%8.2lf%s" \
GPRINT:cdefd:MAX:"Maximum\:%8.2lf%s" \
LINE1:cdefd#00BF47:"MAX" \
LINE1:cdeff#000000:"30min moving avg\n" \
AREA:cdefg#00BED9:"Outbound" \
GPRINT:cdefh:LAST:"Current\:%8.2lf%s" \
GPRINT:cdefi:AVERAGE:"Average\:%8.2lf%s" \
GPRINT:cdefi:MAX:"Maximum\:%8.2lf%s" \
LINE1:cdefba#005199:"MAX" \
LINE1:cdefbb#000000:"30min moving avg\n"

Can anyone tell me what i've done wrong?

Thanks

User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

#5 Post by gandalf » Thu Dec 01, 2005 2:23 pm

cdefbb should point to the NEGATIVE TREND CDEF. In your code, there's only the multiplication by -1. Look at that CDEF to see, if it needs correction
Reinhard

bdiez
Posts: 14
Joined: Thu Sep 08, 2005 8:09 am
Location: Spain

Re: Post subject: CACTI newbie has template problem!

#6 Post by bdiez » Thu Dec 01, 2005 2:28 pm

Hello edged.

As far as I can see your problem is in the following point:

...
CDEF:cdefba=e,8,*,-1,* \
CDEF:cdefbb=,-1,* \
AREA:cdefa#00FF00:"Inbound" \
...

As you can see this CDEF is not correctly constructed, and a DEF is missed here. It's for this reason that you're having the error.

This CDEF is related to the very last entry in your graph definition:

...
LINE1:cdefba#005199:"MAX" \
LINE1:cdefbb#000000:"30min moving avg\n"

...so dig in this point. First try to delete it and see the results, and then try to add it again with special care of the CDEF selection.

Good luck !! :)

CeeS
Posts: 2
Joined: Thu Feb 02, 2006 10:07 am

Re: Howto View Historical Data after Consolidation (MAX)

#7 Post by CeeS » Wed Apr 05, 2006 2:06 am

lvm wrote:When you look at your rrd's, you will notice that often MAXimum is already defined. To display these values, nothing has to be modified at those rrd's. And there is no additional disk space required compared to methods, that keep data without consolidation.
While graphing the MAXimum values along with the AVERAGE ones, you'll be able to discover the strength of the rrdtool principles.
How about the MIN value ?
I like to display both MAX and MIN in a graph where the measured-data variates between +1 and -1. (The measured data should be normally 0, and I am interested in the average, but I also like to know how "noisy" the measurement is, I could use unfiltered data, but displaying the MAX and MIN value of the "average period" is also enough for me.)

MAX works fine, but when I tried to use MIN, there is an error. After investigating I found that the RRD's made by Cacti do not contain any MIN, but only MAX, LAST and the normal AVERAGE. Is there something I could set in Cacti to have a MIN cf in the RRD's too ??

User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

#8 Post by gandalf » Thu Apr 06, 2006 7:14 am

Please find this at RRAs definition (select Data Sources to see RRAs below. Click RRAs and find those definitions. Select them and find, which consolidation functions are use (multi-select))
But be aware of the fact, that this affect ALL FURTHER graph generation only. This will not change existing rrd files.
Reinhard

CeeS
Posts: 2
Joined: Thu Feb 02, 2006 10:07 am

#9 Post by CeeS » Fri Apr 07, 2006 5:06 am

lvm wrote:Please find this at RRAs definition (select Data Sources to see RRAs below. Click RRAs and find those definitions. Select them and find, which consolidation functions are use (multi-select))
But be aware of the fact, that this affect ALL FURTHER graph generation only. This will not change existing rrd files.
Reinhard
Thanks for the info !

Post Reply