Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Fri Apr 28, 2017 10:33 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 102 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7
Author Message
 Post subject: Re: RRA and Data Template Settings for 0.8.7
PostPosted: Wed Oct 26, 2011 6:32 pm 
Offline

Joined: Sat Jul 31, 2010 9:38 am
Posts: 11
yuval_ba wrote:
Hi tekbot,
First of all, thanks for your great post!, it helped me allot to configure my 1min interval polling.
I still have one question regarding Rows calculation.

I understand the math behind Rows calculation, but I did not undersand why you choose longer periods than the period represented by Timespan value.

For example: In your 1min poll / 24 hours view, why did you choose Rows value of 4320 which is equal to 3 days if due to the Timespan value only 24hours will be displayed in the graph? my common sense would be to choose Rows value of 1440 in this case. what am I missing?

thanks
Yuval


Likewise, I believe I understand everything except this last bit... I'm not sure of the reasoning behind keeping extra rows beyond the timespan values Cacti will use when rendering graphs (in most cases double the number of rows needed to represent the specified timespan).

For example, with step adjusted for 1 minute polling, the default RRAs might look something like this:

Code:
Hourly (1 Minute Average)         1       500      14400
Daily (5 Minute Average)          6       600      86400
Weekly (30 Minute Average)        30      700      604800
Monthly (2 Hour Average)          120     775      2678400
Yearly (1 Day Average)            1440    797      33053184


Taking the Daily (5 Minute Average) as an example. The timespan specifies that when this graph is displayed, it will show 1 day's worth of data; so in theory, the number of rows needed to represent this would be 288 (the number of 5 minute intervals in one day) - but the RRA is storing 600 rows of data (giving it a history of over two days).

My question is, what is the reasoning behind Cacti storing twice the data needed (and tekbot 3 times or more) when the timespan setting means this superfluous data wont be used to render the standard graphs?

Could it be that this extra data is used when custom graph view settings are used. For example, if I ask Cacti to show me graphs spanning the past two days instead of one, will it use this extra data being stored when generating this custom graph for me?


If so, it got me thinking about this a bit further... Again using the RRAs above as an example; what if I asked Cacti to show me custom graphs spanning the past 4 days (exceeding the available information contained in the Hourly or Daily RRAs).

Does Cacti use the more accurate Hourly RRA data to render the first 8 hours or so, the Daily RRA data to render up until the past 2 days, and then fill in the rest using the less accurate Weekly (30 Minute Average) RRA? Or does it instead use the RRA data from the Weekly RRA because that's the only one that contains enough data spanning the requested timespan?

Thank you in advanced for your time and consideration.

Edit: I think this from the Cacti manual kind of answers my question: "The data retrieval function of RRDtool automatically selects the archive with the highest resolution that still covers the requested timespan. This mechanism is also used by RRDtool's graphing subsystem."


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Thu Oct 27, 2011 7:00 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
In order to do 1 minute polling, your Data Template's that are going to be polled at one minute, need to have their STEP values changed to 60 from 300. However, you must also understand the relationship between the step and the RRA settings so that you don't get graphs that end after a few months.

Cacti is installed to be run at 5 minute polling. Lot's of customers poll cacti at 1 minutes or less for critical hardware monitoring. But keep in mind, this is an advanced subject. I wish it were easier, but for the experienced RRDtool user, it's not so bad.

Finally, after your Data Templates have been altered, you will have to delete your impacted RRDfiles, and repopulate the poller cache.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Fri Dec 23, 2011 8:10 am 
Offline

Joined: Sat Sep 24, 2011 12:52 pm
Posts: 1
Is there any chance for someone to actually post step by step transform from fresh installed cacti at 5 min to 1 min, including all needed things. It is really hard to read 7 pages of discussions. Or insert in first post.


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Fri Dec 23, 2011 9:38 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
There are a number both in the How-To section of the forum, and in the documentation. Please use Google.

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Fri Dec 23, 2011 12:08 pm 
Offline

Joined: Mon Sep 05, 2011 3:13 am
Posts: 2
lupetalo wrote:
Is there any chance for someone to actually post step by step transform from fresh installed cacti at 5 min to 1 min, including all needed things. It is really hard to read 7 pages of discussions. Or insert in first post.


hi lupetalo, i hope this link would help you..

http://dikehtonon.blogspot.com/2011/11/cacti-1-minute-polling-using-spine.html

compiling all information, I've search from cacti.net and other blogs, I'm now able to understand how to make it work.


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Thu Jul 19, 2012 2:25 am 
Offline

Joined: Tue Apr 24, 2012 5:52 am
Posts: 5
Fantastic guide by Tekbot. Followed this on a fresh 8.8a install and have 1 minute polling working perfectly on bits per second graphs on cisco devices.

There's just one small question now.

Data Templates = When I got the box, the graphs for the actual Cacti Server had already been setup. (mysql etc)

After changing poller (not cron) from 5 mins to 1 min, I expected these graphs to start acting odd. They have not. Do I need to change the step and heartbeat on ALL templates that are active/i want to use in the future? Should I just then delete and re-create these graphs?

Also, if My poller interval is now 1 minute, I assume all hosts will poll on all graphs setup every 1 minute, and I can't mix and match 1 minute & 5 minute polling per device? Graphs obviously I could make a 5 minute average RRA template, but the actual SNMP query would still hit the box every minute? We'd still store the data points every minute also?


Top
 Profile  
 
 Post subject: Re: RRA and Data Template Settings for 0.8.7
PostPosted: Fri Sep 07, 2012 5:04 pm 
Offline
Cacti User
User avatar

Joined: Thu Nov 12, 2009 3:07 pm
Posts: 105
moggie wrote:
My question is, what is the reasoning behind Cacti storing twice the data needed (and tekbot 3 times or more) when the timespan setting means this superfluous data wont be used to render the standard graphs?



My understanding is that this allows you to display the higher resolution graphs further back in time.

So if you're looking at the last 1 hour you can "shift left" several hours and still get the highest resolution information.


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Fri Sep 07, 2012 6:25 pm 
Offline
Cacti User
User avatar

Joined: Thu Nov 12, 2009 3:07 pm
Posts: 105
willieb wrote:
Here's my actual RRA settings. I wanted 1 minute averages on all and I added a 3 year graph.
Code:
Name                                Steps          Rows          Timespan
Hourly  (1 Minute Average)           1              300           14400
Daily   (1 Minute Average)           1              1500          86400
Weekly  (1 Minute Average)           1              11000         604800
Monthly (1 Minute Average)           1              45000         2678400
Yearly  (1 Minute Average)           1              526000        31536000
3 Years (1 Minute Average)           1              1576800       94608000

I hope this helps someone. Thanks.


This seems wrong.

My understanding is that if you want more 1 minute data stored then you need to add more rows to your Hourly (1 min avg) RRA.

Your current Hourly (1 minute average) RRA would hold 5 hours of 1 minute data, and your 3 Year RRA will hold 3 years of 1 minute data. But your 3 year graph will only be 500 pixels wide so you have way more information than you will be able to see. If you really want 3 years of 1 minute data then I propose something along these lines:

Code:
Name                                Steps          Rows          Timespan
Hourly  (1 Minute Average)           1            1576800           14400
Daily   (5 Minute Average)           5               4320           86400
Weekly  (20 Minute Average)          20              4320          604800
Monthly (60 Minute Average)          60              4320         2678400
Yearly  (1 day Average)              1440            1095        31536000


This would result in RRDs that hold:
3 years of 1 minute data
15 days of 5 minute averaged data
60 days of 20 minute averaged data
180 days of 60 minute averaged data
1095 days (3 years) of 1 day (1440 minute) averaged data


or if you want 3 years of data for each consolidation level then something like this:
Code:
Name                                Steps          Rows              Timespan
Hourly  (1 Minute Average)           1            1576800(3y)           14400
Daily   (5 Minute Average)           5             315360(3y)           86400
Weekly  (20 Minute Average)          20             78840(3y)          604800
Monthly (60 Minute Average)          60             26280(3y)         2678400
Yearly  (1440 Minute Average)        1440            1095(3y)        31536000


However, this will require ~122Mb for a simple Interface RRD such as made from teh command below.

Code:
rrdtool create \
test.rrd \
--step 60  \
DS:traffic_in:COUNTER:120:0:1000000000 \
DS:traffic_out:COUNTER:120:0:1000000000 \
RRA:AVERAGE:0.5:1:1576800 \
RRA:AVERAGE:0.5:5:315360 \
RRA:AVERAGE:0.5:20:78840 \
RRA:AVERAGE:0.5:60:26280 \
RRA:AVERAGE:0.5:1440:1095 \
RRA:MIN:0.5:1:1576800 \
RRA:MIN:0.5:5:315360 \
RRA:MIN:0.5:20:78840 \
RRA:MIN:0.5:60:26280 \
RRA:MIN:0.5:1440:1095 \
RRA:MAX:0.5:1:1576800 \
RRA:MAX:0.5:5:315360 \
RRA:MAX:0.5:20:78840 \
RRA:MAX:0.5:60:26280 \
RRA:MAX:0.5:1440:1095 \
RRA:LAST:0.5:1:1576800 \
RRA:LAST:0.5:5:315360 \
RRA:LAST:0.5:20:78840 \
RRA:LAST:0.5:60:26280 \
RRA:LAST:0.5:1440:1095


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Mon Dec 31, 2012 9:56 am 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 276
Location: Melbourne, Australia
daveynihil wrote:
There's just one small question now.

Data Templates = When I got the box, the graphs for the actual Cacti Server had already been setup. (mysql etc)

After changing poller (not cron) from 5 mins to 1 min, I expected these graphs to start acting odd. They have not. Do I need to change the step and heartbeat on ALL templates that are active/i want to use in the future? Should I just then delete and re-create these graphs?

Also, if My poller interval is now 1 minute, I assume all hosts will poll on all graphs setup every 1 minute, and I can't mix and match 1 minute & 5 minute polling per device? Graphs obviously I could make a 5 minute average RRA template, but the actual SNMP query would still hit the box every minute? We'd still store the data points every minute also?


In regards to the part I "bolded" above, I assumed this too. However, my graphs remained in tact just as yours did. After studying the whole process for a while I realised that it all comes down to the Data Template's Step value. It determines the time at which the hosts will be polled, regardless of whether you set your poller time to 1 minute or 5 minutes.

Because all of your original Data Templates have a Step value of 300 and heartbeat value of 600, the corresponding hosts will be polled every 5 minutes even after the poller has been changed to run every one minute.

The same is true for your newly created one minute Data Templates. As your Step value is 60 and your heartbeat value is 120, those hosts will be updated every minute.

In other words, so long as you've got the correct Data Template Step and Heartbeat values configured, your One Minute and Five Minute graphs will work perfectly well in harmony.

classen wrote:
moggie wrote:
My question is, what is the reasoning behind Cacti storing twice the data needed (and tekbot 3 times or more) when the timespan setting means this superfluous data wont be used to render the standard graphs?



My understanding is that this allows you to display the higher resolution graphs further back in time.

So if you're looking at the last 1 hour you can "shift left" several hours and still get the highest resolution information.


This is correct.


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Mon Dec 31, 2012 10:17 am 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 276
Location: Melbourne, Australia
classen wrote:
willieb wrote:
Here's my actual RRA settings. I wanted 1 minute averages on all and I added a 3 year graph.
Code:
Name                                Steps          Rows          Timespan
Hourly  (1 Minute Average)           1              300           14400
Daily   (1 Minute Average)           1              1500          86400
Weekly  (1 Minute Average)           1              11000         604800
Monthly (1 Minute Average)           1              45000         2678400
Yearly  (1 Minute Average)           1              526000        31536000
3 Years (1 Minute Average)           1              1576800       94608000

I hope this helps someone. Thanks.


This seems wrong.

My understanding is that if you want more 1 minute data stored then you need to add more rows to your Hourly (1 min avg) RRA.


The part I have bolded above is correct.


classen wrote:
Your current Hourly (1 minute average) RRA would hold 5 hours of 1 minute data


This is also correct.

classen wrote:
and your 3 Year RRA will hold 3 years of 1 minute data.


Correct.

classen wrote:
But your 3 year graph will only be 500 pixels wide so you have way more information than you will be able to see.


This is not quite true. Yes, the graph will display three years worth of data in the a 500 pixel picture, however, if he uses Cacti's zoom option he will be able to view the 1 minute data without any averaging/consolidation. Therefore it will be as crystal clear as the day the data was captured three years prior.

classen wrote:
If you really want 3 years of 1 minute data then I propose something along these lines:

Code:
Name                                Steps          Rows          Timespan
Hourly  (1 Minute Average)           1            1576800           14400
Daily   (5 Minute Average)           5               4320           86400
Weekly  (20 Minute Average)          20              4320          604800
Monthly (60 Minute Average)          60              4320         2678400
Yearly  (1 day Average)              1440            1095        31536000


This would result in RRDs that hold:
3 years of 1 minute data
15 days of 5 minute averaged data
60 days of 20 minute averaged data
180 days of 60 minute averaged data
1095 days (3 years) of 1 day (1440 minute) averaged data


or if you want 3 years of data for each consolidation level then something like this:
Code:
Name                                Steps          Rows              Timespan
Hourly  (1 Minute Average)           1            1576800(3y)           14400
Daily   (5 Minute Average)           5             315360(3y)           86400
Weekly  (20 Minute Average)          20             78840(3y)          604800
Monthly (60 Minute Average)          60             26280(3y)         2678400
Yearly  (1440 Minute Average)        1440            1095(3y)        31536000


Both of these options would work, but would not provide the granular level of data that is provided with 1 Step, as was originally planned.


classen wrote:
However, this will require ~122Mb for a simple Interface RRD such as made from teh command below.

Code:
rrdtool create \
test.rrd \
--step 60  \
DS:traffic_in:COUNTER:120:0:1000000000 \
DS:traffic_out:COUNTER:120:0:1000000000 \
RRA:AVERAGE:0.5:1:1576800 \
RRA:AVERAGE:0.5:5:315360 \
RRA:AVERAGE:0.5:20:78840 \
RRA:AVERAGE:0.5:60:26280 \
RRA:AVERAGE:0.5:1440:1095 \
RRA:MIN:0.5:1:1576800 \
RRA:MIN:0.5:5:315360 \
RRA:MIN:0.5:20:78840 \
RRA:MIN:0.5:60:26280 \
RRA:MIN:0.5:1440:1095 \
RRA:MAX:0.5:1:1576800 \
RRA:MAX:0.5:5:315360 \
RRA:MAX:0.5:20:78840 \
RRA:MAX:0.5:60:26280 \
RRA:MAX:0.5:1440:1095 \
RRA:LAST:0.5:1:1576800 \
RRA:LAST:0.5:5:315360 \
RRA:LAST:0.5:20:78840 \
RRA:LAST:0.5:60:26280 \
RRA:LAST:0.5:1440:1095


This is really interesting. I'd understanding how different STEP and ROW values affect the size of an RRD file. Can you please explain to me how the above works?


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Tue Jan 01, 2013 4:30 am 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 276
Location: Melbourne, Australia
By the way, I have started a new 1 minute polling thread [url="http://forums.cacti.net/viewtopic.php?f=6&t=49190&p=237488"]over here.[/url] I have written a tutorial in the thread and have also created a calculator which can be used to take all of the manual work out of calculating Steps, Heatbeats, Rows, Timespans, etc.

_________________
[HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.7 and 1 minute polling
PostPosted: Tue Jan 01, 2013 2:30 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22461
Location: Muenster, Germany
Voiper99 wrote:
By the way, I have started a new 1 minute polling thread [url="http://forums.cacti.net/viewtopic.php?f=6&t=49190&p=237488"]over here.[/url] I have written a tutorial in the thread and have also created a calculator which can be used to take all of the manual work out of calculating Steps, Heatbeats, Rows, Timespans, etc.

Great job! Thx
R.

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 102 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group