Cacti (home)ForumsRepositoryDocumentation
Cacti: offical forums and support
It is currently Fri Oct 24, 2014 6:08 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!
PostPosted: Mon Dec 31, 2012 11:14 am 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 275
Location: Melbourne, Australia
Three hours in to 2013 and here I am on the Cacti forums :)

This thread was inspired by the [HOWTO] 0.8.7 and 1 minute polling page, and in particular, tekbot's fantastic post.

Before I begin, I must repeat tekbot's suggestion:

tekbot wrote:
Also, send the developers some love (and / or money). As mentioned in the 0.8.7 release notes, they all have Day Jobs and contribute to Cacti because they WANT TO.


Cacti is an excellent, free product, so I feel the least we can do as users is donate to those who have dedicated so much of their free time in to giving us this software.

Now, let's begin!

Why the new thread?

There are two reasons for this new thread. The first reason is because I am going to create an updated version of gandalf's excellent tutorial and thought it would be better suited in a new thread rather than tacked on to an existing thread.

The second reason is because I have created a calculator that takes all the hassle out of creating new polling intervals (e.g one minute polling). Rather than tack it on to an existing thread, putting it in a new thread will allow me to track and fix issues, feature requests, etc.



What exactly does the calculator do?

Whether you have configured One Minute Polling before, or you are looking to implement it, you will most probably be familiar with terms such as Data Source Steps, Heartbeats, RRA Steps, Rows and Timespans. Calculating these can be quite difficult (especially for those who are just starting out), and, if you have no idea what they all mean, it can make enabling One Minute Polling very challenging - however, this is where the Calculator comes in. By answering four simple questions the calculator will tell you which values you need to enter in to each of the above mentioned fields and will also estimate the amount of space you will need per Data Source.

Note: Although this calculator was built to simplify the One Minute Polling process, I do urge all readers to understand the process fully. Doing so will ensure that you get the most out of this feature.

Two Part Calculator

The calculator is broken in to two parts. The "Forward" calculator and the "Reverse" calculator. Everything written in the previous section applies to the "Forward" calculator. The "Reverse" calculator allows you to turn "Steps", "Rows" and "Timespans" in to easily understandable information, without having to perform any manual calculations.

Reverse Calculator Examples

When you first install Cacti, all Data Templates have a Step value of 300 and the below RRAs are preconfigured:

Code:
Name                       Steps    Rows       Timespan
Hourly (1 Minute Average)    1       500        14400
Daily (5 Minute Average)     1       600        86400
Monthly (2 Hour Average)    24       775      2678400
Weekly (30 Minute Average)   6       700       604800
Yearly (1 Day Average)     288       797     33053184


Using the Calculator, simply plug in the above details and it will translate the information to an easily understandable format.

Let's dissect a few of the above mentioned RRAs:

Code:
Name                       Steps    Rows       Timespan
Daily (5 Minute Average)     1       600        86400


Image

The calculator tells us that the Daily (5 Minute Average) does the following:

    * It keeps data for 2 days before overwriting it.
    * It averages data over a 5 minute period.
    * It displays a graph that spans 1 day.

OK, time for the next one:

Code:
Name                       Steps    Rows       Timespan
Monthly (2 Hour Average)    24       775      2678400


Image

The calculator tells us that the Monthly (2 Hour Average) does the following:

    * It keeps data for just over 2 months before overwriting it.
    * It averages data over a 2 hour period.
    * It displays a graph that spans 1 month.

And finally:

Code:
Name                       Steps    Rows       Timespan
Yearly (1 Day Average)     288       797     33053184


Image

The calculator tells us that the Yearly (1 Day Average) does the following:

    * It keeps data for just over 2years before overwriting it.
    * It averages data over a 1 day period.
    * It displays a graph that spans just over a 1 year.

Forward Calculator Examples

Now that we've seen how the Reverse Calculator works, it should make understanding the Forward Calculator easier.

Below is a list of the One Minute RRAs I wanted to create for myself:

    * Daily (1 Minute Average) - Display a 1 day graph that polls hosts every minute and does not consolidate the data. The data is to be overwritten after 1 week.
    * Weekly (5 Minute Average) - Display a 1 week graph that polls hosts every minute and consolidates the data every 5 minutes. The data is to be overwritten after one month.
    * Monthly (30 Minute Average) - Display a 1 month graph that polls hosts every minute and consolidates the data every 30 minutes. The data is to be overwritten after three months.

So let's get started.

The below example shows how the Daily (1 Minute Average) RRA was configured:

Image

The below example shows how the Weekly (5 Minute Average) RRA was configured:

Image

The below example shows how the Monthly (30 Minute Average) RRA was configured:

Image

Why the long data retention?

A common question I have seen is, "why would you have a longer data retention period than the timespan of the graph?". For example, as mentioned above, one of the RRAs that I wanted was:

    * Daily (1 Minute Average) - Display a 1 day graph that polls hosts every minute and does not consolidate the data. The data is to be overwritten after 1 week.

It may sound silly to store 7 days worth of data and only display 1 day's worth of graph. However, there is method behind this madness.

Using Cacti's "Graph Filters", I can look at any time frame in the past week and view the graphs at a one minute polled interval. If I had not done this, Cacti would have no choice but to display the graphs in my "next best" RRA configuration, which was the Weekly (5 Minute Average) graph. This is not ideal because the data is averaged over 5 minutes, therefore it will effectively have five times less detail than my Daily (1 Minute Average) graphs.

An alternative to Cacti's "Graph Filters" option is its "Zoom Graph" option. Cacti automatically displays graphs using the most detailed graph possible. For example, if a time period is covered by both the Daily (1 Minute Average) and Weekly (5 Minute Average) RRAs, it will present the graph using the Daily (1 Minute Average) RRA.

As per the graphs below, I can zoom in to my "Yearly (1 Day Average)" to the point where the data will be displayed using the Daily (1 Minute Average) RRA.

Fig 1: Zooming in to the "Yearly (1 Day Average)" graph:

Image

Fig 2: After the first zoom, the graph is drawn using the Weekly (5 Minute Average) graph:

Image

Note: The reason why the graph drawing skipped the Monthly (2 Hour Average) RRA is because it had the specified time frame covered by the Weekly (5 Minute Average) RRA, and, as mentioned above, Cacti will always use the RRA which has the most detail in it. If I were to choose a date that was over 1 month ago, the data would not be available in the Weekly (5 Minute Average) RRA (because it only stores data for one month before it is overwritten) and therefore Cacti would use the Monthly (2 Hour Average) RRA instead.

Fig 3: Finally, after zooming in a third time, the graph is drawn using the Daily (1 Minute Average) RRA. You can confirm this is true by counting the amount of times the graphs data changes between two time frames. For example, you can see that the "Inbound" and "Outbound" lines change ten times (once per minute) between 10:50am and 11:am:

Image

HOWTO: One Minute Polling

(Work In Progress).

That's All Folks!

I will update this thread again in the near future to include the above mentioned HOWTO: One Minute Polling section.

Thank you for reading my lengthy post, and I wish you all a Happy New Year!

Please let me know if you find any bugs or have any feature requests.


Attachments:
CactiToolv1.0.zip [17.4 KiB]
Downloaded 475 times

_________________
[HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!


Last edited by Voiper99 on Wed Mar 27, 2013 4:28 am, edited 7 times in total.
Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + Calculator!
PostPosted: Mon Jan 28, 2013 9:36 pm 
Offline

Joined: Wed Oct 11, 2006 2:40 am
Posts: 13
Hi,

Any update to this?
I would like to see the calculator and the How-To.

Regards

-Hurgh-


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + Calculator!
PostPosted: Thu Feb 07, 2013 4:46 am 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 275
Location: Melbourne, Australia
hurgh wrote:
Hi,

Any update to this?
I would like to see the calculator and the How-To.

Regards

-Hurgh-


Sorry about the delay. I have finally completed and uploaded it. The download link can be found in the bottom of my first post of this thread.

_________________
[HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Sun Mar 24, 2013 4:31 pm 
Offline
User avatar

Joined: Mon Nov 15, 2010 3:46 pm
Posts: 17
Location: London, UK
Excellent work, Voiper99, and hugely useful. Thank you.


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Sun Mar 24, 2013 5:47 pm 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 275
Location: Melbourne, Australia
Not a problem at all, I'm glad I could help :)

_________________
[HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Wed Mar 27, 2013 4:30 am 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 275
Location: Melbourne, Australia
Thanks to phalek, I have now been able to attach the spreadsheet to my post above :) I have for that reason removed the external download link.

_________________
[HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Wed Mar 27, 2013 5:21 am 
Offline
User avatar

Joined: Mon Nov 15, 2010 3:46 pm
Posts: 17
Location: London, UK
Most RRD files have multiple RRAs, and multiple time spans. For instance, my "Interface Traffic" RRDs have:

Hourly (1 Minute Average) 1 500 14400
Daily (5 Minute Average) 1 600 86400
Weekly (30 Minute Average) 6 700 604800
Monthly (2 Hour Average) 24 775 2678400
Yearly (1 Day Average) 288 797 33053184

Each of those RRAs has four consolidation functions: AVERAGE, MIN, MAX, and LAST.

That's 5 RRAs, 4 consolidations, and two data sources (bits in and out). However, your file size calculator assumes only one RRA. By hand, I can do 5 calculations, add them up, and remove the 4 extra headers. However, it shouldn't be too hard to add this to your calculator. Are you interested in doing it?


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Fri Apr 05, 2013 3:51 pm 
Offline
Cacti User

Joined: Thu Mar 01, 2007 5:43 pm
Posts: 275
Location: Melbourne, Australia
Thanks for the feedback. I'd be more than happy to update the tool but I'm not too sure if I understand what you're saying.

Tolaris wrote:
Most RRD files have multiple RRAs, and multiple time spans. For instance, my "Interface Traffic" RRDs have:

Hourly (1 Minute Average) 1 500 14400
Daily (5 Minute Average) 1 600 86400
Weekly (30 Minute Average) 6 700 604800
Monthly (2 Hour Average) 24 775 2678400
Yearly (1 Day Average) 288 797 33053184


Agreed, though each one of the above RRAs will result in a different file size due to their polling intervals and data retention times, so you would need to input their details separately to find out how big each individual RRA file is going to be.

Tolaris wrote:
Each of those RRAs has four consolidation functions: AVERAGE, MIN, MAX, and LAST.

That's 5 RRAs, 4 consolidations, and two data sources (bits in and out).


My tool allows you to specify how many consolidations and data sources you're using. However, as mentioned above (and please correct me if I'm wrong), as each RRA's file size is going to be different depending on how much detail you want stored and how long you want the data stored for, the tool requires you to input the details of each RRA separately. For example, if you enter "Yearly" RRA's data in to the tool, that information cannot be used to determine the "Hourly" RRA's file size.

_________________
[HOWTO] 0.8.8 and 1 minute polling + File Size Calculator!


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Fri Apr 05, 2013 4:46 pm 
Offline
User avatar

Joined: Mon Nov 15, 2010 3:46 pm
Posts: 17
Location: London, UK
Voiper99 wrote:
Agreed, though each one of the above RRAs will result in a different file size due to their polling intervals and data retention times, so you would need to input their details separately to find out how big each individual RRA file is going to be.


Thanks for the help. You are correct that each RRA would differ. The problem is nearly all RRD files in Cacti have multiple RRAs in them. One for each span of time (Hourly, Daily, ...), for instance.

Could I calculate each RRA and add them up to calculate the entire RRD file total? IE- which of the following is true?

1. An RRD file has no header, just a series of concatenated RRAs with headers and data.
2. The RRD file has just one header, and a series of RRAs with only data.
3. The RRD file has a header itself, and a series of concatenated RRAs each with headers and data.

Is that more clear?


Top
 Profile  
 
 Post subject: Re: [HOWTO] 0.8.8 and 1 minute polling + File Size Calculato
PostPosted: Mon Jul 14, 2014 11:25 am 
Offline

Joined: Fri Sep 01, 2006 2:23 pm
Posts: 6
Location: Oak Brook, IL
Hello, the screenshots of using the Forward Calculator have gone missing so I'm not entirely sure I'm tracking on the right path as I use it. Could somebody share them (if they still have them)? Or repost them to this thread from their own calculation? Voiper99, perhaps?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group