Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Fri Mar 24, 2017 12:59 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 58 posts ]  Go to page 1, 2, 3, 4  Next
Author Message
 Post subject: Holt-Winters functionality (forecasting) with Cacti
PostPosted: Wed Sep 12, 2007 4:20 pm 
Offline

Joined: Sat May 28, 2005 2:58 pm
Posts: 28
When will Holt-Winters forecast be implemented in cacti?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Feb 21, 2008 5:39 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
sveioen wrote:
When will Holt-Winters forecast be implemented in cacti?


I've had some success in getting some Holt-Winters functionality into Cacti.

All you need to do is:

(a) Convert your RRA files to have the HWPREDICT etc... functions. This is easy with a perl tool called rrd_hwreapply, which you can find as part of the rrdman toolkit on sourceforge http://rrfw.sourceforge.net/rrdman

(b) Edit include/config_arrays.php - where the consolidation_functions array is defined. Add the following to the array - HWPREDICT, SEASONAL, DEVPREDICT, DEVSEASONAL, FAILURES

Now from the Brutlag document you just need to create some CDEFs to use these variables.


Attachments:
File comment: Patch for include/config_arrays.php
holt-winters.patch [423 Bytes]
Downloaded 550 times


Last edited by gilesc on Fri Feb 22, 2008 5:02 am, edited 1 time in total.
Top
 Profile  
 
 Post subject:
PostPosted: Thu Feb 21, 2008 5:46 am 
Offline
Cacti Guru User
User avatar

Joined: Fri Sep 19, 2003 8:36 am
Posts: 2326
Location: Sophia-Antipolis, France
Hi

I've created a new topic because IMHO a lot of people - including me :) - are interested in forecasting with Cacti...

_________________

HOWTOs :
Templates :


Top
 Profile  
 
 Post subject: Confidence Bands
PostPosted: Thu Feb 21, 2008 10:42 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
OK, Well - I'm getting only so so far with this... I seem to have a reasonable HWPREDICT value, but I do have some problems with my CDEF for confidence bands - perhaps I'm being stupid with RPN... It is a bit messy, but I'm just trying to get something to work... here is my graph:

Code:
/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="Holt-Winters Testing - Giles" \
--base=1000 \
--height=350 \
--width=700 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bits per second" \
--slope-mode \
DEF:a="/var/www/sites/stats/web/rra/bst_snmp_oid_2577.rrd":snmp_oid:AVERAGE \
DEF:b="/var/www/sites/stats/web/rra/bst_snmp_oid_2578.rrd":snmp_oid:AVERAGE \
DEF:c="/var/www/sites/stats/web/rra/bst_snmp_oid_2577.rrd":snmp_oid:HWPREDICT \
DEF:d="/var/www/sites/stats/web/rra/bst_snmp_oid_2577.rrd":snmp_oid:DEVPREDICT \
CDEF:cdefa=a,8,* \
CDEF:cdefe=b,8,* \
CDEF:cdefi=d,2,*,c,+,8,* \
CDEF:cdefj=d,-2,*,c,+,8,* \
CDEF:cdefbb=d,8,* \
CDEF:cdefbc=c,8,* \
AREA:cdefa#00CF00:"Inbound"  \
GPRINT:cdefa:LAST:"           Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:cdefe#4444FF:"Outbound"  \
GPRINT:cdefe:LAST:"                  Current\:%8.2lf %s"  \
GPRINT:cdefe:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefe:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:cdefi#FF0000:"Confidence Bands (Testing)\n"  \
LINE1:cdefj#FF0000:""  \
LINE1:d:""  \
LINE1:cdefbb#000000:"Predicted Value"  \
GPRINT:cdefbc:HWPREDICT:"           %8.2lf %s\n"  \
GPRINT:cdefbb:DEVPREDICT:"Deviation\:%8.2lf %s"

My SNMP values are in bytes and I want bits.

My understanding is that if I take HWPREDICT, I can get two confidence bands by plotting HWPREDICT + (2*DEVPREDICT) and HWPREDICT - (2*DEVPREDICT)

The problem I have with the above graph is that the Confidence bands do not appear to be equidistant from the HWPREDICT graph, and in some places the confidence bands actually cross the HWPREDICT graph. :x

I've tried the following RPN notations for CDEFs to calculate the confidence bands - (CDS = Current Datasource))

CDS,d,2,*,+,8,* & CDS,d,-2,*,+,8,*
CDS,8,*,d,16,*,+ & CDS,8,*,d,-16,*,+

am I doing something (RPN) stupid, or is there a deeper problem here?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Feb 22, 2008 3:45 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
Ignore me, the line:

Code:
LINE1:cdefbb#000000:"Predicted Value"  \


Was plotting DEVPREDICT, not HWPREDICT...

So far it appears to work a treat, now all we need is TICK support to be able to highlight anomallies.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Feb 22, 2008 3:54 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
Attached example - the RRD that I'm using is one with only recent data and isn't as predictable as say our backbone traffic would be, but I didn't want to test on one of our more important RRD files as the process involves the insertion of the Holt-Winters RRAs in the RRD.

But as you can see, Holt-Winters in Cacti isn't all that hard after all.


Attachments:
File comment: Example of holt-winters showing the anomalies and smoothing over time.
Clipboard02.png
Clipboard02.png [ 57.32 KiB | Viewed 19947 times ]


Last edited by gilesc on Mon Feb 25, 2008 9:40 am, edited 1 time in total.
Top
 Profile  
 
 Post subject:
PostPosted: Sat Feb 23, 2008 12:26 am 
Offline

Joined: Sat May 28, 2005 2:58 pm
Posts: 28
Hello gilesc,

I havent checked the forums in a while so thats why I havent replied until now. Thats really awesome that you have found a way to make it work! I have tried rrd_hwreapply and experimenting with it on RRD-files, but I never played with it within Cacti.

The screenshot you posted; do you get HW-values for all your data-history? One of the problems I couldnt figure out was that I only got predicted values from "Now - 2 hours", if you know what I mean? It was like a tail that constantly moved along with the time axis, so I never had previous predicted values for more then 2 hours back.

Anyway, I'm going to try this as soon as I can!


Top
 Profile  
 
 Post subject:
PostPosted: Sat Feb 23, 2008 11:02 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
sveioen wrote:
The screenshot you posted; do you get HW-values for all your data-history? One of the problems I couldnt figure out was that I only got predicted values from "Now - 2 hours", if you know what I mean? It was like a tail that constantly moved along with the time axis, so I never had previous predicted values for more then 2 hours back.


This depends on your season, I had my season for this graph set to 288 - so I missed the first 24 hours due to lack of data to make the prediction, but get subsequent predictions.
I've now modified two more RRD files with a weekly season (2016) of Holt Winters, and hope to start seeing predictions on them from next Friday. :D It's going to be a bit of a wait, but rrdtool dumps show that information is being logged. We shall see when that happens, I'll post information as these graphs have much more predictable trends in them.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 07, 2008 10:30 am 
Offline
Cacti User

Joined: Fri Oct 10, 2003 7:23 am
Posts: 91
This looks interesting. I'll have to convert one of my rrds and do some testing myself. Hopefully I can get started tonight :)

/Carl


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 07, 2008 10:33 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
cwahlin wrote:
This looks interesting. I'll have to convert one of my rrds and do some testing myself. Hopefully I can get started tonight :)

/Carl


Do check your RRAs in your RRDs before you use the tool to add the Holt-Winders RRAs.

I have found that if you have RRAs in a strange order in your RRD file that you may lose some historical information, might be best to create a new data source for testing.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 07, 2008 11:39 am 
Offline

Joined: Sat May 28, 2005 2:58 pm
Posts: 28
gilesc wrote:
sveioen wrote:
The screenshot you posted; do you get HW-values for all your data-history? One of the problems I couldnt figure out was that I only got predicted values from "Now - 2 hours", if you know what I mean? It was like a tail that constantly moved along with the time axis, so I never had previous predicted values for more then 2 hours back.


This depends on your season, I had my season for this graph set to 288 - so I missed the first 24 hours due to lack of data to make the prediction, but get subsequent predictions.
I've now modified two more RRD files with a weekly season (2016) of Holt Winters, and hope to start seeing predictions on them from next Friday. :D It's going to be a bit of a wait, but rrdtool dumps show that information is being logged. We shall see when that happens, I'll post information as these graphs have much more predictable trends in them.


Any nice new graphs you want to show us? :D


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 07, 2008 12:47 pm 
Offline
Cacti User

Joined: Fri Oct 10, 2003 7:23 am
Posts: 91
gilesc wrote:
cwahlin wrote:
This looks interesting. I'll have to convert one of my rrds and do some testing myself. Hopefully I can get started tonight :)

/Carl


Do check your RRAs in your RRDs before you use the tool to add the Holt-Winders RRAs.

I have found that if you have RRAs in a strange order in your RRD file that you may lose some historical information, might be best to create a new data source for testing.


I created some new bandwidth sources earlier today to use for testing. I am reading through the brutlag doc now (is this the one you refer to earlier?).

Need to put the kids to sleep before I start playing around :)

/Carl


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 07, 2008 1:10 pm 
Offline
Cacti User

Joined: Fri Oct 10, 2003 7:23 am
Posts: 91
I am using 0.8.7b which does not have config_arrays.php. It uses global_arrays.php. I added HWPREDICT etc to the array, but I see in your patch you have added " do_hook("config_arrays"); " somewhere? Where does this go? (if it is needed)

/Carl


Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 08, 2008 9:29 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
cwahlin wrote:
I am using 0.8.7b which does not have config_arrays.php. It uses global_arrays.php. I added HWPREDICT etc to the array, but I see in your patch you have added " do_hook("config_arrays"); " somewhere? Where does this go? (if it is needed)

/Carl


More than likely not needed, if it's in the patch it's just a context reference (i.e. doesn't have a + or - at the beginning of the line)


Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 08, 2008 9:32 am 
Offline

Joined: Thu Feb 07, 2008 4:01 am
Posts: 21
Quote:
Any nice new graphs you want to show us? :D


I will have one in a few weeks, it has a season of 2016 (1 week) and the confidence bands are still all over the place due to lack of data. I will post it when it looks ready :wink:


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 58 posts ]  Go to page 1, 2, 3, 4  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group