|
|
| Author |
Message |
dus001
Joined: 07 Aug 2005
Posts: 40
|
 Posted: Sat Jun 07, 2008 5:29 am Post subject: |
 |
|
| Quote: | | Is there a way to avoid recreating 1500+ graphs that I have if I want to have 1 min resolution? |
I've not tried it, but maybe something can be done with the "rrdtool tune" command (and the update of datasources or data templates in cacti accordingly) ?
| Code: |
$ rrdtool tune
RRDtool 1.2.19 Copyright 1997-2007 by Tobias Oetiker <tobi@oetiker.ch>
Compiled May 22 2007 13:48:39
Usage: rrdtool [options] command command_options
* tune - Modify some basic properties of an RRD
rrdtool tune filename
[--heartbeat|-h ds-name:heartbeat]
....
|
That would of course need some serious testing before application.
|
|
| Back to top |
|
 |
gandalf
Developer
Joined: 02 Dec 2004
Posts: 12295
Location: Muenster, Germany
|
| Posted: Sun Jun 08, 2008 3:13 am Post subject: |
|
|
No, sorry. No chance to change step size using rrdtool tune or stuff. Using official stuff only, it is NOT POSSIBLE to convert a 5 min absed rrd file to 1 min based.
If you don't trust me  please as at rrdtool-users mailing list
Reinhard
|
|
| Back to top |
|
|
tephra
Joined: 09 Dec 2007
Posts: 24
|
| Posted: Thu Jun 12, 2008 9:56 pm Post subject: |
|
|
Hey Guys,
I need help with this, my setup:
2300ish hosts - mainly traffic stats
2250 of these hosts are setup to be polled every 15 mins
the other 50 are setup for 5 min polling.
Now I want to run the cronjob every minute, I have set both poller and cron interval to 1 minute.
Is Cacti going to automatically split my 2250 hosts into groups so that all of them are polled every 15 mins? ie polling 2250/15 every min?
Cheers
D.
|
|
| Back to top |
|
|
kharford
Joined: 07 Jul 2005
Posts: 40
Location: Mass, USA
|
| Posted: Mon Jun 16, 2008 9:36 am Post subject: |
|
|
06/16/2008 09:20:14 AM - POLLER: Poller[0] WARNING: Cron is out of sync with the Poller Interval! The Poller Interval is '60' seconds, with a maximum of a '300' second Cron, but 313 seconds have passed since the last poll!
06/16/2008 09:19:17 AM - SYSTEM STATS: Time:15.6657 Method:spine Processes:4 Threads:20 Hosts:470 HostsPerProcess:118 DataSources:10146 RRDsProcessed:0
06/16/2008 09:18:56 AM - SYSTEM BOOST STATS: Time:41.1010 RRDUpdates:209666
Cacti Version 0.8.7b
Cacti OS unix
SNMP Version net-snmp
RRDTool Version RRDTool 1.2.x
Hosts 470
Graphs 4485
Data Sources SNMP: 3956
SNMP Query: 1691
Script Query - Script Server: 1566
Total: 7213
Interval 60
Type spine
Items Action[0]: 7338
Action[2]: 2808
Total: 10146
Concurrent Processes 4
Max Threads 20
PHP Servers 8
Script Timeout 35
Max OID 60
*/5 * * * * cactiuser /usr/bin/php /var/www/html/cacti/poller.php > /var/log/poller.log 2>&1
0 */1 * * * /usr/bin/php -q /var/www/cacti-0.8.7b/plugins/boost/poller_boost.php -f -d
Any ideas of why I keep getting INTERMITTENT gaps in my graphs and I get the INTERMITTENT Cron out of sync message?
Any help greatly appreciated
Ken
|
|
| Back to top |
|
|
tephra
Joined: 09 Dec 2007
Posts: 24
|
| Posted: Mon Jun 16, 2008 7:27 pm Post subject: |
|
|
not sure on your particular case but my spine processes ALWAYS take up the maximum amount of time, ie 297seconds or 57seconds.. even though they are idle they dont automatically die off...
strange... i hope it gets fixed 
|
|
| Back to top |
|
|
tekbot
Joined: 07 Jun 2005
Posts: 45
Location: Venice, CA
|
| Posted: Wed Jul 02, 2008 11:14 am Post subject: |
|
|
| gandalf wrote: | No, sorry. No chance to change step size using rrdtool tune or stuff. Using official stuff only, it is NOT POSSIBLE to convert a 5 min absed rrd file to 1 min based.
If you don't trust me please as at rrdtool-users mailing list
Reinhard |
Guys,
I have written a script that reformats old 5 minute RRD files into 1 minute RRD files. I ran this against an infrastructure and it worked a charm. I will attach the scripts below. There is some information you need to make this work, and you need to be familiar navigating around Cacti. I'll be as verbose as I can to ease you through this.
(PS - I don't want any code monkeys telling me how sub-optimal this script is. Feel free and improve on it, but don't tell me it's "bad code", because as far as I'm aware, I'm the only one who's successfully done this -- Ever).
The script has a few parts, which should be explained here.
GetAndConvert - this is the wrapper script. This script takes one argument, which should be your INPUT_FILE described below. There are constants defined at the top of the script. This is where you tell the script important information about your environmeent like where to find the RRD files to convert, and where to dump them when they're done. Modify this to suit your needs. Also, since I built my 0.8.7 on a new machine, I used SCP to get the files from a remote host. If you're not doing this, you'll need to edit the script accordingly. (I've put a note on how to get your SCP working without entering a password for each file, if you don't know how to use Public Key Auth).
INPUT_FILE - YOU create this file. This file should be a list of all of the files you want to bring over and convert. You need to strip the .rrd from the filenames for the script to work properly (or you can fix this bug -- code monkey!). An easy way to do this in vim would be or you could run something like this on your old rra dir | Code: | | ls | awk -F. '{print $1}' > INPUT_FILE |
Whatever works for you.
FormatRRA - This is the guy that does all the work. This requires information from you as well to properly set the multipliers. Basically, what this script does is takes the raw XML of your RRD and reads through it until it finds it's first data block. We then adjust the step value in the data definition and we print every line n times (depending on your multiplier - also discussed later). In simpler terms, the first block of a 5 minute graph has a line that defines STEP, which is set to 300 (seconds). We replace that with a new STEP because we want a 1 minute graph (60 seconds). And, since our step is different, we need more data points to make up for the missing space, so we take each value and print it out 5 times. We do this for every existing line until we encounter a new STEP definition, and we then adjust our multiplier accordingly. Making sense?
Misc Notes:
All of these scripts should be put into the same directory. You should run them all as the cacti user. You need to tell your 0.8.7n installation about your new RRD formats. This can be done under Data Sources -> RRAs. I wrote a post a few months ago describing how to configure your new RRAs for optimal storage at high-granularity. That post is here: http://forums.cacti.net/viewtopic.php?t=23885&postdays=0&postorder=asc&start=15. I know it's long, but Read It! If you use the format I suggest there, you do not have to edit your multipliers. The only "drawback" to using the formats as I've defined them is, I want more granular data for longer, therefor my RRD files are bigger than the default. MBs are cheap these days.
Output:
This script will churn for a good few minutes on every RRD file, and I didn't bother writing in any MySQL injection (feel free). The output will give you the perfect text string to paste into your Data Source field. Here is your basic order of operations (this is in the README in the gzip attached):
* Create your RRD Structure as defined in my post on 1-minute polling (I'm tekbot).
* Create new Graph for existing device (This creates the datasource in the Database)
* Create input file for list of existing Graphs (.rrd files) from old cacti installation to convert (see section on INPUT_FILE in the post where you found these scripts).
* Edit the Constants at the top of GetAndConvert and FormatRRA to suit your needs.
* Setup SSH Private Key Authentication (Optional)
* Run the script (Syntax: ./GetAndConvert INPUT_FILE)
* Update Data source to point to new RRD.
* Place on Tree!
Have fun kids, and be safe!
--tekbot
| Description: |
|
Download |
| Filename: |
Convert5minTo1minRRD.tar.gz |
| Filesize: |
1.7 KB |
| Downloaded: |
113 Time(s) |
|
|
| Back to top |
|
|
it
Cacti User
Joined: 17 Jun 2008
Posts: 118
|
| Posted: Wed Jul 16, 2008 8:59 am Post subject: |
|
|
Hourly (1 Minute Average) 1 500 14400
Daily (5 Minute Average) 1 600 86400
Weekly (30 Minute Average) 6 700 604800
Monthly (2 Hour Average) 24 775 2678400
Yearly (1 Day Average) 288 797 33053184
this is my rra settings , can u tell me how to fix this in order to get the 1 min polling? i have done the whole settings i still have the rra??
thx in advance
|
|
| Back to top |
|
|
tekbot
Joined: 07 Jun 2005
Posts: 45
Location: Venice, CA
|
| Posted: Mon Jul 21, 2008 8:11 am Post subject: |
|
|
@IT:
These RRA settings need to be adjusted slightly. I'm assuming you're doing this work in a development or pre-production environment?
Here's how I'd set it and why:
| Code: |
Name / Steps / Rows / Timespan
Hourly / 1 / 500 / 14400 (This is fine)
Daily / 5 / 600 / 86400 (Adjust your steps since you're polling at 1 minute now -- 5 polls every 5 minutes)
Weekly / 30 / 700 / 604800 (Adjust your steps since you're polling at 1 minutes -- 30 polls per half hour)
Monthly / 120 / 775 / 2678400 (Adjust your steps -- 120 polls every 2 hours)
Yearly / 1440 / 797 / 33053184 (Adjust your steps -- 1440 minutes in a day, so 1440 polls to average to get your day value).
|
After you've adjusted your RRAs you *must* recreate the graphs for them to take effect. So, a basic order of operations would be:
* Adjust your RRA settings in a NEW/Dev/Pre-Production Cacti 0.8.7a Environment
* Verify your Data Templates (This is required if you've added new RRA settings for 1-minute or 10-second polling) -- In other words, if you've added a new RRA type, you need to make sure your DataSources are expecting this (see screenshot below for my Network 1 minute Data Template).
* Create NEW graphs for your host. If you've followed the 2 steps immediately previous, you should start seeing 1 minute data on your new graphs.
* Convert your old Graphs using the scripts I've provided. The scripts will output a string you can paste into the Data Source field. Updating this tells cacti to start populating the converted RRD File with the 1 minute data, instead of the graph you created in the previous step.
* When you reload the screen containing this graph, you should see all of your old data "converted" and you'll be able to update the new RRD files every one minute.
Of course, you need to ensure you take the advice of the other posters in this thread regarding how to configure your CRON, and the settings on the Poller Tab within Cacti.
Here's the Screenshot of the Data Template I mentioned above:
As you can see, I've associated the 1 min RRAs with the Data Source so that Cacti knows how to format the new RRD file.
Hope this helps. I wish I had more time to write out more definitive documentation, unfortunately, as great as cacti is, it doesn't pay the bills.
Back to work!
--tekbot
| Description: |
| Network Traffic Data Source with Associated 1 minute RRA Files |
|
| Filesize: |
105.87 KB |
| Viewed: |
1853 Time(s) |
|
|
|
| Back to top |
|
|
Gavin
Joined: 13 Aug 2008
Posts: 6
|
| Posted: Thu Aug 14, 2008 4:38 am Post subject: |
|
|
1.To complete all task, should I reschdule the task of runing "php poller.php" to 1 minute either?
2.I create two RRAs :
Name / Steps / Rows / Timespan
1 min - 1 Minute Average - 24 Hour View 1 4320 86400
1 min - 1 Minute Average - 7 Day View 1 44640 604800
Then recreate the graph.
Is there a method I can take to make sure that 1 minute polling is working and new graph is of 1minute ?
3. You said clear snmp and poller cache.,but how to do it? I entered "system utilities"-"View SNMP cache", then clicked "clear",but nothing happened. Do you mean to click "rebuild poller cache " in system utilities ?
4.According to your instruction, I modified the Interface - Traffic (default) Data template.
When I clicked save, I got some warning as following. However,I rechecked the Data template, all modification had been saved. Although the graph can display ,I want to know how to deal with these warning ?
Warning: Division by zero in C:\AppServ\www\cacti\lib\api_poller.php on line 119
Warning: Division by zero in C:\AppServ\www\cacti\lib\api_poller.php on line 119
Warning: Division by zero in C:\AppServ\www\cacti\lib\api_poller.php on line 119
Warning: Division by zero in C:\AppServ\www\cacti\lib\api_poller.php on line 119
Warning: Cannot modify header information - headers already sent by (output started at C:\AppServ\www\cacti\lib\api_poller.php:119) in C:\AppServ\www\cacti\data_templates.php on line 249
|
|
| Back to top |
|
|
tekbot
Joined: 07 Jun 2005
Posts: 45
Location: Venice, CA
|
| Posted: Thu Aug 14, 2008 5:26 am Post subject: |
|
|
Hey Gavin. I have some answers for you.
1. Leave your cron at 5 minutes. In your Poller Settings you have an option for cron interval and polling interval. Set cron interval to 5 and polling interval to 1.
2. There are a lot of ways to check that your new device is being polled every minute. You can check your cacti.log file and see how often the poller is running and polling your device(s). You can also zoom in on your new graph or view the "last 30 minutes" and see if you're getting updates every one minute, or still every 5. If you've created the 1 minute RRA, changed the data template and recreated the graph, you should have 1 minute data.
3. They've renamed it from Clear Poller Cache to Rebuild Poller Cache. This step is probably unnecessary, but it's a default for general cacti troubleshooting.
4. So long as when you verified your changes afterwards you're fine. I've posted about that bug as well -- it's benign. You can avoid seeing that in the future by going into your php.ini file and turning off 'display_errors'. (You need to restart apache after modifying php.ini for that change to take effect).
Hope that helps! Happy Polling!
--tekbot
|
|
| Back to top |
|
|
Gavin
Joined: 13 Aug 2008
Posts: 6
|
| Posted: Thu Aug 14, 2008 10:47 am Post subject: |
|
|
1. I have set cron interval to 5 and polling interval to 1 in the Polling Setting web page.
In cacti document "Install_windows.html", I was told to schedule a task to run poller.php every 5 minutes. My question is whether I should modify the task to run every minute now.
2. If the answer to my first question is yes ,in other words, I should run poller.php every minute, I think that I will learn that poller is running every minute from cacti.log file, no matter which polling interval I set in the Polling Setting web page.
My second question may be not very clear. In fact ,I want to know whether the polled data is of 1 minute but not of original 5 minutes.
Therefore, viewing the "last half hour" is a good answer to my second question.
4. Could you give the link to your post about this bug? I want to read it,but I can not find it .
|
|
| Back to top |
|
|
tekbot
Joined: 07 Jun 2005
Posts: 45
Location: Venice, CA
|
| Posted: Thu Aug 14, 2008 11:00 am Post subject: |
|
|
1. No, leave your scheduled task at 5 minutes. Cacti will schedule the polls on its own based on the poller interval setting. You don't need to change anything further here.
2. Even though you're set with Cron at 5 and Polling Interval at 1, you'll still see polls every one minute in the log file if your data source is setup properly. Check your graph. How many discreet data points do you have over the half hour? 30? or 6? If there's only 6, you've missed something. If you have 30, you're good.
4. http://forums.cacti.net/about20954.html&highlight=display_errors+php.ini
There's the link to my reporting the division by 0. Again, it's benign. Just find your php.ini, set display_errors to Off and restart apache and you're good to go.
Let me know how it goes.
--tekbot
|
|
| Back to top |
|
|
Gavin
Joined: 13 Aug 2008
Posts: 6
|
| Posted: Fri Aug 15, 2008 2:40 am Post subject: |
|
|
1.Do you mean that even if the scheduled task runs at other interval ,for example 10 mintues or 20 mintues, Cacti will poll on its own based on the polling interval setting ? Then the scheduled task seems insignificant.
Nevertheless,if I stop the scheduled task,the graph will stop updating with new data.I guess that creating this scheduled task must have its meaning.
In addition,chapter5 in The Cacti Manual , Principles of Operation, says that Cacti will retrieve data using its Poller. The Poller is executed from the operating system's scheduler, e.g. crontab for Unix flavored OSes.
If you know and have time ,could you tell me how to understand scheduled task, Cacti's polling task and Cron task ? How do the three tasks work together ?
|
|
| Back to top |
|
|
johnnylately
Joined: 11 Aug 2008
Posts: 1
Location: Maryville, TN, USA
|
| Posted: Tue Aug 19, 2008 11:27 am Post subject: |
|
|
I would like to make an attempt to explain scheduled task/cron and polling:
(Scheduled task=5 min, polling=5 min)
(Scheduled task executes the poller)
Time
11:00 Scheduled task runs, poller polls, then scheduled task ends
11:05 Scheduled task runs, poller polls, then scheduled task ends
11:10 Scheduled task runs, poller polls, then scheduled task ends
etc...
(Scheduled task=5 min, polling=1 min)
Time
11:00 Scheduled task runs, poller polls (#1), scheduled task stays running
11:01 (task still running) poller polls (#2)
11:02 (task still running) poller polls (#3)
11:03 (task still running) poller polls (#4)
11:04 (task still running) poller polls (#5), scheduled task ends
11:05 Scheduled task runs, poller polls (#1), scheduled task stays running
11:06 (task still running) poller polls (#2)
11:07 (task still running) poller polls (#3)
11:08 (task still running) poller polls (#4)
11:09 (task still running) poller polls (#5), scheduled task ends
11:10 repeats...
I will say that I am a beginner with Cacti and php language so any corrections/enhancements are appreciated. The times are not exact, I'm using them to show each step. This was the most confusing terminology until I read tekbot's posts, changed Cacti, and watched the windows open and close (I'm running Cacti on a Win 2003 server). I've got everything to work on a test server but haven't changed the production server (yet). Cacti installs perfectly from the Beta installer, but my boss wants 1 minute graphs like we have on our existing Cricket grapher, so it has been a slow process. For me at least.
johnny
*********
|
|
| Back to top |
|
|
baxford
Joined: 20 Nov 2008
Posts: 20
|
| Posted: Thu Nov 27, 2008 7:01 pm Post subject: |
|
|
| tekbot wrote: | | gandalf wrote: | No, sorry. No chance to change step size using rrdtool tune or stuff. Using official stuff only, it is NOT POSSIBLE to convert a 5 min absed rrd file to 1 min based.
If you don't trust me :wink: please as at rrdtool-users mailing list
Reinhard |
Guys,
I have written a script that reformats old 5 minute RRD files into 1 minute RRD files. I ran this against an infrastructure and it worked a charm. I will attach the scripts below. There is some information you need to make this work, and you need to be familiar navigating around Cacti. I'll be as verbose as I can to ease you through this.
(PS - I don't want any code monkeys telling me how sub-optimal this script is. Feel free and improve on it, but don't tell me it's "bad code", because as far as I'm aware, I'm the only one who's successfully done this -- Ever). :)
The script has a few parts, which should be explained here.
GetAndConvert - this is the wrapper script. This script takes one argument, which should be your INPUT_FILE described below. There are constants defined at the top of the script. This is where you tell the script important information about your environmeent like where to find the RRD files to convert, and where to dump them when they're done. Modify this to suit your needs. Also, since I built my 0.8.7 on a new machine, I used SCP to get the files from a remote host. If you're not doing this, you'll need to edit the script accordingly. (I've put a note on how to get your SCP working without entering a password for each file, if you don't know how to use Public Key Auth).
INPUT_FILE - YOU create this file. This file should be a list of all of the files you want to bring over and convert. You need to strip the .rrd from the filenames for the script to work properly (or you can fix this bug -- code monkey!). An easy way to do this in vim would be or you could run something like this on your old rra dir | Code: | | ls | awk -F. '{print $1}' > INPUT_FILE |
Whatever works for you.
FormatRRA - This is the guy that does all the work. This requires information from you as well to properly set the multipliers. Basically, what this script does is takes the raw XML of your RRD and reads through it until it finds it's first data block. We then adjust the step value in the data definition and we print every line n times (depending on your multiplier - also discussed later). In simpler terms, the first block of a 5 minute graph has a line that defines STEP, which is set to 300 (seconds). We replace that with a new STEP because we want a 1 minute graph (60 seconds). And, since our step is different, we need more data points to make up for the missing space, so we take each value and print it out 5 times. We do this for every existing line until we encounter a new STEP definition, and we then adjust our multiplier accordingly. Making sense?
Misc Notes:
All of these scripts should be put into the same directory. You should run them all as the cacti user. You need to tell your 0.8.7n installation about your new RRD formats. This can be done under Data Sources -> RRAs. I wrote a post a few months ago describing how to configure your new RRAs for optimal storage at high-granularity. That post is here: http://forums.cacti.net/viewtopic.php?t=23885&postdays=0&postorder=asc&start=15. I know it's long, but Read It! If you use the format I suggest there, you do not have to edit your multipliers. The only "drawback" to using the formats as I've defined them is, I want more granular data for longer, therefor my RRD files are bigger than the default. MBs are cheap these days.
Output:
This script will churn for a good few minutes on every RRD file, and I didn't bother writing in any MySQL injection (feel free). The output will give you the perfect text string to paste into your Data Source field. Here is your basic order of operations (this is in the README in the gzip attached):
* Create your RRD Structure as defined in my post on 1-minute polling (I'm tekbot).
* Create new Graph for existing device (This creates the datasource in the Database)
* Create input file for list of existing Graphs (.rrd files) from old cacti installation to convert (see section on INPUT_FILE in the post where you found these scripts).
* Edit the Constants at the top of GetAndConvert and FormatRRA to suit your needs.
* Setup SSH Private Key Authentication (Optional)
* Run the script (Syntax: ./GetAndConvert INPUT_FILE)
* Update Data source to point to new RRD.
* Place on Tree!
Have fun kids, and be safe!
--tekbot |
Any chance you have this conversion tool for Windows?
|
|
| Back to top |
|
|
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Register
Profile
Log in to check your private messages
Log in
