Cacti (home)ForumsRepositoryDocumentation
Cacti: offical forums and support
It is currently Mon Jul 28, 2014 1:14 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 53 posts ]  Go to page 1, 2, 3, 4  Next
Author Message
 Post subject: Submit your BOOST Statistics Here
PostPosted: Sat Dec 09, 2006 3:55 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
All,

I have released the Boost Plugin, version 1.0. Once you have something good to say about boost, or bad for that matter, please post your before and after statistics as followups to this post.

Here are the main features:
On-Demand RRD Updating You don't have to update 40k RRD files every 5 minutes, you can wait hours before doing updates. If a user accesses a Graph before a major update, the "On-Demand" updater takes care of it.
Network Service to handle RRD Updating This service is handy when your Web Server does not have RW access to the RRA folder. This provides an added benefit of allowing for multiple Web Servers to be deployed within this framework to host Cacti.
PNG Caching Yes, that's right. This plugin will cache PNG's for as long as they are viable. Once they expire, the first person to view the graph uses the overhead of creating the next good image.

Regards,

TheWitness

http://forums.cacti.net/viewtopic.php?p=86053#86053

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 15, 2006 1:01 am 
Offline

Joined: Mon Jul 18, 2005 10:42 pm
Posts: 43
Location: Melbourne, Australia
Seems to be working fine.

12/15/2006 05:00:23 PM - SYSTEM BOOST STATS: Time:4.2656 RRDUpdates:1022

Love the cache feature :-)

Cheers

PS. View Boost Status not working.


Top
 Profile  
 
 Post subject:
PostPosted: Sun Dec 17, 2006 2:35 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22461
Location: Muenster, Germany
DAP wrote:
PS. View Boost Status not working.
This requires a patch to plugin arch. In the meantime, you should be able to find it on the utilities page on the very left bottom side
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 25, 2007 10:08 am 
Offline

Joined: Fri May 19, 2006 1:17 pm
Posts: 16
Location: Lake St Louis, MO USA
Boost is my new best friend! :) I had been struggling to tweak cacti to handle a huge amount of data sources. Thanks devs!

This system is monitoring nearly 70 Cisco 65xx routers, with a handful of other devices thrown in. This makes for a LOT of data sources.

SYSTEM: BOOST STATS: Time:985.7940 RRDUpdates:988752

SYSTEM: STATS: Time:59.0062 Method:cactid Processes:1 Threads:32 Hosts:74 HostsPerProcess:74 DataSources:75951 RRDsProcessed:0


Total Data Sources: 75951
Total Boost Records: 758095
Current Boost Table Size: 43 MBytes
Avg Bytes/Record: 45 Bytes
Max Allowed Boost Table Size: 4 GBytes
Estimated Maximum Records: 95443718 Records

_________________
William Riley
http://workbench.freetcp.com/


Last edited by wriley on Mon Feb 12, 2007 1:38 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 25, 2007 1:55 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22461
Location: Muenster, Germany
You may want to use my new Poller Stats, featuring Boost Statistics (runtime, amount of BOOST) http://forums.cacti.net/viewtopic.php?t=18057
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 26, 2007 9:36 am 
Offline

Joined: Fri May 19, 2006 1:17 pm
Posts: 16
Location: Lake St Louis, MO USA
gandalf wrote:
You may want to use my new Poller Stats, featuring Boost Statistics (runtime, amount of BOOST) http://forums.cacti.net/viewtopic.php?t=18057
Reinhard


I was already using your templates for poller statistics and subsequently turned on the boost stats as well. Thanks for the templates!


These graphs show the improvement when Boost was activated (at about two-thirds point). I also added the final 20 hosts, hence the jump in data sources at the same time.


Attachments:
File comment: Boost Statistics
boost_statistics.png
boost_statistics.png [ 20.13 KiB | Viewed 28439 times ]
File comment: Poller Runtime Statistics
poller_runtime.png
poller_runtime.png [ 41.59 KiB | Viewed 28439 times ]
File comment: Poller Statistics
poller_stats.png
poller_stats.png [ 27.45 KiB | Viewed 28439 times ]

_________________
William Riley
http://workbench.freetcp.com/
Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 26, 2007 2:04 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22461
Location: Muenster, Germany
Do not worry, that the green line in the last image stops. This is now reported in the boost stats
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 26, 2007 8:49 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
That is one BIG system. Glad to see it come back down to earth. I am sure the web site behaves better now too. I hope you enjoy this plugin I wrote for you. Not a lot of host's, but Lot's of Data Sources...

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 13, 2007 2:50 am 
Offline
Cacti User

Joined: Sun Aug 28, 2005 12:51 pm
Posts: 137
Depends my configuration i test cactid polling vs. booster.
On both settings, timers are same, about 366 sec. (~6 min.)

Polling interval 10 min.

Below are some from my system graphs (cpu, load average).

Current Booster settings:

Current Boost Status
Boost On Demand Updating: Idle
Total Data Sources: 38379
Total Boost Records: 22966

Table Size Estimates
Current Boost Table Size: 16 MBytes
Avg Bytes/Record: 127 Bytes
Max Allowed Boost Table Size: 4 GBytes
Estimated Maximum Records: 33818640 Records

Previous Runtime
Last Start Time: 2007-03-13 8:36:07
Last Run Duration: 319.6620 Seconds
RRD's Updated: 28646

Run Time Configuration
Update Frequency: 20 Minutes
Next Start Time: 2007-03-13 8:56:07
Maximum Records: 1000000 Records
Maximum Allowed Runtime: 20 Minutes

Boost Server Details
Server Config Status: Enabled
Multiprocess Server: Single Process
Update Timeout: 2 Seconds
Server/Port: localhost@9050
Authorized Update Web Servers: 127.0.0.1
RRDtool Binary Used: /usr/bin/rrdtool

Seems, that Booster eat more system than cactid poller.
Its normal?

Best Regards,
Alex.


Attachments:
File comment: Monthly, from Week 10 system works on booster.
load03.PNG
load03.PNG [ 46.5 KiB | Viewed 27994 times ]
File comment: Daily, before 12h works on booster.
load01.PNG
load01.PNG [ 34.35 KiB | Viewed 27994 times ]
File comment: Weekly.
cpu02.PNG
cpu02.PNG [ 41.03 KiB | Viewed 27994 times ]
File comment: Daily, before a gap works on booster.
cpu01.PNG
cpu01.PNG [ 26.43 KiB | Viewed 27994 times ]
Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 13, 2007 11:18 am 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22461
Location: Muenster, Germany
From my point of view, boost is not a replacement for cactid. IMHO, the sequence would be

1) Use cmd.php as a start
If your systems grows, you may hit the wall unless ...
2) Use cactid as a fast (by order of a magnitude) replacement next
If your system still grows (we are speaking of at least 20,000 to 30,000 to 40,000 and more data sources) this will not be fast enough. rrdtool update consumes most of the time. But rrdtool features sth. like "multi-row insert". Boost will use this. If you hit the wall again ...
3) use cactid in combination with boost
This is now the ultimate step.
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 13, 2007 3:31 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Concurring with Gandolf.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 14, 2007 3:10 am 
Offline
Cacti User

Joined: Sun Aug 28, 2005 12:51 pm
Posts: 137
Yes Gandalf you have right, as you see now i have about 40k Data Sources on 10k Hosts, and this number will grows..
Up i post my stats cacti+cactid working on normal polling vs. cacti+cactid+boost..

Graphs shows that in the same poll time (normal and booster check hosts in ~6 min) system utilization (CPU, Memory, HDD) is higher when work boost. I still search a 'gold center' to minimize a polling time (to 5 min.) && system use.

I will post there my test results, if you don't have nothing against that.
Thank you for advice..

Best Regards,
Alex.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 12, 2007 6:43 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Alex,

I would move your update frequency to more like every 1-3 hours.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 18, 2007 2:22 pm 
Offline

Joined: Wed Feb 02, 2005 5:26 pm
Posts: 22
Location: Zurich, Switzerland
This plugin is just amazing, finally I'm able to write all my data into the rra's without getting gaps in the graphs.

cacti: 0.8.6j
cactid: 0.8.6i

BOOST Stats:
***********
Code:
04/18/2007 06:49:12 PM - SYSTEM BOOST STATS: Time:380.0422 RRDUpdates:548371
04/18/2007 07:49:09 PM - SYSTEM BOOST STATS: Time:377.6698 RRDUpdates:543419


POLLER Stats:
***********
Code:
04/18/2007 08:32:53 PM - SYSTEM STATS: Time:51.5142 Method:cactid Processes:1 Threads:15 Hosts:251 HostsPerProcess:251 DataSources:50366 RRDsProcessed:0
04/18/2007 08:37:54 PM - SYSTEM STATS: Time:52.6713 Method:cactid Processes:1 Threads:15 Hosts:251 HostsPerProcess:251 DataSources:50366 RRDsProcessed:0
04/18/2007 08:42:51 PM - SYSTEM STATS: Time:49.4776 Method:cactid Processes:1 Threads:15 Hosts:251 HostsPerProcess:251 DataSources:50366 RRDsProcessed:0


Thanks a lot 'TheWitness', this definitly r0cks!!

cheers
:lol:


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 23, 2007 3:15 am 
Offline

Joined: Wed Feb 02, 2005 5:26 pm
Posts: 22
Location: Zurich, Switzerland
I would like to show you the impact of boost plugin on my disk i/o waittime. I also attachted a graph that show my loadavg. The first part of the graph shows statistics with the boost plugin. As soon as I switched back to the regular cactid poller method without the boost plugin, the red area (i/o wait) shows up...

cheers :roll:


Attachments:
File comment: another shot of the loadaverage
cacti_boost_plugin_loadavg.png
cacti_boost_plugin_loadavg.png [ 58.67 KiB | Viewed 27198 times ]
File comment: iowait (red)
cacti_boost_plugin_iowait.png
cacti_boost_plugin_iowait.png [ 38.61 KiB | Viewed 27198 times ]
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 53 posts ]  Go to page 1, 2, 3, 4  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group