All,

I have released the Boost Plugin, version 1.0. Once you have something good to say about boost, or bad for that matter, please post your before and after statistics as followups to this post.

Here are the main features:
On-Demand RRD Updating You don't have to update 40k RRD files every 5 minutes, you can wait hours before doing updates. If a user accesses a Graph before a major update, the "On-Demand" updater takes care of it.
Network Service to handle RRD Updating This service is handy when your Web Server does not have RW access to the RRA folder. This provides an added benefit of allowing for multiple Web Servers to be deployed within this framework to host Cacti.
PNG Caching Yes, that's right. This plugin will cache PNG's for as long as they are viable. Once they expire, the first person to view the graph uses the overhead of creating the next good image.

Regards,

TheWitness

http://forums.cacti.net/viewtopic.php?p=86053#86053

Seems to be working fine.

12/15/2006 05:00:23 PM - SYSTEM BOOST STATS: Time:4.2656 RRDUpdates:1022

Love the cache feature

Cheers

PS. View Boost Status not working.

This requires a patch to plugin arch. In the meantime, you should be able to find it on the utilities page on the very left bottom side
Reinhard

Boost is my new best friend! I had been struggling to tweak cacti to handle a huge amount of data sources. Thanks devs!

This system is monitoring nearly 70 Cisco 65xx routers, with a handful of other devices thrown in. This makes for a LOT of data sources.

SYSTEM: BOOST STATS: Time:985.7940 RRDUpdates:988752

SYSTEM: STATS: Time:59.0062 Method:cactid Processes:1 Threads:32 Hosts:74 HostsPerProcess:74 DataSources:75951 RRDsProcessed:0


Total Data Sources: 75951
Total Boost Records: 758095
Current Boost Table Size: 43 MBytes
Avg Bytes/Record: 45 Bytes
Max Allowed Boost Table Size: 4 GBytes
Estimated Maximum Records: 95443718 Records

You may want to use my new Poller Stats, featuring Boost Statistics (runtime, amount of BOOST) http://forums.cacti.net/viewtopic.php?t=18057
Reinhard

I was already using your templates for poller statistics and subsequently turned on the boost stats as well. Thanks for the templates!


These graphs show the improvement when Boost was activated (at about two-thirds point). I also added the final 20 hosts, hence the jump in data sources at the same time.

Do not worry, that the green line in the last image stops. This is now reported in the boost stats
Reinhard

That is one BIG system. Glad to see it come back down to earth. I am sure the web site behaves better now too. I hope you enjoy this plugin I wrote for you. Not a lot of host's, but Lot's of Data Sources...

TheWitness

Depends my configuration i test cactid polling vs. booster.
On both settings, timers are same, about 366 sec. (~6 min.)

Polling interval 10 min.

Below are some from my system graphs (cpu, load average).

Current Booster settings:

Current Boost Status
Boost On Demand Updating: Idle
Total Data Sources: 38379
Total Boost Records: 22966

Table Size Estimates
Current Boost Table Size: 16 MBytes
Avg Bytes/Record: 127 Bytes
Max Allowed Boost Table Size: 4 GBytes
Estimated Maximum Records: 33818640 Records

Previous Runtime
Last Start Time: 2007-03-13 8:36:07
Last Run Duration: 319.6620 Seconds
RRD's Updated: 28646

Run Time Configuration
Update Frequency: 20 Minutes
Next Start Time: 2007-03-13 8:56:07
Maximum Records: 1000000 Records
Maximum Allowed Runtime: 20 Minutes

Boost Server Details
Server Config Status: Enabled
Multiprocess Server: Single Process
Update Timeout: 2 Seconds
Server/Port: localhost@9050
Authorized Update Web Servers: 127.0.0.1
RRDtool Binary Used: /usr/bin/rrdtool

Seems, that Booster eat more system than cactid poller.
Its normal?

Best Regards,
Alex.

From my point of view, boost is not a replacement for cactid. IMHO, the sequence would be

1) Use cmd.php as a start
If your systems grows, you may hit the wall unless ...
2) Use cactid as a fast (by order of a magnitude) replacement next
If your system still grows (we are speaking of at least 20,000 to 30,000 to 40,000 and more data sources) this will not be fast enough. rrdtool update consumes most of the time. But rrdtool features sth. like "multi-row insert". Boost will use this. If you hit the wall again ...
3) use cactid in combination with boost
This is now the ultimate step.
Reinhard

Concurring with Gandolf.

TheWitness

Yes Gandalf you have right, as you see now i have about 40k Data Sources on 10k Hosts, and this number will grows..
Up i post my stats cacti+cactid working on normal polling vs. cacti+cactid+boost..

Graphs shows that in the same poll time (normal and booster check hosts in ~6 min) system utilization (CPU, Memory, HDD) is higher when work boost. I still search a 'gold center' to minimize a polling time (to 5 min.) && system use.

I will post there my test results, if you don't have nothing against that.
Thank you for advice..

Best Regards,
Alex.

Alex,

I would move your update frequency to more like every 1-3 hours.

TheWitness

This plugin is just amazing, finally I'm able to write all my data into the rra's without getting gaps in the graphs.

cacti: 0.8.6j
cactid: 0.8.6i

BOOST Stats:
***********


POLLER Stats:
***********


Thanks a lot 'TheWitness', this definitly r0cks!!

cheers

I would like to show you the impact of boost plugin on my disk i/o waittime. I also attachted a graph that show my loadavg. The first part of the graph shows statistics with the boost plugin. As soon as I switched back to the regular cactid poller method without the boost plugin, the red area (i/o wait) shows up...

cheers