Submit Your CMD.PHP vs. CACTID Metrics Here

[gandalf]

Per theory, it should be set to 60 if all targets are working fine with that setting. It surely will reduce network load at minimum. If it also will reduce polling time will be clear if you add more devices.
Reinhard

[ben_c]

New install of cacti/cactid

10/15/2007 02:10:32 PM - SYSTEM STATS: Time:360.5152 Method:cactid Processes:2 Threads:5 Hosts:170 HostsPerProcess:85 DataSources:30912 RRDsProcessed:15748

Not getting the performance I had hoped, and I still haven't migrated all hosts over to it

IBM x346 Xeon 3.4Ghz
4Gb RAM
1 x raided 10k SCSI HDD's
Redhat AS 4

[gandalf]

IS this a dual CPU system? Which version of rrdtool used? We have about 7ß% of your numbers ds/rrd and require 40 sec polling time on a dual XEON. Please increase threads to 12-15. Most of the ds are SNMP?
Reinhard

[ben_c]

Hi Reinhard,

Yeah dual Xeon.

I've had some issues with the latest MySQL that was placing a lot of load on the system, i've done a kernel upgrade and all seems well.

[hylaride]

10/17/2007 01:26:35 PM - SYSTEM STATS: Time:34.7723 Method:cactid Processes:4 Threads:14 Hosts:229 HostsPerProcess:58 DataSources:4604 RRDsProcessed:2901

This is from 4-5 minutes from poller.php


Attached is the image as it went to cactid (the huge spike at the beginning). Since cacti is running of this machine, it's always running a load when checking itself I had to explain that to our nitwits in the NOC that the other half of the time the computer is idle...

[antifaction]

Didn't get much better. I'm lookign through my config now to check settings.

12/11/2007 01:10:16 PM - SYSTEM STATS: Time:14.1418 Method:spine Processes:16 Threads:4 Hosts:52 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1279
12/11/2007 01:05:14 PM - SYSTEM STATS: Time:13.0296 Method:spine Processes:16 Threads:4 Hosts:52 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1279
12/11/2007 01:00:14 PM - SYSTEM STATS: Time:13.2933 Method:spine Processes:16 Threads:4 Hosts:52 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1279
12/11/2007 12:50:15 PM - SYSTEM STATS: Time:14.1304 Method:spine Processes:16 Threads:1 Hosts:52 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1279


12/11/2007 12:40:16 PM - SYSTEM STATS: Time:15.3111 Method:cmd.php Processes:16 Threads:N/A Hosts:55 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1270
12/11/2007 12:35:17 PM - SYSTEM STATS: Time:16.4388 Method:cmd.php Processes:16 Threads:N/A Hosts:55 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1270
12/11/2007 12:30:17 PM - SYSTEM STATS: Time:15.7749 Method:cmd.php Processes:16 Threads:N/A Hosts:55 HostsPerProcess:4 DataSources:1950 RRDsProcessed:1270

[briang]

I'm currently seeing exceptional performance for a large windows based cacti install using cacti 0.8.7a/spine 0.8.7b/boost. Just wanted to post my configuration to hopefully save others the time and effort that it took me to get this right....

Current stats:

12/17/2007 09:44:23 AM - SYSTEM STATS: Time:23.6648 Method:spine Processes:1 Threads:8 Hosts:265 HostsPerProcess:265 DataSources:13930 RRDsProcessed:0

12/17/2007 09:44:42 AM - SYSTEM BOOST STATS: Time:282.2263 RRDUpdates:273150

My poller is running at a 1 minute interval with an RRA that keeps 30 days of granular data. This results in RRD files that are fairly large ~35 MB. Part of my initial performance problems were due to the amount of time needed to actually update the RRDs - about 80% of the polling cycle. All of my data is snmp (no scripts). The boost plugin addresssed the length of the update cycle.

My hardware specs:
CPU 2 x dual core - Xeon @ 2.33ghz; 4 GB RAM OS is w2k03

I'm using the latest version of boost from the svn as I was seeing a known issue with "Exceeded Run time limits" errors in my cacti.log using 1.7.

Here's the config that worked for me:
I have two scheduled tasks - standard one for the poller.php (5 min interval) and one for the boost (boost_poller.php -f -d) at a 20 min interval.

My poller settings:
Poller Type - spine
Poller Interval - Every Minute
Cron Interval - Every 5 minutes
Maximum Concurrent Poller Processes - 1
Maximum Threads Per Process - 8
Number of PHP Script Servers - 10
Script Server Timeout Value - 30
Maximum SNMP OIDs per SNMP get request - 60
Downed Host Detection - SNMP
(Important that Downed Host Detection is SNMP when definining an individual device as well.)

My.ini customizations (needed to allocate sufficient memory for boost.)
I used the mysql-huge.ini from the standard distribution with the following additional updates:
max_connections=250
max_user_connections=230
wait_timeout=600
max_heap_table_size=1500000000
tmp_table_size=1500000000
key_buffer_size=12000000

I also allocated more memory to PHP.
memory_limit = 384M

boost settings:
Enabled on Demand RRD Updating
How Often Should Boost Update All RRDs: 6 Hours
Maximum Records: 1000000
Maximum Records Per Pass: 100,000
Maximum MySQL insert string: 100000
Update String Length: 8100
Maximum RRD Update Script Run Time: 1 Hour

[koolaidrocks]

Here are the stats for my Cacti test box:

Cacti 0.8.7a and Plugin Arch 1.4 w/ Monitor, MacTrack, Discovery, Weathermap and Thold

All of this runs on an old Compaq Deskpro EN sporting a 1GHz Pentium III w/ 512MB of RAM running Fedora Core 7. Needless to say, I was quite surprised with the results:

12/31/2007 10:37:18 AM - SYSTEM STATS: Time:16.3401 Method:spine Processes:1 Threads:5 Hosts:101 HostsPerProcess:101 DataSources:5352 RRDsProcessed:2823

Currently, only our switch infrastructure is added into Cacti. But the wireless and WAN devices will be added in soon enough.

[keefe007]

PHP 5.2.5
Cacti 0.8.7a

AMD Athlon 2800+
2 GB Ram


01/04/2008 12:22:14 AM - RECACHE STATS: RecacheTime:10.2970 HostsRecached:1
01/04/2008 12:22:03 AM - SYSTEM STATS: Time:2.3959 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:58 RRDsProcessed:31
01/04/2008 12:21:50 AM - SYSTEM STATS: Time:48.0235 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:638 RRDsProcessed:383
01/04/2008 12:20:05 AM - SYSTEM STATS: Time:4.3200 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:41 RRDsProcessed:20
01/04/2008 12:19:04 AM - SYSTEM STATS: Time:2.3350 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:41 RRDsProcessed:20
01/04/2008 12:18:07 AM - SYSTEM STATS: Time:6.3048 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:41 RRDsProcessed:20
01/04/2008 12:17:14 AM - RECACHE STATS: RecacheTime:10.0997 HostsRecached:1
01/04/2008 12:17:04 AM - SYSTEM STATS: Time:2.3558 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:58 RRDsProcessed:31
01/04/2008 12:16:48 AM - SYSTEM STATS: Time:47.2248 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:638 RRDsProcessed:383
01/04/2008 12:15:04 AM - SYSTEM STATS: Time:2.3276 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:41 RRDsProcessed:20
01/04/2008 12:11:04 AM - SYSTEM STATS: Time:2.4391 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:41 RRDsProcessed:20
01/04/2008 12:10:08 AM - SYSTEM STATS: Time:7.7261 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:41 RRDsProcessed:20
01/04/2008 12:09:03 AM - SYSTEM STATS: Time:2.2893 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:58 RRDsProcessed:31
01/04/2008 12:08:50 AM - SYSTEM STATS: Time:48.9131 Method:spine Processes:2 Threads:15 Hosts:42 HostsPerProcess:21 DataSources:638 RRDsProcessed:383

How are these stats?

[koolaidrocks]

[gandalf]

I won't bother with that few data sources. Most of the runtime will be dedicated to startup issues, I suppose
Reinhard

[antifaction]

02/11/2008 11:15:16 AM - SYSTEM STATS: Time:14.6410 Method:spine Processes:6 Threads:7 Hosts:154 HostsPerProcess:26 DataSources:5030 RRDsProcessed:3024
02/11/2008 11:10:14 AM - SYSTEM STATS: Time:13.6882 Method:spine Processes:6 Threads:7 Hosts:154 HostsPerProcess:26 DataSources:5030 RRDsProcessed:3024
02/11/2008 11:05:14 AM - SYSTEM STATS: Time:12.8746 Method:spine Processes:6 Threads:7 Hosts:154 HostsPerProcess:26 DataSources:5030 RRDsProcessed:3024
02/11/2008 11:00:16 AM - SYSTEM STATS: Time:14.6267 Method:spine Processes:6 Threads:7 Hosts:154 HostsPerProcess:26 DataSources:5030 RRDsProcessed:3024

Linux
0.8.7a
1GB memory

[bsdboy]

I started pushing the envelope a bit and my cacti box let me know today that it was time to look at something other then cmd.php. Seems to run much better, thanks!

Specs:
Cent/OS 4.4 on a Sun v60
dual-core hyper-threaded 2.8Ghz Xeon (4 cores total)
2G RAM

[gandalf]

I'd recommend to lower the proccesses a bit an double the threads instead. More memory would be fine (expect 30% reduction of the current runtime). Latest rrdtool (1.2.26 or later) along with a current 2.6.x kernel should reduce by another 30% or the like.
Reinhard

[bsdboy]

Went ahead and did some upgrades, still the same hardware but using 0.8.7b with spine 0.8.7a.