[INFO] Speed Boost for CMD.PHP / Increase MySQL Performance

If you figure out how to do something interesting/cool in Cacti and want to share it with the community, please post your experience here.

Moderators: Moderators, Developers

Post Reply
Author
Message
User avatar
TheWitness
Developer
Posts: 14804
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

[INFO] Speed Boost for CMD.PHP / Increase MySQL Performance

#1 Post by TheWitness » Thu Sep 16, 2004 4:26 am

Speed Boost in CMD.PHP and MySQL:

Based upon two user suggestions, the following configuration changes should enhance Cacti performance.

First, if you are using CMD.PHP:

Remove your MIBS environment variable, if it is on the system. According to the user, it spead up polling significantly.

Second, if you are using MySQL 4.x, you should change your my.cnf or my.ini to include the following directives. You will need to restart MySQL to implement. This will reduce MySQL load drastically.

set-variable=key_buffer=32M
query_cache_size=64M

Also, MySQL Administrator is a great realtime tool for viewing load while polling and viewing cache and key buffer hit rates.

TheWitness
Last edited by TheWitness on Thu Nov 25, 2004 9:12 am, edited 1 time in total.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

JoeNetwork
Posts: 38
Joined: Mon Feb 16, 2004 2:01 pm
Location: Westland, MI

#2 Post by JoeNetwork » Fri Sep 17, 2004 12:34 am

What about if we have MySQL 3.x? :)

Is it worth upgrading to 4.x, or is there a different variable we can tweak?

User avatar
TheWitness
Developer
Posts: 14804
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#3 Post by TheWitness » Fri Sep 17, 2004 7:02 am

Not applicable to 3.x. Sorry. I suggest a meaningfull and well thought out upgrade. It significantly reduced my Disk IO and CPU (120 Devices, ~2000 Data Sources).

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

karlkatzke
Posts: 10
Joined: Sun Apr 03, 2005 4:29 pm
Location: Tigard, OR
Contact:

#4 Post by karlkatzke » Sun Apr 03, 2005 5:03 pm

I'd beware of setting the values that high if you're running on a machine with limited memory. For instance, the box I have cacti running on only has 256MB of Ram... using that much of it just for queryspace on MySQL might hurt it. Only way to tell is to do it, though... and report your results back here. ;)

gandolfi
Posts: 2
Joined: Thu Jul 28, 2005 5:41 pm

Just Newbie...

#5 Post by gandolfi » Thu Jul 28, 2005 5:55 pm

TheWitness wrote:Not applicable to 3.x. Sorry. I suggest a meaningfull and well thought out upgrade. It significantly reduced my Disk IO and CPU (120 Devices, ~2000 Data Sources).

TheWitness
I'm starting to use cacti, and I was planning to install it in a win2k server, but I have 1187 Devices (most of them routers) but but no more than 3000 data sources...

I have php5 and mysql 4. What CPU should I use to get it working fine (and how many RAM) ? (It's a P4 2.0 with 512 enough?)

I was thinking that a linux box maybe better...

Another question... Should I put the poller to run every 10 minutes instead of 5 ? (just to give it time to finish)

Thanks in advance...

Ariel

User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

#6 Post by gandalf » Sun Jul 31, 2005 6:23 am

Ariel,

have a look at the metrics posted at http://forums.cacti.net/viewtopic.php?t=6206. Your timing will depend on the data gathering method, e.g. better if you use SNMP, slower if you use lots of scripts. Use cactid as a data gatherer.

happy "cacti"ing
Reinhard

dgerow
Posts: 6
Joined: Mon Aug 07, 2006 5:58 am

#7 Post by dgerow » Thu Aug 10, 2006 5:30 am

(Hopefully people don't mind me dredging up an old thread.)

I've been playing around with our cacti installation, using cmd.php, as cactid segfaults as soon as it reads the maximum SNMP OIDs per request, and I've not had the time to figure out why yet. I just re-compiled PHP to include php-snmp, instead of snmpget. Our polling time dropped from well over two minutes to less than thirty seconds, and the system load dropped from ~10 to >0.5 while polling.

Additionally, I'm finding that a /smaller/ number of concurrent poller processes is actually better. When using 50 processes, the total runtime was reliably around 47 seconds. Now that I've dropped it to 10 processes, we're looking at a runtime of 10 seconds. This is with 80 hosts and 1800 data sources.

Combining these two, the system load has dropped to >0.1, and CPU utilization has dropped from 100% to a maximum spike of ~25%, all while doing polling. This is on a quad 3.6GHz Xeon (yeah, a touch overkill) with 4GB RAM, running MySQL 4.1.19.

Suicidal
Posts: 6
Joined: Mon Jan 23, 2006 3:21 am
Contact:

#8 Post by Suicidal » Thu Nov 02, 2006 4:29 pm

I found the absolute best improvement is to use the devices actual ip address vs. fqdn:


11/02/2006 12:34:22 PM - SYSTEM STATS: Time:59.8656 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:153
11/02/2006 12:35:30 PM - SYSTEM STATS: Time:60.0798 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:113
11/02/2006 12:37:25 PM - SYSTEM STATS: Time:62.7944 Method:cactid Processes:2 Threads:4 Hosts:6 HostsPerProcess:3 DataSources:238 RRDsProcessed:113
11/02/2006 12:38:59 PM - SYSTEM STATS: Time:56.7890 Method:cactid Processes:8 Threads:4 Hosts:6 HostsPerProcess:1 DataSources:238 RRDsProcessed:113
11/02/2006 12:40:14 PM - SYSTEM STATS: Time:62.0994 Method:cactid Processes:1 Threads:4 Hosts:6 HostsPerProcess:6 DataSources:238 RRDsProcessed:113
11/02/2006 12:42:01 PM - SYSTEM STATS: Time:55.6983 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:113
11/02/2006 12:43:15 PM - SYSTEM STATS: Time:54.7663 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:113
11/02/2006 12:45:49 PM - SYSTEM STATS: Time:55.7001 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:90
<---- changed to ip addressing here ---->
11/02/2006 12:48:03 PM - SYSTEM STATS: Time:6.2655 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:113
11/02/2006 12:53:18 PM - SYSTEM STATS: Time:7.2820 Method:cactid Processes:4 Threads:8 Hosts:6 HostsPerProcess:2 DataSources:238 RRDsProcessed:113
11/02/2006 12:54:32 PM - SYSTEM STATS: Time:3.2186 Method:cactid Processes:2 Threads:4 Hosts:6 HostsPerProcess:3 DataSources:238 RRDsProcessed:113
11/02/2006 12:54:45 PM - SYSTEM STATS: Time:2.1760 Method:cactid Processes:1 Threads:4 Hosts:6 HostsPerProcess:6 DataSources:238 RRDsProcessed:113
11/02/2006 12:55:02 PM - SYSTEM STATS: Time:2.2006 Method:cactid Processes:1 Threads:6 Hosts:6 HostsPerProcess:6 DataSources:238 RRDsProcessed:113
[color=green][size=84]Cacti-0.8.6i|Gentoo-2006.1|kernel-x86_64-2.6.17-hardened-r1|mysql-5.0.26|rrdtool-1.2.6|glibc-2.3.6-r4|gcc-3.4.6|net-snmp-5.2.1.2|php-5.1.6|apache-2.0.58[/size][/color]

User avatar
BSOD2600
Cacti Moderator
Posts: 12160
Joined: Sat May 08, 2004 12:44 pm
Location: USA

#9 Post by BSOD2600 » Thu Nov 02, 2006 5:48 pm

Suicidal wrote:I found the absolute best improvement is to use the devices actual ip address vs. fqdn:
Wow. You sure thats just not a DNS problem on your side?

User avatar
SyxPak
Posts: 9
Joined: Fri Mar 24, 2006 7:56 am
Location: Ireland
Contact:

#10 Post by SyxPak » Thu Nov 23, 2006 5:26 am

Maybe try using a local dns caching proxy perhaps, or nscd?

That said, I use IP only too. Our hostnames here are too long to type ;)

_Dmitry_
Posts: 36
Joined: Tue Sep 18, 2007 4:58 am
Location: DE

#11 Post by _Dmitry_ » Wed Oct 17, 2007 8:48 am

Small Tip to increase your CACTI Performance.

If your Poller Statistic looks like this one in attachment (always slowly increasing, without any new Data sources). Check your poller_output table !!!

And if there are any old values then delete from poller_output :) And after that fix Data source which is spamming this table.

Gives a lot of Performance boost :D
Attachments
cacti_pooler.png
cacti_pooler.png (50.45 KiB) Viewed 44577 times

User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

#12 Post by gandalf » Thu Oct 18, 2007 2:39 pm

Please increase php memory settings in this case as given in the second link of my signature
Reinhard

_Dmitry_
Posts: 36
Joined: Tue Sep 18, 2007 4:58 am
Location: DE

#13 Post by _Dmitry_ » Fri Oct 19, 2007 1:56 am

gandalf wrote:Please increase php memory settings in this case as given in the second link of my signature
Reinhard
That's was not the problem. I have memory_limit = 128M.
That was buggy Script, and after few week I've had in this table ~50000-80000 rows.

Post Reply