Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Mon Mar 18, 2019 1:00 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 354 posts ]  Go to page Previous  1 ... 15, 16, 17, 18, 19, 20, 21 ... 24  Next
Author Message
 Post subject:
PostPosted: Tue Feb 03, 2009 5:26 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Please pay special attention to scripts with undefined timeouts. This happened to me lately
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu Feb 05, 2009 3:43 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
jbarbieri wrote:
cmd.php
Edit-----

Spine is having hiccups too :(


Update to 0.8.7c

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Feb 05, 2009 3:45 pm 
Offline
Cacti User

Joined: Tue May 24, 2005 8:46 am
Posts: 101
Location: Rochester, New Hampshire
TheWitness wrote:
jbarbieri wrote:
cmd.php
Edit-----

Spine is having hiccups too :(


Update to 0.8.7c

TheWitness


It's on 0.8.7c


Had a helluva time getting spine to compile on solaris.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 04, 2009 12:05 pm 
Offline

Joined: Mon Mar 10, 2008 5:49 am
Posts: 20
03/04/2009 05:57:11 PM - SYSTEM STATS: Time:9.8844 Method:spine Processes:2 Threads:40 Hosts:145 HostsPerProcess:73 DataSources:31187 RRDsProcessed:10786
03/04/2009 05:56:12 PM - SYSTEM STATS: Time:10.0587 Method:spine Processes:2 Threads:40 Hosts:145 HostsPerProcess:73 DataSources:31187 RRDsProcessed:10786
03/04/2009 05:55:14 PM - SYSTEM STATS: Time:9.3282 Method:spine Processes:2 Threads:40 Hosts:145 HostsPerProcess:73 DataSources:31187 RRDsProcessed:10786
03/04/2009 05:54:13 PM - SYSTEM STATS: Time:10.9903 Method:spine Processes:2 Threads:40 Hosts:145 HostsPerProcess:73 DataSources:31187 RRDsProcessed:10786
03/04/2009 05:53:10 PM - SYSTEM STATS: Time:9.1489 Method:spine Processes:2 Threads:40 Hosts:145 HostsPerProcess:73 DataSources:31187 RRDsProcessed:10786




only 1/3 of hosts have been added at the moment, going slow to make sure nothing goes wrong


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 04, 2009 1:40 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
At some point you will need boost.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 12:24 am 
Offline

Joined: Tue Sep 30, 2008 11:38 pm
Posts: 41
Location: Sofia, Bulgaria
Code:
03/05/2009 07:04:22 AM - SYSTEM STATS: Time:20.3342 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58904 RRDsProcessed:0
03/05/2009 07:03:21 AM - SYSTEM STATS: Time:20.2148 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58903 RRDsProcessed:0
03/05/2009 07:02:22 AM - SYSTEM STATS: Time:20.5017 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58903 RRDsProcessed:0
03/05/2009 07:01:21 AM - SYSTEM STATS: Time:20.3234 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58903 RRDsProcessed:0
03/05/2009 07:00:22 AM - SYSTEM STATS: Time:20.3463 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58903 RRDsProcessed:0
03/05/2009 06:59:21 AM - SYSTEM STATS: Time:20.3226 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58904 RRDsProcessed:0
03/05/2009 06:58:22 AM - SYSTEM STATS: Time:20.2752 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58903 RRDsProcessed:0
03/05/2009 06:57:21 AM - SYSTEM STATS: Time:20.1203 Method:spine Processes:4 Threads:10 Hosts:1494 HostsPerProcess:374 DataSources:58903 RRDsProcessed:0


Code:
03/05/2009 06:54:48 AM - SYSTEM BOOST STATS: Time:261.9606 RRDUpdates:3583637
03/05/2009 05:53:30 AM - SYSTEM BOOST STATS: Time:248.3017 RRDUpdates:3584929
03/05/2009 04:52:27 AM - SYSTEM BOOST STATS: Time:244.7419 RRDUpdates:3525397
03/05/2009 03:52:27 AM - SYSTEM BOOST STATS: Time:245.2317 RRDUpdates:3524961
03/05/2009 02:52:40 AM - SYSTEM BOOST STATS: Time:258.7262 RRDUpdates:3584674


CPU: Core2Duo E8600 @3.3Ghz (this is the bottleneck sometimes)
RAM: 8G
HDD: 2xWDC WD1500HLFS @10k RPM in RAID
Supermicro motherboard

Running Debian 5.0-lenny with kernel 2.6.26-1-686-bigmem
RRDTool 1.2.28.

Do you think switching to RRDTool 1.3.x will improve something?

_________________
"Ubuntu" - an African word, meaning "Slackware is too hard for me".


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 4:48 am 
Offline
Cacti User

Joined: Tue May 24, 2005 8:46 am
Posts: 101
Location: Rochester, New Hampshire
I am beginning to think my slowdown is because my RRAs are on an NFS mount :/


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 7:42 am 
Offline

Joined: Tue Sep 30, 2008 11:38 pm
Posts: 41
Location: Sofia, Bulgaria
Storage speed is crucial for cacti performance, unless you use the boost plugin.
Do "iostat 1" on the NFS host and see how many TPS it's doing when cacti poller runs. If it's something below 300 this may be your problem.
Usually PATA/SATA disks at 7200RPM make about 300-400 transactions per second.
My 10k RPM Velociraptors top out at about 1000-1300 TPS each, which gives an average of 2000-2100 TPS when working in RAID, and still this isn't enough for cacti when writing 50k+ DSs.
Also look at your iowaits when cacti polls. If you have large percent of iowaits during polls this means your storage is slowing the poller process.

_________________
"Ubuntu" - an African word, meaning "Slackware is too hard for me".


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 8:16 am 
Offline
Cacti User

Joined: Tue May 24, 2005 8:46 am
Posts: 101
Location: Rochester, New Hampshire
iostat 1 doesn't work for NFS mounts.


had to do iostat -xn 5 to get it to display:


Code:
                    extended device statistics
    r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
    0.0    6.2    0.0  118.5  0.0  0.0    0.3    0.6   0   0 172.17.4.6:/vol/mrtg01/mrtg01
  131.3  160.5  630.1 1372.4  0.0  0.7    0.0    2.3   0  62 172.17.4.6:/vol/mrtg01/mrtg01
  194.4  198.8  973.6 1625.0  0.0  1.0    0.0    2.7   0  95 172.17.4.6:/vol/mrtg01/mrtg01
   51.7   79.6  446.1  808.4  0.0  0.3    0.0    2.5   0  27 172.17.4.6:/vol/mrtg01/mrtg01
  188.6  145.8  937.5  728.3  0.0  1.0    0.0    3.0   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  188.0  135.2 1009.6  648.1  0.0  1.0    0.0    3.1   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  206.0  150.0  990.5  708.3  0.0  0.9    0.0    2.6   0  87 172.17.4.6:/vol/mrtg01/mrtg01
  181.8  138.4  923.2  707.2  0.0  1.0    0.0    3.1   0  94 172.17.4.6:/vol/mrtg01/mrtg01
  231.2  173.4 1130.5  789.3  0.0  1.0    0.0    2.5   0  95 172.17.4.6:/vol/mrtg01/mrtg01
  208.4  152.6 1012.8  711.0  0.0  1.0    0.0    2.8   0  95 172.17.4.6:/vol/mrtg01/mrtg01
  253.0  205.0 1307.3  946.4  0.0  1.0    0.0    2.2   0  94 172.17.4.6:/vol/mrtg01/mrtg01
  181.4  132.2  899.2  628.4  0.0  1.0    0.0    3.2   0  95 172.17.4.6:/vol/mrtg01/mrtg01
  146.8  165.8  753.7 1359.5  0.0  1.0    0.0    3.1   0  90 172.17.4.6:/vol/mrtg01/mrtg01
  186.4  160.0  958.4 1079.5  0.0  1.0    0.0    2.9   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  146.8  105.2  787.3  501.8  0.0  1.0    0.0    3.9   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  293.0  228.0 1508.0  994.7  0.0  1.0    0.0    2.0   0  93 172.17.4.6:/vol/mrtg01/mrtg01
  189.2  138.6  912.1  731.1  0.0  1.0    0.0    3.1   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  204.2  156.8 1012.8  707.0  0.0  1.0    0.0    2.8   0  95 172.17.4.6:/vol/mrtg01/mrtg01
  202.8  149.6 1044.9  698.2  0.0  1.0    0.0    2.8   0  95 172.17.4.6:/vol/mrtg01/mrtg01
  240.6  189.4 1175.2  898.8  0.0  1.0    0.0    2.3   0  93 172.17.4.6:/vol/mrtg01/mrtg01
  204.6  147.6 1024.1  700.9  0.0  1.0    0.0    2.9   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  199.8  144.2 1028.8  691.0  0.0  1.0    0.0    2.9   0  95 172.17.4.6:/vol/mrtg01/mrtg01
  183.8  150.8  968.8  904.2  0.0  1.0    0.0    2.9   0  91 172.17.4.6:/vol/mrtg01/mrtg01
  133.8  102.0  708.9  460.6  0.0  1.0    0.0    4.2   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  139.2  153.2  803.3 1435.6  0.0  1.0    0.0    3.6   0  88 172.17.4.6:/vol/mrtg01/mrtg01
  180.6  135.4  918.4  707.4  0.0  1.0    0.0    3.2   0  96 172.17.4.6:/vol/mrtg01/mrtg01
  164.0  123.8  811.2  566.5  0.0  1.0    0.0    3.5   0  94 172.17.4.6:/vol/mrtg01/mrtg01
  138.0  102.4  708.7  483.6  0.0  0.8    0.0    3.5   0  81 172.17.4.6:/vol/mrtg01/mrtg01
    0.4    5.0    0.1   89.1  0.0  0.0    0.0    0.6   0   0 172.17.4.6:/vol/mrtg01/mrtg01
    0.0    9.6    0.0    3.6  0.0  0.0    0.0    0.3   0   0 172.17.4.6:/vol/mrtg01/mrtg01


This is for one poll cycle, every 5 seconds.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 8:30 am 
Offline

Joined: Tue Sep 30, 2008 11:38 pm
Posts: 41
Location: Sofia, Bulgaria
That's why i said to run iostat on the NFS host :)
Here's my -xn output (only the important lines), while boost is updating the rrd files (similar to when cacti poller runs w/o boost):
md0 is the raid volume.

Code:
Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00   541.00    0.00 1723.00     0.00 18232.00    10.58    98.94   57.01   0.58 100.00
sdb               0.00   571.00    0.00 1690.00     0.00 18080.00    10.70    98.50   56.62   0.59 100.00
md0               0.00     0.00    0.00 4473.00     0.00 35784.00     8.00     0.00    0.00   0.00   0.00

I think "w/s" is similar to "tps", but i'm not sure. I've never user NFS before.

_________________
"Ubuntu" - an African word, meaning "Slackware is too hard for me".


Last edited by Jacket on Thu Mar 05, 2009 8:34 am, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 8:37 am 
Offline
Cacti User

Joined: Tue May 24, 2005 8:46 am
Posts: 101
Location: Rochester, New Hampshire
I couldn't run it on the NFS host.


w/s and r/s is write transactions per second, and read transactions per second, so yes, I am guessing the same as TPS.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 05, 2009 9:21 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
The only way to boost performance on NFS is to parallelize the update process. I did write a patch for just that, but did not go further with it due simply to the issues with disk caching with NFS.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 13, 2009 11:15 am 
Offline
Cacti User

Joined: Tue May 24, 2005 8:46 am
Posts: 101
Location: Rochester, New Hampshire
New Server, new results (still migrating all my devices, one at a time)


Code:
04/13/2009 03:22:11 PM - SYSTEM STATS: Time:129.0801 Method:cmd.php Processes:15 Threads:N/A Hosts:24 HostsPerProcess:2 DataSources:5048 RRDsProcessed:2163


04/13/2009 04:10:07 PM - SYSTEM STATS: Time:6.0652 Method:spine Processes:15 Threads:25 Hosts:24 HostsPerProcess:2 DataSources:5048 RRDsProcessed:2163
04/13/2009 04:15:12 PM - SYSTEM STATS: Time:11.0515 Method:spine Processes:15 Threads:25 Hosts:24 HostsPerProcess:2 DataSources:5048 RRDsProcessed:2163



Quad Xeon 3.4Ghz with 2GB RAM.
Cacti 0.8.7d and Spine 0.8.7c


Top
 Profile  
 
 Post subject: Super Fast!?
PostPosted: Tue Jun 02, 2009 3:31 pm 
Offline

Joined: Mon Jun 01, 2009 12:13 pm
Posts: 5
Location: York, Nebraska
IBM x345, 2xXeon 2.8GHz w/hyperthreading (4 total procs)
Ubuntu 7.10, 4GB RAM, U320 10k SCSI

Cacti 0.8.7b, RRDTool 1.2, Hosts 178, Graphs 5053
Poller Interval 300, Processes 2, Threads 20, PHP Servers 10, Max OID 25

Time:11.4981 Method:spine Processes:2 Threads:20 Hosts:177 HostsPerProcess:89 DataSources:10502 RRDsProcessed:5341

I tried 3 processes and that tripled the time, so did threads at 30.

The machine is running nagios and PHP Syslog for most of these devices, so I can't say I'm unhappy with these results at all!

(I was at 200 seconds a few days ago with 2GB RAM)

_________________
Thomas Schenck


Top
 Profile  
 
 Post subject: advice on my poor performance?!
PostPosted: Mon Jun 22, 2009 2:56 am 
Offline
Cacti User
User avatar

Joined: Thu May 03, 2007 3:17 pm
Posts: 244
Location: Italy
Hi guys,

any advice for my situation?

06/22/2009 09:49:30 AM - SYSTEM STATS: Time:269.0361 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7065

DELL Poweredge 1950
2xQUADCORE 2.00 GHZ XEON
8GB RAM
2xHDD 250GB RAID1
MYSQL ACTIVE ON ANOTHER SERVER TUNED

I don't know if will be better activate boost on my side ... the system is on production and fundamental for all my bussiness. There's the way for do that without downtime?

bye

sam


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 354 posts ]  Go to page Previous  1 ... 15, 16, 17, 18, 19, 20, 21 ... 24  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group