Cacti (home)ForumsRepositoryDocumentation
Cacti: offical forums and support
It is currently Tue Jul 29, 2014 7:44 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: increased cpu load and runtime with the new spine version
PostPosted: Thu Jun 10, 2010 5:46 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
Hello,

i have testet the new cacti-0.8.7g-BETA1 with spine-0.8.7g and also spine-0.8.7e .

The duration and the CPU load has increased but very strong.

cacti-0.8.7g-BETA1 spine-0.8.7g
Time:91.4288 Method:spine Processes:4 Threads:8 Hosts:793 HostsPerProcess:199 DataSources:20619 RRDsProcessed:11849

cacti-0.8.7g-BETA1 spine-0.8.7e
Time:17.6848 Method:spine Processes:4 Threads:8 Hosts:793 HostsPerProcess:199 DataSources:20619 RRDsProcessed:11829

Bug or Configuration Problem ?

thx
UK


Attachments:
cacti-cpu.png
cacti-cpu.png [ 28.62 KiB | Viewed 5453 times ]
cacti-poller-runtime.png
cacti-poller-runtime.png [ 24.57 KiB | Viewed 5453 times ]
Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 10, 2010 6:24 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
I may have missed a commit, I don't think so though. I'll have to check.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 10, 2010 6:27 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Try this modified spine.c and let me know the impact.

TheWitness


Attachments:
spine.zip [7.99 KiB]
Downloaded 180 times

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository
Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 10, 2010 6:28 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Also, PM me the following:

Code:
./spine -V 5 -R -S


For both versions.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 4:13 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
Hallo TheWitness,


the error I found in the file spine.c

The problem is that there are in the table "host" no entry device_threads .

>line 470 spine.c

/* obtain the list of hosts to poll */
if (set.device_threads_exists) {
qp += sprintf(qp, "SELECT id, device_threads FROM host");
}else{
qp += sprintf(qp, "SELECT id, '1' as device_threads FROM host");
}


This causes the high cpu utilization and the time delay.

Unfortunately, the problem occurs in the cacti-0.8.7g-BETA2 version and svn as well.

by UK


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 7:06 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
There is a beta2? Did not see the note. Hopefully the order bit was removed. I'll have to check (was on a plane). That should not cause high CPU, it's the order statement.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 7:10 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Remove line 486 from spine.c
Code:
qp += sprintf(qp, " GROUP BY host.id");


Let me know if that fixes it. Also, do you have time for a GotoMeeting this morning?

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 8:47 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
TheWitness wrote:
Remove line 486 from spine.c
Code:
qp += sprintf(qp, " GROUP BY host.id");


Let me know if that fixes it. Also, do you have time for a GotoMeeting this morning?

TheWitness


Unfortunately, no change.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 9:18 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
GotoMeeting?

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 9:24 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
bfek-18 wrote:
TheWitness wrote:
Remove line 486 from spine.c
Code:
qp += sprintf(qp, " GROUP BY host.id");


Let me know if that fixes it. Also, do you have time for a GotoMeeting this morning?

TheWitness


Unfortunately, no change.

the problem is the select statement.

orginal sql command:

SELECT id, '1' as device_threads FROM host LEFT JOIN poller_item ON poller_item.host_id=host.id WHERE disabled='';


my environment:

host.id = 800 rows
poller_item.host_id = 20.000 rows

Runs = 800 x 20.000 = 16.000.000

It takes a long time.

bye Uk


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 9:26 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
TheWitness wrote:
GotoMeeting?

sorry but this is my enlisch too bad


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 9:28 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Don't need to talk, just "see".

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 9:36 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
TheWitness wrote:
Don't need to talk, just "see".

TheWitness

Unfortunately, not possible.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 9:45 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14861
Location: MI, USA
Well, send me a copy of the output of this:

Code:
#!/bin/bash
while [ 1 ]; do
  clear
  mysql -e "show processlist" | grep -v Sleep | grep -v NULL
  sleep 1
done


While the poller is starting up.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 14, 2010 10:03 am 
Offline

Joined: Mon Oct 27, 2003 9:13 am
Posts: 37
Location: Germany, Potsdam
TheWitness wrote:
Well, send me a copy of the output of this:

Code:
#!/bin/bash
while [ 1 ]; do
  clear
  mysql -e "show processlist" | grep -v Sleep | grep -v NULL
  sleep 1
done


While the poller is starting up.

TheWitness


Query Log:

Id User Host db Command Time State Info
767 cacti localhost cactineu Query 0 Sending data SELECT id, '1' as device_threads FROM host LEFT JOIN poller_item ON poller_item.host_id=host.id WHER
...
767 cacti localhost cactineu Query 45 Sending data SELECT id, '1' as device_threads FROM host LEFT JOIN poller_item ON poller_item.host_id=host.id WHER

bye Uk


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group