|
|
| Author |
Message |
rvd
Joined: 21 Nov 2007
Posts: 7
Location: France
|
 Posted: Fri May 16, 2008 2:20 am Post subject: Script Server Data slowing Spine |
 |
|
Hi everybody,
I created a new host template for weblogic hosts. Graph and data templates for this host type are linked to a perl script - I can't write in php.
This script gives output values on 9 various data (heapused, heaptotal, threadtotal an so on) for a weblogic host, indexed on weblogic server name.
I followed then the HowTo and created xml file.
When I call the script in command line with get option, the answer is quite fast.
But when I finally added a new host of this type and created graphs for it the poller is getting MUCH slower.
The host contains 12 WL servers and the 9 values are asked, so 108 more DS.
This is the difference between before and after adding the host
| Quote: | 05/16/2008 08:50:26 AM - SYSTEM STATS: Time:24.3523 Method:spine Processes:2 Threads:20 Hosts:85 HostsPerProcess:43 DataSources:1210 RRDsProcessed:726
05/16/2008 08:56:03 AM - SYSTEM STATS: Time:62.0576 Method:spine Processes:2 Threads:20 Hosts:86 HostsPerProcess:43 DataSources:1318 RRDsProcessed:798 |
My interrogation is about spine. Launching debug mode shows me a unique spine process executes the 108 calls to the script sequentially, whatever is the number of threads. It takes about 36 seconds.
Is there a way to have the 108 calls executed not sequentially ?
My other interrogation is about 'Script Data' and 'Script Server Data' in 'Data Queries'. I don't really understand the difference ?
Using spine 0.8.7 and cacti 0.8.7b |
|
| Back to top |
|
 |
nate450
Joined: 13 Mar 2006
Posts: 16
Location: Seattle, WA
|
| Posted: Fri May 16, 2008 8:36 pm Post subject: |
|
|
try re-writing the script so you only have 1 DS per host instead of 9.
I have several scripts that return a dozen or more data points to cacti for one data source, which seems to allow it to scale much higher than it otherwise would be able to. Extra work is involved in creating the graphs and stuff though.
My cacti install:
| Code: | 05/16/2008 06:35:13 PM - SYSTEM STATS: Time:11.7292 Method:spine Processes:1 Threads:30 Hosts:305 HostsPerProcess:305 DataSources:337 RRDsProcessed:333
|
Doesn't look like much, but when you see how many data points are actually in those 333 data files, there are 6,773 points of data, or an average of 20 points per file. I have cacti running every minute. It's on a dual proc quad core system 16GB memory and is averaging 11% cpu usage. |
|
| Back to top |
|
|
gandalf
Developer
Joined: 02 Dec 2004
Posts: 12295
Location: Muenster, Germany
|
| Posted: Sun May 18, 2008 8:44 am Post subject: Re: Script Server Data slowing Spine |
|
|
| rvd wrote: | My interrogation is about spine. Launching debug mode shows me a unique spine process executes the 108 calls to the script sequentially, whatever is the number of threads. It takes about 36 seconds.
Is there a way to have the 108 calls executed not sequentially ?
My other interrogation is about 'Script Data' and 'Script Server Data' in 'Data Queries'. I don't really understand the difference ? |
That's by design; and sometimes I hate it.
Cacti splits up all hosts between the spine process. This results in executing all Scripts/Script Server Scripts/SNMP stuff in a single spine process.
Difference between Scrip and Script Server is:
1. Script Server is available for PHP scripts only
2. Script Server starts a single PHP execution environment and then LOADs all Script Server Scripts into it (include!). The Script Server then calls the function that is provided by the Script Server Script
Reinhard |
|
| Back to top |
|
|
rvd
Joined: 21 Nov 2007
Posts: 7
Location: France
|
| Posted: Tue May 20, 2008 1:47 am Post subject: |
|
|
| Quote: | | Cacti splits up all hosts between the spine process. This results in executing all Scripts/Script Server Scripts/SNMP stuff in a single spine process |
A pity cacti doesn't dispatch job by DS rather than by host. You may actually have hosts with a few DS and other ones with a hundred ones. 
| Quote: | Difference between Scrip and Script Server is:
1. Script Server is available for PHP scripts only
2. Script Server starts a single PHP execution environment and then LOADs all Script Server Scripts into it (include!). The Script Server then calls the function that is provided by the Script Server Script |
Thx for this explanation |
|
| Back to top |
|
|
gandalf
Developer
Joined: 02 Dec 2004
Posts: 12295
Location: Muenster, Germany
|
| Posted: Sat May 24, 2008 3:18 pm Post subject: |
|
|
I'm quite sure you're right.
Especially with the new cli functions, it will be a common issue, that hosts of same kind have id's in increasing sequence. This is a good chance to have multiple spine processes in a very unbalanced way.
Please create a feature request at http://bugs.cacti.net
Reinhard |
|
| Back to top |
|
|
aleu
Cacti User
Joined: 11 Dec 2006
Posts: 108
|
| Posted: Mon May 26, 2008 3:22 pm Post subject: |
|
|
| gandalf wrote: | I'm quite sure you're right.
Especially with the new cli functions, it will be a common issue, that hosts of same kind have id's in increasing sequence. This is a good chance to have multiple spine processes in a very unbalanced way.
Please create a feature request at http://bugs.cacti.net
Reinhard |
rvd, have you created this feature request? It seems that I am facing exactly the same problem... |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Register
Profile
Log in to check your private messages
Log in
