Cacti 0.8.7 Beta 4, Spine Beta 4 and PIA Beta 4 Problems

[chrisgapske]

When I set the poller to spine I came across this error. It looks like it polling but fails to update the rrd's ?

What Should I look at ?




Log File [Total Lines: 10000 - All Items Shown]

10/02/2007 04:00:01 PM - POLLER: Poller[0] DEBUG: Poller Interval: '300', Cron Interval: '300', Max Poller Runtime '298', Poller Runs: '1'
10/02/2007 04:00:01 PM - MANAGE: Poller[0] Checking Manage
10/02/2007 04:00:01 PM - SYSTEM STATS: Time:298.8753 Method:spine Processes:4 Threads:1 Hosts:160 HostsPerProcess:40 DataSources:1366 RRDsProcessed:0
10/02/2007 04:00:01 PM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
10/02/2007 03:55:02 PM - POLLER: Poller[0] DEBUG: Poller Interval: '300', Cron Interval: '300', Max Poller Runtime '298', Poller Runs: '1'
10/02/2007 03:55:01 PM - MANAGE: Poller[0] Checking Manage
10/02/2007 03:55:01 PM - SYSTEM STATS: Time:298.9209 Method:spine Processes:4 Threads:1 Hosts:160 HostsPerProcess:40 DataSources:1366 RRDsProcessed:0
10/02/2007 03:55:01 PM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
10/02/2007 03:50:02 PM - POLLER: Poller[0] DEBUG: Poller Interval: '300', Cron Interval: '300', Max Poller Runtime '298', Poller Runs: '1'
10/02/2007 03:50:01 PM - MANAGE: Poller[0] Checking Manage
10/02/2007 03:50:01 PM - SYSTEM STATS: Time:298.9830 Method:spine Processes:4 Threads:1 Hosts:160 HostsPerProcess:40 DataSources:1366 RRDsProcessed:0
10/02/2007 03:50:01 PM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
10/02/2007 03:45:02 PM - POLLER: Poller[0] DEBUG: Poller Interval: '300', Cron Interval: '300', Max Poller Runtime '298', Poller Runs: '1'
10/02/2007 03:45:02 PM - MANAGE: Poller[0] Checking Manage
10/02/2007 03:40:24 PM - POLLER: Poller[0] Plugin Update Check is now processing
10/02/2007 03:40:23 PM - SYSTEM STATS: Time:22.4501 Method:cmd.php Processes:4 Threads:N/A Hosts:160 HostsPerProcess:40 DataSources:1366 RRDsProcessed:822

[cacti4ever]

Same problem here ... poller time it isn't constant:

10/05/2007 12:10:00 PM - SYSTEM STATS: Time:299.2472 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:494
10/05/2007 12:14:59 PM - SYSTEM STATS: Time:299.1710 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:496
10/05/2007 12:15:13 PM - SYSTEM STATS: Time:11.3785 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:546
10/05/2007 12:25:00 PM - SYSTEM STATS: Time:298.8185 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:492
10/05/2007 12:25:10 PM - SYSTEM STATS: Time:9.9563 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:546
10/05/2007 12:35:00 PM - SYSTEM STATS: Time:299.5067 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:495
10/05/2007 12:35:11 PM - SYSTEM STATS: Time:10.4565 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:546
10/05/2007 12:45:00 PM - SYSTEM STATS: Time:299.5358 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:494
10/05/2007 12:45:22 PM - SYSTEM STATS: Time:22.0495 Method:spine Processes:4 Threads:8 Hosts:114 HostsPerProcess:29 DataSources:761 RRDsProcessed:546

When Spine spent 298 seconds in their polling cycle log is full of entries like that:

10/05/2007 12:09:03 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_id,end_time from poller_time where poller_id=0"
10/05/2007 12:09:03 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) LIMIT 10000"


Cacti version: cacti-0.8.7-beta4-PAv1.2
Spine version: cacti-0.8.7-beta3 or cacti-0.8.7-beta4 (same results)

Can any help with this? comments welcome!

[TheWitness]

Check your /var/log/messages for spine segmentation faults. I need to know the address of them. If you can confirm that is what is happending, then we need to work together to isolate.

Please Advise,

TheWitness

[TheWitness]

Here is the procedure BTB. You will see segmentation fault's at such and such an address. Take that information and goto the spine "build" directory and do the following:

gdb ./sping
b *0x<the_address_from_the_log>

It should return a line in once of the *.c files. Please provide that information.

TheWitness