Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Sun May 28, 2017 12:00 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 33 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: [RESOLVED] Spine 0.8.7 SVN: Duplicate entry in poller_table
PostPosted: Wed Mar 24, 2010 1:27 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
Hi Folks,

I started testing out spine SVN rev 5706 on a dev box to evaluate the fix to the 'cannot open an ICMP socket' problem as described here: http://forums.cacti.net/viewtopic.php?t=34650 and started seeing the following duplicate entry in poller_table errors appear intermittently in my logs:


Quote:
03/24/2010 09:25:03 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '429-avg-2010-03-24 09:25:02' for key 1', SQL Fragment:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (429,'avg','2010-03-24 09:25:02','avg:348.25')'
03/24/2010 09:25:05 AM - SYSTEM STATS: Time:3.6164 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:99
03/24/2010 09:30:05 AM - SYSTEM STATS: Time:3.5168 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 09:35:06 AM - SYSTEM STATS: Time:4.4518 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 09:40:02 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '428-avg-2010-03-24 09:40:02' for key 1', SQL Fragment:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (428,'avg','2010-03-24 09:40:02','avg:239.07')'
03/24/2010 09:40:05 AM - SYSTEM STATS: Time:3.4949 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:99
03/24/2010 09:45:05 AM - SYSTEM STATS: Time:4.5009 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 09:50:05 AM - SYSTEM STATS: Time:3.4521 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 09:55:04 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '3-mem_buffers-2010-03-24 09:55:02' for key 1', SQL Fragment:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (3,'mem_buffers','2010-03-24
09:55:02','283980'),(4,'mem_swap','2010-03-24 09:55:02','2031608'),(5,'','2010-03-24 09:55:02','1min:0.06 5min:0.06 10min:0.06'),(6,'users','2010-03-24 09:55:02','5'),(7,'proc','2010-03-24 09:55:02','200'),(71,'traffic_in','2010-03-24 09:55:02','262929287'),(71,'traffic_out','2010-03-24 09:55:02','154143048'),(72,'hdd_free','2010-03-24 09:55:02','873796816'),(72,'hdd_used','2010-03-24 09:55:02','21568976'),(166,'','2010-03-24 09:55:02','change_db:1319 delete:32384 insert:3104 select:86854 update:13665'),(167,'connections','2010-03-24 09:55:02','13701'),(168,'','2010-03-24 09:55:02','delete:32657 read_first:5927 read_key:283428 read_next:321330 read_prev:0 read_rnd:57261 read_rnd_next:3891979 update:121101 write:3526030'),(169,'questions','2010-03-24 09:55:02','1'),(170,'','2010-03-24 09:55:02','connected:23 running:1 cached:0'),(297,'','2010-03-24 09:55:02','Time:3.4521 Method:spine Processes:4 Threads:10 H'
03/24/2010 09:55:06 AM - SYSTEM STATS: Time:4.4581 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 10:00:05 AM - SYSTEM STATS: Time:4.5230 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 10:05:05 AM - SYSTEM STATS: Time:4.4387 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:54
03/24/2010 10:10:06 AM - SYSTEM STATS: Time:4.4571 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 10:15:06 AM - SYSTEM STATS: Time:4.5156 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 10:20:04 AM - SYSTEM STATS: Time:3.4560 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 10:25:05 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '3-mem_buffers-2010-03-24 10:25:02' for key 1', SQL Fragment:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (3,'mem_buffers','2010-03-24 10:25:02','300180'),(4,'mem_swap','2010-03-24 10:25:02','2031608'),(5,'','2010-03-24 10:25:02','1min:0.53 5min:0.31 10min:0.16'),(6,'users','2010-03-24 10:25:02','5'),(7,'proc','2010-03-24 10:25:02','200'),(71,'traffic_in','2010-03-24 10:25:02','266494374'),(71,'traffic_out','2010-03-24 10:25:02','163615558'),(72,'hdd_free','2010-03-24 10:25:02','873796756'),(72,'hdd_used','2010-03-24 10:25:02','21569036'),(166,'','2010-03-24 10:25:02','change_db:196 delete:319 insert:31 select:5182 update:176'),(167,'connections','2010-03-24 10:25:02','171'),(168,'','2010-03-24 10:25:02','delete:362 read_first:262 read_key:23063 read_next:21178 read_prev:0 read_rnd:5350 read_rnd_next:48270 update:2163 write:47705'),(169,'questions','2010-03-24 10:25:02','1'),(170,'','2010-03-24 10:25:02','connected:22 running:1 cached:0'),(297,'','2010-03-24 10:25:02','Time:3.4560 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProc'03/24/2010 10:25:06 AM - SYSTEM STATS: Time:4.5240 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/24/2010 10:30:06 AM - SYSTEM STATS: Time:4.4748 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100



Code:
Technical Support

General Information

Date    Wed, 24 Mar 2010 11:24:44 -0700
Cacti Version    0.8.7e
Cacti OS    unix
SNMP Version    NET-SNMP version: 5.3.2.2
RRDTool Version    RRDTool 1.2.x
Hosts    8
Graphs    101
Data Sources    Script/Command: 34
SNMP: 19
SNMP Query: 48
Script Query: 2
Script - Script Server (PHP): 1
Script Query - Script Server: 6
Total: 110


Poller Information

Interval    300
Type    spine
Items    Action[0]: 72
Action[1]: 38
Action[2]: 11
Total: 121
Concurrent Processes    4
Max Threads    10
PHP Servers    1
Script Timeout    120
Max OID    10
Last Run Statistics    Time:4.4581 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100


PHP Information

PHP Version    5.1.6
PHP OS    Linux
PHP uname    Linux brudy-dev1 2.6.18-164.15.1.el5xen #1 SMP Wed Mar 17 12:04:23 EDT 2010 x86_64
PHP SNMP    Installed
max_execution_time    30
memory_limit    256M


Previously I was running the 0.8.7a of spine with the ping.c patch and hadn't seen anything like this. Just to be sure it wasn't something with MySQL, I bumped my max_connections to 1000 and still see these intermittent errors.


Last edited by Setarcos on Mon Apr 26, 2010 1:16 am, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 24, 2010 10:51 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Turn off process leveling at Console->Poller. There is a bug there.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 25, 2010 10:43 am 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
Turn off process leveling at Console->Poller. There is a bug there.

TheWitness


Thanks Larry,

Testing now...

I had a quick poke through Mantis looking for references to the process leveling bug and only saw 0001250. Is this captured elsewhere?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 25, 2010 11:00 am 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
Unchecking Balance Process Load doesn't appear to have improved anything:

Quote:
03/25/2010 08:15:05 AM - SYSTEM STATS: Time:3.4996 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/25/2010 08:20:05 AM - SYSTEM STATS: Time:3.5075 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/25/2010 08:25:05 AM - SYSTEM STATS: Time:3.6044 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/25/2010 08:30:05 AM - SYSTEM STATS: Time:3.4516 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/25/2010 08:35:04 AM - SYSTEM STATS: Time:3.4845 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/25/2010 08:40:05 AM - SYSTEM STATS: Time:3.4301 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:54
03/25/2010 08:45:05 AM - SYSTEM STATS: Time:3.5541 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100
03/25/2010 08:50:04 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '3-mem_buffers-2010-03-25 08:50:01' for key 1', SQL Fragment:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (3,'mem_buffers','2010-03-25 08:50:01','264644'),(4,'mem_swap','2010-03-25 08:50:01','2031608'),(5,'','2010-03-25 08:50:01','1min:0.10 5min:0.03 10min:0.01'),(6,'users','2010-03-25 08:50:01','4'),(7,'proc','2010-03-25 08:50:01','199'),(71,'traffic_in','2010-03-25 08:50:01','398277407'),(71,'traffic_out','2010-03-25 08:50:01','322179216'),(72,'hdd_free','2010-03-25 08:50:01','873791544'),(72,'hdd_used','2010-03-25 08:50:01','21574248'),(166,'','2010-03-25 08:50:01','change_db:3858 delete:27763 insert:2452 select:133084 update:12062'),(167,'connections','2010-03-25 08:50:01','12333
'),(168,'','2010-03-25 08:50:01','delete:27806 read_first:7595 read_key:492637 read_next:450390 read_prev:0 read_rnd:73372 read_rnd_next:3496827 update:136854 write:3408978'),(169,'questions','2010-03-25 08:50:01','1'),(170,'','2010-03-25 08:50:01','connected:25 running:1 cached:0'),(297,'','2010-03-25 08:50:01','Time:3.5541 Method:spine Processes:4 Threads:'
03/25/2010 08:50:04 AM - SYSTEM STATS: Time:3.4637 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:95
03/25/2010 08:55:03 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '461-ActiveConnections-2010-03-25 08:55:02' for key 1', SQL Fragment:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (461,'ActiveConnections','2010-03-25 08:55:02','0'),(462,'ActiveConnections','2010-03-25 08:55:02','0'),(463,'ActiveConnections','2010-03-25 08:55:02','0'),(464,'ActiveConnections','2010-03-25 08:55:02','0'),(465,'ActiveConnections','2010-03-25 08:55:02','0'),(466,'L4TotalConnections','2010-03-25 08:55:02','0'),(470,'DynamicWeight','2010-03-25 08:55:02','0'),(471,'DynamicWeight','2010-03-25 08:55:02','0'),(472,'DynamicWeight','2010-03-25 08:55:02','0'),(473,'DynamicWeight','2010-03-25 08:55:02','0'),(474,'DynamicWeight','2010-03-25 08:55:02','0'),(475,'HitRate','2010-03-25 08:55:02','0'),(476,'HitRate','2010-03-25 08:55:02','0'),(477,'HitRate','2010-03-25 08:55:02','0'),(478,'Load','2010-03-25 08:55:02','0'),(479,'Load','2010-03-25 08:55:02','0'),(480,'Load','2010-03-25 08:55:02','0'),(481,'Load','2010-03-25 08:55:02','0'),(482,'Load','2010-03-25 08:55:02','0'),(483,'ClusterLoad','2010-03-25 08:55:02','0'),(484,'ClusterLoad','2010-03-25'
03/25/2010 08:55:05 AM - SYSTEM STATS: Time:3.4595 Method:spine Processes:4 Threads:10 Hosts:9 HostsPerProcess:3 DataSources:121 RRDsProcessed:100


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 25, 2010 11:43 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Interesting this is that this has not changed. So, I find it hard to believe, unless you have two cron's running performing the update simultaneously.

This problem is notorious. You are using stock cacti with/without PIA right?

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 25, 2010 11:59 am 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
Interesting this is that this has not changed. So, I find it hard to believe, unless you have two cron's running performing the update simultaneously.

This problem is notorious. You are using stock cacti with/without PIA right?

TheWitness


Yep, I went through all my crons to be sure and confirmed that the following is the only entry that exists:

# Cacti poller
*/5 * * * * cactiuser php /var/www/html/cacti/poller.php > /dev/null 2>&1

This is with a totally stock Cacti 0.8.7e and the latest set of official patches. No PIA, Boost, etc...

After disabling Balance Process Load I enabled/disabled the poller, cleared the poller output and rebuild the poller cache to make sure things started with a clean slate.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 25, 2010 12:03 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
PM you log at debug level for poller.php

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 25, 2010 1:21 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
PM you log at debug level for poller.php

TheWitness


PM sent


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 07, 2010 11:40 am 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
Anything further I can do to help track this down Larry?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 08, 2010 5:32 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Just getting back from vacation. I'll be looking at this soon.

Larry

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 09, 2010 12:39 am 
Offline
Cacti User

Joined: Fri Sep 12, 2008 2:41 am
Posts: 168
Location: Sweden
If you remember TheWitness, i have the same problem, with or without an unchecked Balance Process Load.

Tho i cant say its a problem, it just fills the logfile :)

I tried the latest rev of spine (5731) but only got a mysql error:
Code:
FATAL: MySQL Error:'1054', Message:'Unknown column 'device_id' in 'where clause'' (Spine parent)


I suppose this could be because of an updated db with cacti0.8.8.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 09, 2010 7:09 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Yes, use branches/0.8.7

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 09, 2010 7:22 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Ok, I'm not sure this is spine related. Can you please run poller.php with debug mode set as well as force, and send the stdout here.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 09, 2010 8:42 am 
Offline
Cacti User

Joined: Fri Sep 12, 2008 2:41 am
Posts: 168
Location: Sweden
TheWitness wrote:
Yes, use branches/0.8.7

rev5731 is part of branches/main and /branches/0.8.7 :-?

Ill have to get back to you with poller.php on monday, unless Setarcos manages before me.

//David


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 09, 2010 9:39 am 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
toe_cutter wrote:
TheWitness wrote:
Yes, use branches/0.8.7

rev5731 is part of branches/main and /branches/0.8.7 :-?

Ill have to get back to you with poller.php on monday, unless Setarcos manages before me.

//David


When I make changes to 0.8.7, I also have to keep up with main too. Getting tiring to have to maintain two releases though.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 33 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group