Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Wed May 24, 2017 2:56 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 4 posts ] 
Author Message
 Post subject: how to handle sporadic spine error
PostPosted: Fri Feb 17, 2017 7:55 am 
Offline
Cacti User

Joined: Wed Dec 07, 2011 9:19 am
Posts: 184
How can I handle this kind of trouble, how can I track down what/whereis the trouble
I'm getting spine error on my log of this type:

02/17/2017 01:47:06 PM - SYSTEM STATS: Time:5.3733 Method:spine Processes:25 Threads:20 Hosts:870 HostsPerProcess:35 DataSources:10759 RRDsProcessed:7388
02/17/2017 01:47:06 PM - SYSTEM THOLD STATS: Time:6.0040 Tholds:1616 TotalHosts:869 DownHosts:1 NewDownHosts:0
02/17/2017 01:47:01 PM - POLLER: Poller[0] WARNING: There are '12' detected as overrunning a polling process, please investigate
02/17/2017 01:47:00 PM - SYSTEM STATS: Time:58.4133 Method:spine Processes:25 Threads:20 Hosts:870 HostsPerProcess:35 DataSources:10759 RRDsProcessed:5157
02/17/2017 01:47:00 PM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.
02/17/2017 01:46:07 PM - SPINE: Poller[0] FATAL: Connection Failed, Error:'2002', Message:'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (11)' (Spine thread)
02/17/2017 01:45:17 PM - SYSTEM THOLD STATS: Time:8.8480 Tholds:2286 TotalHosts:869 DownHosts:1 NewDownHosts:0
02/17/2017 01:45:08 PM - SYSTEM STATS: Time:6.4391 Method:spine Processes:25 Threads:20 Hosts:870 HostsPerProcess:35 DataSources:10757 RRDsProcessed:7384
02/17/2017 01:44:15 PM - SYSTEM THOLD STATS: Time:8.7473 Tholds:2287 TotalHosts:869 DownHosts:1 NewDownHosts:0
02/17/2017 01:44:06 PM - SYSTEM STATS: Time:4.7885 Method:spine Processes:25 Threads:20 Hosts:870 HostsPerProcess:35 DataSources:10757 RRDsProcessed:7387
.
.
.
02/17/2017 01:15:00 PM - SYSTEM STATS: Time:58.1940 Method:spine Processes:25 Threads:20 Hosts:870 HostsPerProcess:35 DataSources:10757 RRDsProcessed:4116
02/17/2017 01:15:00 PM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.
02/17/2017 01:14:07 PM - SPINE: Poller[0] FATAL: Connection Failed, Error:'1040', Message:'Too many connections' (Spine thread)
.
.
.
02/17/2017 01:14:07 PM - SPINE: Poller[0] FATAL: Connection Failed, Error:'1040', Message:'Too many connections' (Spine thread)
02/17/2017 01:13:15 PM - SYSTEM THOLD STATS: Time:7.3882 Tholds:2287 TotalHosts:869 DownHosts:1 NewDownHosts:0
02/17/2017 01:13:08 PM - SYSTEM STATS: Time:6.6720 Method:spine Processes:25 Threads:20 Hosts:870 HostsPerProcess:35 DataSources:10757 RRDsProcessed:7387

_________________
CentOS
Cacti 0.8.8h
Spine 0.8.8h
PIA 3.1
Aggregate 0.75
Monitor 1.3
Settings 0.71
Weathermap 0.98
Thold 0.5
rrdclean 0.41

Own plugin: LinkDiscovery 0.3


Top
 Profile  
 
 Post subject: Re: how to handle sporadic spine error
PostPosted: Fri Feb 17, 2017 12:57 pm 
Offline
Developer

Joined: Thu Apr 07, 2005 3:29 pm
Posts: 2534
Location: B/CS Texas
Increase the number of allowed mysql connections.

_________________
CactiEZ CD


Top
 Profile  
 
 Post subject: Re: how to handle sporadic spine error
PostPosted: Sat Feb 18, 2017 1:03 pm 
Offline
Cacti User

Joined: Wed Dec 07, 2011 9:19 am
Posts: 184
I have 1000, isn't that enough for the size of my network ?

_________________
CentOS
Cacti 0.8.8h
Spine 0.8.8h
PIA 3.1
Aggregate 0.75
Monitor 1.3
Settings 0.71
Weathermap 0.98
Thold 0.5
rrdclean 0.41

Own plugin: LinkDiscovery 0.3


Top
 Profile  
 
 Post subject: Re: how to handle sporadic spine error
PostPosted: Mon Feb 20, 2017 12:17 am 
Offline
Developer

Joined: Thu Apr 07, 2005 3:29 pm
Posts: 2534
Location: B/CS Texas
Just your spine threads are consuming (25+1) * 20 = 520 plus the poller process, script servers, etc... you are chewing threw a lot.

_________________
CactiEZ CD


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group