Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Tue May 23, 2017 1:55 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 74 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
 Post subject:
PostPosted: Mon Apr 04, 2005 12:24 pm 
Offline
Cacti User

Joined: Tue Jan 27, 2004 5:32 pm
Posts: 77
I used that latest version you recommended, and am still gettting the same error.

04/04/2005 01:00:04 PM - CACTID: Poller[0] ERROR: MySQL Connection failed aft
er 10 attempts: Can't connect to local MySQL server through socket '/tmp/mysq
l.sock' (4)
04/04/2005 01:00:04 PM - PHPSVR: Poller[0] ERROR: Input Expected, Script Serv
er Terminating

I changed my cacti threads from 2 to 1, and restarted mysql. Is cacti trying to make too many connections to mysql so that mysql is denying them?


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 04, 2005 2:34 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
- You should check for hung processes first. "ps -ef | grep cactid"

- If you don't have any, then check to see how many concurrent connections mysql is configured for. You can check that in my.conf/my.ini.

- If then you don't get happy with those two checks, my question would be: "does cactid only run intermittantly or not at all?"

- If it does not run at all, check your mysql library version vs. the installed version only if you are running MySQL 4.1 or higher.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 04, 2005 4:20 pm 
Offline
Cacti User

Joined: Tue Jan 27, 2004 5:32 pm
Posts: 77
- You should check for hung processes first. "ps -ef | grep cactid"

Everytime the cactid fails it gets hung. I wrote a script to kill the hung processes once an hour.

- If you don't have any, then check to see how many concurrent connections mysql is configured for. You can check that in my.conf/my.ini.

I was running mysql in safe mode with no .cnf. I have now changed this and am running it with the 'my_huge.cnf" as my /etc/my.cnf file.

- If then you don't get happy with those two checks, my question would be: "does cactid only run intermittantly or not at all?"

Cactid fails about 10 percent of the times it runs, at various points during the poll.

- If it does not run at all, check your mysql library version vs. the installed version only if you are running MySQL 4.1 or higher.



Ill see if my changes help out.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 05, 2005 5:11 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
I'm working with a user who was having problems and he is ok now and had to perform an upgrade to net-snmp 5.1.1 to stop the segfaults.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 12, 2005 4:17 pm 
Offline
Cacti User

Joined: Tue Jan 27, 2004 5:32 pm
Posts: 77
I changed the threading from 2 to 1, and it reduced the number of MYSQL errors from a few per hour, to 1 or 2 per day. I let it run like this for a few days, then upgraded from UCD-SNMP 4.2.3 to NET-SNMP-5.1.1. I am still seeing the MYSQL Errors occuring frequently when I have 2 threads enabled:

04/12/2005 04:13:29 PM - CACTID: Poller[0] ERROR: MySQL Connection failed after 10 attempts: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (4)
04/12/2005 04:30:49 PM - CACTID: Poller[0] ERROR: MySQL Connection failed after 10 attempts: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (4)
04/12/2005 04:50:03 PM - CACTID: Poller[0] ERROR: MySQL Connection failed after 10 attempts: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (4)


The only difference I see is that my perfomance acutally slowed from 130 sec, to 140 sec.

How many connections is cactid attempting to make to MYSQL during the polling period? Is it possible MYSQL is not allowing the connections? Here is some stuff from my my.cnf file:

port = 3306
socket = /tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-locking
#set-variable = key_buffer=384M
set-variable = key_buffer=128M
set-variable = max_allowed_packet=1M
set-variable = max_connection=200
set-variable = table_cache=512
set-variable = sort_buffer=2M
set-variable = record_buffer=2M
set-variable = thread_cache=8
# Try number of CPU's*2 for thread_concurrency
set-variable = thread_concurrency=8
set-variable = myisam_sort_buffer_size=64M
#log-bin
server-id = 1


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 12, 2005 4:32 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
The maximum connections that Cacti will use is (Processes*Treads)+1, but likely less. Could it be that there are issues with the MySQL client libraries and the release of connections after a process ends. Can you look at MySQL Admin and see if you are simply hanging on to old connections and if you can see what they were doing before hanging?

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 12, 2005 4:35 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Also, when you see these errors, are the processes still locking up?

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 12, 2005 5:13 pm 
Offline
Cacti User

Joined: Tue Jan 27, 2004 5:32 pm
Posts: 77
Yes. I will see some hung rrdtool updates and php scripts.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 12, 2005 5:15 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Which scripts? Which updates?

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 13, 2005 10:38 am 
Offline
Cacti User

Joined: Tue Jan 27, 2004 5:32 pm
Posts: 77
The failures occur at different places during the poll, I was not able to find any sort of trend. See my previous posts on this topic from Nov 23 and Feb17/18 for an example of what hangs.

Since I changed to 1 thread yesterday, I have not had a MySQL Connection failure, however, the poll is taking as long as 150 sec. I will run with 1 thread for a while, and then switch back to 2 to see the difference in performance and rate of failure.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 13, 2005 5:30 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
This then sounds like a pthread issue with Solaris. Can you look up bugs for your version of the OS relative to pthread support on the Sun web site? In addition, review the net-snmp bug pages on sourceforge.net for references to memory leaks/segfaults. My PC died yesterday, and I don't know when I will be back upto full speed.

TheWitness :oops:

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: mysql threads approach
PostPosted: Thu Apr 14, 2005 9:17 am 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22461
Location: Muenster, Germany
To all cacti fans!
Yes, we also suffered from those gaps. But meanwhile, all graphs are clean. We have about 5580 poller_items in about 2375 rrds. The solution seemd to be the "tuning" of some mysql parms:
Code:
max_connections = 500
key_buffer=32M
query_cache_size=64M

max_connections seems to be exessive (default is 100), just a shot. But from that very moment: no more gaps!
Ok, apart from a lot of snmp queries there are lots of wgets with perl parsing, some "smokepings" that last 2 sec each, some snmp execs that will last some time.
I don't have an "exact" explanation, but folks, it works!
RHEL 4.0, mysql 4.1.10a, php 4.3.9, rrdtool 1.0.49, perl v5.8.5, apache 2.0.52
A great thanks to the cacti developers

Reinhard


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 19, 2005 3:48 pm 
Offline
Cacti User

Joined: Tue Jan 27, 2004 5:32 pm
Posts: 77
I have made the mysql changes recommended above. Some of those features are not in our current version.

bash-2.03$ /usr/local/mysql/bin/mysql --version
/usr/local/mysql/bin/mysql Ver 11.18 Distrib 3.23.53, for sun-solaris2.8 (sparc)

We are able to get rid of the gaps, but the key seems to be the threading. With 2 threads enabled, I was getting 1-5 per MySQL connection failures per hour. After switching to 1 thread, I have only had 3 failures in 5 days. Any idea why threading would cause this problem? The machine is not under alot of load.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 19, 2005 3:52 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Poor implementation of threading in solaris. Don't forget to re-maike php_snmp per my other thread. You can simply download the PHP CVS version for snmp.c and remake the entire package.

Larry

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Sat Dec 03, 2005 6:10 pm 
Offline
Cacti User
User avatar

Joined: Wed Jul 20, 2005 10:48 am
Posts: 165
Location: UK - Cambridge
Hi Yardus,

Did you ever get this resolved? I ask as I am having the timeout messages and also the PHP crash?

Many thanks,

Larry

_________________
---- lard007skype ----


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 74 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group