Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Mon May 20, 2019 6:08 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: cmd.php dying in 0.8.6j
PostPosted: Fri Jan 19, 2007 12:56 pm 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
Hi all

Here I have CentOS 4.4, php 4.3.9, net-snmp 5.1.2.
I just upgraded to 0.8.6j to fix the recent security issues.
Immediately after it, my drawings ceased being updated, and I found that poller.php is timing out.
Here's my log:

01/19/2007 06:38:28 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_id,end_time from poller_time where poller_id = 0"
01/19/2007 06:38:28 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) LIMIT 10000"
01/19/2007 06:38:29 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_id,end_time from poller_time where poller_id = 0"
01/19/2007 06:38:29 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) LIMIT 10000"
...
LOTS of these lines...
...
01/19/2007 06:39:51 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_id,end_time from poller_time where poller_id = 0"
01/19/2007 06:39:51 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) LIMIT 10000"
01/19/2007 06:39:51 PM - POLLER: Poller[0] Maximum runtime of 292 seconds exceeded. Exiting.
01/19/2007 06:39:51 PM - SYSTEM STATS: Time:293.0723 Method:cmd.php Processes:1 Threads:N/A Hosts:8 HostsPerProcess:8 DataSources:97 RRDsProcessed:5
01/19/2007 06:39:51 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "replace into settings (name,value) values ('stats_poller','Time:293.0723 Method:cmd.php Processes:1 Threads:N/A Hosts:8 HostsPerProcess:8 DataSources:97 RRDsProcessed:5')"
01/19/2007 06:39:51 PM - CMDPHP: Poller[0] DEBUG: SQL Cell: "select count(*) from poller_command"

I investigated the cause of this behaviur, and found that:
1) cmd.php dies without any warning soon after being run by poller.php.
2) poller.php tries forever to read poller_time, which cmd.php should update, but timeouts because cmd.php is just dead.
3) cmd.php dies when trying to ping a host (in this case localhost), more precisely it dies exactly when when it calls @snmpgetnext() in cacti_snmp_getnext() in snmp.php. In fact, the call to @snmpgetnext has been added in 0.8.6j.
4) @snmpgetnext is only available in PHP5, not in 4.
5) reverting snmpgetnext to snmpget as it used to be, solved the problem.

I think that in the code we should at least track the case of PHP4 and use snmpget instead of snmpgetnext;
and maybe poller.php should look for cmd.php dying abnormally, and report it in the log.
If cmd.php could die with some sort of debug message, I would appreciate it, but I really don't know if this can be done in PHP.

Comments are welcome.

Thank you for your attention
Luigi


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 6:06 pm 
Offline

Joined: Fri Jan 19, 2007 6:06 pm
Posts: 1
Location: Heath, OH
Having the same issue... I am running CentOS 4.4, PHP 4.3.,NET-SNMP 5.3.1

-CBC-


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 2:57 am 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
I wrote that changing @snmpgetnext in @snmpget solved the problem. I had been optimistic. It only stops cmd.php to suddenly die.
I also had to change the line in ping.php saying:
$oid = ".1";
into:
$oid = ".1.3.6.1.2.1.1.3.0";
because if I do snmpget instead of snmpgetnext, I have to point to an available oid. I noticed I could have the same result using a retry count higher than 1, since the code switches to the ".1.3.6.1.2.1.1.3.0" oid when it retries.

So, a feature request is added to the ones in my previous post:
can't we have a new property for the host, saying what oid we should ask for pinging it, just as we can modify the SNMP port of the host?
In my opinion, it could be also helpful to have a similar property for the UDP port we use to ping the host, instead of the hard coded 33439. I have cacti in a firewalled environment, and could not use UDP ping because the firewall blocks this port.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 1:07 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
1) Please verify if php_snmp is installed. Post your findings.
2) Please verify that you have a valid path the the binary snmpgetnext. Post your findings.
3) Please revert to ".1" and sniff utilizing Wireshark (Ethereal) and PM me the capture file.

This works perfectly fine on other systems.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 5:12 pm 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
TheWitness wrote:
1) Please verify if php_snmp is installed. Post your findings.


It's installed. Sorry for not metioning it explicitly. The fact that with 0.8.6i (and with 0.8.6j, with the little mod I was talking in my previous post) I had no problems should nearly show that the problem is in some change between cacti 0.8.6i and 0.8.6j.
The thing (I think) is that in php-snmp version 4.3, snmpgetnext is not implemented. I had a look at php source code and, althought I'm not a programmer, I found no mention about it in the snmp module. Instead, in php version 5, snmpgetnext is implemented.
In 0.8.6i we did not use snmpgetnext, so there was no problem even using php 4.3. Now in 0.8.6j, since bug#0000886, we use snmpgetnext. But it is not implemented in php-snmp v4.3, and my cmd.php crashes.

TheWitness wrote:
2) Please verify that you have a valid path the the binary snmpgetnext. Post your findings.


It's in /usr/bin, and it works. But I'm using snmp v1 so in lib/snmp.php I'm using SNMP_METHOD_PHP, and not SNMP_METHOD_BINARY.

TheWitness wrote:
3) Please revert to ".1" and sniff utilizing Wireshark (Ethereal) and PM me the capture file.


I sniffed using the filter 'port 33439 or port 161'. The first being the UDP port used for UDP pinging, the second the SNMP port. Absolutely no packet was captured.
Not surprinsing for me, because of the way cmd.php works: it first pings the host, then it retrieves the data it needs. My cmd.php dies as soon as it tries to ping the host (the first is localhost) with a call to snmpgetnext (which is not implemented in php 4.3, and cmd.php suddenly dies).
If I should sniff using a different filter please ask me. My system is in production, so sniffing without a filter produces a lot of unuseful, and potentially private, infos.
<opinion>
I love network analyzers very much, but unfortunately in this case this kind of tool is of little help, since cmd.php dies just before sending its first packet.
</opinion>

TheWitness wrote:
This works perfectly fine on other systems.


Hmm.. with cacti = 0.8.6j and PHP < 5? Because only in this situation this problem arises.
Another info: I tried to use both the options "Ping and SNMP" and "SNMP" for "Downed Host Detection" in the poller options. Since the SNMP ping is tried in both cases, cmd.php dies in both cases.

On my side, things return to work perfectly if I only apply the following patch that reverts to the old-style (pre-0.8.6j) SNMP ping style if we are on php < 5:

Code:
--- lib/ping.php.lux    2007-01-20 08:49:25.000000000 -0500
+++ lib/ping.php        2007-01-20 12:21:05.000000000 -0500
@@ -289,6 +289,7 @@
                                return false;
                        }

+                       if (PHP_VERSION >= 5) {
                        $output = cacti_snmp_getnext($this->host["hostname"],
                                $this->host["snmp_community"],
                                $oid,
@@ -298,6 +299,18 @@
                                $this->host["snmp_port"],
                                $this->host["snmp_timeout"],
                                SNMP_CMDPHP);
+                       }else{
+                               /* no snmpgetnext here :( */
+                               $output = cacti_snmp_get($this->host["hostname"],
+                                       $this->host["snmp_community"],
+                                       ".1.3.6.1.2.1.1.3.0" ,
+                                       $this->host["snmp_version"],
+                                       $this->host["snmp_username"],
+                                       $this->host["snmp_password"],
+                                       $this->host["snmp_port"],
+                                       $this->host["snmp_timeout"],
+                                       SNMP_CMDPHP);
+                       }

                        /* determine total time +- ~10% */
                        $this->time = $this->get_time($this->precision);


Thank you.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 5:47 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
Looking at the 4.4.4 source, it's there. I will go back to 4.3.9 and check as well.

TheWitness

Code:
      } else if (st >= 2) {
         pdu = snmp_pdu_create(SNMP_MSG_GETNEXT);
         snmp_add_null_var(pdu, name, name_length);
      }

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 5:49 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
It's also in php 4.3.9. Going back even further...

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 5:51 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
It's in 4.3.0

Try the following. Create a and run a test script:

Code:
<?php

$get_next = function_exists("snmpgetnext");

print "The getnext function '" . $get_next . "'\n";

?>


TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 5:52 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
Hmm, even though it's in the source code, it does not appear to be in use.. :(

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 6:02 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
Ok, try this...

TheWitness


Attachments:
ping.zip [3.94 KiB]
Downloaded 554 times

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.
Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 6:30 pm 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
TheWitness wrote:
It's in 4.3.0

Try the following. Create a and run a test script:

Code:
<?php

$get_next = function_exists("snmpgetnext");

print "The getnext function '" . $get_next . "'\n";

?>


TheWitness


The output is:

Content-type: text/html
X-Powered-By: PHP/4.3.9

The getnext function ''




If I change snmpgetnext in snmpget I obtain:

Content-type: text/html
X-Powered-By: PHP/4.3.9

The getnext function '1'


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 6:35 pm 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
TheWitness wrote:
Hmm, even though it's in the source code, it does not appear to be in use.. :(


I noticed that in the php source package, is the file ext/snmp/snmp.c. There I found the array snmp_functions. In this array, looking in php 5, I found snmpgetnext; in php 4.3.9, I did not.
A look to the function php_snmp and to the documentation seems to confirm that snmpgetnext support began in v5 of php.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 6:56 pm 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
TheWitness wrote:
Ok, try this...

TheWitness


This does not work as is, but maybe with
(version_compare("5.0", phpversion(), ">="))
you wanted to say
(version_compare("5.0", phpversion(), "<="))

With this change, the patch works. Similar to my little patch, seems we had the same idea :wink:


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 7:47 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14804
Location: MI, USA
How does this work?

Larry


Attachments:
ping.zip [3.94 KiB]
Downloaded 604 times

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.
Top
 Profile  
 
 Post subject:
PostPosted: Sun Jan 21, 2007 1:12 am 
Offline

Joined: Fri Jan 19, 2007 12:00 pm
Posts: 7
TheWitness wrote:
How does this work?

Larry


Now it works.
Any hope to have this included in the mainstream package?

Thanks.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group