[solved] gaps in graph due to never sent requests

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Moderators, Developers

Post Reply
Author
Message
Muffinman
Posts: 17
Joined: Thu Apr 12, 2007 1:48 am

[solved] gaps in graph due to never sent requests

#1 Post by Muffinman » Fri Apr 13, 2007 5:54 am

Hi out there,

(sorry for the long posting!)

I observe some gaps in the graphs. I captured the SNMP and ICMP packages (via tcpdump) and discovered (via wireshark), that in the polling rounds sometimes requests are send late (3-4 minutes after the round began) and sometimes (very rarely) they are never sent.

I'll express it:
I run a polling every 5 minutes. A normal polling contains 10 SNMP and many ICMP requests. The polling begins with 2 sysUpTime- and a ifInOctet-requests. Then follow many ICMP (=PING) packages. The poll ends with 6 mib-2.25.3.x.x.x-requests and a ssCpuRawUser-0-request. The whole poll normaly doesn't need more than one minute.

But sometimes the requests after the PING are sent with a delay of 3-4 minutes (relatively to the beginning of the sequence).

And sometimes they are even never sent. I assume that they would be sent, if not the new polling round would come in the way and therefor they will never be sent.

Is this assumption right? How can I prove or disprove it? If the assumption is right, how can I avoid that? Can I explicitly say in which order cacti the information polls? Can I see in which order cacti polls the information at all?
Perhaps I should mention, that there are <5 server, which are connected via VPN and have average PING times of 800-1100ms.


TIA for your patience and your answers

Kai

P.S.: Did it come clear, what I wanted to explain? :-?
P.P.S.: Shall I post the wireshark screenshots?
P.P.P.S.: Is there a recommendation how the bandwidth, the number of data sources, the number of devices, and the RAM/CPU of the cacti server should be scaled?
Last edited by Muffinman on Wed Apr 25, 2007 4:10 am, edited 1 time in total.

User avatar
gandalf
Developer
Posts: 22375
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Re: gaps in graph due to never sent requests

#2 Post by gandalf » Mon Apr 16, 2007 5:51 am

Muffinman wrote: Perhaps I should mention, that there are <5 server, which are connected via VPN and have average PING times of 800-1100ms.
The number of servers is no problem. But the round trip time is insane. Ensure, that timeout settings cover this timespan. Cacti may declare these hosts as down. Find more information when using logging verbosity "DEBUG". Some help is found at the second link of my signature
Reinhard

User avatar
TheWitness
Developer
Posts: 14834
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#3 Post by TheWitness » Mon Apr 16, 2007 10:02 pm

cmd.php or cactid?

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

Rossco
Cacti User
Posts: 76
Joined: Sat Jan 15, 2005 4:52 pm
Location: London, England

snmp not polling despite successfull pings

#4 Post by Rossco » Tue Apr 17, 2007 4:05 am

Hi

I'm getting exactly this problem, but *only* on three ucd/net hosts (freebsd 6.0) boxes I have just added as hosts. The cacti server is Windows 2003, PHP439, mysql4, cacti 0.8.6i and all other graphs are fine - no gaps.

An ethereal dump on the server combined with a tcpdump on the polled clients shows that the icmp requests and replies are both sent/received by both parties, but no snmp polling follows. Then, occasionally it will start polling snmp correctly. I can have gaps of 5-10 polling cycles, then get a couple of succesfull polls, then back to nothing again. I also find that I get only partial polls, for instance when it pings successfully, then I get a few snmp stats requested and replied(viewed via ethereal/tcpdump) and then nothing more - only say 10 or 12 of 70+ oids are polled. From this point on the debug log displays entries like:

04/17/2007 09:51:14 AM - CACTID: Poller[0] Host[611] DS[26077] WARNING: SNMP timeout detected [8000 ms], ignoring host 'xxx.xxx.xxx.xxx'

I have tried rebuilding the poller cache with no noticable difference.

Any thoughts? Its bizarre that this only apppears to affect the three ucd/net hosts (note that I am polling them via snmpv1). I dont see how the clients can be at fault as I dont see the requests being generated by the server.

Any thoughts are welcome :-)

Cheers

Ross

User avatar
TheWitness
Developer
Posts: 14834
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#5 Post by TheWitness » Tue Apr 17, 2007 10:02 am

Well, if you see this:
04/17/2007 09:51:14 AM - CACTID: Poller[0] Host[611] DS[26077] WARNING: SNMP timeout detected [8000 ms], ignoring host 'xxx.xxx.xxx.xxx'
Then there is no need to rebuild the poller cache. Have you tried the SVN version of cactid "branch_0_8_6"? I suspect a bug. :(

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

Rossco
Cacti User
Posts: 76
Joined: Sat Jan 15, 2005 4:52 pm
Location: London, England

svn version of cactid

#6 Post by Rossco » Thu Apr 19, 2007 3:11 am

Hi Witness

Thanks for your message. I can see the files via the view cvs, but I'm a windows user and have no way to compile them.

Is there a precompiled version anywhere I could try?

Many thanks

Rossco

Muffinman
Posts: 17
Joined: Thu Apr 12, 2007 1:48 am

#7 Post by Muffinman » Mon Apr 23, 2007 2:22 am

Sorry for the late reply.
TheWitness wrote:cmd.php or cactid?
cactid with:
Maximum Concurrent Poller Processes 1
Maximum Threads per Process 10
Number of PHP Script Servers 10
Script and Script Server Timeout Value 250
The Maximum SNMP OID's Per SNMP Get Request 10
Ping Timeout Value 5000


BTW:
My problem is solved!!! 8) :lol: 8)

In the Data Sources -> MyHost -> Advanced Ping I changed the number of ping to the host from 200 to 20.
Apparently the host wasn't able to answer the SNMP requests due to the huge number of pings.

Post Reply