Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Tue May 23, 2017 1:54 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: [RESOLVED] Spine 0.8.7 SVN: returns 0 for some data sources
PostPosted: Thu May 06, 2010 2:15 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
Hi Folks,

I am sure that all of you (particularly TheWitness) are getting tired of me by now ;)

I noticed that after addressing the spine quote trimming bug with the updates in Spine SVN 5871, I was still seeing intermittent gaps in some of my graphs as I had seen in earlier spine SVN versions. After doing some additional digging, it appears that the gaps did not correspond with NaNs, but with 0's that were erroneously being returned during the spine polling cycle.

Here are some samples comparing the same data sources with 0.8.7a with SVN 5871:

0.8.7a with ping.c patch
Code:
[root@DL360-3 0.8.7]# /usr/local/bin/spine-087a-patched -R -V 5 -f 13 -l 13 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[13] DS[20674] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.6, value: 10
Host[13] DS[20671] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 8
Host[13] DS[20672] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 10
Host[13] DS[20673] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 10
Host[13] DS[20670] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 10
Host[13] DS[20669] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 9


Spine SVN 5871 run 1
Code:
[root@DL360-3 0.8.7]# ./spine -R -V 5 -f 13 -l 13 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[13] TH[1] DS[20671] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 0
Host[13] TH[1] DS[20672] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 0
Host[13] TH[1] DS[20673] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 10
Host[13] TH[1] DS[20674] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.6, value: 10
Host[13] TH[1] DS[20670] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 0
Host[13] TH[1] DS[20669] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 0

Note that this version is returning 0 for indexes 1,2,3 and 4 but 5 and 6 are fine.

Spine SVN 5871 run 2
Code:
[root@DL360-3 0.8.7]# ./spine -R -V 5 -f 13 -l 13 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[13] TH[1] DS[20674] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.6, value: 10
Host[13] TH[1] DS[20671] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 8
Host[13] TH[1] DS[20672] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 10
Host[13] TH[1] DS[20673] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 10
Host[13] TH[1] DS[20670] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 0
Host[13] TH[1] DS[20669] SNMP: v2: 10.40.10.1, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 9

This time only index 2 is 0, but the rest are fine.

I have only seen this issue on my larger environment, and have not managed to reproduce it on my smaller dev system (same host/templates/settings/versions):
Code:
[root@brudy-dev1 0.8.7]# ./spine -R -V 5 -f 51 -l 51 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[51] TH[1] DS[549] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 9
Host[51] TH[1] DS[550] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 10
Host[51] TH[1] DS[551] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 8
Host[51] TH[1] DS[552] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 10
Host[51] TH[1] DS[553] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 10
Host[51] TH[1] DS[554] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.6, value: 10


Last edited by Setarcos on Fri May 07, 2010 9:44 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 2:37 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Set MAX_OID's to 1 for this host and continue your testing. If it does not show up there, then I have a good idea where it is, although the code was contributed and would take some time to review.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 2:44 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
Set MAX_OID's to 1 for this host and continue your testing. If it does not show up there, then I have a good idea where it is, although the code was contributed and would take some time to review.

TheWitness


Interesting... Setting the Max OIDs to anything greater than 6 on the larger environment seems to cause this. Previous testing had Max OIDs set for 10 for this host.


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 3:32 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
Another host (353) that exhibited this issue seems to have a different sweet spot for Max OIDs with the SVN spine. Anything greater than 10 causes 0s.

This run has Max OIDs set to 11 (10+1):
Code:
[root@DL360-3 0.8.7]# ./spine -R -V 5 -f 353 -l 353 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[353] TH[1] DS[19892] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.12, value: 13
Host[353] TH[1] DS[19893] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.8, value: 16
Host[353] TH[1] DS[19891] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.25, value: 15
Host[353] TH[1] DS[10562] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.11, value: 48
Host[353] TH[1] DS[10563] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.28, value: 16
Host[353] TH[1] DS[10566] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.9, value: 13
Host[353] TH[1] DS[10567] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.13, value: 15
Host[353] TH[1] DS[10568] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.27, value: 13
Host[353] TH[1] DS[10569] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 51
Host[353] TH[1] DS[10570] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.7, value: 13
Host[353] TH[1] DS[10571] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.16, value: 15
Host[353] TH[1] DS[10572] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 13
Host[353] TH[1] DS[10613] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.32, value: 15
Host[353] TH[1] DS[10614] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.22, value: 13
Host[353] TH[1] DS[10615] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.18, value: 15
Host[353] TH[1] DS[10617] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.29, value: 12
Host[353] TH[1] DS[10618] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.17, value: 14
Host[353] TH[1] DS[10619] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 14
Host[353] TH[1] DS[10621] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 16
Host[353] TH[1] DS[10622] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.21, value: 13
Host[353] TH[1] DS[10625] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.31, value: 14
Host[353] TH[1] DS[10626] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.24, value: 13
Host[353] TH[1] DS[10627] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.23, value: 15
Host[353] TH[1] DS[10629] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.19, value: 50
Host[353] TH[1] DS[10630] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.10, value: 15
Host[353] TH[1] DS[19901] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.14, value: 0
Host[353] TH[1] DS[19902] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.30, value: 13
Host[353] TH[1] DS[19903] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 14
Host[353] TH[1] DS[19904] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.20, value: 16
Note the 0 at index 14.

This time with Max OIDs set to 12 (10+2)
Code:
[root@DL360-3 0.8.7]# ./spine -R -V 5 -f 353 -l 353 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[353] TH[1] DS[19892] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.12, value: 13
Host[353] TH[1] DS[19893] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.8, value: 15
Host[353] TH[1] DS[19891] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.25, value: 14
Host[353] TH[1] DS[10562] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.11, value: 48
Host[353] TH[1] DS[10563] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.28, value: 15
Host[353] TH[1] DS[10566] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.9, value: 13
Host[353] TH[1] DS[10567] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.13, value: 14
Host[353] TH[1] DS[10568] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.27, value: 14
Host[353] TH[1] DS[10569] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 51
Host[353] TH[1] DS[10570] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.7, value: 13
Host[353] TH[1] DS[10571] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.16, value: 15
Host[353] TH[1] DS[10572] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 13
Host[353] TH[1] DS[10613] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.32, value: 14
Host[353] TH[1] DS[10614] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.22, value: 13
Host[353] TH[1] DS[10615] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.18, value: 14
Host[353] TH[1] DS[10617] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.29, value: 13
Host[353] TH[1] DS[10618] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.17, value: 13
Host[353] TH[1] DS[10619] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 14
Host[353] TH[1] DS[10621] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 15
Host[353] TH[1] DS[10622] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.21, value: 13
Host[353] TH[1] DS[10625] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.31, value: 14
Host[353] TH[1] DS[10626] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.24, value: 14
Host[353] TH[1] DS[10627] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.23, value: 14
Host[353] TH[1] DS[10629] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.19, value: 50
Host[353] TH[1] DS[10630] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.10, value: 15
Host[353] TH[1] DS[19901] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.14, value: 0
Host[353] TH[1] DS[19902] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.30, value: 0
Host[353] TH[1] DS[19903] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 13
Host[353] TH[1] DS[19904] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.20, value: 15
Note the two 0s in indexes 14 and 30.

..and finally with max OIDs set to 10:
Code:
[root@DL360-3 0.8.7]# ./spine -R -V 5 -f 353 -l 353 -S | grep .1.3.6.1.4.1.xxxx.1.1.1.9
Host[353] TH[1] DS[19892] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.12, value: 13
Host[353] TH[1] DS[19893] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.8, value: 15
Host[353] TH[1] DS[19891] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.25, value: 14
Host[353] TH[1] DS[10562] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.11, value: 48
Host[353] TH[1] DS[10563] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.28, value: 15
Host[353] TH[1] DS[10566] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.9, value: 13
Host[353] TH[1] DS[10567] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.13, value: 14
Host[353] TH[1] DS[10568] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.27, value: 14
Host[353] TH[1] DS[10569] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.1, value: 51
Host[353] TH[1] DS[10570] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.7, value: 13
Host[353] TH[1] DS[10571] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.16, value: 15
Host[353] TH[1] DS[10572] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.3, value: 12
Host[353] TH[1] DS[10613] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.32, value: 14
Host[353] TH[1] DS[10614] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.22, value: 13
Host[353] TH[1] DS[10615] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.18, value: 14
Host[353] TH[1] DS[10617] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.29, value: 13
Host[353] TH[1] DS[10618] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.17, value: 13
Host[353] TH[1] DS[10619] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.2, value: 14
Host[353] TH[1] DS[10621] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.5, value: 15
Host[353] TH[1] DS[10622] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.21, value: 13
Host[353] TH[1] DS[10625] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.31, value: 14
Host[353] TH[1] DS[10626] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.24, value: 14
Host[353] TH[1] DS[10627] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.23, value: 13
Host[353] TH[1] DS[10629] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.19, value: 50
Host[353] TH[1] DS[10630] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.10, value: 15
Host[353] TH[1] DS[19901] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.14, value: 13
Host[353] TH[1] DS[19902] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.30, value: 14
Host[353] TH[1] DS[19903] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.4, value: 13
Host[353] TH[1] DS[19904] SNMP: v2: 10.248.17.10, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.1.1.9.20, value: 15
No 0s

IMHO this looks like an array over/under run.


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 4:54 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
I finally managed to find a combination that could reproduce this on my smaller environment. (Host 51 on the smaller environment is the same as host 13 on the bigger environment.)

Max OIDs = 5:
Code:
[root@brudy-dev1 0.8.7]# ./spine -R -V 5 -f 51 -l 51 -S | grep .1.3.6.1.4.1.xxxx.1.2.1.3
Host[51] TH[1] DS[556] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.1, value: U
Host[51] TH[1] DS[558] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.2, value: U
Host[51] TH[1] DS[560] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.3, value: U
Host[51] TH[1] DS[562] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.4, value: U
Host[51] TH[1] DS[564] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.5, value: 0
Host[51] TH[1] DS[566] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.6, value: U



Max OIDs = 4:
Code:
[root@brudy-dev1 0.8.7]# ./spine -R -V 5 -f 51 -l 51 -S | grep .1.3.6.1.4.1.xxxx.1.2.1.3
Host[51] TH[1] DS[556] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.1, value: U
Host[51] TH[1] DS[558] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.2, value: U
Host[51] TH[1] DS[560] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.3, value: U
Host[51] TH[1] DS[562] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.4, value: U
Host[51] TH[1] DS[564] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.5, value: 29
Host[51] TH[1] DS[566] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.6, value: U


Note: The U's in the other indexes are expected as no data is returned for these OIDs.

Here is the Tech Support dump for this system:
Code:
General Information

Date    Thu, 06 May 2010 14:52:52 -0700
Cacti Version    0.8.7e
Cacti OS    unix
SNMP Version    NET-SNMP version: 5.3.2.2
RRDTool Version    RRDTool 1.2.x
Hosts    10
Graphs    205
Data Sources    Script/Command: 35
SNMP: 28
SNMP Query: 138
Script Query: 6
Script - Script Server (PHP): 1
Script Query - Script Server: 14
Total: 222


Poller Information

Interval    300
Type    spine
Items    Action[0]: 223
Action[1]: 45
Action[2]: 23
Total: 291
Concurrent Processes    4
Max Threads    10
PHP Servers    2
Script Timeout    120
Max OID    35
Last Run Statistics    Time:3.5638 Method:spine Processes:4 Threads:10 Hosts:11 HostsPerProcess:3 DataSources:291 RRDsProcessed:209


PHP Information

PHP Version    5.1.6
PHP OS    Linux
PHP uname    Linux brudy-dev1 2.6.18-164.15.1.el5xen #1 SMP Wed Mar 17 12:04:23 EDT 2010 x86_64
PHP SNMP    Installed
max_execution_time    30
memory_limit    256M


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 6:40 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Quote:
Interesting... Setting the Max OIDs to anything greater than 6 on the larger environment seems to cause this. Previous testing had Max OIDs set for 10 for this host.


You have to be very careful with Max OID's. If you increase the value, you also have to increase the timeout's. Also, having lot's of OID's with "U" tends to cause issues with polling times. I'll look a little more closely at this.

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 6:41 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Setarcos wrote:
I finally managed to find a combination that could reproduce this on my smaller environment. (Host 51 on the smaller environment is the same as host 13 on the bigger environment.)

Max OIDs = 5:
Code:
[root@brudy-dev1 0.8.7]# ./spine -R -V 5 -f 51 -l 51 -S | grep .1.3.6.1.4.1.xxxx.1.2.1.3
Host[51] TH[1] DS[556] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.1, value: U
Host[51] TH[1] DS[558] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.2, value: U
Host[51] TH[1] DS[560] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.3, value: U
Host[51] TH[1] DS[562] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.4, value: U
Host[51] TH[1] DS[564] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.5, value: 0
Host[51] TH[1] DS[566] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.6, value: U



Max OIDs = 4:
Code:
[root@brudy-dev1 0.8.7]# ./spine -R -V 5 -f 51 -l 51 -S | grep .1.3.6.1.4.1.xxxx.1.2.1.3
Host[51] TH[1] DS[556] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.1, value: U
Host[51] TH[1] DS[558] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.2, value: U
Host[51] TH[1] DS[560] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.3, value: U
Host[51] TH[1] DS[562] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.4, value: U
Host[51] TH[1] DS[564] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.5, value: 29
Host[51] TH[1] DS[566] SNMP: v2: x.x.x.x, dsname: StxProcessorCPU, oid: .1.3.6.1.4.1.xxxx.1.2.1.3.6, value: U


Note: The U's in the other indexes are expected as no data is returned for these OIDs.

Here is the Tech Support dump for this system:
Code:
General Information

Date    Thu, 06 May 2010 14:52:52 -0700
Cacti Version    0.8.7e
Cacti OS    unix
SNMP Version    NET-SNMP version: 5.3.2.2
RRDTool Version    RRDTool 1.2.x
Hosts    10
Graphs    205
Data Sources    Script/Command: 35
SNMP: 28
SNMP Query: 138
Script Query: 6
Script - Script Server (PHP): 1
Script Query - Script Server: 14
Total: 222


Poller Information

Interval    300
Type    spine
Items    Action[0]: 223
Action[1]: 45
Action[2]: 23
Total: 291
Concurrent Processes    4
Max Threads    10
PHP Servers    2
Script Timeout    120
Max OID    35
Last Run Statistics    Time:3.5638 Method:spine Processes:4 Threads:10 Hosts:11 HostsPerProcess:3 DataSources:291 RRDsProcessed:209


PHP Information

PHP Version    5.1.6
PHP OS    Linux
PHP uname    Linux brudy-dev1 2.6.18-164.15.1.el5xen #1 SMP Wed Mar 17 12:04:23 EDT 2010 x86_64
PHP SNMP    Installed
max_execution_time    30
memory_limit    256M


Give me two tcpdumps with this exact scenario. You can PM them to me.

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 06, 2010 7:28 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
Give me two tcpdumps with this exact scenario. You can PM them to me.


PM sent.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 07, 2010 11:17 am 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
Thanks again TheWitness!

SVN 5874 looks good so far on both my small and larger environments. I haven't managed to locate any graph gaps yet with the Max OID settings for the hosts cranked back up to 35.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 07, 2010 1:47 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Please test 5877. I made the algorithm a bit easier for me to swallow. It should work just fine. Keep in mind, if you are polling a bunch of bad OID's, it causes a huge delay in polling, especially if the MAX OID's is high. You should likely stop polling things if they have an error like "noSuch"... in the response.

Maybe I should do that with Cacti ;)

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 07, 2010 1:48 pm 
Offline
Developer
User avatar

Joined: Tue May 14, 2002 5:08 pm
Posts: 14863
Location: MI, USA
Oh, one other thing, when I am finished with this version of spine, you will only be able to run on Cacti 0.8.7f and above. I have not made that change yet, but it's coming.

TheWitness

_________________
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
Gandalfs Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 07, 2010 2:15 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
Please test 5877. I made the algorithm a bit easier for me to swallow. It should work just fine. Keep in mind, if you are polling a bunch of bad OID's, it causes a huge delay in polling, especially if the MAX OID's is high. You should likely stop polling things if they have an error like "noSuch"... in the response.

Maybe I should do that with Cacti ;)

TheWitness


Yea, some of these data sources don't have parity between the indexible input field and the output field. Bad design, I know, but not something I can fix.

I will bear in mind the max OIDs limitations. Some of these hosts are far afield, so the latency is high. Increasing max OIDs has helped a bunch though.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 07, 2010 3:10 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
TheWitness wrote:
Oh, one other thing, when I am finished with this version of spine, you will only be able to run on Cacti 0.8.7f and above. I have not made that change yet, but it's coming.

TheWitness


Understood. As soon as 0.8.7f drops I can put it on my small dev environment, but the bigger environment will need to wait a week or so until any potential issues can be shaken out.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 07, 2010 9:44 pm 
Offline
Cacti User
User avatar

Joined: Mon Dec 13, 2004 2:55 pm
Posts: 145
Location: San Jose, CA
After reverting the 5877 changes with 5885, things are working fine. Marking this issue resolved.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group