Graph with NaN value and datasource not as expected

Post support questions that directly relate to Linux/Unix operating systems.

Moderators: Moderators, Developers

Post Reply
Author
Message
antokapo
Posts: 47
Joined: Fri Nov 20, 2015 4:06 am

Graph with NaN value and datasource not as expected

#1 Post by antokapo » Wed Oct 16, 2019 6:18 am

Hi

I'm experiencing a crazy behaviour on my cact installation :

General Information
Date Wed, 16 Oct 2019 12:48:20 +0200
Cacti Version 0.8.8b
Cacti OS unix
SNMP Version NET-SNMP version: 5.4.3
RRDTool Version RRDTool 1.4.x
Hosts 1428
Graphs 43500
Data Sources Script/Command: 5
SNMP: 1950
SNMP Query: 124602
Script Query: 3
Script Query - Script Server: 378
Total: 126938
Poller Information
Interval 300
Type SPINE 0.8.8b Copyright 2002-2013 by The Cacti Group
Items Action[0]: 246712
Action[1]: 11
Action[2]: 582
Total: 247305
Concurrent Processes 4
Max Threads 5
PHP Servers 4
Script Timeout 180
Max OID 15
Last Run Statistics Time:111.7686 Method:spine Processes:4 Threads:5 Hosts:1427 HostsPerProcess:357 DataSources:247305 RRDsProcessed:0
PHP Information
PHP Version 5.3.10-1ubuntu3.24
PHP OS Linux
PHP uname Linux netmonitor 3.13.0-96-generic #143~precise1-Ubuntu SMP Tue Aug 30 14:38:06 UTC 2016 x86_64
PHP SNMP Installed
max_execution_time 300
memory_limit 2560M

I created a new graph , but it doesn't populate, all NaN values in the rrd , i suspect that database got corrupted because of many inconstistencies found by the script repair_database.php ,
and also the data source that has been generated is different from the other ones related to the same data template.

I can give further details if you need them.

One thing i've done is to change the data template associated with the graph, maybe this is messing all up.

Thanks for your help

Antonello

antokapo
Posts: 47
Joined: Fri Nov 20, 2015 4:06 am

Re: Graph with NaN value and datasource not as expected

#2 Post by antokapo » Wed Oct 16, 2019 9:52 am

antokapo wrote:
Wed Oct 16, 2019 6:18 am
Hi

I'm experiencing a crazy behaviour on my cact installation :

General Information
Date Wed, 16 Oct 2019 12:48:20 +0200
Cacti Version 0.8.8b
Cacti OS unix
SNMP Version NET-SNMP version: 5.4.3
RRDTool Version RRDTool 1.4.x
Hosts 1428
Graphs 43500
Data Sources Script/Command: 5
SNMP: 1950
SNMP Query: 124602
Script Query: 3
Script Query - Script Server: 378
Total: 126938
Poller Information
Interval 300
Type SPINE 0.8.8b Copyright 2002-2013 by The Cacti Group
Items Action[0]: 246712
Action[1]: 11
Action[2]: 582
Total: 247305
Concurrent Processes 4
Max Threads 5
PHP Servers 4
Script Timeout 180
Max OID 15
Last Run Statistics Time:111.7686 Method:spine Processes:4 Threads:5 Hosts:1427 HostsPerProcess:357 DataSources:247305 RRDsProcessed:0
PHP Information
PHP Version 5.3.10-1ubuntu3.24
PHP OS Linux
PHP uname Linux netmonitor 3.13.0-96-generic #143~precise1-Ubuntu SMP Tue Aug 30 14:38:06 UTC 2016 x86_64
PHP SNMP Installed
max_execution_time 300
memory_limit 2560M

I created a new graph , but it doesn't populate, all NaN values in the rrd , i suspect that database got corrupted because of many inconstistencies found by the script repair_database.php ,
and also the data source that has been generated is different from the other ones related to the same data template.

I can give further details if you need them.

One thing i've done is to change the data template associated with the graph, maybe this is messing all up.

Thanks for your help

Antonello
Even if i simply save the data_template , data_source changes ... that drives me crazy .. probably it's better to upgrade this CACTI instance

antokapo
Posts: 47
Joined: Fri Nov 20, 2015 4:06 am

Re: Graph with NaN value and datasource not as expected

#3 Post by antokapo » Thu Oct 17, 2019 5:38 am

I found a possibile problem when querying the remote Cisco router:

[email protected]:~$ sudo snmpwalk -v 2c -c CiscovieW xx.xxx.xxx.xxx .1.3.6.1.2.1.31.1.1.1.6.53
IF-MIB::ifHCInOctets.53 = No Such Instance currently exists at this OID
[email protected]:~$ sudo snmpwalk -v 2c -c CiscovieW xxx.xxx.xxx.xxx .1.3.6.1.2.1.31.1.1.1.10.53
IF-MIB::ifHCOutOctets.53 = No Such Instance currently exists at this OID
[email protected]:~$

But if i do a check on the router i get :

ifName : GigabitEthernet0/2/0/38 ifIndex : 53

as excerpt of the output of " sh snmp interface " command ...

antokapo
Posts: 47
Joined: Fri Nov 20, 2015 4:06 am

Re: Graph with NaN value and datasource not as expected

#4 Post by antokapo » Thu Oct 17, 2019 6:25 am

and finally i had to restart the mibd_interface process on the router to get it working!!

ONE MAN SHOW

Bye

Antonello

netniV
Cacti Guru User
Posts: 3132
Joined: Sun Aug 27, 2017 12:05 am

Re: Graph with NaN value and datasource not as expected

#5 Post by netniV » Sun Oct 20, 2019 6:32 am

Well done on diagnosing that, I doubt I would have found it as I wasn't even aware you could just restart processes on a router.

antokapo
Posts: 47
Joined: Fri Nov 20, 2015 4:06 am

Re: Graph with NaN value and datasource not as expected

#6 Post by antokapo » Mon Oct 21, 2019 2:12 am

To tell you the truth ,
after i set the poller in DEBUG mode,
I noticed this row :

SPINE: Poller[0] Host[1419] TH[1] DS[362819] SNMP: v2: xxx.xxx.xxx.xxx, dsname: traffic_in, oid: .1.3.6.1.2.1.31.1.1.1.6.53, value: U

and it reminded me to a similar problem we had in the past with the CISCO ASR9K family ,
as a member of the network department i could restart any process :D

Regards

Antonello

netniV
Cacti Guru User
Posts: 3132
Joined: Sun Aug 27, 2017 12:05 am

Re: Graph with NaN value and datasource not as expected

#7 Post by netniV » Mon Oct 21, 2019 5:03 am

Lucky to have that prior knowledge. I was more meaning that I didn't realise you could restart a process within a router without fully rebooting. I just would never image that they'd allow the end user to do that as I've not had a device allowing it. :)

antokapo
Posts: 47
Joined: Fri Nov 20, 2015 4:06 am

Re: Graph with NaN value and datasource not as expected

#8 Post by antokapo » Tue Oct 22, 2019 5:03 am

Hi

Please excuse me, i misunderstood what you wrote.

CISCO IOS XR allow you to have more granularity on processes control, as you can do in UNIX platforms, you can even run a shell on those enterprise level devices.

My team in in charge for interface and statistics graph on many network devices, so we often face that kind of problem ;-)

Regards

Antonello

Post Reply