problem with rrd - nan

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Moderators, Developers

Post Reply
Author
Message
radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

problem with rrd - nan

#1 Post by radczenko » Fri Mar 01, 2019 7:10 am

Hello.
I write this, because i'm going to make headshout myself
I have a strong will finally do it monitoring.
Since 2 weeks I can't create a working graph on my cacti.
I have read dozens of help pages / problems and the proposed diagnosis - and I am in the corner

My Cacti don't make graphs - but working on realtime. Now I know i have problem with rrd.

What I have:
Router on ip address 10.2.7.254 with snmp v2 - it's Mikrotik CCR - I using tamplate - but it's not important i thing.
i'm ok with permission becaus my rrd file ar update

rrdtool last czesiek_traffic_in_65.rrd
1551440402

date +%s
1551440628


Snmp it's ok to:

Cacti read device info : System: RouterOS CCR1036-12G-4S
Uptime: 46869900 (5days, 10hours, 11minutes)

Cacti something reading to (Cacti/data source/edit:
traffic_in COUNTER 120 0.00 953.67 M 894.29 M NaN 2.00
traffic_out COUNTER 120 0.00 953.67 M 3.29 G NaN 2.00

snmpwalk and snmpget from shell reading snmp from this device good.

cacti log :
2019-03-01 13:00:02 - SYSTEM STATS: Time:1.1530 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:2 RRDsProcessed:1
2019-03-01 13:00:02 - POLLER: Poller[1] Time: 0.2322 s, Poller: , Theads: N/A, Devices: 1
2019-03-01 13:00:02 - PING Device[7] Description[czesiek] SNMP: Device responded to SNMP
2019-03-01 13:00:01 - POLLER: Poller[1] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '300.67', Max Runtime '298', Poller Runs: '1'


But problem is someware on rrd.
I have -Nan value

rdtool info czesiek_traffic_in_65.rrd filename = "czesiek_traffic_in_65.rrd"
rrd_version = "0003"
step = 60
last_update = 1551441901
header_size = 4940
ds[traffic_in].index = 0
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 120
ds[traffic_in].min = 0,0000000000e+00
ds[traffic_in].max = 1,0000000000e+09
ds[traffic_in].last_ds = "1930266784"
ds[traffic_in].value = NaN
ds[traffic_in].unknown_sec = 1
ds[traffic_out].index = 1
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 120
ds[traffic_out].min = 0,0000000000e+00
ds[traffic_out].max = 1,0000000000e+09
ds[traffic_out].last_ds = "870095790"
ds[traffic_out].value = NaN
ds[traffic_out].unknown_sec = 1
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].cur_row = 35
rra[0].pdp_per_row = 1
rra[0].xff = 5,0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].cur_row = 115
rra[1].pdp_per_row = 6
rra[1].xff = 5,0000000000e-01
rra[1].cdp_prep[0].value = 0,0000000000e+00
rra[1].cdp_prep[0].unknown_datapoints = 5
rra[1].cdp_prep[1].value = 0,0000000000e+00
rra[1].cdp_prep[1].unknown_datapoints = 5
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].cur_row = 392
rra[2].pdp_per_row = 24
rra[2].xff = 5,0000000000e-01
rra[2].cdp_prep[0].value = 0,0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 5
rra[2].cdp_prep[1].value = 0,0000000000e+00
rra[2].cdp_prep[1].unknown_datapoints = 5
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].cur_row = 186
rra[3].pdp_per_row = 288
rra[3].xff = 5,0000000000e-01
rra[3].cdp_prep[0].value = 0,0000000000e+00
rra[3].cdp_prep[0].unknown_datapoints = 149
rra[3].cdp_prep[1].value = 0,0000000000e+00
rra[3].cdp_prep[1].unknown_datapoints = 149
rra[4].cf = "MIN"
rra[4].rows = 600
rra[4].cur_row = 334
rra[4].pdp_per_row = 1
rra[4].xff = 5,0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[4].cdp_prep[1].value = NaN
rra[4].cdp_prep[1].unknown_datapoints = 0
rra[5].cf = "MIN"
rra[5].rows = 700
rra[5].cur_row = 220
rra[5].pdp_per_row = 6
rra[5].xff = 5,0000000000e-01
rra[5].cdp_prep[0].value = inf
rra[5].cdp_prep[0].unknown_datapoints = 5
rra[5].cdp_prep[1].value = inf
rra[5].cdp_prep[1].unknown_datapoints = 5
rra[6].cf = "MIN"
rra[6].rows = 775
rra[6].cur_row = 321
rra[6].pdp_per_row = 24
rra[6].xff = 5,0000000000e-01
rra[6].cdp_prep[0].value = inf
rra[6].cdp_prep[0].unknown_datapoints = 5
rra[6].cdp_prep[1].value = inf
rra[6].cdp_prep[1].unknown_datapoints = 5
rra[7].cf = "MIN"
rra[7].rows = 797
rra[7].cur_row = 436
rra[7].pdp_per_row = 288
rra[7].xff = 5,0000000000e-01
rra[7].cdp_prep[0].value = inf
rra[7].cdp_prep[0].unknown_datapoints = 149
rra[7].cdp_prep[1].value = inf
rra[7].cdp_prep[1].unknown_datapoints = 149
rra[8].cf = "MAX"
rra[8].rows = 600
rra[8].cur_row = 399
rra[8].pdp_per_row = 1
rra[8].xff = 5,0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[8].cdp_prep[1].value = NaN
rra[8].cdp_prep[1].unknown_datapoints = 0
rra[9].cf = "MAX"
rra[9].rows = 700
rra[9].cur_row = 23
rra[9].pdp_per_row = 6
rra[9].xff = 5,0000000000e-01
rra[9].cdp_prep[0].value = -inf
rra[9].cdp_prep[0].unknown_datapoints = 5
rra[9].cdp_prep[1].value = -inf
rra[9].cdp_prep[1].unknown_datapoints = 5
rra[10].cf = "MAX"
rra[10].rows = 775
rra[10].cur_row = 547
rra[10].pdp_per_row = 24
rra[10].xff = 5,0000000000e-01
rra[10].cdp_prep[0].value = -inf
rra[10].cdp_prep[0].unknown_datapoints = 5
rra[10].cdp_prep[1].value = -inf
rra[10].cdp_prep[1].unknown_datapoints = 5
rra[11].cf = "MAX"
rra[11].rows = 797
rra[11].cur_row = 121
rra[11].pdp_per_row = 288
rra[11].xff = 5,0000000000e-01
rra[11].cdp_prep[0].value = -inf
rra[11].cdp_prep[0].unknown_datapoints = 149
rra[11].cdp_prep[1].value = -inf
rra[11].cdp_prep[1].unknown_datapoints = 149
rra[12].cf = "LAST"
rra[12].rows = 600
rra[12].cur_row = 535
rra[12].pdp_per_row = 1
rra[12].xff = 5,0000000000e-01
rra[12].cdp_prep[0].value = NaN
rra[12].cdp_prep[0].unknown_datapoints = 0
rra[12].cdp_prep[1].value = NaN
rra[12].cdp_prep[1].unknown_datapoints = 0
rra[13].cf = "LAST"
rra[13].rows = 700
rra[13].cur_row = 626
rra[13].pdp_per_row = 6
rra[13].xff = 5,0000000000e-01
rra[13].cdp_prep[0].value = NaN
rra[13].cdp_prep[0].unknown_datapoints = 5
rra[13].cdp_prep[1].value = NaN
rra[13].cdp_prep[1].unknown_datapoints = 5
rra[14].cf = "LAST"
rra[14].rows = 775
rra[14].cur_row = 560
rra[14].pdp_per_row = 24
rra[14].xff = 5,0000000000e-01
rra[14].cdp_prep[0].value = NaN
rra[14].cdp_prep[0].unknown_datapoints = 5
rra[14].cdp_prep[1].value = NaN
rra[14].cdp_prep[1].unknown_datapoints = 5
rra[15].cf = "LAST"
rra[15].rows = 797
rra[15].cur_row = 8
rra[15].pdp_per_row = 288
rra[15].xff = 5,0000000000e-01
rra[15].cdp_prep[0].value = NaN
rra[15].cdp_prep[0].unknown_datapoints = 149
rra[15].cdp_prep[1].value = NaN
rra[15].cdp_prep[1].unknown_datapoints = 149



Could give me some tips whare searching problem ?
I have some system problem with snmp
snmpget -v2c -c public 10.2.7.254 1.3.6.1.2.1.4.24.3.0
Cannot find module (NET-SNMP-PASS-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-EXTEND-MIB): At line 0 in (none)
Cannot find module (SNMP-NOTIFICATION-MIB): At line 0 in (none)
Cannot find module (SNMPv2-TM): At line 0 in (none)
Cannot find module (NET-SNMP-VACM-MIB): At line 0 in (none)
iso.3.6.1.2.1.4.24.3.0 = Gauge32: 1466040

but snmp reads the data : iso.3.6.1.2.1.4.24.3.0 = Gauge32: 1466040 at last line.


radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#3 Post by radczenko » Mon Mar 04, 2019 6:40 am

Hello,
thx for replay.
30 min ego I delate/add device and create 64bit graph, but it's still don't working. I still have empty grap with -nan value


radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#5 Post by radczenko » Mon Mar 04, 2019 12:32 pm

No, I have only one graph for solving the problem but before that I had 30 of them - and all graphs are empty with -nan value.

Except for the local machine graph (ram,cpu, disk) - they normally drew - of course they did not use snmp.

netniV
Cacti Guru User
Posts: 2928
Joined: Sun Aug 27, 2017 12:05 am

Re: problem with rrd - nan

#6 Post by netniV » Mon Mar 04, 2019 2:46 pm

If you put the device into debug mode, do you get any info in the logs?

radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#7 Post by radczenko » Mon Mar 04, 2019 5:37 pm

2019-03-04 23:35:03 - SYSTEM STATS: Time:1.1756 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:2 RRDsProcessed:1
2019-03-04 23:35:02 - POLLER: Poller[1] Time: 0.5528 s, Poller: , Theads: N/A, Devices: 1
2019-03-04 23:35:02 - POLLER: Poller[1] Device[8] Description[routerek] DS[67] Graphs['routerek - Traffic - sfp1-S-NET'] SNMP: v2: 10.2.7.254, dsname: traffic_out, oid: .1.3.6.1.2.1.31.1.1.1.10.1, output: 2631385801837
2019-03-04 23:35:02 - POLLER: Poller[1] Device[8] Description[routerek] DS[67] Graphs['routerek - Traffic - sfp1-S-NET'] SNMP: v2: 10.2.7.254, dsname: traffic_in, oid: .1.3.6.1.2.1.31.1.1.1.6.1, output: 22736922078423
2019-03-04 23:35:02 - POLLER: Poller[1] Device[8] Description[routerek] RECACHE DQ[1] OID: .1.3.6.1.2.1.1.3.0, output: 76664400
2019-03-04 23:35:02 - POLLER: Poller[1] Device[8] Description[routerek] RECACHE: Processing 1 items in the auto reindex cache for '10.2.7.254'.
2019-03-04 23:35:02 - POLLER: Poller[1] Device[8] Description[routerek] STATUS: Device '10.2.7.254' is UP.
2019-03-04 23:35:02 - PING Device[8] Description[routerek] SNMP: Device responded to SNMP
2019-03-04 23:35:01 - POLLER: Poller[1] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '300.9', Max Runtime '298', Poller Runs: '1'

netniV
Cacti Guru User
Posts: 2928
Joined: Sun Aug 27, 2017 12:05 am

Re: problem with rrd - nan

#8 Post by netniV » Tue Mar 05, 2019 7:05 am

What do you get if you use the Data Source Info / Debug options when editing the data source?

radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#9 Post by radczenko » Tue Mar 05, 2019 4:37 pm

Data Source Debug
/usr/bin/rrdtool create \
/var/www/cacti/rra/routerek_traffic_in_67.rrd \
--step 60 \
DS:traffic_in:COUNTER:120:0:1000000000 \
DS:traffic_out:COUNTER:120:0:1000000000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MIN:0.5:1:600 \
RRA:MIN:0.5:6:700 \
RRA:MIN:0.5:24:775 \
RRA:MIN:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \
RRA:LAST:0.5:1:600 \
RRA:LAST:0.5:6:700 \
RRA:LAST:0.5:24:775 \
RRA:LAST:0.5:288:797 \

Data source info in attach
Attachments
cacti_data_info.png
cacti_data_info.png (66.64 KiB) Viewed 601 times

radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#10 Post by radczenko » Fri Mar 08, 2019 12:56 pm

netniV wrote:What do you get if you use the Data Source Info / Debug options when editing the data source?
Hello,
DO You have any idea ?

netniV
Cacti Guru User
Posts: 2928
Joined: Sun Aug 27, 2017 12:05 am

Re: problem with rrd - nan

#11 Post by netniV » Sat Mar 09, 2019 10:49 am

I believe your issue is a value of 2,631,385,801,837 vs 953,000,000

You have basically tried to put a 2 TB value into something that maxes at less than 1GB

User avatar
Osiris
Cacti Pro User
Posts: 863
Joined: Mon Jan 05, 2015 10:10 am

Re: problem with rrd - nan

#12 Post by Osiris » Sat Mar 09, 2019 11:32 am

The issue is that the max value is incorrect. Likely that the default was bad. Update to develop and see if the recommendations on the Data Source Info mode tell you it's broken.
Before history, there was a paradise, now dust.

radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#13 Post by radczenko » Mon Mar 11, 2019 4:46 pm

ok, but why I have TB ? it's seem snmp reading total trafic bytes - not bit/s

radczenko
Posts: 9
Joined: Thu Feb 28, 2019 6:58 pm

Re: problem with rrd - nan

#14 Post by radczenko » Mon Mar 11, 2019 6:30 pm

I have modified in templates/data source/interface-traffic
Maximum Value ("U" for No Maximum)
The maximum value of data that is allowed to be collected.
I set "U"

and
Data Source Type
How data is represented in the RRA.

I change counter/guage

And it's worked a few time

#rrdtool fetch routerek_traffic_in_73.rrd AVERAGE

1552343400: 1,1431833971e+06 -nan
1552343700: 1,6867333433e+06 -nan
1552344000: 1,5315022703e+06 -nan
1552344300: 9,4801562813e+05 -nan
1552344600: 5,7662011007e+05 -nan
1552344900: 4,0040436662e+05 -nan
1552345200: 1,4729121616e+06 -nan
1552345500: 1,8519644961e+06 -nan
1552345800: 2,6898732031e+05 8,4179177000e+07
1552346100: 4,2258613433e+05 2,2366400422e+08
1552346400: 9,4610464345e+05 3,4260649308e+08
1552346700: -nan -nan

Post Reply