Virtual counter64 SNMP support

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Post Reply
Author
Message
wheelear
Posts: 2
Joined: Fri Oct 21, 2005 1:43 pm

Virtual counter64 SNMP support

#1 Post by wheelear » Fri Oct 21, 2005 1:52 pm

Most of the computers I monitor with Cacti do not seem to support 64bit counters. And, the switches I have which do support them seem to have a bug that makes the data unreliable.
So, I decided to fix this by writing a server that would run the needed queries once every 30 seconds, keeping track of the data using a 64bit counter, so I could effectively monitor gigabit interfaces.
Then when Cacti wanted the data, present it the locally maintained 64bit value.

This has been imlemented using perl for the server, and MySQL to hold the bits to query.

Is this something anyone else would be interested in?

The way it works now, when SNMP queries for either of the 2 counter64 interface OIDs are generated, they are converted to the 32bit queries, and the results stored in the database, which the server then updates every 30 seconds.

So, now I have counter64 support for all my systems.

If there is interest, I'll make the patch/server available here.

klavs
Posts: 18
Joined: Tue Oct 01, 2002 8:36 am

#2 Post by klavs » Thu Dec 01, 2005 3:05 am

I would very much like that patch/script. Have all Gigabit here interfaces here - and no graphs that works with 64-bit counters :(

wheelear
Posts: 2
Joined: Fri Oct 21, 2005 1:43 pm

#3 Post by wheelear » Fri Dec 09, 2005 11:42 am

Ok, here you go.
There are 4 files in the tarball:
SnmpProxy2.pl - The Perl script which runs a query every 30 seconds
snmp_counter64.sql - MySQL Code to put a new table in the database
cacti64php.tgz - Tarball containing replacement php files
README - Installation instructions and notes

Let me know if you have any questions.
As it says in the README, this should only affect queries that are for the 64bit counters. Your current 32bit counters should remain unaffected.

You must also be using the cmd.php poller. I think I'm going to modify cactid, but for now, it requires cmd.php.
Attachments
cacti64.tar.gz
Adds virtual 64bit counter support to cacti.
(18.56 KiB) Downloaded 1431 times

prjctgeek
Posts: 1
Joined: Fri Jan 13, 2006 6:12 pm

recompile net-snmp?

#4 Post by prjctgeek » Fri Jan 13, 2006 6:24 pm

very Creative workaround..

While looking at this idea I checked out the net-snmp site and discovered that the snmpd that comes with RH/Fedora(others?) doesn't have a flag that enables the IfXTable (64bit counters for interfaces). Reconfigure/recompile with the --enable-mfd-rewrites flag and eliminate the linux hosts from your problem list.

look at: http://www.net-snmp.org/about/ChangeLog.html.

snippet:
Ports:
Linux:
- new experimental tables/rewrites for Linux, including:
ifTable, ifXTable, inetCidrRouteTable, ipCidrRouteTable,
ipAddressTable, ipSystemStatsTable, ipNetToPhysicalTable.
- Enable these talbles by specifying --enable-mfd-rewrites to
configure.

fletch
Cacti User
Posts: 132
Joined: Mon Oct 06, 2003 5:40 pm
Location: Stanford, CA

#5 Post by fletch » Sun Jan 15, 2006 11:04 pm

I'm about to try this, but am not clear on how to "tell cacti that you want to collect 64bit network statistics for some servers" - from your README.

How do I tell cacti to collect the 64 bit counters?

thanks,
fletch.

fletch
Cacti User
Posts: 132
Joined: Mon Oct 06, 2003 5:40 pm
Location: Stanford, CA

#6 Post by fletch » Sun Jan 15, 2006 11:55 pm

answer: choose the DataSource and update bottom dropdown list to "In/Out Bits (64 Bit counters)

its working great - brilliant workaround - although I am afraid to hammer everything every 30 seconds...

thanks,
fletch

fletch
Cacti User
Posts: 132
Joined: Mon Oct 06, 2003 5:40 pm
Location: Stanford, CA

#7 Post by fletch » Tue Jan 17, 2006 2:32 pm

Ok, there is a problem with some idle servers.

the graphs for these servers using your proxy script are showing a steady level of > 100Mbit traffic when there is really next to none.

I've guessing the is due to the logic in the script assuming the initial readings wrapped - when really they are not?

Any fix for this? (reset the DB counter?)

Below is an example of the issue - one reading taken from an unmodified 32 bit cacti installation - the other from the 64-bit SnmpProxy workaround modified version. The network is not carrying 200Mbit on this server...its not computing this correctly.

thanks,
fletch.
Attachments
ds32.png
32-bit counter
ds32.png (6.72 KiB) Viewed 31145 times
ds64.png
64-bit SnmpProxy workaround
ds64.png (29.06 KiB) Viewed 31145 times

fletch
Cacti User
Posts: 132
Joined: Mon Oct 06, 2003 5:40 pm
Location: Stanford, CA

#8 Post by fletch » Tue Jan 17, 2006 6:58 pm

The first graph is 32 bit counter (pre-patch) the second is the same web server network traffic graph post 64-bit patch - something is not right in the 64 bit version
Attachments
web04post.png
64 bit (not OK)
web04post.png (37.97 KiB) Viewed 31131 times
web04pre.png
32 bit (OK)
web04pre.png (55.03 KiB) Viewed 31131 times

fletch
Cacti User
Posts: 132
Joined: Mon Oct 06, 2003 5:40 pm
Location: Stanford, CA

#9 Post by fletch » Wed Jan 18, 2006 2:09 am

Figured it out

the last32 field in the snmp_counter64 table is an int(32) which does not accept the updates from the pl script (mysql warns - data truncated) - therefore the subsequent differences between current and last grow and grow .
The fix - re-do the last32 as a varchar(20):

drop table snmp_counter64;
create table snmp_counter64(
host varchar(20) not null,
community varchar(20) not null,
oid varchar(255) not null,
version int default 1,
last32 varchar(20) not null default '0',
last64 varchar(20) not null default '0'
);

update the .pl update line to have single quotes around the last32 value:

$Query="UPDATE snmp_counter64 SET last32='$current32', last64='$last64' WHERE host='$host' AND oid='$oid'";

Cheers!
Fletch.

kbartoletta
Cacti User
Posts: 146
Joined: Tue Oct 04, 2005 4:20 pm
Location: suwanee, ga
Contact:

#10 Post by kbartoletta » Tue Apr 04, 2006 1:33 pm

Just to chime in here.... along these same lines, I am trying to graph some items on an F5 BigIP load balencer - all of the values are reported in counter64 data type (64 bit counter) - Am I hearing from your discussion above that there is no way to tell cacti that these are 64 bit counters??? Currently, my graphs are all crazy and inaccurate...

robn
Posts: 9
Joined: Wed Jan 25, 2006 3:33 pm

#11 Post by robn » Wed May 03, 2006 3:01 am

There seems to be a bug somewhere - I ended up with 1000s of duplicate rows for one hosts - enough to ensue mysql took up 90% of the cpu time on a dual P3 1Ghz server :o

Anyway for now I've added a primary key to the database untill I can look through the code.

Code: Select all

CREATE TABLE `snmp_counter64` (
  `host` varchar(20) NOT NULL default '',
  `community` varchar(20) NOT NULL default '',
  `oid` varchar(255) NOT NULL default '',
  `version` int(11) default '1',
  `last32` varchar(20) NOT NULL default '0',
  `last64` varchar(20) NOT NULL default '0',
  PRIMARY KEY  (`host`,`oid`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8
The only obvious difference is that its an SNMP v1 host with a FQDN

robn
Posts: 9
Joined: Wed Jan 25, 2006 3:33 pm

#12 Post by robn » Thu May 04, 2006 6:06 am

It def doesn't like this server - everytime the poller runs I'm getting this error.

SQL Exec Failed "INSERT INTO snmp_counter64(host, community, oid, version, last32, last64) VALUES('ilab.midyorks.nhs.uk:161', 'public', '.1.3.6.1.2.1.2.2.1.16.2', 1, 108951049, 108951049)"

The graph for this host still works fine - but it would be nice to get to the bottom of this and find out why this is happening

robn
Posts: 9
Joined: Wed Jan 25, 2006 3:33 pm

#13 Post by robn » Fri Jul 28, 2006 7:34 am

I've not got arround to fixing that bug yet, but I have made a few changes to the script

Basically I've changed it so it will run every ~60seconds no matter how long it takes to poll all the devices.

If it takes longer than 60seconds to poll the devices it will simply repeat as soon as its finished.

Why? Will we had an issue with it taking too long with the 30 seconds pause in it. I removed the pause and it was fine, untill this week when it started to take about 8seconds and was constantly updating the database.

These changes should mean that it will repeat striaght away when devices are slow to respond, but still delay things when all devices are responding normally.
Attachments
SnmpProxy2.pl.txt
(1.39 KiB) Downloaded 876 times

robn
Posts: 9
Joined: Wed Jan 25, 2006 3:33 pm

#14 Post by robn » Tue Sep 26, 2006 7:14 am

I've made a change to the table to reduce disk i/o by using the heap storage method. All the data is lost on mysql restart, but this has little impact on the graphs (plus solves the issue of deleted devices not getting removed from the database).
alter table cacti.snmp_counter64 engine=heap;

Faruk
Posts: 10
Joined: Tue Apr 04, 2006 8:20 am

#15 Post by Faruk » Mon Oct 09, 2006 11:25 pm

robn wrote:I've made a change to the table to reduce disk i/o by using the heap storage method. All the data is lost on mysql restart, but this has little impact on the graphs (plus solves the issue of deleted devices not getting removed from the database).
alter table cacti.snmp_counter64 engine=heap;

Hi,

I have loaded all the scripts/files for counter 64 bit support. I want to poll some specific OID using 'SNMP-Generic OID Template'. Problem is, I can not figure out any way to tell cacti to use the data as 64 bit counter.

Need help...

Faruk

Post Reply