Universal Log Processing Script (tail) / Bind query log

[white-jeroen]

Hi Guys,

I've written a script in PHP to check what lines are added to a logfile since the last run of the script, and processes them.
The script checks the added lines against the regular expressions set, and increases counters.
These counters can be used in Cacti as 'DERIVE'.

In the attached tgz I use the script to process the Bind querylog.
To be able to use the script for an other log, you need to understand regular epressions.

If you want to use this script exactly as provided please edit 'bindlogstats.php' and check if the settings match your configuration, especially the dirs and files.

I hope the script is usefull for someone else. Please let me know.

Kind regards, Jeroen.

[white-jeroen]

I'll add a Graph Template export from Cacti of my Graph, so if someone wants to use this script exactly as is, you can use this template.

I didn't create the graph all by myself, I have customized a template I did find somewhere on this forum.

The graph looks like this:

[white-jeroen]

Installation Tips

• Go to the "scripts" dir of your Cacti installation
  
• Download the TGZ
  
• # tar -xpzf bindlogstats.tgz
  
• # mkdir data
  
• Edit the bindlogstats.php and update the location of the Bind query log file
  
• Also check the if the path of the data dir is correct
  
• Make sure the query log is readable by the user that executes the Cacti cron job
  
• Make sure the data dir is writable by the user that executes the Cacti cron job
  
• Test the script: # ./bindlogstats.php
  
• The script creates a file in the data directory. Make sure it is writable by the user that executes the Cacti cron job
  
• If you don't have a Bind query log file, add the following to you Bind config:
  

  Code:

  logging {   channel queries_file_cacti { file "/var/log/named/queries_cacti.log" versions 3 size 5m; };   category queries           { queries_file_cacti; }; };

  
  Make sure the user 'named' can write tot the queries_cacti.log file.
  Configure the same file in bindlogstats.php as $sLogFile.
  

[/b]

[sizulku]

Hi White-jeroen,

Thanks for the script and template. But it won't work on my system. I've check the query log is readable and data dir is writeable. Got this result below

[white-jeroen]

I uploaded a new version of the TGZ with updated regular expressions to match your query log.

[sizulku]

Thanks for upcate. But got error dependency while importing to cacti-0.8.7b.

[white-jeroen]

Ai... I don't know how to fix this at the moment. I exported my graph from Cacti Version 0.8., with dependencies. Maybe I'll test the export on a clean install and find out how to fix it later.

Edit: I have found the problem. Some of the datafields were connected to the wrong data input method. I updated the XML. Have fun.

[white-jeroen]

I've written an [HOWTO] to be able to use this script to query a remote server. You need to put the scripts from the TGZ on the remote server, and you need to change the Data Input Method from the XML Import to match the settings explained in the Howto.
http://forums.cacti.net/viewtopic.php?p=130433

[white-jeroen]

Sizulku told me he had problems using the script with Spine, so I took some time figuring out what the problem was. I discovered Spine doesn't like a line-end after the output, so I removed it. I uploaded a new version of bindlogstats.tgz (V6).

[mkoninkx]

Hey White-Jeroen,

First of all, thanks for creating the script!

I've got a small problem with it though, I'm using the script over a SSH RSA tunnel, all outputs give numbers, but the Graph in Cacti show only all that have ever been. When I check the output from the script, the Graph matched all the numbers in there. I've attached a screenshot of my graph, maybe you can have a look at it? and tell me in which way I have to think to solve this.

Second question:
How was your script designed? I'm searching for an explaination for the numbers generated by the bindlogstats.php, shouldn't the numbers output by the script be the difference with the last poll? I'm a bit confused here.

Thanks for you help!

[white-jeroen]

Hello mkoninkx,

I will first answer your first question:

I see the numbers in your graph only increasing, never decreasing.
I think you need to check the following setting:

-Log into Cacti as Admin
-Click "Data Templates" from the right menu
-Click the right Data Template
-Look at the lower half of the page
-For each "Data Source Item" check the following setting:
-The "Data Source Type" must be "DERIVE"

[edit: answer to second question]

I think this is also the answer to your first question:

A good way to measure increasing values is using the "DERIVE".
When a DNS request is done, for example an A-record, the value is increased by one.
When Cacti asks for an update you just the current value is sent.
Cacti compares it to the last value it received, and devides the difference by the number of seconds passed between the previous measure. Now Cacti knows the average number of DNS request per second during this period. It is no problem when the value gets reset, for example once a day, Cacti detects this and skips that measure.

Kind regards,
Jeroen