Syslog monitor addon beta

General discussion about Plugins for Cacti

Moderators: Moderators, Developers

Post Reply
Author
Message
harlequin
Posts: 13
Joined: Wed Nov 09, 2005 11:46 pm

Syslog monitor addon beta

#1 Post by harlequin » Thu Nov 10, 2005 10:16 am

<Modified 2005-12-03> ver 0.1.2b has been posted - please check further down in this topic

h.aloe is a modified version of sidewinder's aloe addon.
It's been completely revamped and updated to work with Cacti 8.6g

In brief, it's a Cacti addon that provides a color-coded, searchable front-end for a mysql syslog / eventlog database [The database can be populated by Kiwi's syslog daemon, syslog-ng, etc...].
It includes an option to integrate with Cacti's graph timespan, so you can correlate graphed items with syslog events, and has an option to output filtered data to a comma delimited text file.
  • Installation Level: (Easy)
    Installation Time: 5 Minutes
    Files To Edit: 4
thanks to sidewinder for the original aloe: http://forums.cacti.net/viewtopic.php?t=3993

This is a beta version. Comments, criticisms, additions, etc. are welcome, but don't blame me if it breaks something (unlikely) or doesn't work (more likely) ;)

Hope it's useful to someone. Cheers,
Harlequin
Attachments
h.aloe_v0.1.1b.zip
old version - please check further down in this topic for new files
(19.74 KiB) Downloaded 2350 times
h.aloe_v0.1.2b.jpg
pic is ver 0.1.2b - please check further down in this topic for new files
h.aloe_v0.1.2b.jpg (127.59 KiB) Viewed 64498 times
Last edited by harlequin on Sat Dec 03, 2005 3:50 am, edited 1 time in total.
mrmee, mrmee, mrmee...

User avatar
TheWitness
Developer
Posts: 14817
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#2 Post by TheWitness » Thu Nov 10, 2005 10:57 am

Niiiiicccceee. Can we integrate into the full product?

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

Phobos182
Cacti User
Posts: 65
Joined: Tue Sep 21, 2004 2:22 pm
Location: Madison, WI

#3 Post by Phobos182 » Thu Nov 10, 2005 11:04 am

Bravo.
[size=84][color=red] Electronic Frontier Foundation[/color]
[url]http://eff.org[/url][/size]

harlequin
Posts: 13
Joined: Wed Nov 09, 2005 11:46 pm

#4 Post by harlequin » Thu Nov 10, 2005 11:29 am

TheWitness wrote:Niiiiicccceee. Can we integrate into the full product?

TheWitness
Absolutely. You may want to check the code over - it's pobably a mite bit sloppy... Several things could be simplified/improved with better integration, but I tried to modify Cacti files as little as possible. Glad you like it
Harlequin
mrmee, mrmee, mrmee...

User avatar
rony
Developer/Forum Admin
Posts: 6016
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

#5 Post by rony » Thu Nov 10, 2005 11:51 am

Accually, I will be contacting you about writing it as a plugin for 0.9.0.

No planned intergration into 0.8.6.

That doesn't stop you from offering it as a addon for 0.8.6, just will not be intergrated into the 0.8.6 code tree.. :)
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]

tommyj
Posts: 43
Joined: Thu Jun 23, 2005 5:16 pm
Location: Stockholm, Sweden

#6 Post by tommyj » Thu Nov 10, 2005 6:15 pm

Looks amazing! :D I got it up&running but I don't get the nice color coding, how do I get that? I'm using syslog-ng for information.

Also, how about some tail -f function, would that be possible to implement?

harlequin
Posts: 13
Joined: Wed Nov 09, 2005 11:46 pm

#7 Post by harlequin » Thu Nov 10, 2005 9:09 pm

rony wrote:No planned intergration into 0.8.6.
That was my assumption. Let me know about the 0.9.0 plugin :D
tommyj wrote:Looks amazing! :D I got it up&running but I don't get the nice color coding, how do I get that?
Thanks much. Edit the ./include/haloe-config.php file and change the ["names"] in the color section to match what shows in your 'priorities' dropdown. For example, if you have a priority listed as 'emerg', then change
$haloe_colors["Emergency"] = "FF0000"; to
$haloe_colors["emerg"] = "FF0000";
Let me know if that helps.
tommyj wrote:Also, how about some tail -f function, would that be possible to implement?
Hmmm. As it reads and sorts from a database, not really, but it basically does the same thing with the meta-refresh. You could shorten the refresh time to reload the page every couple seconds - currently it pulls this from whatever you have set for your graph refresh time - I've included files with changes for a separate setting for the syslogs refresh rate - just replace the files and edit the new setting in haloe-config.php to your liking.
Harlequin
Attachments
update_refresh.zip
no longer needed - please check further down in this topic for new files
(3.52 KiB) Downloaded 1008 times
Last edited by harlequin on Sat Dec 03, 2005 3:51 am, edited 3 times in total.
mrmee, mrmee, mrmee...

tommyj
Posts: 43
Joined: Thu Jun 23, 2005 5:16 pm
Location: Stockholm, Sweden

#8 Post by tommyj » Fri Nov 11, 2005 2:21 am

Thanks much. Edit the ./include/haloe-config.php file and change the ["names"] in the color section to match what shows in your 'priorities' dropdown. For example, if you have a priority listed as 'emerg', then change
$haloe_colors["Emergency"] = "FF0000"; to
$haloe_colors["emerg"] = "FF0000";
Let me know if that helps.
Yes, that helped, an easy one. Don't know how I could miss that :oops: . Thanks a lot!

Another thing, would it be possible to show all entries above one severity level so it shows all entries except for example info or debug messages?

harlequin
Posts: 13
Joined: Wed Nov 09, 2005 11:46 pm

#9 Post by harlequin » Fri Nov 11, 2005 3:39 am

Glad you got the colors fixed - I should probably document that a bit better.
I'll look into adding an 'and above' option to the priority select - makes sense. Cheers,
Harlequin
mrmee, mrmee, mrmee...

Devil
Posts: 21
Joined: Sun Oct 03, 2004 2:14 am
Contact:

#10 Post by Devil » Fri Nov 11, 2005 12:06 pm

harelquin really cool add-on.

i get the following errors when i load the syslog page:

Code: Select all

Notice: Undefined index: haloe_pdt_change in /usr/share/webapps/cacti/0.8.6f-r1/htdocs/haloe.php on line 38

Notice: Undefined index: button_clear_x in /usr/share/webapps/cacti/0.8.6f-r1/htdocs/haloe.php on line 46
and it would be nice to have a documentation tha told me how to add hosts to monitor.

Regards
Devil

harlequin
Posts: 13
Joined: Wed Nov 09, 2005 11:46 pm

#11 Post by harlequin » Fri Nov 11, 2005 2:39 pm

Thanks. For a quick fix on the 'Notice: Undefined...' errors, you could try editing your php.ini file and setting:
display_errors = Off
(this should be Off in a production server anyway) - or - setting:
error_reporting = E_ALL & ~E_NOTICE
(not really recommended in a production server, but it'll do for a test environment).
I will fix this in the next release. If you can't change the php.ini file, let me know and I will send you a 'hack' fix.
it would be nice to have a documentation tha told me how to add hosts to monitor
Hosts are pulled from whatever is in the haloe/syslog database - any hosts that are sending logging to the db will be in the list (you need to be use an external application like Kiwi syslog deamon or syslog-ng to collect syslog info and populate the database). Hope that helps...
Harlequin
mrmee, mrmee, mrmee...

cigamit
Developer
Posts: 2782
Joined: Thu Apr 07, 2005 3:29 pm
Location: B/CS Texas
Contact:

#12 Post by cigamit » Fri Nov 11, 2005 5:58 pm

harlequin wrote:Thanks. For a quick fix on the 'Notice: Undefined...' errors, you could try editing your php.ini file and setting:
display_errors = Off
(this should be Off in a production server anyway) - or - setting:
error_reporting = E_ALL & ~E_NOTICE
(not really recommended in a production server, but it'll do for a test environment).
I will fix this in the next release. If you can't change the php.ini file, let me know and I will send you a 'hack' fix.
it would be nice to have a documentation tha told me how to add hosts to monitor
Hosts are pulled from whatever is in the haloe/syslog database - any hosts that are sending logging to the db will be in the list (you need to be use an external application like Kiwi syslog deamon or syslog-ng to collect syslog info and populate the database). Hope that helps...
Harlequin
First off, I would like to say thanks for the great add-on. Its very similar to the Syslog plugin I have been working on (but not even close to finishing with the everything else I have to do).

I hope you don't mind, but I took the liberty of converting your add-on into the plugins format, it really only took about 15 minutes to do. I have also added the setting for custom refresh time. I went ahead and fixed several index errors (its good practice to disable E_ALL in production, but its also good practice to code with it on). I also fixed several other minor issues. It wasn't correctly outputting to file format for me (no database call), and the page selector was passing a variable that didn't exist.

This is fairly close to what I been hoping to for. I do see a few features that I would like see eventually added. Mainly I am looking at writing another script that runs every 5 minutes (right after normal pollings) which goes through and scans all "new" events and searches for specified ones to alert on (using user customized regex or just simple string comparisons). Possibly at the same time, have it go through and purge different ones from the database that we don't deem important (same regex concept), and also purge all events that are over XX days old (simple setting).

Overall, its looking really nice so far, and I hope you keep up the good work!
Attachments
haloe.zip
Syslog add-on in Plugin Format
(16.39 KiB) Downloaded 1820 times

Devil
Posts: 21
Joined: Sun Oct 03, 2004 2:14 am
Contact:

#13 Post by Devil » Sat Nov 12, 2005 5:00 am

I installed cigamits modified version and now it works like a charm.

Just one little thing. could you change so that to time field says now instead of a specific time. then it works better.

User avatar
egarnel
Cacti Pro User
Posts: 703
Joined: Thu Nov 21, 2002 8:55 am
Location: Austin, TX

#14 Post by egarnel » Sat Nov 12, 2005 11:53 am

This is awesome.
I was wondering if I could get a little assistance with the syslog-ng setup?

Here is the syslog-ng.conf to push into the haloe db:

Code: Select all

# Log syslog-ng to mysql database
                                    ##
                                        destination d_mysql {
                                            pipe("/tmp/mysql.pipe"
                                            template("INSERT INTO logs (host, facility, priority, level, tag, date,
                                            time, program, msg,seq) VALUES ( '$HOST', '$SEQ',  '$PROGRAM', '$TIME', '$DATE', '$PRIORITY',
                                            '$FACILITY') ;\n") template-escape(yes));
                                             };
                                        log { source(net); destination(d_mysql);
                                        };

and here is the fifo to route syslog messages into syslog-ng

Code: Select all

#!/bin/bash

if [ -e /tmp/mysql.pipe ]; then
        while [ -e /tmp/mysql.pipe ]
                do
                        mysql -u haloe --password=haloepassword haloe < /tmp/mysql.pipe
        done
else
        mkfifo /tmp/mysql.pipe
fi
The logs table never gets populated for some reason....

Thanks for your help
Cacti1 OS: CentOS 5.6 | 300+ devices
Cacti2 OS: CentOS 5.6 | 300+ devices
King of the Elves
Local Anarchists Union #427
"Anarchism is founded on the observation that since few men are wise enough to rule themselves, even fewer are wise enough to rule others." -Edward Abbey

Devil
Posts: 21
Joined: Sun Oct 03, 2004 2:14 am
Contact:

#15 Post by Devil » Sat Nov 12, 2005 5:40 pm

you have some errors in you syslog-ng config.

Code: Select all

# Log syslog-ng to mysql database
                                    ##
                                        destination d_mysql {
                                            pipe("/tmp/mysql.pipe"
                                            template("INSERT INTO logs (host, facility, priority, level, tag, date,
                                            time, program, msg,seq) VALUES ( '$HOST', '$SEQ',  '$PROGRAM', '$TIME', '$DATE', '$PRIORITY',
                                            '$FACILITY') ;\n") template-escape(yes));
                                             };
                                        log { source(net); destination(d_mysql);
                                        };
should be changed to:

Code: Select all

destination d_mysql { 
pipe("/var/log/mysql.pipe" 
template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) 
VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); };
You see you have to match the variables with the columns in the table (basic sql). The data get inserted in the wrong columns with your declaration.

How have you declared the source net in sysloc-ng.conf?
have you created the fifo file?
have you restated the syslog-ng process?

Post Reply