Syslog viewer in Cacti

knobdy · Cacti User Joined: 28 Sep 2005 Posts: 495

It seems that this project (http://lofimo.anzac.at/) might provide my team with what they're looking for which is a replacement to their desktop logserver - but its java based.

We have a centralized log server now and I'd prefer they use it to review logs but they also need to be able to see them "live".

So, that said, I'm wondering if one of the lovely developers here might be able to incorporate this into Cacti as a plug-in/add-on. I'd probably take a stab at it myself but the length of time it would take me to do it right would be pathetic - and besides, someone here might have a better way. I thought there had to be plenty of open source projects that basically do a "tail -f" on a log file and feed it to a web interface, but if there are I haven't found them. Placing the log files into a database is not a remote possibility either - WAY too much data.

adrianmarsh · Cacti User Joined: 17 Aug 2005 Posts: 417 Location: UK

Well... Cacti already has a syslog plugin..

It looks to me as though LoFiMo is a workaround-solution, rather than those apps simply using the syslog system anyway (I'm not dissing LoFiMo)..

Have you checked out the plugin?

knobdy · Cacti User Joined: 28 Sep 2005 Posts: 495

Yeah, I believe I have - or at least an early one. The problem with it is that it utilizes a database and I have way too much data to be logging into a database. The plugin I'm looking for needs to be as simple as a "tail -f" command from the terminal - but fed to a web interface.

adrianmarsh · Cacti User Joined: 17 Aug 2005 Posts: 417 Location: UK

But if you've so much data coming in... then surely tail won't suffice either ?

Outputting to the mysql database, then reviewing the db is a lot faster than tail -f would be..

TheWitness · Posted: Sat Jan 27, 2007 9:44 am Post subject:

The syslog_ng utilizes a separate database. IMHO, and I have +++experience, this is the only way to go, for many reasons. Just look at the competition (best of breed). This is what they do.

What Haloe may lack is Database management and Optimization routines to keep performance optimal. Not performing optimization, will, over time destroy any performance gains of using a database.

Since I have not dissected Haloe yet, I can not comment on such matters.

TheWitness

knobdy · Cacti User Joined: 28 Sep 2005 Posts: 495

Tail -f is fast because its LIVE data, I'm only seeing the last 10 to 20 lines and then every new line after that. Compare that to importing each line into a database and then printing it - still think tailing is slower? If so, you simply havne't dealt with a busy log server.

Further, I'm using syslog-ng to split up the log files based on device type - and other things as needed - so I'm tailing a small subset of all the data coming in. I realize that syslog-ng can utilize mysql, and I've tried that, and its slow. For that matter, just feeding a mysql database with 15-20 gigs of data with nothing optimizing that data (as has just been said) simply KILLS my PPC box...

TheWitness · Posted: Mon Jan 29, 2007 7:26 pm Post subject:

Hmm. Well, I will know more in the months to come. Jimmy and I will have a few conversations on scalability.

Larry