Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Wed Jul 17, 2019 10:23 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 142 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 10  Next
Author Message
 Post subject:
PostPosted: Tue Feb 06, 2007 11:02 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
I think I have a fix for this now too. Sorry about that. I must have an older version of Perl because I don't get those warnings. At any rate, see if this works.

Find,
Code:
open(MAILLOG, "tail -n 0 -f /var/log/maillog|") or die "Cannot open maillog: $!.";
while(!$time_to_die){
And replace it with,
Code:
open(MAILLOG, "tail -n 0 -f /var/log/maillog|") or die "Cannot open maillog: $!.";
my $line="";
while(!$time_to_die){
Essentially you need to add my $line=""; on a line before the "while...." statement to initialize the $line variable that is used in the pattern matching.

Thanks for finding this. I'll update the main post too.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 07, 2007 6:58 am 
Offline

Joined: Mon Feb 05, 2007 11:40 am
Posts: 6
I'm sorry to have to say that but I still have the same error :(
Adding this new line had not change the error.
Anything else ?
(Thx for helping me) :wink:


Top
 Profile  
 
 Post subject:
PostPosted: Tue Feb 13, 2007 11:27 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
Okay I've been digging away again. Unfortunately I can't replicate the problem on my Fedora Core installation. :( But how about we try this,

Find,
Code:
while(!$time_to_die){
        $line=<MAILLOG>;
        # Look for received messages where the sender is not from our domain(s)
And replace it with,
Code:
while(!$time_to_die && ( $line = <MAILLOG> )){
        # Look for received messages where the sender is not from our domain(s)
If that works then I'll make the changes permanent on the main page and upload a new version of the script.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Feb 26, 2007 3:38 pm 
Offline

Joined: Tue Aug 10, 2004 5:44 pm
Posts: 17
Hi!

Graph template from Sven worked indeed for me! :D (FC6, Cacti 0.8.6i)

The other problem I have is that i'm running the watchmaillog.sh script remotely on our mailserver. (FC5, Plesk8.1). On this server we've setup SpamAssassin which is running fine. It produces output in the maillog like
Code:
Feb 26 21:23:39 giant spamd[21797]: spamd: processing message <[email protected]> for [email protected]:110
Feb 26 21:23:51 giant spamd[21797]: spamd: clean message (3.1/7.0) for [email protected]:110 in 12.0 seconds, 35663 bytes.
Feb 26 21:23:51 giant spamd[21797]: spamd: result: . 3 - EXTRA_MPART_TYPE,HTML_10_20,HTML_MESSAGE,RCVD_IN_BL_SPAMCOP_NET scantime=12.0,size=35663,[email protected],uid=110,required_score=7.0,rhost=localhost,raddr=127.0.0.1,rport=/tmp/spamd_full.sock,mid=<[email protected]>,autolearn=no


So I changed this::
Code:
   # Look for MailScanner spam scanning batch results
   if($line=~/Spam\ Checks\:\ Found/){
      $item="spam";
      $spam_count_pos = index($line,"Spam\ Checks\:\ Found");
      $spam_count_pos2 = index($line, "\ spam\ messages");
      $spam_count = substr($line,($spam_count_pos+19),($spam_count_pos2-($spam_count_pos+19)));
      &readcounterfile;
      $counter{$item}=$counter{$item}+$spam_count;
      if($debug){print("Found $spam_count SPAM in the MailScanner batch, incrementing the spam counter to $counter{$item}.\n");}
      &writecounterfile;
   }


to this::

Code:
        # Look for MailScanner spam scanning batch results
        if($line=~/is\ spam/ || $line=~/spamd\: identified spam/){
                $item="spam";
                $spam_count_pos = index($line,"Spam\ Checks\:\ Found");
                $spam_count_pos2 = index($line, "\ spam\ messages");
                $spam_count = substr($line,($spam_count_pos+19),($spam_count_pos2-($spam_count_pos+19)));
                &readcounterfile;
                $counter{$item}=$counter{$item}+$spam_count;
                if($debug){print("Found $spam_count SPAM in the MailScanner batch, incrementing the spam counter to $counter{$item}.\n");}
                &writecounterfile;
        }


But still all my counters are 0!!! :s
At plesk the maillog is at /usr/local/psa/var/log/maillog, so I symlinked /var/log/maillog -> /usr/local/psa/var/log/maillog

I also did a
Code:
cat /var/log/maillog | grep "spamd: identified spam"
which gave me around 50 lines of found spam entries.
So I seems to work ...

Anyone an idea??

[EDIT]
I'm using ClamAV as virus-scanner. It is reporting to /var/log/clamav/clamd.log Can I change that somewhere??

Thanks!
Onno.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Feb 26, 2007 10:22 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
It might be that the script is expecting numbers at certain positions. The "Spam Checks: Found" that spamd generated was the SpamAssassin results from a batch scan. So if you modify that search to look for individual "spamd: identified spam" messages then the numbers that the script is looking for (because it thinks the results are from a batch scan) aren't there. Can you post a copy of a syslog message that has your "spamd: identified spam" message? I can offer some suggestions for a mod.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 01, 2007 7:21 am 
Offline

Joined: Tue Aug 10, 2004 5:44 pm
Posts: 17
warnesj wrote:
It might be that the script is expecting numbers at certain positions. The "Spam Checks: Found" that spamd generated was the SpamAssassin results from a batch scan. So if you modify that search to look for individual "spamd: identified spam" messages then the numbers that the script is looking for (because it thinks the results are from a batch scan) aren't there. Can you post a copy of a syslog message that has your "spamd: identified spam" message? I can offer some suggestions for a mod.


Hi!

Thanks for your reply. Since the maillog is symlinked: /var/log/maillog -> /usr/local/psa/var/log/maillog
Here's the output of [cat /var/log/maillog |grep "spamd: identified spam"]:

Code:
Mar  1 10:20:55 giant spamd[17402]: spamd: identified spam (7.8/7.0) for [email protected]:110 in 22.3 seconds, 3385 bytes.
Mar  1 11:06:48 giant spamd[17402]: spamd: identified spam (24.3/7.0) for [email protected]:110 in 20.4 seconds, 13965 bytes.
Mar  1 11:12:45 giant spamd[17402]: spamd: identified spam (19.8/7.0) for [email protected]:110 in 20.3 seconds, 1202 bytes.
Mar  1 11:41:46 giant spamd[17402]: spamd: identified spam (11.7/7.0) for [email protected]:110 in 20.6 seconds, 2856 bytes.
Mar  1 12:20:30 giant spamd[17402]: spamd: identified spam (29.6/7.0) for [email protected]:110 in 21.3 seconds, 12643 bytes.
Mar  1 12:49:13 giant spamd[17402]: spamd: identified spam (11.2/7.0) for [email protected]:110 in 20.3 seconds, 1548 bytes.


Thanks!!
But note, all other counters also remain 0, and last type after a day I had around 20 perl processes... is that ok??

Greetz.
Onno.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 02, 2007 4:15 pm 
Offline

Joined: Tue Aug 10, 2004 5:44 pm
Posts: 17
Hi there!!

I'd made some changes by myself, and have at least a graph that's working I think. (see attachment)

I know it's not a really interesting image, this because I only had some of all domains added for testing... but it's looking good I think.

Any comments are welcome!

Thanks.
Onno.


Attachments:
mailstats.JPG
mailstats.JPG [ 113.67 KiB | Viewed 8567 times ]
Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 23, 2007 12:13 pm 
Offline

Joined: Thu Mar 25, 2004 11:17 am
Posts: 4
Location: Netherlands
Yep,
I have got it working with ClamAV and Spamassassin.
Here is the snippet of my watchmaillog.sh (i have the orriginal lines commented out)

#!/usr/bin/perl
#
# Daemon used to watch the maillog messages for certain messages and trigger events when
# certain messages occur
#
# By Jason Warnes
#
# Change Log
# ~~~~~~~~~~
# 2006-08-22: Initial release
# 2006-09-05: Fixed signal handling
# Added /var/run/watchmaillog.pid file for logrotate usage
# 2006-09-07: Added proper signal handling (Thanks pvenezia!)
# Added SpamAssassin spamd checking support for SPAM (Thanks raiten!)
# 2006-09-18: Added new item mess_waiting, which is the number of messages MailScanner
# detects when a new batch is started.
# 2006-11-02: Changed the way SPAM messages were detected so the script works
# on servers configured for less verbose logging. (Thanks sdetroch!)
# 2006-11-08: Added new item mess_rejected, which is the number of rejected
# messages by Sendmail.
# 2007-02-06: Fixed <MAILLOG> close statement at end of main program. (Thanks Avenger!)
# Fixed warning messages about uninitialized $line used in pattern
# matching (Thanks Avenger!)


$debug=0; # 1=Debug messages are displayed, 0=No debug messages are displayed
$daemon=1; # 1=Daemonize the program, 0=Run interactive
$syslog=1; # 1=Log stuff to syslog, 0=No logging to syslog
$self="/home/scripts/reliant/cacti/watchmaillog_scripts/watchmaillog.sh"; # Location of this script
$counterfile="/home/scripts/reliant/cacti/watchmaillog_scripts/watchmaillog_counters"; # Location to store the counter file
$resetfile="/home/scripts/reliant/cacti/watchmaillog_scripts/watchmaillog_reset"; # Location of the reset counter flag file
$pidfile="/var/run/watchmaillog.pid"; # Location of the running process ID file (used in logrotate)

use Sys::Syslog;
use POSIX;
use Time::HiRes qw( gettimeofday tv_interval );

$|=1;

my $sigset = POSIX::SigSet->new();
my $hupaction = POSIX::SigAction->new('hup_signal_handler',
$sigset,
&POSIX::SA_NODEFER);
my $osigaction = POSIX::SigAction->new('signal_handler',
$sigset,
&POSIX::SA_NODEFER);
POSIX::sigaction(&POSIX::SIGHUP, $hupaction);
POSIX::sigaction(&POSIX::SIGINT, $osigaction);
POSIX::sigaction(&POSIX::SIGTERM, $osigaction);


if($daemon){
$pid=fork;
if($pid) {
open(PID,">".$pidfile) or die "Cannot open PID file: $!.";
print PID ("$pid\n"); # Write the PID out to the PID file for logrotate
close(PID);
}
exit if $pid;
die "Couldn't fork : $!" unless defined($pid);
setsid() or die "Can't start a new session: $!";
$time_to_die=0;
}

sub signal_handler {
$time_to_die=1;
}

sub hup_signal_handler {
if($debug){print "got SIGHUP\n";}
exec($self) or die "Couldn't restart: $!\n";
}

if($syslog){openlog("watchmaillog","pid","daemon");}
if($syslog){syslog("notice","Starting.");}
if($debug){print("watchmaillog is starting.\n");}

# Main part of the program
open(MAILLOG, "tail -n 0 -f /var/log/maillog|") or die "Cannot open maillog: $!.";
my $line="";
while(!$time_to_die){
$line=<MAILLOG>;
# Look for received messages where the sender is not from our domain(s)
if(($line=~/from\=/) && ($line!~/\@sonnehoek.nl|\@home.org|\@reliant.home.org/)){
$item="mess_recv";
&readcounterfile;
$counter{$item}++;
if($debug){print("Found an inbound message, incrementing the message recieve counter to $counter{$item}.\n");}
&writecounterfile;
}
# Look for messages sent to our domain(s), indicates an inbound message relayed to an internal server
if(($line=~/stat\=Sent/) && ($line=~/\@sonnehoek.nl|\@home.org|\@reliant.home.org/)){
$item="mess_relay";
&readcounterfile;
$counter{$item}++;
if($debug){print("Found an clean inbound message, incrementing the clean message recieve counter to $counter{$item}.\n");}
&writecounterfile;
}
# Look for sent messages to NOT our email domain(s), indicates an outbound message
if(($line=~/stat\=Sent/) && ($line!~/\@sonnehoek.nl|\@home.org|\@reliant.home.org/)){
$item="mess_sent";
&readcounterfile;
$counter{$item}++;
if($debug){print("Found an outbound message, incrementing the message sent counter to $counter{$item}.\n");}
&writecounterfile;
}
# Look for rejected messages
if((($line=~/ruleset/) && ($line=~/reject\=/)) || ($line =~/rejecting/)){
$item="mess_rejected";
&readcounterfile;
$counter{$item}++;
if($debug){print("Found a rejected message, incrementing the message rejected counter to $counter{$item}.\n");}
&writecounterfile;
}
# Look for Spamassassin spam results
# if($line=~/Spam\ Checks\:\ Found/){
if($line=~/X\-Spam\-Status\:\ Yes/){
$item="spam";
# $spam_count_pos = index($line,"Spam\ Checks\:\ Found");
# $spam_count_pos = index($line,"Spam\ Status\:\ Yes");
# $spam_count_pos2 = index($line, "\ spam\ messages");
# $spam_count = substr($line,($spam_count_pos+19),($spam_count_pos2-($spam_count_pos+19)));
$spam_count = 1;
&readcounterfile;
$counter{$item}=$counter{$item}+$spam_count;
if($debug){print("Found $spam_count SPAM in the Maillog, incrementing the spam counter to $counter{$item}.\n");}
&writecounterfile;
}
# Look for ClamAV virus scanning results
# if($line=~/Virus\ Scanning\:\ Found/){
if($line=~/Blocked\ INFECTED/){
$item="virus";
# $virus_count_pos = index($line,"Virus\ Scanning\:\ Found");
# $virus_count_pos2 = index($line, "\ viruses");
# $virus_count = substr($line,($virus_count_pos+22),($virus_count_pos2-($virus_count_pos+22)));
$virus_count = 1;
&readcounterfile;
$counter{$item}=$counter{$item}+$virus_count;
if($debug){print("Found $virus_count viruses in the Maillog, incrementing the virus counter to $counter{$item}.\n");}
&writecounterfile;
}

# Look for MailScanner waiting messages
if($line=~/New\ Batch\:\ Found/){
$item="mess_waiting";
$mess_waiting_pos = index($line,"New\ Batch\:\ Found");
$mess_waiting_pos2 = index($line,"\ messages\ waiting");
$mess_waiting = substr($line,($mess_waiting_pos+17),($mess_waiting_pos2-($mess_waiting_pos+17)));
&readcounterfile;
$counter{$item}=$mess_waiting;
if($debug){print("Mailscanner found $mess_waiting messages waiting, setting the mess_waiting counter to $counter{$item}.\n");}
&writecounterfile;
}
}
close(MAILLOG);
if($debug){print("watchmaillog is ending.\n");}
if($syslog){syslog("notice","Ending.");}
unlink($pidfile);

# Subroutine to read the contents of the counter file
sub readcounterfile {
# Read the counter values from the file
if($debug){print("Reading contents of counter file.\n");}
open(COUNTER,$counterfile);
while($line=<COUNTER>){
@line=split(/\:/,$line);
chop($line[1]); # Drop the trailing LF off the value
# Check for reset counter flag file
if(-e $resetfile."_".$line[0]){
if($debug){print("Reset counter flag file found for counter $line[0], resetting counter value to 0.\n");}
$counter{$line[0]}=0;
unlink($resetfile."_".$line[0]);
} else {
$counter{$line[0]}=$line[1];
}
if($debug){print("Counter $line[0] = $counter{$line[0]}.\n");}
}
close(COUNTER);
}

# Subrouting to write the contents of the counter file
sub writecounterfile {
if($debug){print("Writing counter values to counter file.\n");}
open(COUNTER,">".$counterfile);
# Write each counter item out to the counter file
foreach $item (sort keys(%counter)) {
print COUNTER ($item."\:".$counter{$item}."\n");
}
close(COUNTER);
chmod(0666,$counterfile);
}


Gr. Peer
http://peer.no-ip.org


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 06, 2007 11:52 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
Sorry everyone, I've been unavailable lately for a while. :(

I'd made some changes by myself, and have at least a graph that's working I think. (see attachment)
Looks good [email protected] I'd be curious to see your changes if you have some time. The way the unmodified script is right now, the snippet of the log file you showed above wouldn't have triggered any SPAM counters. Usually I have the script look for the end of the MailScanner batch which has the summary of how many SPAM messages were found in the MailScanner batch. Usually those messages start with "Spam Checks: Found" and then it should have a number of some sort. Does you log file have any of those?

Peer wrote:
I have got it working with ClamAV and Spamassassin.
I'm guessing that you're not running MailScanner right? Most of the messages I'm triggering on for the SPAM and virus scanning are generated by MailScanner. It's nice to see that the script is being moded to work for other environments successfully. Thanks for sharing the changes Peer.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 09, 2007 7:26 pm 
Offline

Joined: Wed Mar 28, 2007 5:48 pm
Posts: 22
I have successfully imported all of the latest graphs and data templates without error. Also, I have snmpwalked my sendmail host and verified the OIDs increment correctly with the amount of mail being received.

But, after I associate them with my sendmail host, I only see a blank entry in my graphs section. I don't even see an empty picture box, just the text for "E-Mail Statistics".

I'm on RHEL ES4, with Apache 2.2.x, Cacti 0.8.6j
, PHP 5.2.1, and no other problems. I think I've just missed a piece. Any ideas?

Thanks,

Greg


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 10, 2007 10:18 am 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
gpetme wrote:
But, after I associate them with my sendmail host, I only see a blank entry in my graphs section. I don't even see an empty picture box, just the text for "E-Mail Statistics".
Can you include a picture of what you're seeing? I'm not exactly sure what it could be yet.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 10, 2007 1:10 pm 
Offline

Joined: Wed Mar 28, 2007 5:48 pm
Posts: 22
Sure, here's the screenshot. Also, I noticed yesterday that under Data Sources -> Add -> Selected Data Template none of the watchmaillog sources were listed. This morning they were. I've added them in, but the same behavior is being exhibited (see screenshot).

Thanks,

Greg


Attachments:
File comment: screenshot
nograph.PNG
nograph.PNG [ 6.03 KiB | Viewed 8195 times ]
Top
 Profile  
 
 Post subject: A twist of the same problem posted previously
PostPosted: Thu Apr 26, 2007 10:20 pm 
Offline

Joined: Fri Sep 30, 2005 5:12 pm
Posts: 11
Location: Madison, WI
I have at least scanned through each of the postings on this topic. I am running 0.8.6j on RHEL v4 update 2 on a 32bit CPU system.

I have the SNMP side of things working, and I was having the same import problem, but I used the template provided by Sven and that fixed the issue I was having with the one graph template not loading. I checked the cacti interface to make sure that all of the templates were there.

The problem I am currently having is that I can't select the data and graph items when in the host area of Cacti. That is to say that I created a generic ucd/snmp host in cacti (which is the box I am monitoring sendmail on.) and I can create, much less monitor, those aspects of the host template just fine. But I can't add the sendmail-centric data templates but I can add the graph templates to the host.

What can I do to fix this (ie.: what I am doing wrong)?


Top
 Profile  
 
 Post subject:
PostPosted: Tue May 29, 2007 10:54 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
Okay sorry all for my absence. I'm back again. Can anyone having problems with this script post again? I'm not sure the status of them. Thanks.


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 31, 2007 10:27 am 
Offline

Joined: Thu May 31, 2007 10:22 am
Posts: 1
Hi!

warnesj wrote:
Okay sorry all for my absence. I'm back again. Can anyone having problems with this script post again? I'm not sure the status of them. Thanks.


I'm using watchmaillog.sh on FreeBSD. Unfortunally the tail process isn't killed on restart via HUP. So after n restarts n+1 tail processes are running.

Adding the following patch solves the problem for me. Has anyone else seen this?

Code:
--- watchmaillog.sh.new 2007/05/31 15:20:58     1.1
+++ watchmaillog.sh.new 2007/05/31 15:21:16
@@ -68,6 +68,7 @@

sub hup_signal_handler {
       if($debug){print "got SIGHUP\n";}
+      close(MAILLOG);
       exec($self) or die "Couldn't restart: $!\n";
}


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 142 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 10  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group