Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Sun Apr 21, 2019 9:44 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 46 posts ]  Go to page 1, 2, 3, 4  Next
Author Message
 Post subject: Sendmail & Mailscanner (alternate)
PostPosted: Mon May 30, 2005 11:11 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
Long time listener, first time caller ;). Anyway I thought I'd throw in a few little scripts that I whiped up that allow me to get Sendmail and Mailscanner stats through SNMP and come up with cool graphs using Cacti like the one shown below.
Image
Ignore the little gap in the graph, I had to move my server and so it was down for a bit.

Net-SNMP has a nice little feature that allows you to execute a script and return the result back when an SNMP OID is queried. I should mention that my scripts run a bit different than those posted by africanw in that there is no requirement for Mailscanner to be logging to a database. Instead I opted to scrape through the last 5000 lines of the /var/log/maillog file for entries added in the last 5 minutes, so as a result it is more disk and CPU intensive than africanw's method but it makes it pretty easy to query from any SNMP manager. And Cacti loves SNMP (right???).

NOTE: If you're email server is a high-volume email server and your load average is running a little high the scripts may not complete in time before SNMP times out (we are scraping through syslog files after all). What will happen is that no data will be returned by Net-SNMP and you're graphs may be missing bits. Just thought I'd mention it before I get flamed for stuff that is out of my control. :)

Sendmail Stats
You need to create a couple of scripts, one to count the recieved mail and another to count the sent mail. For simplicity sakes I stored mine in /opt. Each script is shown below,
/opt/count_recv.sh,
Code:
#!/bin/sh
echo .1.3.6.1.4.100.5
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
        search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
        search_sent = "stat=Sent";
        search_domain_1 = "@domain.com";
        search_domain_2 = "@domain2.com";
        recv_count = 0;
        }
{
        if ($0 ~ search_time && $0 ~ search_sent) time_trigger = 1;
        if ($0 ~ search_sent) is_sent_trigger = 1;
        if ($0 ~ search_domain_1) is_domain_trigger = 1;
        if ($0 ~ search_domain_2) is_domain_trigger = 1;
        if (time_trigger && is_sent_trigger && is_domain_trigger) recv_count++;
        is_sent_trigger = 0;
        is_domain_trigger = 0;
}
END { print recv_count; }'
exit
NOTE:You'll need to change domain.com and domain2.com to whatever your domain(s) are. If you only have one domain then you can take out the domain2.com search.
Now for a brief explanation. Everyone likes AWK right? :) It's a pretty simple script, it examines the last 5000 lines of /var/log/maillog for entries added in the last 5 minutes and looks for ones that has a Sendmail status of "Sent" to your domain(s) and then increments a counter if there is one.

/opt/count_sent.sh,
Code:
#!/bin/sh
echo .1.3.6.1.4.100.6
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
        search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
        search_sent = "stat=Sent";
        search_domain_1 = "@domain.com";
        search_domain_2 = "@domain2.com";
        recv_count = 0;
        }
{
        if ($0 ~ search_time && $0 ~ search_sent) time_trigger = 1;
        if ($0 ~ search_sent) is_sent_trigger = 1;
        if ($0 ~ search_domain_1) is_domain_trigger = 1;
        if ($0 ~ search_domain_2) is_domain_trigger = 1;
        if (time_trigger && is_spam_trigger) spam_count++;
        if (time_trigger && is_not_spam_trigger) not_spam_count++;
        if (time_trigger && is_sent_trigger && !is_domain_trigger) sent_count++;
        is_sent_trigger = 0;
        is_domain_trigger = 0;
}
END { print sent_count; }'
exit
NOTE:You'll need to change domain.com and domain2.com to whatever your domain(s) are. If you only have one domain then you can take out the domain2.com search.
This script is similar to the count_recv.sh script except for one key difference. Basically this script looks for the same Sendmail status of "Sent" for any domain that isn't yours.

Now with the scripts in place there are some changes you need to make to your snmpd.conf file so that when the .1.3.6.1.4.100.5 and .1.3.6.1.4.100.6 OIDs are queried the scripts get run.
/etc/snmp/snmpd.conf additions,
Code:
# Count the number of received messages in the last 5 minutes
pass .1.3.6.1.4.100.5 /bin/sh /opt/count_recv.sh
# Count the number of sent messages in the last 5 minutes
pass .1.3.6.1.4.100.6 /bin/sh /opt/count_sent.sh
As you can probably guess now when OID .1.3.6.1.4.100.5 is queried Net-SNMP runs the /opt/count_recv.sh script and when OID .1.3.6.1.4.100.6 is queried Net-SNMP runs the /opt/count_sent.sh script.

Mailscanner Stats
africanw's method has an advantage over mine here in that SPAM isn't identified as HIGHSPAM or LOWSPAM, it's only SPAM. As well viruses include both blocked files and viruses. No distiction. Sorry, this is a first version. I might work on fixing that later. Now again you'll need to create a couple of scripts, one to count the number of viruses that Mailscanner has detected and another to count the number of messages detected as SPAM.
/opt/count_viruses.sh
Code:
#!/bin/sh
echo .1.3.6.1.4.100.4
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
        search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
        search_virus = "Virus Scanning: Found";
        virus_count = 0;
        }
{
        if ($0 ~ search_time && $0 ~ search_virus) time_trigger = 1;
        if ($0 ~ search_virus) is_virus_trigger = 1;
        if (time_trigger && is_virus_trigger) virus_count = virus_count + $9;
        is_virus_trigger = 0;
}
END { print virus_count; }'
exit
This script scrapes through the last 5000 lines of /var/log/maillog for entries added in the last 5 minutes and looks for the "Virus Scanning: Found" line that indicates how many viruses Mailscanner found in it's scan batch. Then adds that number to a counter.

/opt/count_spam.sh
Code:
#!/bin/sh
echo .1.3.6.1.4.100.2
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
        search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
        search_is_spam = "is spam";
        spam_count = 0;
        }
{
        if ($0 ~ search_time && $0 ~ search_is_spam) time_trigger = 1;
        if ($0 ~ search_is_spam) is_spam_trigger = 1;
        if (time_trigger && is_spam_trigger) spam_count++;
        is_spam_trigger = 0;
}
END { print spam_count; }'
exit
This script looks for the "is spam" tag in /var/log/maillog to indicate that the message that Mailscanner is examining (usually with SpamAssassin) is SPAM.

And again you'll need to modify your snmpd.conf file so that the scripts get run when the OIDs are queried,
/etc/snmp/snmpd.conf additions,
Code:
# Count the number of SPAM messages in the last 5 minutes
pass .1.3.6.1.4.100.2 /bin/sh /opt/count_spam.sh
# Count the number of viruses detected in the last 5 minutes
pass .1.3.6.1.4.100.4 /bin/sh /opt/count_viruses.sh
An now when .1.3.6.1.4.100.2 or .1.3.6.1.4.100.4 OIDs are queried Net-SNMP will run the count_spam.sh or count_viruses.sh script respectively.

Cacti Templates
Below is the Cacti Graph Template that I made up too (the Data Templates are included in the XML file since they are dependents). Now all you need to do is setup Cacti to start querying the above SNMP OIDs and then graph them using the template.


Attachments:
File comment: Sendmail and Mailscanner Data and Graph Templates.
Sendmail_Mailscanner_Template_0.8.6d.zip [3.53 KiB]
Downloaded 1809 times


Last edited by warnesj on Tue Jun 07, 2005 12:33 pm, edited 2 times in total.
Top
 Profile  
 
 Post subject: huh... suddenly script returning correct values?
PostPosted: Wed Jun 01, 2005 2:22 am 
Offline

Joined: Wed Jun 01, 2005 2:09 am
Posts: 7
Hey great stuff!

I like the flat file approach, it's not that slow at all, I expected to wait a few seconds before a value is returned but hey, I doubt if one full second is needed.

I have a question though. I'm using Postfix instead of Sendmail. That shouldn't make any difference since the only changes needed are:
stat -> status and Sent -> sent. Pretty simple and straightforward, I like that :wink:

I added the stuff to my cacti host and my mailserver (other machine) and gather the data using SNMP. I did this yesterday around 5:00 PM and came in this morning, expecting to see beautiful graphs. I do see nice graphs, but for some strange reason, my sent script doesn't return anything anymore. And my received script always returns a zero for some reason. I'm not able to think of any logical reason why the scripts don't return the values anymore. This happened overnight. It seems that just when the clock ticked 0:00, the magic stopped working.

It's not related to SNMP, because I get the same result when I myself execute the scripts as a regular user (non-root) I was able to get great results yesterday as a regular user. I don't know why I can't get results anymore from a simple script that does nothing more than a tail of the last 5000 lines, and then throws some awk lines at it... I'm only using the sent and received scripts that way. They both worked fine yesterday, but suddenly stopped working around midnight. I can't imagine any special cronjob doing anything...

Any clues of what I might be overseeing here?

Thnx!

D


!! EDIT !!

I found the problem. The cause of the problem lies in the string format used by awk. On my server, the format used is different than the format on the server on which these scripts were made.

The problem was discovered pretty soon luckily because the date has changed from may 31 to june 1. The scripts will format the time string to search for: May 31 09:34:56. Last night our servers turned to june. The format returned in maillog was: Jun 1 09:34:56. The script will format the string to Jun 01 09:34:56. The leading zero (or space) is never matched, thus the script will either return a zero or nothing.

To change this behaviour change the following line:
Code:
        search_time = "^" strftime("%b %d %H:%M", systime() - 5 * 60);

into
Code:
        search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);


Then it works again. :D

The other thing you can do is wait for 10 days :wink:

Thanks to my collegue for helping me out so quickly


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jun 01, 2005 7:59 am 
Offline

Joined: Wed Mar 16, 2005 4:16 am
Posts: 7
I can not establish a template. It can with a mistake?


Top
 Profile  
 
 Post subject: Re: huh... suddenly script returning correct values?
PostPosted: Wed Jun 01, 2005 9:21 am 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
D43m0n wrote:
They both worked fine yesterday, but suddenly stopped working around midnight. I can't imagine any special cronjob doing anything...

Any clues of what I might be overseeing here?

!! EDIT !!

I found the problem. The cause of the problem lies in the string format used by awk. On my server, the format used is different than the format on the server on which these scripts were made.

The problem was discovered pretty soon luckily because the date has changed from may 31 to june 1. The scripts will format the time string to search for: May 31 09:34:56. Last night our servers turned to june. The format returned in maillog was: Jun 1 09:34:56. The script will format the string to Jun 01 09:34:56. The leading zero (or space) is never matched, thus the script will either return a zero or nothing.

Yeah I noticed all my graphs stopped at midnight too. Damn, I should have waited for a month turn-over before posting my scripts. ;) I made the same change you did and edited my original post so nobody else runs into that. 'man date' is now my new friend. :) Sorry about that. Sweet mod for Postfix too!

anTIDot wrote:
I can not establish a template. It can with a mistake?
I'm not sure what you're asking here. Are you having problems installing the Template through the Import Template option in Cacti? Or are you having problems with one of the scripts and getting Net-SNMP to run.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 02, 2005 3:05 am 
Offline

Joined: Wed Mar 16, 2005 4:16 am
Posts: 7
Sorry for my bad English.
At me a problem import of a template file cacti_graph_template_sendmail_mailscanner.xml
Cacti does not wish it to import.
Cacti version 0.8.6d.
In what there can be a problem?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 02, 2005 9:48 am 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
No appologies required.

Do you get any specific error from Cacti when you try to import the template? Is there anything in the Cacti log file?

Thanks for the info.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 03, 2005 6:49 am 
Offline

Joined: Wed Mar 16, 2005 4:16 am
Posts: 7
I do all as usually:
Console > Import templates > Browse > cacti_graph_template_sendmail_mailscanner.xml > SAVE
And nothing occurs.

Poller Logging Level - DEBUG (poller errors, poller warnings )
In cacti.log - nothing.


Top
 Profile  
 
 Post subject: Import problem
PostPosted: Mon Jun 06, 2005 6:22 am 
Offline

Joined: Mon Jun 06, 2005 6:13 am
Posts: 4
I've got the same problem


Top
 Profile  
 
 Post subject: Re: Import problem
PostPosted: Mon Jun 06, 2005 12:08 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
yianniska wrote:
I've got the same problem

This is typically a cacti version mismatch between exporter and importer. Importer must have same or higher (?) level than exporter.

HTH :wink:
Reinhard


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 06, 2005 5:47 pm 
Offline
Cacti User

Joined: Thu Mar 31, 2005 4:24 pm
Posts: 124
Location: Sydney, Asutralia
Yep same prob with no errors. Running version d.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 06, 2005 10:08 pm 
Offline
Cacti User

Joined: Sun May 29, 2005 7:34 pm
Posts: 173
Hmmm, strange. I'm running v0.8.6d on Fedora Core 3. I've updated using the RPM that's posted on the main page, but I've redone the update using the tarball that's there. I've re-exported the template and included it below. See if that one works for you.

Sorry all for the problems with the template, hopefully this one works better.


Attachments:
File comment: Second attempt at the Graph & Data Source template. Exported from Cacti v0.8.6d
Sendmail_Mailscanner_Template_0.8.6d.zip [3.53 KiB]
Downloaded 1315 times
Top
 Profile  
 
 Post subject:
PostPosted: Tue Jun 07, 2005 6:35 am 
Offline

Joined: Mon Jun 06, 2005 6:13 am
Posts: 4
work's for me

thanks


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jun 07, 2005 6:35 am 
Offline

Joined: Mon Jun 06, 2005 6:13 am
Posts: 4
work's for me
thanks


Top
 Profile  
 
 Post subject: ok
PostPosted: Tue Jun 07, 2005 6:36 am 
Offline

Joined: Mon Jun 06, 2005 6:13 am
Posts: 4
import is ok but still i cannot get any graphs
when i try to create the graphs

i get this error

RRDTool Says:

ERROR: opening '/var/www/html/cacti0.8.6d/rra/eml2_mess_recv_62.rrd': No such file or directory.

i 've put the scripts to the mail server
i also put the extra lines in my ucdsnmp.conf
but no luck.

could you please help me with this because i find this graphs very
nice

thanks


Last edited by yianniska on Tue Jun 07, 2005 9:13 am, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Tue Jun 07, 2005 7:46 am 
Offline
Cacti User

Joined: Thu Mar 31, 2005 4:24 pm
Posts: 124
Location: Sydney, Asutralia
Yep now imports.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 46 posts ]  Go to page 1, 2, 3, 4  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: Google Adsense [Bot] and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group