|
|
| Author |
Message |
warnesj
Cacti User
Joined: 29 May 2005
Posts: 169
|
 Posted: Mon May 30, 2005 11:11 pm Post subject: Sendmail & Mailscanner (alternate) |
 |
|
Long time listener, first time caller  . Anyway I thought I'd throw in a few little scripts that I whiped up that allow me to get Sendmail and Mailscanner stats through SNMP and come up with cool graphs using Cacti like the one shown below.
Ignore the little gap in the graph, I had to move my server and so it was down for a bit.
Net-SNMP has a nice little feature that allows you to execute a script and return the result back when an SNMP OID is queried. I should mention that my scripts run a bit different than those posted by africanw in that there is no requirement for Mailscanner to be logging to a database. Instead I opted to scrape through the last 5000 lines of the /var/log/maillog file for entries added in the last 5 minutes, so as a result it is more disk and CPU intensive than africanw's method but it makes it pretty easy to query from any SNMP manager. And Cacti loves SNMP (right???).
NOTE: If you're email server is a high-volume email server and your load average is running a little high the scripts may not complete in time before SNMP times out (we are scraping through syslog files after all). What will happen is that no data will be returned by Net-SNMP and you're graphs may be missing bits. Just thought I'd mention it before I get flamed for stuff that is out of my control. 
Sendmail Stats
You need to create a couple of scripts, one to count the recieved mail and another to count the sent mail. For simplicity sakes I stored mine in /opt. Each script is shown below,
/opt/count_recv.sh, | Code: | #!/bin/sh
echo .1.3.6.1.4.100.5
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
search_sent = "stat=Sent";
search_domain_1 = "@domain.com";
search_domain_2 = "@domain2.com";
recv_count = 0;
}
{
if ($0 ~ search_time && $0 ~ search_sent) time_trigger = 1;
if ($0 ~ search_sent) is_sent_trigger = 1;
if ($0 ~ search_domain_1) is_domain_trigger = 1;
if ($0 ~ search_domain_2) is_domain_trigger = 1;
if (time_trigger && is_sent_trigger && is_domain_trigger) recv_count++;
is_sent_trigger = 0;
is_domain_trigger = 0;
}
END { print recv_count; }'
exit |
NOTE:You'll need to change domain.com and domain2.com to whatever your domain(s) are. If you only have one domain then you can take out the domain2.com search.
Now for a brief explanation. Everyone likes AWK right? It's a pretty simple script, it examines the last 5000 lines of /var/log/maillog for entries added in the last 5 minutes and looks for ones that has a Sendmail status of "Sent" to your domain(s) and then increments a counter if there is one.
/opt/count_sent.sh, | Code: | #!/bin/sh
echo .1.3.6.1.4.100.6
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
search_sent = "stat=Sent";
search_domain_1 = "@domain.com";
search_domain_2 = "@domain2.com";
recv_count = 0;
}
{
if ($0 ~ search_time && $0 ~ search_sent) time_trigger = 1;
if ($0 ~ search_sent) is_sent_trigger = 1;
if ($0 ~ search_domain_1) is_domain_trigger = 1;
if ($0 ~ search_domain_2) is_domain_trigger = 1;
if (time_trigger && is_spam_trigger) spam_count++;
if (time_trigger && is_not_spam_trigger) not_spam_count++;
if (time_trigger && is_sent_trigger && !is_domain_trigger) sent_count++;
is_sent_trigger = 0;
is_domain_trigger = 0;
}
END { print sent_count; }'
exit |
NOTE:You'll need to change domain.com and domain2.com to whatever your domain(s) are. If you only have one domain then you can take out the domain2.com search.
This script is similar to the count_recv.sh script except for one key difference. Basically this script looks for the same Sendmail status of "Sent" for any domain that isn't yours.
Now with the scripts in place there are some changes you need to make to your snmpd.conf file so that when the .1.3.6.1.4.100.5 and .1.3.6.1.4.100.6 OIDs are queried the scripts get run.
/etc/snmp/snmpd.conf additions, | Code: | # Count the number of received messages in the last 5 minutes
pass .1.3.6.1.4.100.5 /bin/sh /opt/count_recv.sh
# Count the number of sent messages in the last 5 minutes
pass .1.3.6.1.4.100.6 /bin/sh /opt/count_sent.sh |
As you can probably guess now when OID .1.3.6.1.4.100.5 is queried Net-SNMP runs the /opt/count_recv.sh script and when OID .1.3.6.1.4.100.6 is queried Net-SNMP runs the /opt/count_sent.sh script.
Mailscanner Stats
africanw's method has an advantage over mine here in that SPAM isn't identified as HIGHSPAM or LOWSPAM, it's only SPAM. As well viruses include both blocked files and viruses. No distiction. Sorry, this is a first version. I might work on fixing that later. Now again you'll need to create a couple of scripts, one to count the number of viruses that Mailscanner has detected and another to count the number of messages detected as SPAM.
/opt/count_viruses.sh | Code: | #!/bin/sh
echo .1.3.6.1.4.100.4
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
search_virus = "Virus Scanning: Found";
virus_count = 0;
}
{
if ($0 ~ search_time && $0 ~ search_virus) time_trigger = 1;
if ($0 ~ search_virus) is_virus_trigger = 1;
if (time_trigger && is_virus_trigger) virus_count = virus_count + $9;
is_virus_trigger = 0;
}
END { print virus_count; }'
exit |
This script scrapes through the last 5000 lines of /var/log/maillog for entries added in the last 5 minutes and looks for the "Virus Scanning: Found" line that indicates how many viruses Mailscanner found in it's scan batch. Then adds that number to a counter.
/opt/count_spam.sh | Code: | #!/bin/sh
echo .1.3.6.1.4.100.2
echo gauge
tail -n 5000 /var/log/maillog|awk '
BEGIN {
search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60);
search_is_spam = "is spam";
spam_count = 0;
}
{
if ($0 ~ search_time && $0 ~ search_is_spam) time_trigger = 1;
if ($0 ~ search_is_spam) is_spam_trigger = 1;
if (time_trigger && is_spam_trigger) spam_count++;
is_spam_trigger = 0;
}
END { print spam_count; }'
exit |
This script looks for the "is spam" tag in /var/log/maillog to indicate that the message that Mailscanner is examining (usually with SpamAssassin) is SPAM.
And again you'll need to modify your snmpd.conf file so that the scripts get run when the OIDs are queried,
/etc/snmp/snmpd.conf additions, | Code: | # Count the number of SPAM messages in the last 5 minutes
pass .1.3.6.1.4.100.2 /bin/sh /opt/count_spam.sh
# Count the number of viruses detected in the last 5 minutes
pass .1.3.6.1.4.100.4 /bin/sh /opt/count_viruses.sh |
An now when .1.3.6.1.4.100.2 or .1.3.6.1.4.100.4 OIDs are queried Net-SNMP will run the count_spam.sh or count_viruses.sh script respectively.
Cacti Templates
Below is the Cacti Graph Template that I made up too (the Data Templates are included in the XML file since they are dependents). Now all you need to do is setup Cacti to start querying the above SNMP OIDs and then graph them using the template.
| Description: |
| Sendmail and Mailscanner Data and Graph Templates. |
|
Download |
| Filename: |
Sendmail_Mailscanner_Template_0.8.6d.zip |
| Filesize: |
3.53 KB |
| Downloaded: |
1142 Time(s) |
Last edited by warnesj on Tue Jun 07, 2005 12:33 pm; edited 2 times in total |
|
| Back to top |
|
 |
D43m0n
Joined: 01 Jun 2005
Posts: 4
|
| Posted: Wed Jun 01, 2005 2:22 am Post subject: huh... suddenly script returning correct values? |
|
|
Hey great stuff!
I like the flat file approach, it's not that slow at all, I expected to wait a few seconds before a value is returned but hey, I doubt if one full second is needed.
I have a question though. I'm using Postfix instead of Sendmail. That shouldn't make any difference since the only changes needed are:
stat -> status and Sent -> sent. Pretty simple and straightforward, I like that
I added the stuff to my cacti host and my mailserver (other machine) and gather the data using SNMP. I did this yesterday around 5:00 PM and came in this morning, expecting to see beautiful graphs. I do see nice graphs, but for some strange reason, my sent script doesn't return anything anymore. And my received script always returns a zero for some reason. I'm not able to think of any logical reason why the scripts don't return the values anymore. This happened overnight. It seems that just when the clock ticked 0:00, the magic stopped working.
It's not related to SNMP, because I get the same result when I myself execute the scripts as a regular user (non-root) I was able to get great results yesterday as a regular user. I don't know why I can't get results anymore from a simple script that does nothing more than a tail of the last 5000 lines, and then throws some awk lines at it... I'm only using the sent and received scripts that way. They both worked fine yesterday, but suddenly stopped working around midnight. I can't imagine any special cronjob doing anything...
Any clues of what I might be overseeing here?
Thnx!
D
!! EDIT !!
I found the problem. The cause of the problem lies in the string format used by awk. On my server, the format used is different than the format on the server on which these scripts were made.
The problem was discovered pretty soon luckily because the date has changed from may 31 to june 1. The scripts will format the time string to search for: May 31 09:34:56. Last night our servers turned to june. The format returned in maillog was: Jun 1 09:34:56. The script will format the string to Jun 01 09:34:56. The leading zero (or space) is never matched, thus the script will either return a zero or nothing.
To change this behaviour change the following line:
| Code: | | search_time = "^" strftime("%b %d %H:%M", systime() - 5 * 60); |
into
| Code: | | search_time = "^" strftime("%b %e %H:%M", systime() - 5 * 60); |
Then it works again. 
The other thing you can do is wait for 10 days
Thanks to my collegue for helping me out so quickly
|
|
| Back to top |
|
|
anTIDot
Joined: 16 Mar 2005
Posts: 7
|
| Posted: Wed Jun 01, 2005 7:59 am Post subject: |
|
|
| I can not establish a template. It can with a mistake?
|
|
| Back to top |
|
|
warnesj
Cacti User
Joined: 29 May 2005
Posts: 169
|
| Posted: Wed Jun 01, 2005 9:21 am Post subject: Re: huh... suddenly script returning correct values? |
|
|
| D43m0n wrote: | They both worked fine yesterday, but suddenly stopped working around midnight. I can't imagine any special cronjob doing anything...
Any clues of what I might be overseeing here?
!! EDIT !!
I found the problem. The cause of the problem lies in the string format used by awk. On my server, the format used is different than the format on the server on which these scripts were made.
The problem was discovered pretty soon luckily because the date has changed from may 31 to june 1. The scripts will format the time string to search for: May 31 09:34:56. Last night our servers turned to june. The format returned in maillog was: Jun 1 09:34:56. The script will format the string to Jun 01 09:34:56. The leading zero (or space) is never matched, thus the script will either return a zero or nothing. |
Yeah I noticed all my graphs stopped at midnight too. Damn, I should have waited for a month turn-over before posting my scripts. I made the same change you did and edited my original post so nobody else runs into that. 'man date' is now my new friend. Sorry about that. Sweet mod for Postfix too!
| anTIDot wrote: | | I can not establish a template. It can with a mistake? |
I'm not sure what you're asking here. Are you having problems installing the Template through the Import Template option in Cacti? Or are you having problems with one of the scripts and getting Net-SNMP to run.
|
|
| Back to top |
|
|
anTIDot
Joined: 16 Mar 2005
Posts: 7
|
| Posted: Thu Jun 02, 2005 3:05 am Post subject: |
|
|
Sorry for my bad English.
At me a problem import of a template file cacti_graph_template_sendmail_mailscanner.xml
Cacti does not wish it to import.
Cacti version 0.8.6d.
In what there can be a problem?
|
|
| Back to top |
|
|
warnesj
Cacti User
Joined: 29 May 2005
Posts: 169
|
| Posted: Thu Jun 02, 2005 9:48 am Post subject: |
|
|
No appologies required.
Do you get any specific error from Cacti when you try to import the template? Is there anything in the Cacti log file?
Thanks for the info.
|
|
| Back to top |
|
|
anTIDot
Joined: 16 Mar 2005
Posts: 7
|
| Posted: Fri Jun 03, 2005 6:49 am Post subject: |
|
|
I do all as usually:
Console > Import templates > Browse > cacti_graph_template_sendmail_mailscanner.xml > SAVE
And nothing occurs.
Poller Logging Level - DEBUG (poller errors, poller warnings )
In cacti.log - nothing.
|
|
| Back to top |
|
|
yianniska
Joined: 06 Jun 2005
Posts: 4
|
| Posted: Mon Jun 06, 2005 6:22 am Post subject: Import problem |
|
|
| I've got the same problem
|
|
| Back to top |
|
|
gandalf
Developer
Joined: 02 Dec 2004
Posts: 14053
Location: Muenster, Germany
|
| Posted: Mon Jun 06, 2005 12:08 pm Post subject: Re: Import problem |
|
|
| yianniska wrote: | | I've got the same problem |
This is typically a cacti version mismatch between exporter and importer. Importer must have same or higher (?) level than exporter.
HTH
Reinhard
|
|
| Back to top |
|
|
africanw
Cacti User
Joined: 31 Mar 2005
Posts: 116
Location: Sydney, Asutralia
|
| Posted: Mon Jun 06, 2005 5:47 pm Post subject: |
|
|
| Yep same prob with no errors. Running version d.
|
|
| Back to top |
|
|
warnesj
Cacti User
Joined: 29 May 2005
Posts: 169
|
| Posted: Mon Jun 06, 2005 10:08 pm Post subject: |
|
|
Hmmm, strange. I'm running v0.8.6d on Fedora Core 3. I've updated using the RPM that's posted on the main page, but I've redone the update using the tarball that's there. I've re-exported the template and included it below. See if that one works for you.
Sorry all for the problems with the template, hopefully this one works better.
| Description: |
| Second attempt at the Graph & Data Source template. Exported from Cacti v0.8.6d |
|
Download |
| Filename: |
Sendmail_Mailscanner_Template_0.8.6d.zip |
| Filesize: |
3.53 KB |
| Downloaded: |
820 Time(s) |
|
|
| Back to top |
|
|
yianniska
Joined: 06 Jun 2005
Posts: 4
|
| Posted: Tue Jun 07, 2005 6:35 am Post subject: |
|
|
work's for me
thanks
|
|
| Back to top |
|
|
yianniska
Joined: 06 Jun 2005
Posts: 4
|
| Posted: Tue Jun 07, 2005 6:35 am Post subject: |
|
|
work's for me
thanks
|
|
| Back to top |
|
|
yianniska
Joined: 06 Jun 2005
Posts: 4
|
| Posted: Tue Jun 07, 2005 6:36 am Post subject: ok |
|
|
import is ok but still i cannot get any graphs
when i try to create the graphs
i get this error
RRDTool Says:
ERROR: opening '/var/www/html/cacti0.8.6d/rra/eml2_mess_recv_62.rrd': No such file or directory.
i 've put the scripts to the mail server
i also put the extra lines in my ucdsnmp.conf
but no luck.
could you please help me with this because i find this graphs very
nice
thanks
Last edited by yianniska on Tue Jun 07, 2005 9:13 am; edited 1 time in total |
|
| Back to top |
|
|
africanw
Cacti User
Joined: 31 Mar 2005
Posts: 116
Location: Sydney, Asutralia
|
| Posted: Tue Jun 07, 2005 7:46 am Post subject: |
|
|
| Yep now imports.
|
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Register
Profile
Log in to check your private messages
Log in
