Walkthrough on Postifx monitoring with Cacti

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Author
Message
otaviof
Posts: 1
Joined: Tue Apr 21, 2009 6:41 am
Contact:

#76 Post by otaviof » Tue Apr 21, 2009 7:11 am

Mathieu wrote:Hi there,

i made this script work for my configuration of Postfix/ClamAv/Amavisd-New/SpamAssasin this morning (i am using a Sarge Debian fully upgraded). I had to modify both amavisd logformat and fetch_mail_statistics.pl scripts. Here are my modifications :

/etc/amavisd/amavisd.conf

Code: Select all

$log_templ = '[?%#D|Not-Delivered|Passed] [? %#V |[? %#F |[? %#X |CLEAN|BAD-HEADER]|BANNED name/type (%F)]|INFECTED (%V)], #
[?%o|(?)|<%o>] -> [<%R>|,][? %i ||, quarantine %i], Message-ID: %m, Hits: %c';
fetch_mail_statistics.pl

Code: Select all

  while( <LOGFILE> ) {
    if ( / postfix\/smtpd/ && /client=/ && ! /127.0.0.1/ ) { $num_rec++; }
    elsif ( /status=sent/ && ! /relay=(mailfilter|procmail|local|127.0.0.1)/ ) { $num_sent++; }
    elsif ( /status=deferred/ ) { $num_deferred++; }
    elsif ( /status=bounced/ ) { $num_bounced++; }
    elsif ( /: NOQUEUE: reject: / ) { $num_reject++; }
    elsif ( / Passed CLEAN/  && /Hits: ([-+]?[0-9]*\.?[0-9]+)/) {
        $hits = $1;
        $num_clean++ if ($hits < $tag_level);
        $num_pspam++ if ($hits > $tag_level);
        }

    elsif ( / Not-Delivered CLEAN/  && /Hits: ([-+]?[0-9]*\.?[0-9]+)/) {
        $hits = $1;
        $num_clean++ if ($hits < $tag_level);
        $num_spam++ if ($hits > $tag_level);
        }

    elsif ( / Passed INFECTED/ ) { $num_infected++; }
    elsif ( / Not-Delivered INFECTED/ ) { $num_infected++; }
    elsif ( / Passed BAD-HEADER/ ) { $num_header++; }
    elsif ( / Not-Delivered BAD-HEADER/ ) { $num_header++; }
    elsif ( / Blocked BANNED/ ) { $num_banned++; }
    elsif ( / Not-Delivered BANNED/ ) { $num_banned++; }

    }


To use this modification, you must add at the beginning of the script a SPAMLVL variable with your local value. You can find this value in /etc/amavisd/amavisd.conf if you look for
$tag2_level
parameter.
Here is an exemple of such a line :

Code: Select all

$SPAMLVL  = 2.0;
I hope this will help...

Mathieu
Hey Mathieu,

Very nice job. But I have an suggestion:

You can make your script read amavisd.conf $sa_tag2_level_deflt variable automatically, if you include this modification:

<code>
#
# This defines how many seconds we will cache calculated values.
#
our $sa_tag2_level_deflt;
require q{/usr/local/etc/amavisd.conf};

$MINRESET = 50;
$SPAMLVL = $sa_tag2_level_deflt;
</code>

thank you indeed.

--
Otávio Fernandes <otaviof at gmail dot com>
http://otaviof.blogspot.com

enash
Posts: 25
Joined: Tue Oct 04, 2005 9:41 am
Location: Clearwater, FL
Contact:

#77 Post by enash » Wed Jun 24, 2009 12:15 pm

Can someone help here, I have read all 6 pages of this thread and found a couple other people with the same problem as I do and they have yet to get an answer. I hae a random dtat graph. as attached at the bottom. Since i am not using Spam filter on the postfix server and its handled internally. But I do use my access.db file to block out all interrnational IP addresses. I am not much of an XML guy, and when I made the changes I try to re add that xml file so it would update the template, and bam got an XML parse error. I also have helo and header restrictions on. As for the helo u must use your FQDN etc.. I wanted to replace the spam field with the addition all mail my config filters. Here is my XML. What i want is to monitor how much mail is incoming from the external, not just whats coming in from my internal exchange server and external, and be able to graph all my blocks to international IP's, bad helo responses, etc....

After i loaded this XML i had a Mail Rcvd graph. I went in and pulled the OID out of the graph temp and ran a walk against that OID and i was returning the following..

[[email protected] bin]# snmpwalk -v2c -c zzzzzz localhost .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 530

[[email protected] bin]# snmpwalk -v2c -c zzzzzz mysyslogsrv01 .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 341

[[email protected] bin]# snmpwalk -v2c -c zzzzzz mymail01 .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 556

[[email protected] bin]# snmpwalk -v2c -c zzzzzz mymail02 .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 507
Attachments
mailrcvd.png
mailrcvd.png (21.04 KiB) Viewed 6741 times
cacti_host_template_postfix_mailserver.xml
(58.8 KiB) Downloaded 523 times
graph_image.png
graph_image.png (37.56 KiB) Viewed 6751 times

enash
Posts: 25
Joined: Tue Oct 04, 2005 9:41 am
Location: Clearwater, FL
Contact:

#78 Post by enash » Fri Jul 17, 2009 10:07 am

Trying to bump this up for some help.. I still have these gaps in my graph.. Is this due to the poller running every 5 mins? Here is another example. Our company is a pretty big one. And we are one of the bigger online retailers. So we have a lot of mail that is flowing in and out or our shop. In my postfix config I am blocking a ton of international IP ranges. and Im using RBL lists as well. I have a ton of hits in my log file like this

postfix/smtpd[10338]: NOQUEUE: reject: RCPT from unknown[78.172.90.86]: 504 5.5.2 <78.172.90.86>: Helo command rejected: need fully-qualified hostname;

so I know i should have more stats than this in my graph. Can anyone point me in the right directions. whether I have to adjust the graph sizes, or something on the host side. etc..

thanks...
Attachments
mail_transporting_graph.JPG
mail_transporting_graph.JPG (40.05 KiB) Viewed 6596 times

eugenevdm
Posts: 8
Joined: Wed Aug 31, 2005 3:40 am

Re: mailstat.db

#79 Post by eugenevdm » Wed Feb 03, 2010 5:12 am

Here are some troubleshooting tips:

1. Turning on debugging in Caciti / Console / Settings / Poller Logging Level from "LOW - Statistics and Errors" to the next level "MEDIUM..." is very useful.

2. Check under which user your SNMPD is running. On Ubuntu check /etc/default/snmpd to see which user it's running under

3. Be sure to restart snmpd after making changes

4. For all those troubleshooting just using 'snmpwalk', snmpwalk returns empty if you haven't run the script first. So while you're testing run the script first and then snmpwalk.

5. If you're running default polling at 5 minutes I agree with the one author that "$MINRESET = 298;" should be changed from the script writer's default of 50.

6. If you get "WARNING: Result from SNMP not valid. Partial Result: No Such Instance cur" it could be that the script is not running properly. The script might not run properly due to permissions. Permissions might be off due to context of SNMPD user or script / file permissions. Run the script on the console and see if on the next poll that messages disappears. If so examine permissions.

7. Check the rights on the mailstat.db file

8. Check the rights on the script

martin_nl
Posts: 2
Joined: Sun Mar 28, 2010 8:04 am

#80 Post by martin_nl » Sun Mar 28, 2010 8:12 am

I have installed this script, but have one problem.

I get only results in my cacti log when i add this line to a cron job, and
run it once a minute.

/usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/tmp/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1


When i don't add this line to a cron job the cacti log only shows value: U

jangrewe
Posts: 4
Joined: Fri Sep 25, 2009 4:41 am

Re: Walkthrough on Postifx monitoring with Cacti

#81 Post by jangrewe » Mon Jan 10, 2011 5:48 pm

Thanks to everybody for their work, i finally got the script to run under Debian "squeeze".
The trick i used was adding the 'snmp' user, as which snmpd runs, to the 'adm' group - this allowed the script to read the mail log.

Code: Select all

usermod -aG adm snmp
You can find out if your snmpd's user has enough permissions to read the logfile like this (change to your respective paths):

Code: Select all

sudo -u snmp /usr/bin/fetch_mail_stats.pl /var/log/mail.log /var/lib/cacti/mailstats.db .1.3.6.1.4.1.2021.255 -g .1.3.6.1.4.1.2021.255

Creaton
Posts: 2
Joined: Mon Oct 24, 2011 1:32 am

Re: Walkthrough on Postifx monitoring with Cacti

#82 Post by Creaton » Mon Oct 24, 2011 1:35 am

Useful stuff. I need both for the practical results. I’ve seperated certain log functions using the "localX" logging facility to better control the log data obtained on a loaded production server.

pexotinec
Posts: 2
Joined: Thu Jun 28, 2012 10:52 pm

Re: Strange problem

#83 Post by pexotinec » Thu Jun 28, 2012 11:33 pm

Guys any idea ?
snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.255
/usr/local/bin/snmpwalk: line 1: 3310246: command not found
/usr/local/bin/snmpwalk: line 2: 0: command not found
/usr/local/bin/snmpwalk: line 3: 463: command not found
/usr/local/bin/snmpwalk: line 4: 91: command not found
/usr/local/bin/snmpwalk: line 5: 437: command not found
/usr/local/bin/snmpwalk: line 6: 2667: command not found
/usr/local/bin/snmpwalk: line 7: 0: command not found
/usr/local/bin/snmpwalk: line 8: 0: command not found
/usr/local/bin/snmpwalk: line 9: 0: command not found
/usr/local/bin/snmpwalk: line 10: 0: command not found
/usr/local/bin/snmpwalk: line 11: 0: command not found
/usr/local/bin/snmpwalk: line 12: 0: command not found

Lewis
Posts: 35
Joined: Thu May 19, 2005 8:04 pm

Re: Walkthrough on Postifx monitoring with Cacti

#84 Post by Lewis » Sun Aug 02, 2015 10:46 am

hello all,

installed the script, installed the template.. changed $MINRESET = 298;
checked permissions, etc..

i get these on my cacti's log file..

Code: Select all

08/02/2015 05:35:27 PM - CMDPHP: Poller[0] Host[14] DS[171] SNMP: v1: xx.xx.xx.xx, dsname: postfix_sent, oid: .1.3.6.1.4.1.2021.255.1, output: U
08/02/2015 05:35:27 PM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'xx.xx.xx.xx', and OID:'.1.3.6.1.4.1.2021.255.1'
when i run the script locally, i get:

Code: Select all

[email protected]:/usr/local/bin# /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/tmp/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1
.1.3.6.1.4.1.2021.255.2
integer
49


Lewis
Posts: 35
Joined: Thu May 19, 2005 8:04 pm

Re: Walkthrough on Postifx monitoring with Cacti

#85 Post by Lewis » Sun Aug 02, 2015 12:10 pm

its driving me mad... i'm trying the last 8 hours to get some data :(

Code: Select all

[email protected]:~# sudo -u snmp /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/log/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1
.1.3.6.1.4.1.2021.255.2
integer
967
[email protected]:~# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.255
[email protected]:~# 

Lewis wrote:hello all,

installed the script, installed the template.. changed $MINRESET = 298;
checked permissions, etc..

i get these on my cacti's log file..

Code: Select all

08/02/2015 05:35:27 PM - CMDPHP: Poller[0] Host[14] DS[171] SNMP: v1: xx.xx.xx.xx, dsname: postfix_sent, oid: .1.3.6.1.4.1.2021.255.1, output: U
08/02/2015 05:35:27 PM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'xx.xx.xx.xx', and OID:'.1.3.6.1.4.1.2021.255.1'
when i run the script locally, i get:

Code: Select all

[email protected]:/usr/local/bin# /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/tmp/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1
.1.3.6.1.4.1.2021.255.2
integer
49


Post Reply