mod_watch - virtual website statistics

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Author
Message
zman818
Posts: 10
Joined: Sun Aug 13, 2006 5:30 pm

mod_watch - virtual website statistics

#1 Post by zman818 » Wed Aug 16, 2006 10:17 am

mod_watch/Cacti v0.9

- implemented by zman818 ([email protected])
- based on original work by Anthony Howe (snert.com)


mod_watch/Cacti (mwc) is a Cacti implementation of Anthony Howe's most excellent mod_watch module for Apache/MRTG. Mod_watch will watch and collect the bytes, requests, and documents in & out per virtual host, file owner, remote-ip address, directory or location, and the web server as a whole (configured through the use of httpd.conf directives).

A more detailed overview of mod_watch itself can be viewed at http://www.snert.com/Software/mod_watch/

The example below shows simple traffic statistics for 3 individual, virtual sites running under the same apache server. Other statistics are also available (hits, documents, etc). It's a simple adaptation of the stock bandwidth graph. Obviously such things as total traffic could be added to dress it up.

Please note that this is my first attempt at packaging and distributing a template. If I've borked it, I apologize in advance. :)
Attachments
mod_watch.png
mod_watch.png (81.13 KiB) Viewed 54226 times
modwatch_cacti.zip
(5.56 KiB) Downloaded 2631 times

jmeyerdo
Posts: 26
Joined: Mon May 02, 2005 7:11 am
Location: Dortmund, Germany

#2 Post by jmeyerdo » Mon Jan 08, 2007 12:31 pm

Hi!

Great script/module.
But unfortunately website of mod_watch is not available these days (no longer available?).

Does anyone know if the module (as RPM for Fedora Core or as Source) is available anywhere else?

Is there any other solution to graph the traffic for different virtual hosts?

Kind regards, Jens

unclecameron
Posts: 1
Joined: Thu Jan 11, 2007 12:16 pm
Contact:

#3 Post by unclecameron » Thu Jan 11, 2007 2:12 pm

Yeah, I found the same problem with snert not serving up pretty much anything these days, for no apparent reason. Anyway, I found this version of mod_watch lurking around somewhere, YMMV though as I haven't been able to install it yet, since I don't want to recompile and apxs wasn't loaded originally on the server I'm testing it on. I'm building a new server this weekend that I'll try to build this module into, I'll let you know if that works.

Anyway, here it is, let me know if you get it working :)

Cameron
San Diego
Attachments
mod_watch403.tgz
mod_watch4.03, this one is for Apache2, I don't have the older version for Apache 1.3
(97.25 KiB) Downloaded 2474 times

WimanX
Posts: 4
Joined: Wed Jan 17, 2007 3:57 am

#4 Post by WimanX » Fri Jan 19, 2007 6:58 am

Does a similar template exist for IIS?

drewhka
Posts: 1
Joined: Thu Apr 19, 2007 7:37 am

#5 Post by drewhka » Thu Apr 19, 2007 7:44 am

Corrected version of mod_watch_4.3 for Apache 2.2 (old sources contain deprecated macro & functions). Successfuly tested on Apache/2.2.4 (FreeBSD)
Attachments
mod_watch-4.3_apache22_mod.tar.gz
(95.74 KiB) Downloaded 4340 times

Indiana
Posts: 13
Joined: Fri Jul 06, 2007 6:28 am

#6 Post by Indiana » Fri Jul 06, 2007 6:35 am

Hi all,

I use mod_watch4.03 with apache 2.0 (on debian sarge) but /watch-list didn't show all virtual hosts.
Only 18 of 21 :-(
The missed three are equal the other vhosts entries (all are automatic generated out of a webfrontend).
[b]Have a N.I.C.E. day![/b]
[img]http://www.bluepaw.de/projects/dynbar/indiana.1.png[/img]
Real programmers don't comment their code - it was hard to write, it should be hard to understand.

Cacti 0.8.6i on Debian Etch

Indiana
Posts: 13
Joined: Fri Jul 06, 2007 6:28 am

#7 Post by Indiana » Fri Nov 09, 2007 9:34 am

Hmm .. still get some crazy results .. some vhosts are listed some not :-(
Moved now from apache2.0 to 2.2 (and mod watch 4.03 to 4.3).

Edit: Ok, now I got it .. It there is no bit of the vhost requestet it isn't in the watch-hash.
[b]Have a N.I.C.E. day![/b]
[img]http://www.bluepaw.de/projects/dynbar/indiana.1.png[/img]
Real programmers don't comment their code - it was hard to write, it should be hard to understand.

Cacti 0.8.6i on Debian Etch

firehole
Posts: 1
Joined: Tue May 05, 2009 6:54 pm

Missing vhosts

#8 Post by firehole » Tue May 05, 2009 6:55 pm

Indiana -

How did you fix your problem with missing vhosts? I have the same issue.

ziggyz27
Posts: 2
Joined: Wed Feb 17, 2010 4:11 am

Modified apache2mrtg.pl for includes in Apache2.x

#9 Post by ziggyz27 » Wed Feb 17, 2010 4:24 am

Ok i hope this helps. Unlike normal Apache 2 my setup uses Includes to include Virtual Hosts for different domains. So the apache2mrtg.pl script that comes with mod_watch-4.3 did not work for me, it was just unable to find my virtual_hosts.

So modified the script to parse Includes.
NOTE: This is a quick and dirty modification but works for me.
I have also added few print statements commented out here to debug.

Code: Select all

#!/usr/bin/perl
#
# apache2mrtg.pl
#
# Copyright 2001, 2003 by Anthony Howe.  All rights reserved.
#

# Where the scripts should live
$SCRIPTDIR = "/usr/local/sbin";

##########################################################################
#       Nothing to be configured below this point.                       #
##########################################################################
$AUTHOR = '[email protected]';
$VERSION = '2.0';


unless (@ARGV == 1) {
        print STDERR "usage: apache2mrtg.pl path/to/httpd.conf\n";
        exit 2;
}

%seen = ();

sub target {
        my $name = shift;

        if ($seen{$name}) {
                print STDERR "DUPLICATE entry \"$name\"\n";
                return;
        }

        $seen{$name} = 1;

        print <<EOT;
Title[$name]: $name Data Traffic
Target[$name]: `${SCRIPTDIR}/mod_watch.pl http://$name/watch-info`
MaxBytes[$name]: 1250000
PageTop[$name]: <h2><a href="http://$name/">$name</a> Data Traffic</h2>

EOT
}

#########################################################################

$x=0;
while (<ARGV>) {
        # Skip commented blocks
        next if m!\#.*</?virtualhost!i;

        if (m!<virtualhost\s!i .. m!</virtualhost>!i) {
                $name = $1 if m!ServerName\s+(\S+)!;
        }

        if (m!</virtualhost>!i) {
                target($name);
        }

#############################################################################
#     Added to make script compliant to httpd.conf that uses Includes to    #
#     include virtual hosts. For example: Direct Admin type environments.   #
#############################################################################
        if(m!Include\s!i) {

                                my($text) = $_;
        #                       print "$_\n";
                                my($directory, $filename)= $text  =~ m/(.*\/)(.*)$/;
                                $directory =~ s/^\s+|\s+$//g ;
                                $directory =~ s/Include //;
        #                       print "D=$directory, F=$filename\n";
                                $mypath = $directory . "/" . $filename;
                                if($filename =~ m/httpd.conf/){

        #                               print "Processing $mypath we need it\n===============================================\n";
                                        $ALLHTTPDCONFS = $mypath;
                                        open(ALLHTTPDCONFS) or die("Could not open log file.");

                                        while(<ALLHTTPDCONFS>){

                                                 # Skip commented blocks
                                                        next if m!\#.*</?virtualhost!i;

                                                        if (m!<virtualhost\s!i .. m!</virtualhost>!i) {
                                                                $name = $1 if m!ServerName\s+(\S+)!;
                                                        }

                                                        if (m!</virtualhost>!i) {
        #                                                        print "Processing for: $name \n***************************************\n";
                                                                target($name);
                                                        }



                                        }

                                                $x++;
                                } else {
        #                               print "Skipping $filename we dont need it\n==============================================\n";
                                }

        }
        #       print "Processed total $x domains from includes!\n";
}

deksar
Posts: 2
Joined: Fri Feb 26, 2010 4:47 pm

#10 Post by deksar » Fri Feb 26, 2010 4:52 pm

I would like to limit monthly bandwidth usage of some of my (apache) virtualhost accounts (shared IP - domain based) and check their current bandwidth usage to via a URL.
I mean a limit like: 20 GB/Month. Is this possible with mod_watch/Cacti?

(for Apache 2.2.x.)

By the way, thanks a lot for the corrections, drewhka and ziggyz27.
Thank you all!
Last edited by deksar on Fri Feb 26, 2010 5:09 pm, edited 1 time in total.

ziggyz27
Posts: 2
Joined: Wed Feb 17, 2010 4:11 am

Use mod_cband

#11 Post by ziggyz27 » Fri Feb 26, 2010 5:02 pm

You should use an apache add-on like mod_cband

Checkout
http://www.freshports.org/www/mod_cband/

deksar
Posts: 2
Joined: Fri Feb 26, 2010 4:47 pm

Re: Use mod_cband

#12 Post by deksar » Fri Feb 26, 2010 5:07 pm

ziggyz27 wrote:You should use an apache add-on like mod_cband
Checkout
http://www.freshports.org/www/mod_cband/
I tried it already a while ago. Too many errors..
Unfortunately, it is not getting updated anymore. It seems mod_cband died long time ago. Dozens of bugs are waiting:
http://sourceforge.net/tracker/?group_i ... tid=791368
:(

guilgot
Posts: 5
Joined: Fri Mar 19, 2010 4:21 am

Need help for install

#13 Post by guilgot » Fri Mar 19, 2010 4:45 am

I need a solution to view in realtime the bandwidth used by virtualhosts...
This should be the soluion and I am happy to see that I am not the only one trying to use this great module !!!

The instalation with apache2 looks ok (well, no error message at restart...)

BUT...
I still have nothing when I check http://mydomain/watch-info
and ./mod_watch.pl -f ifRequests,ifDocuments http://mydomain/watch-list returns 0 0 0...

Secondly, do you know how to install it as a plugin on Munin (as explain here : http://www.freshnet.org/wordpress/2007/ ... ith-munin/) ?


Regards,

Guillaume

guilgot
Posts: 5
Joined: Fri Mar 19, 2010 4:21 am

day after day....

#14 Post by guilgot » Sun Mar 21, 2010 9:05 am

I think I have read all web pages about mod_watch without any success...

I try to make it run with cacti and munin but the problem seems to come from mod_watch it self which don't want to give me any response to watch-info... (watch-table and watch-list does now...)
And more I have found a recursive error in in my error.log :
shVerifyString(b7fa6c58, b4ac3b18) failed!
[Sun Mar 21 14:55:05 2010] [crit] (20014)Internal error: shGetLockedEntry(b7fa6c58, "SERVER") failed in watchLogHash()
I use Apache/2.2.12 (Ubuntu) PHP/5.2.10-2ubuntu6.4 with Suhosin-Patch with mod_watch-4.3_apache22_mod...

I use several access log files... Do you think it could come from that ?
Do I need to use apache2mrtg.pl ?


Please, could you tell if, finaly, it works on your servers because I become crasy !`

Thanks in advanced,

Guilgot

User avatar
hbokh
Posts: 40
Joined: Thu Feb 24, 2005 4:29 pm
Location: Groningen, NL

#15 Post by hbokh » Wed Apr 14, 2010 9:36 am

Regarding the error log / watchLogHash I found a fix here:
http://www.ovcharov.me/2009/09/01/kak-p ... #more-2295
It's in Russian but it comes to adding /altering a line in "SharedHash.c" at line 102 and should look like this:

Code: Select all

shVerifyString(struct shTable *tp, char *str)
{
    /* if ((char *) tp->shared <= str && str < (char *) tp->eshared) { */
    if ((char *) apr_shm_baseaddr_get(tp->shared) <= str && str < (char *) tp->eshared) {
        if (str + strlen(str) < (char *) tp->eshared)
            return str;
    }
Recompile the whole bunch and it should work somewhat better (read: no more errors).

On this side of the line getting info from "watch-list" DOES work, "watch-info" however gives a 404...
This is my sick nature...

Post Reply