|
|
| Author |
Message |
Loceur
Joined: 09 Mar 2006
Posts: 2
|
 Posted: Thu Mar 09, 2006 10:33 am Post subject: [HOWTO] Netflow |
 |
|
Ok, so here's my first post, but it seems like a needed one.
Things needed:
flow-tools
flowscan
CUFlow
cacti (I'm using 0.8.6h)
I'm using debian, so I was able to apt-get everything. Please note, though, that you need the testing version of flowscan to be able to use flow-tools.
So here's how it works:
| Code: |
Netflow from router -> flow-capture -> flowscan -> rrds -> cacti
CUFlow -^
|
Netflow from Router:
On Cisco -
| Code: |
ip cef
! Do THIS ON EACH INTERFACE
interface e0/0
ip route-cache flow
no ip mroute-cache
!
ip flow-export version 5 peer-as bgp-nexthop
ip flow-export destination 1.2.3.4 9999
! where 1.2.3.4 is your flow colector's IP and 9999 is the port you'll choose
|
Easy enough...
flow-capture:
apt-get install flow-capture #I love good package management
vi /etc/flow-tools/flow-capture.conf
Add the following:
-z0 -V5 -n 288 -N0 -w /home/flows -E2G 0/0/9801
Feel free to man flow-capture for details. You should change these details to suite your needs. Of main interest is the last number, "9801", this is the port that needs to correspond with your previous entry into your cisco router. -z0 -V5 -N0 is important to have to work correctly with flowscan. -n 288 means to make a file capture every 5 minutes. -w is your save directory.
flowscan:
Pick your favorite method and install flowscan (note: needs to support flow-tools, not just cflowd, which ARE different)
apt-get install -t testing flowscan #good_package_management++
edit /etc/flowscan/flowscan.cf (or wherever your flowscan.cf is)
Change the defaults to:
| Code: |
FlowFileGlob /home/flows/ft-v05.*
ReportClasses CUFlow
WaitSeconds 30 # change to 300 if you use a SLOW machine or you're processor paranoid
Verbose 1 # you can change this later when it works for sure
|
Flowscan didn't come with an init script, so I made one.
| Code: |
#!/bin/sh
#Description: Start/stop Flowscan
case "$1" in
'start')
/usr/bin/flowscan >> /var/log/flowscan 2>&1 </dev/null & >/dev/null
;;
'stop')
killall -9 flowscan
;;
*)
echo "Usage: $0 {start | stop }"
;;
esac
exit 0
|
Then, don't forget to add it to your startup scripts.
Optional:
Next, I wanted to keep 95th percentile marks ACCURATE for up to 2 months, so I had to change flowscan's way of making default rrds.
You don't have to do this if you don't care about accurate 95th percentile marks.
Edit your FlowScan.pm
Find the following and change the values to:
RRA:AVERAGE:.5:1:17280
RRA:AVERAGE:.5:6:1536
RRA:AVERAGE:.5:24:2268
RRA:AVERAGE:.5:288:1890
RRA:MAX:.5:24:2268
RRA:MAX:.5:288:1890
This SHOULD keep up to 60 days worth of non-compressed data before it starts to compress it down. Feel free to tell me if I'm wrong about this.
CUFlow:
Install cuflow perl module:
apt-get install -t testing flowscan-cuflow
edit /etc/flowscan/CUFlow.cf
Actually, go man CUFlow on google, as you'll need to customize this, however, I'll give an example:
| Code: |
Subnet 1.2.3.0/24 #my internal networks
Network 1.2.3.128/25 customera # Customer A
Network 1.2.3.0/25 customerb #Customer B
OutputDir /home/flows/rrds
|
Comment out the rest of the crap, unless you want a dozen graphs that do a dozen things. This config is where you make different graphs based on IP assignment. Very useful in an ISP environment.
Now, if all goes well, you should be making rrds in /home/flows/rrds
Also, if you want to save your raw flow files, create a directory called "saved" in your /home/flows directory, as flowscan deletes the processed flows if it doesn't find a ./saved directory.
rrds -> cacti:
Feel free to search for your favorite process to do this. There are lots of tutorials on HOW to do this already. I will, however, link my personal favorite.
http://forums.cacti.net/viewtopic.php?t=12202
Also, I just use duplicated the Traffic - (bits/sec, w/ 95th percentile) graph template for the netflows and it looks great!
| Description: |
|
| Filesize: |
69.58 KB |
| Viewed: |
37385 Time(s) |
|
|
|
| Back to top |
|
 |
Loceur
Joined: 09 Mar 2006
Posts: 2
|
| Posted: Thu Mar 09, 2006 1:43 pm Post subject: |
|
|
Hmm, maybe this should be in the HOWTO section. oops! 
|
|
| Back to top |
|
|
Howie
Cacti Guru User
Joined: 16 Sep 2004
Posts: 2162
Location: United Kingdom
|
| Posted: Thu Mar 09, 2006 3:51 pm Post subject: |
|
|
It's worth mentioning that there are a couple of Unix tools to produce netflow data too (by sniffing a mirrored switch port, for example), if you don't have support for netflow in your routers.
On FreeBSD, check out ng_netflow, and also fprobe on Linux.
I've been using ng_netflow and nfsen for some time to monitor our transit usage better, but I hadn't thought about Cacti integration - that looks cool!
|
|
| Back to top |
|
|
warenet
Cacti User
Joined: 10 Feb 2003
Posts: 53
Location: Southern / CA
|
| Posted: Mon Mar 13, 2006 7:51 pm Post subject: |
|
|
I'm a bit confused on the editing of the RRD metadata.
I've edited the default cacti RRA settings and basically set the monthly one to have "5 minutes" worth of data by changing the "Rows" to 8928.
I guess I'm confused about the relation between rows and timespan, but I'd think you'd have to change the timespan for the daily RRA?
My monthly RRA (that I thought had 5 minute averages) is:
X-files: 0.5
Steps: 1
Rows: 8928
Timespan: 2678400
Is this wrong?
TIA
|
|
| Back to top |
|
|
phiz
Joined: 04 Apr 2006
Posts: 2
Location: Phoenix
|
| Posted: Tue Apr 04, 2006 1:42 am Post subject: Netflow Template |
|
|
Good HOWTO. Thanks for the info. I'm using the | Code: | Netflow from router -> flow-capture -> flowscan -> rrds -> cacti
CUFlow -^ |
setup and it works great. If anyone has any questions with this setup, feel free to e-mail me.
Do you have any more information about duplicating the Traffic - (bits/sec, w/ 95th percentile) graph template for the netflows? I'm having trouble with the process. I have duplicated the graph template, but I can't seem to figure out where to go from there.
Thanks again for the great HOWTO.
C
|
|
| Back to top |
|
|
chercen
Joined: 09 Apr 2006
Posts: 35
Location: Spain
|
| Posted: Sun Apr 09, 2006 4:19 pm Post subject: |
|
|
Nice howto...
However I think using ntop plugin for cacti offers much more information and is pretty easy to setup (enabling netflow add-on). Of course, it always depends on how much information you require 
Regards,
|
|
| Back to top |
|
|
qwertz
Cacti User
Joined: 16 Feb 2006
Posts: 98
|
| Posted: Sun Jun 04, 2006 5:20 am Post subject: |
|
|
Hello,
i tried the Howto in the follonwing link:
http://www.prolixium.com/sitenews.php?id=482
It looks like this tutorial but when i start flowscan, i have the following thing:
flowscan
2006/06/04 12:07:29 working on file /var/lib/netflow/ft/ft-v05.2006-06-04.120501+0200...
/var/lib/netflow/ft/ft-v05.2006-06-04.120501+0200: Invalid index in cflowd flow file: 0xCF100103! Version 5 flow-export is required with *all* fields being saved.2006/06/04 12:07:29 flowscan-1.020 CUFlow: Cflow::find took 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU) for 1980 flow file bytes, flow hit ratio: 0/0
2006/06/04 12:07:29 flowscan-1.020 CUFlow: report took 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
sleep 30...
When i try a rrdtool info on a rrd file just created, there is UNKN values inside.
rrdtool info total.rrd
filename = "total.rrd"
rrd_version = "0003"
step = 300
last_update = 1149416933
ds[in_bytes].type = "ABSOLUTE"
ds[in_bytes].minimal_heartbeat = 400
ds[in_bytes].min = NaN
ds[in_bytes].max = NaN
ds[in_bytes].last_ds = "UNKN"
ds[in_bytes].value = 0.0000000000e+00
ds[in_bytes].unknown_sec = 0
etc ...
Regards
QWertz
|
|
| Back to top |
|
|
qwertz
Cacti User
Joined: 16 Feb 2006
Posts: 98
|
| Posted: Sun Jun 04, 2006 3:05 pm Post subject: |
|
|
hello:
here
http://www.onlamp.com/pub/a/bsd/2005/09/15/Big_Scary_Daemons.html?page=3
I read
If FlowScan complains about an "Invalid index in cflowd flow file," you probably didn't install the newest Flowscan.pm module. This is perhaps the most common error people make with FlowScan. If you have this problem, go get the appropriate version of the module as described earlier.
So what?
what do i have to download or check?
Thanks
Qwertz
|
|
| Back to top |
|
|
tinycamp
Joined: 27 Jul 2006
Posts: 1
|
| Posted: Thu Jul 27, 2006 11:29 pm Post subject: no monthly graphs!!! |
|
|
it worked just fine for 20 days, now i dont have monthly graphs!!!!
heeeeeelp!
|
|
| Back to top |
|
|
qwertz
Cacti User
Joined: 16 Feb 2006
Posts: 98
|
| Posted: Wed Aug 23, 2006 6:23 am Post subject: |
|
|
Nice, i am very happy for you.
Which tutorial did you use?
On my side i had nothing in my rrdfile with ubuntu or Debian
Thanks
qwertz
|
|
| Back to top |
|
|
Skept
Joined: 03 Jul 2005
Posts: 12
|
| Posted: Sun Oct 01, 2006 3:42 pm Post subject: |
|
|
Qwertz,
this error in flowscan logs means that CFlow is not compiled with flowtools support
Invalid index in cflowd flow file: 0xCF100103! Version 5 flow-export is required with *all* fields being saved.
to enable cflow support, ensure that you compile cflow from within flow-tools directory, right after you compile flow-tools. you will also see the -OSU flag during the compile process.
|
|
| Back to top |
|
|
Skept
Joined: 03 Jul 2005
Posts: 12
|
| Posted: Sun Oct 01, 2006 11:55 pm Post subject: |
|
|
qwertz,
[skept@stopgap flow-tools-0.68]$ cd contrib/
[skept@stopgap contrib]$ tar xzf Cflow-1.051.tar.gz
[skept@stopgap contrib]$ cd Cflow-1.051
[skept@stopgap Cflow-1.051]$ perl Makefile.PL
Checking if your kit is complete...
Looks good
Found flow-tools... using "-DOSU -I../../lib -I../../lib/.. -L../../lib -lft -lz".
Anyway, i configured yet another system this morning
|
|
| Back to top |
|
|
justinchudgar
Joined: 19 Oct 2006
Posts: 9
Location: Weed, Siskiyou County, CA, USA
|
| Posted: Thu Oct 19, 2006 8:06 pm Post subject: No RRD file output... |
|
|
I've checked twice and I am using the latest (Debian unstable) version of all relevant packages. I've also run though the how-to steps several times and I am convinced that I did not miss anything.
Flowscan runs, and, it shows <i>sleeping 30...</i> in /var/log/flowscan.log on the scheduled intervals. However, nothing shows up in my netflows/rrds directory. I know that I am getting netflow traffic to my box, as well.
What is happening is files are created in netflows/ every 5 minutes ( -n 288). Following is a ls of my netflows/:
<i>
-rw-r--r-- 1 root root 10708 2006-10-19 17:51 ft-v05.2006-10-19.174952-0700
-rw-r--r-- 1 root root 15892 2006-10-19 17:56 ft-v05.2006-10-19.175118-0700
-rw-r--r-- 1 root root 19796 2006-10-19 18:01 ft-v05.2006-10-19.175617-0700
-rw-r--r-- 1 root root 15188 2006-10-19 18:06 ft-v05.2006-10-19.180116-0700
-rw-r--r-- 1 root root 22100 2006-10-19 18:11 ft-v05.2006-10-19.180615-0700
-rw-r--r-- 1 root root 15188 2006-10-19 18:16 ft-v05.2006-10-19.181113-0700
-rw-r--r-- 1 root root 19604 2006-10-19 18:21 ft-v05.2006-10-19.181613-0700
drwxrwxrwx 2 root root 4096 2006-10-19 09:53 rrds
drwxrwxrwx 2 root root 4096 2006-10-19 10:00 saved
-rw-r--r-- 1 root root 84 2006-10-19 18:21 tmp-v05.2006-10-19.182111-0700
</i>
In case it matters, I am running Ubuntu Edgy and am trying to receive flow data from several WRT-54's. DD-WRT's rflow-collector works fine under windows; but, I am transistioning to Linux.
Please direct me where to troubleshoot from. And, please be more verbose than you would with a longtime 'NIX sysadmin since I am not as skilled as most of you.
Thanks.
____________________
Fixed that problem. In CUFlow.cf, the networks must be subsets of the subnets. See http://www.columbia.edu/acis/networks/advanced/CUFlow/CUFlow.html for a good description of the cuflow options.
|
|
| Back to top |
|
|
caseinpoint
Joined: 31 Oct 2006
Posts: 2
|
| Posted: Tue Oct 31, 2006 9:10 pm Post subject: How does flowscan know... |
|
|
How does flowscan know where to load the config file from?
Is it compiled into the binary?
I think I'm starting my flowscan binary, but it's not reading my configuration file.
How do I make sure that my startup script tells flowscan where to look for my config?
|
|
| Back to top |
|
|
caseinpoint
Joined: 31 Oct 2006
Posts: 2
|
| Posted: Wed Nov 01, 2006 9:00 am Post subject: Also... |
|
|
This never worked for me
| Code: | | apt-get install -t testing flowscan-cuflow |
I'm looking at the standard and unvirse apt lists and I can't find those anywhere.
Where did you get that?
|
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Register
Profile
Log in to check your private messages
Log in
