F5 BigIP Virtual Server Query

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Author
Message
jcaesar
Posts: 32
Joined: Mon Nov 01, 2004 5:07 pm
Location: Atlanta, GA

F5 BigIP Virtual Server Query

#1 Post by jcaesar » Thu Dec 30, 2004 10:58 am

I've set up a data query for F5 virtual servers in Cacti 0.8.6c. The only useful info that I could find to graph was traffic and connections on a per-virtual server basis, so that is all these are set up to do. The query xml file and template exports are attached.

All you need to do is put the f5_bigip_vs.xml file into your cacti_directory/resource/snmp_queries/ directory, and then use the template import feature with the cacti_data_query_snmp_f5_bigip_virtual_servers.xml file.

I've never done a template export before, so please let me know if there are any issues. I successfully imported these on my test server so they should be working OK.

Jamie

P.S. Similar query for SSL Proxies can be found here.
Attachments
f5_bigip_vs.xml
F5 Virtual Server Query XML file. Put in your cacti_dir/resource/snmp_queries/ directory.
(2.05 KiB) Downloaded 3351 times
cacti_data_query_snmp_f5_bigip_virtual_servers.xml
F5 Virtual Server template export.
(25.05 KiB) Downloaded 3963 times

duckhead
Cacti User
Posts: 59
Joined: Wed Oct 20, 2004 7:41 pm

#2 Post by duckhead » Mon Jan 03, 2005 3:52 pm

I tested out the import process, and it works fine. It appears to get getting the pools correctly also (but I haven't confirmed that yet.) Thanks for the new feature. I'll provide an update when I've completed the 0.8.6c upgrade process and get the graphs working.

duckhead

duckhead
Cacti User
Posts: 59
Joined: Wed Oct 20, 2004 7:41 pm

#3 Post by duckhead » Mon Jan 03, 2005 4:36 pm

I think I've found an oddity (or maybe it's just my stupidity on F5's, I'm not sure.)

My Virtual servers that are straight IP matches without a rule work great. They appear with the correct pool name, and "none" for the rule name.
For example:
10.19.1.236:80
Pool: CUSTOMER1
192.168.17.36:80
192.168.18.36:80
My virtual servers that are straight IP matches, but with a rule don't have the correct pool name, but have the correct rule name:
10.19.1.236:80
Rule: CUSTOMER1_RULES
Pool: CUSTOMER1
192.168.17.36:80
192.168.18.36:80
This appears in the Create Graph page with the correct rule name, but no pool name.

The big problem is with 1 Rule and multiple pools. This looks like:
10.19.1.247:80
Rule: SMALLCUSTOMER_RULES
Pool: SMALLCUSTOMER3
192.168.17.28:80
192.168.18.28:80
Pool: SMALLCUSTOMER4
192.168.17.14:80
192.168.18.14:80
In this case, I get 1 "Create Graph" line, with the correct rule name, and "none" for the pool name. I would have expected 2 lines, one for each pool.

Like I say, I might be stupid with F5's, but what I thnk I want to monitor is trafiic on each of the pools. I think that what this code is offering me is traffic on each of the virtual servers. If that's the case, I apologize. If not, I'll be happy to help test and get these graphs working.

duckhead.

jcaesar
Posts: 32
Joined: Mon Nov 01, 2004 5:07 pm
Location: Atlanta, GA

#4 Post by jcaesar » Tue Jan 04, 2005 10:17 am

I'm a little confused about what you are seeing. As far as I'm aware, for each virtual server you can either:

1) Assign a pool of addresses to load balance the connections across,
Or
2) Assign a rule that does something with the particular traffic (for example, redirect HTTP to HTTPS)

I'm not sure how you are getting a rule and a pool assigned to the same virtual server (identified by the IP address and port combination), or getting multiple pools assigned to the same virtual server.

We have set up something along the lines of:

Virtual Server: 10.1.1.1 Port 80 -> Rule to Redirect to HTTPS
Virtual Server: 10.1.1.1 Port 443 -> Pool of web servers

In that case, I'd monitor the virtual server with the Index of 10.1.1.1.443 to track bandwidth and connections that are being distributed across that pool.

The Query would show:

10.1.1.1.80 Rule: http_redirect Pool: none
10.1.1.1.443 Rule: none Pool: Server_Pool

Hope that helps. Let me know if I'm confusing your issue.

Jamie

duckhead
Cacti User
Posts: 59
Joined: Wed Oct 20, 2004 7:41 pm

#5 Post by duckhead » Tue Jan 04, 2005 10:56 am

No problem. I've got no doubt that we can work this through. The first example I gave, with CUSTOMER1, is a customer that has it's own IP address. They come in on port 80, and get balanced to 2 nodes on port 80. There's no rule required there, since they're the only customer hitting that 10. IP.

In the second example, we have CUSTOMER2 hitting just that one IP, so there's just one pool. In this case, we have a rule in place that just checks the web site to make sure it's up on a scheduled interval. We have other rules that redirect port 80 to 443.

In the 3rd example, we have 1 IP Address, but many different hostnames. We use the hostname to redirect to the appropriate pool. So, 1 IP address on the outside, 1 rule to determine which pool, and
many pools. one for each customer.

What I think I'm interesting in monitoring is the throughput on each pool, not on the virtual server. After all, even though it's possible to have a 1 to 1 mapping of virtual server to pool (which it sounds like you have), that's certainly not required. And, since it's harder and harder to get discrete IP's for each web server, I think that's a common configuration. I think that your script is just doing the virtual servers, instead of the pools. The list under "Create Graphs" is the same as my virtual server list, with the only difference that the "Create Graphs" page is either displaying pool name or rule name, but not both. From what I can tell, it's favoring the rule name.

duckhead

jcaesar
Posts: 32
Joined: Mon Nov 01, 2004 5:07 pm
Location: Atlanta, GA

#6 Post by jcaesar » Tue Jan 04, 2005 5:12 pm

Ok. Now I understand what you mean. I didn't know that you could set up the F5s like you have them in scenario #3.

This query was created to monitor from the virtual server's point of view, which is a different MIB tree than either SSL Proxies or Pools. I assume that you are using a rule to direct to different pools based on the hostname. If that is the case, there isn't a direct associate between a pool and virtual server so the F5 MIB will contain "none" under the pool name, and that is what cacti will show you. My guess is that you'll only see the associated rule.

Like you said, you'll need to monitor on a per-pool basis, and I didn't write a query that will do that. It will take me a little time to see if I can set up a query for pools. The issue is that the index isn't a fixed length like it is for virtual servers and proxies. With virtual servers, the IP address and port was used for the index, so the index was always going to be 5 OID values long (10 in the case of the SSL proxy, which uses the original and destination IP and port as the index). When it comes to pools, the index is a string of ASCII character numbers based on what the name of the pool is.

So, if the pool is named: ABC, the OID will be:
<POOL BASE OID>.65.66.67

And if you have a second pool named TestPool, the OID will be:
<POOL BASE OID>.84.101.115.116.80.111.111.108

I'll have to fiddle around with the regular expression that identifies which part of the OID is the index. I'll let you know if I figure something out.

jcaesar
Posts: 32
Joined: Mon Nov 01, 2004 5:07 pm
Location: Atlanta, GA

#7 Post by jcaesar » Wed Jan 05, 2005 10:00 am

I'm probably going to need someone who is a lot better with Regular Expressions and/or Cacti internals to make an attempt at this. The other two F5 queries have a fixed length index, so you could just anchor the RegEx to the end of the OID string and then count back however far you need.

Since this one is variable length I can't do it the same way. So in this case you'd need to start matching at the end of the base portion of the OID, and then run to the end of the OID to get the index. I haven't been able to do this without matching a portion of the base OID as well, which makes your index values incorrect.

rcariddi
Posts: 12
Joined: Fri Sep 10, 2004 3:42 pm
Location: Charlotte, NC
Contact:

What version of BigIP ??

#8 Post by rcariddi » Thu Jan 13, 2005 3:47 pm

Hi Jamie,

What version of BigIP did you test this against?
I am having a problem after I input your xml's.
I created a host template using your graph templates:
1) F5 BigIP - Virtual Server Connections
2) F5 BigIP - Virtual Server Traffic (bits/sec)

I copied the other XML files into ~cacti/resource/snmp_query

But when I apply the host template to the device:

BigIP SSL
Warning: Variable passed to each() is not an array or object in
/usr/local/apache2/htdocs/cacti-0.8.6c/lib/data_query.php on line 490

(*For the VS's*)
Warning: Cannot modify header information - headers already sent by
(output started at /usr/local/apache2/htdocs/cacti-0.8.6c/lib/data_query.php:490)
in /usr/local/apache2/htdocs/cacti-0.8.6c/host.php on line 121

(* for the Proxies*)
ECOM Proxies.
Warning: Variable passed to each() is not an array or object in
/usr/local/apache2/htdocs/cacti-0.8.6c/lib/data_query.php on line 490

Warning: Cannot modify header information - headers already sent by
(output started at /usr/local/apache2/htdocs/cacti-0.8.6c/lib/data_query.php:490)
in /usr/local/apache2/htdocs/cacti-0.8.6c/host.php on line 121

When I go to create a graph:
Data Query [SNMP - F5 BigIP - Virtual Servers]
Error in data query.

I have this error and I can't get anything but a couple of broken graphs.

I tested out all your OID's and they seemed to work, except I had to change 2 being that the OID didn't exist, this could be due the version of the BIGIP's.
1.3.6.1.4.1.3375.1.1.3.2.1.38
1.3.6.1.4.1.3375.1.1.3.2.1.39
For instance, I just changed them to
1.3.6.1.4.1.3375.1.1.3.2.1.13
1.3.6.1.4.1.3375.1.1.3.2.1.14
Byte's in and Byte's out respectively.

Anyways, if you have any suggestions, it would be appreciated, else, I just keep trying...
We have something like 900 Virtual Servers/Proxies :-?

Thanks,
Rich
/rjc

jcaesar
Posts: 32
Joined: Mon Nov 01, 2004 5:07 pm
Location: Atlanta, GA

Re: What version of BigIP ??

#9 Post by jcaesar » Mon Jan 17, 2005 4:45 pm

Those OIDs that you changed were for the High Capacity (64-bit) counters on the F5. If you are running on older code that might explain why they didn't exist. I think we are on the latest code rev and using a 2000 series BigIP.

The only problem I had when creating the data query was that I forgot to put read permissions on the .xml file. I don't remember the error message, but I know it wasn't obvious that it was a permissions issue from that output.

I probably should modify the XML and template to include both the 32-bit and 64-bit counters.

gkuchera
Posts: 11
Joined: Fri Feb 04, 2005 5:01 pm
Location: Minneapolis, MN

#10 Post by gkuchera » Fri Feb 04, 2005 5:04 pm

Question:

What are you suposed to put in the fields that is inquires about upon creation of the graphs?

Index Type:
Index Value:
Output Type ID:

Thanks,
Geoff Kuchera

acidic99
Posts: 2
Joined: Mon Feb 14, 2005 2:20 pm

bump

#11 Post by acidic99 » Mon Feb 14, 2005 2:27 pm

I'm not sure what to put in these fields either.

firozrangwala
Posts: 17
Joined: Mon Jan 24, 2005 4:00 pm

error while trying to create the graphs using the F5 templt.

#12 Post by firozrangwala » Fri Feb 25, 2005 4:57 pm

I get the error below while trying to create the graph for the connections

RRDTool Says:

ERROR: can't parse '#311F4E:Connections\:'

I get the following erro while trying to create the graph for the VS servers:

ERROR: invalid rpn expression 'a,8,*'
Firoz Rangwala

rcariddi
Posts: 12
Joined: Fri Sep 10, 2004 3:42 pm
Location: Charlotte, NC
Contact:

Thanks Jamie

#13 Post by rcariddi » Tue Mar 01, 2005 11:50 am

Hey I got your templates working for BigIP Virtual Statistics. They look great, I had to change the OID due to my older version of code for traffic in/out, after that, it was perfect. We have like 1000 Sites behind our LB's, and w/o this it would have been a full time job getting all that in.
Thanks again for the work on these!
Rich Cariddi
/rjc

gkuchera
Posts: 11
Joined: Fri Feb 04, 2005 5:01 pm
Location: Minneapolis, MN

#14 Post by gkuchera » Tue Mar 01, 2005 2:18 pm

Can you let us know what you put in the request fields when you created the devices? See a couple of posts up.. Thanks,

-geoff
Geoff Kuchera
MLT Vacations, Inc.

rcariddi
Posts: 12
Joined: Fri Sep 10, 2004 3:42 pm
Location: Charlotte, NC
Contact:

#15 Post by rcariddi » Tue Mar 01, 2005 3:05 pm

Hey Geof,
I am not sure which fields you are having a request issue/problem with.
My error was an SNMP failure, but that was due to the OID that is in Jamies XML file (*newer version of F5 Code*). I changed that to an OID that was able to walk on the BigIP and it worked great. So now I am getting connection rates and traffice for the 1000 Virtual Servers.

If this is what you are looking for, I can send it, but I think we are having different issues?
If the request field you are talking about is the OID for Interface Stats on the Virtual Server templates, I'd be glad to send you that snippet.

Do you think the issue is the same?
Rich
/rjc

Post Reply