Varnish cache statistics

[dmuntean]

Hello

Here are some varnish cache statistics we did and use for ourselves. Script uses varnish management port to collect stats, so make sure it is enabled and opened for cacti host.

To install it import the templates, place script in cacti/scripts directory. Each graph will ask you for the port of varnish management interface.

Here are the previews:

[dmuntean]

If anyone feels that there should be other stats added, just let me know. There is no problem in adding other stats, those are the ones we needed.

[frafall]

Nice but I thing the exports are missing dependecies? I get "Error: XML: Hash version does not exist" when trying to import...

[dmuntean]

What version of cacti do you use?

Those templates were made with 0.8.7d. I did a fresh install of latest cacti and imported them without any problem.

[jschirle]

I just got my frist varnish box up and running and would really like to use your templates but i am getting an error in the python script that i have no idea what it means. (its the noob in me)

to test the script i ran this command

/usr/bin/python /var/www/cacti/scripts/get_varnish_stats.py -h xxx.xxx.xxx.xxx -p 6082

Traceback (most recent call last):
File "/var/www/cacti/scripts/get_varnish_stats.py", line 24, in ?
req = req.group(0).split()[0]
AttributeError: 'NoneType' object has no attribute 'group'


any help would be awsome.

[dmuntean]

What version of varnish do you run?

The error you get means script can't find stats data from varnish. Can you manually telnet to varnish port (6082) and issue "stats" (without quotes) command and pm me?

[jschirle]

I am running varnish version 2.0.3 and yes i can manually telnet to the box on port 6082 and run the stats command.

[dmuntean]

Please, PM me the output you get when you run "stats" command on varnish management port.

[jschirle]

This is just some of the stats that are returned when i telnet in and rum the stats command.


2202469 Client connections accepted
3981630 Client requests received
2444450 Cache hits
17122 Cache hits for pass
536733 Cache misses
1421087 Backend connections success
0 Backend connections not attempted
0 Backend connections too many
0 Backend connections failures
1359422 Backend connections reuses
1362629 Backend connections recycles
0 Backend connections unused
2343 N struct srcaddr
484 N active struct srcaddr
3915 N struct sess_mem
493 N struct sess
9933 N struct object
10046 N struct objecthead
18396 N struct smf
475 N small free smf

[dmuntean]

OK, but I need to see full output of the "stats" command. Can you please post it without cutting off anything from it?

[jschirle]

Full out put.....


stats200 2957
6815
016 Client connections accepted
11872740 Client requests received
69168
14 Cache hits
51530 Cache hits for pass
1698945 Cache misses
45
54634 Backend connections success
0 Backend connections not attempte
d
0 Backend connections too many
15 Backend connections fai
lures
4378743 Backend connections reuses
4388696 Backend connections r
ecycles
0 Backend connections unused
2429 N struct srcaddr

509 N active struct srcaddr
3915 N struct sess_mem
523 N
struct sess
12014 N struct object
13089 N struct objecthead

18397 N struct smf
186 N small free smf
14 N large free smf

13 N struct vbe_conn
48 N struct bereq
103 N worke
r threads
4976 N worker threads created
248 N worker threads no
t created
147814 N worker threads limited
0 N queued work reque
sts
43230 N overflowed work requests
0 N dropped work requests

4 N backends
1686064 N expired objects
0 N LRU nuked
objects
0 N LRU saved objects
4433980 N LRU moved objects

0 N objects on deathrow
198 HTTP header overflows
0 Ob
jects sent with sendfile
11431569 Objects sent with write
0 Objec
ts overflowing workspace
6814946 Total Sessions
11872724 Total Requests

0 Total pipe
2859499 Total pass
4523861 Total fetch
41881
23473 Total header bytes
288173001494 Total body bytes
894104 Session Clo
sed
89 Session Pipeline
18 Session Read Ahead
0 S
ession Linger
11049813 Session herd
675956394 SHM records
49195007 SH
M writes
20309 SHM flushes due to overflow
17197 SHM MTX contentio
n
287 SHM cycles through buffer
9517233 allocator requests
1
8197 outstanding allocations
395431936 bytes allocated
1215180800 bytes fr
ee
0 SMA allocator requests
0 SMA outstanding allocations

0 SMA outstanding bytes
0 SMA bytes allocated
0
SMA bytes free
432159 SMS allocator requests
0 SMS outstanding
allocations
18446744073709544244 SMS outstanding bytes
198918192 SMS bytes a
llocated
198925104 SMS bytes freed
4523888 Backend requests made

3 N vcl total
3 N vcl available
0 N vcl discarded

1 N total active purges
1 N new purges added
0 N ol
d purges deleted
0 N objects tested
0 N regexps tested aga
inst
0 N duplicate purges removed
0 HCB Lookups without lo
ck
0 HCB Lookups with lock
0 HCB Inserts
0 Obje
cts ESI parsed (unlock)
0 ESI parse errors (unlock)

[dmuntean]

Script reads all the data till

[davideb]

We had a little problem because Python wasn't installed on the Cacti machine, so I took the original python script and rebuild it in Perl. It needs the Net::Telnet perl module to work.

I've also added different graphs.

Here is the whole shebang.

[davideb]

After a while, I didn't liked the idea to have Varnish management's interface exposed to the world. Since it doesn't have any kind of authentication on his own. So I decided to employ a trick and used the varnishstat tool called directly from Xinetd. This way I just need to poke that to get the same informations. Unfortunately, I had to change the script that process the data. Second version with both scripts.

[basic]

I'm not able to import the varnish+xinetd template in Cacti, it's giving me an "Error: XML: Hash version does not exist.". What version of Cacti are you using?

We are still running 0.8.7d here, i'm guessing you're running 0.8.7e and it's not backwards compatible?

Thanks!