ZFS Templates

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Post Reply
Author
Message
X505
Posts: 39
Joined: Fri Jun 29, 2007 3:53 pm
Contact:

ZFS Templates

#1 Post by X505 » Thu Aug 02, 2012 12:48 pm

Hi Folks,
I've been working on those for a while, i hope you'll enjoy them and use it :)
It's all described here but in french http://www.hypervisor.fr/?p=4010

Zpool Usage
Image
graph template: http://files.hypervisor.fr/scripts/cact ... _usage.xml
snmp query: http://files.hypervisor.fr/scripts/zpool_usage.xml
snmp config:

Code: Select all

extend .1.3.6.1.4.1.2021.88 zpool_name /bin/bash -c "zpool list -H -o name"
extend .1.3.6.1.4.1.2021.88 zpool_snap /bin/bash -c "zpool list -Ho name|for zpool in `xargs`;do zfs get -rHp -o value usedbysnapshots $zpool|awk -F: '{sum+=$1} END{print sum}';done"
extend .1.3.6.1.4.1.2021.88 zpool_used /bin/bash -c "zpool list -Ho name|xargs zfs get -Hp -o value used"
extend .1.3.6.1.4.1.2021.88 zpool_data_used /bin/bash -c "zpool list -Ho name|for zpool in `xargs`;do snap=`zfs get -rHp -o value usedbysnapshots $zpool|awk -F: '{sum+=$1} END{print sum}'`;pool=`zfs get -Hp -o value used $zpool`; echo $pool $snap|awk '{print (\$1-\$2);}';done"
extend .1.3.6.1.4.1.2021.88 zpool_available /bin/bash -c "zpool list -Ho name|xargs zfs get -Hp -o value available"
extend .1.3.6.1.4.1.2021.88 zpool_capacity /bin/bash -c "zpool list -H -o capacity"
ARC Usage
Image
graph template: http://files.hypervisor.fr/scripts/cact ... _usage.xml
snmp config:

Code: Select all

extend .1.3.6.1.4.1.2021.85 arc_meta_max /bin/bash -c "echo ::arc | mdb -k| grep arc_meta_max|tr -cd '[:digit:]'"
extend .1.3.6.1.4.1.2021.85 arc_meta_used /bin/bash -c "echo ::arc | mdb -k| grep arc_meta_used|tr -cd '[:digit:]'"
extend .1.3.6.1.4.1.2021.85 arc_size /bin/bash -c "echo ::arc | mdb -k| grep -w size|tr -cd '[:digit:]'"
extend .1.3.6.1.4.1.2021.85 arc_meta_limit /bin/bash -c "echo ::arc | mdb -k| grep arc_meta_limit|tr -cd '[:digit:]'"
extend .1.3.6.1.4.1.2021.85 arc_meta_c_max /bin/bash -c "echo ::arc | mdb -k| grep c_max|tr -cd '[:digit:]'"
ARC Stats
Image
graph template: http://files.hypervisor.fr/scripts/cact ... _stats.xml
snmp config:

Code: Select all

extend .1.3.6.1.4.1.2021.89 arc_hits /bin/bash -c "kstat -p ::arcstats:hits| cut -s -f 2"
extend .1.3.6.1.4.1.2021.89 arc_misses /bin/bash -c "kstat -p ::arcstats:misses| cut -s -f 2"
L2ARC Stats
Image
graph template: http://files.hypervisor.fr/scripts/cact ... _stats.xml
snmp config:

Code: Select all

extend .1.3.6.1.4.1.2021.89 arc_l2_hits /bin/bash -c "kstat -p ::arcstats:l2_hits| cut -s -f 2"
extend .1.3.6.1.4.1.2021.89 arc_l2_misses /bin/bash -c "kstat -p ::arcstats:l2_misses| cut -s -f 2"
VFS IOPS
Image
graph template: http://files.hypervisor.fr/scripts/cact ... s_iops.xml
snmp config:

Code: Select all

extend .1.3.6.1.4.1.2021.90 vopstats_zfs_nread /bin/bash -c "kstat -p ::vopstats_zfs:nread | cut -s -f 2"
extend .1.3.6.1.4.1.2021.90 vopstats_zfs_nwrite /bin/bash -c "kstat -p ::vopstats_zfs:nwrite | cut -s -f 2
VFS Throughput
Image
graph template: http://files.hypervisor.fr/scripts/cact ... ughput.xml
snmp config:

Code: Select all

extend .1.3.6.1.4.1.2021.90 vopstats_zfs_read_bytes /bin/bash -c "kstat -p ::vopstats_zfs:read_bytes | cut -s -f 2"
extend .1.3.6.1.4.1.2021.90 vopstats_zfs_write_bytes /bin/bash -c "kstat -p ::vopstats_zfs:write_bytes | cut -s -f 2"
Cheers 8)

User avatar
Linegod
Developer
Posts: 1630
Joined: Thu Feb 20, 2003 10:16 am
Location: Canada
Contact:

Re: ZFS Templates

#2 Post by Linegod » Fri Aug 03, 2012 2:41 pm

If you get a chance, please post these to the templates site:

http://docs.cacti.net/templates

http://docs.cacti.net/templates.guidelines
--
Live fast, die young
You're sucking up my bandwidth.

J.P. Pasnak,CD
CCNA, LPIC-1
http://www.warpedsystems.sk.ca

DavidTomic
Posts: 7
Joined: Tue Oct 30, 2012 6:30 am

Re: ZFS Templates

#3 Post by DavidTomic » Tue Nov 20, 2012 6:08 pm

Thank you for these X505!

After a little bit of fiddling around, I've just managed to get the first template up and running on my new Solaris 11 ZFS server.

Much appreciated!

EDIT: Although I've just had a look at the Cacti log file, and it seems to be throwing up an error on one particular item:

Code: Select all

11/21/2012 10:20:04 AM - CMDPHP: Poller[0] Host[2] DS[40] WARNING: Result from SNMP not valid. Partial Result: U
11/21/2012 10:20:04 AM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'10.0.0.20', and OID:'.1.3.6.1.4.1.2021.88.4.1.2.15.122.112.111.111.108.95.100.97.116.97.95.117.115.101.100.1'
Any ideas on how I might be able to resolve this?

X505
Posts: 39
Joined: Fri Jun 29, 2007 3:53 pm
Contact:

Re: ZFS Templates

#4 Post by X505 » Tue Nov 20, 2012 6:21 pm

DavidTomic wrote:Thank you for these X505!

After a little bit of fiddling around, I've just managed to get the first template up and running on my new Solaris 11 ZFS server.

Much appreciated!

EDIT: Although I've just had a look at the Cacti log file, and it seems to be throwing up an error on one particular item:

Code: Select all

11/21/2012 10:20:04 AM - CMDPHP: Poller[0] Host[2] DS[40] WARNING: Result from SNMP not valid. Partial Result: U
11/21/2012 10:20:04 AM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'10.0.0.20', and OID:'.1.3.6.1.4.1.2021.88.4.1.2.15.122.112.111.111.108.95.100.97.116.97.95.117.115.101.100.1'
Any ideas on how I might be able to resolve this?
Hi DavidTomic,

Did you tried to execute each command line from the .1.3.6.1.4.1.2021.88 extend manually to check if something go wrong ?
Since i'm not a solaris expert and designed this for NexentaOS (solaris 10), it might be an incompatibility issue.

DavidTomic
Posts: 7
Joined: Tue Oct 30, 2012 6:30 am

Re: ZFS Templates

#5 Post by DavidTomic » Tue Nov 20, 2012 10:48 pm

X505 wrote:
Did you tried to execute each command line from the .1.3.6.1.4.1.2021.88 extend manually to check if something go wrong ?
If I run the commands directly, then zpool_snap and zpool_data_used seem to just hang forever. The others seem to be working OK.

I don't really do much work with scripts, so apologies if I'm missing something simple.

DavidTomic
Posts: 7
Joined: Tue Oct 30, 2012 6:30 am

Re: ZFS Templates

#6 Post by DavidTomic » Tue Nov 20, 2012 11:27 pm

Here's the full output from snmpwalk if it's helpful at all:

http://pastebin.com/8g8bsBCH

b0fh
Posts: 32
Joined: Fri Jul 18, 2003 3:46 am

Re: ZFS Templates

#7 Post by b0fh » Wed Nov 21, 2012 3:16 pm

Great templates, especially the ARC statistics are interesting for me, thanks a lot. Is there any chance to gather deduplication-ratio, too?

b0fh
Posts: 32
Joined: Fri Jul 18, 2003 3:46 am

Re: ZFS Templates

#8 Post by b0fh » Mon Nov 26, 2012 4:42 am

I created a template for ZFS deduplication ratio.
Add

Code: Select all

extend zfs_dedup_ratio /bin/bash -c "zpool list | grep <your pool name> | cut -d \" \" -f 13 | tr -cd '[:digit:]'"
to your snmpd.conf.
Attachments
dedup.png
dedup.png (12.62 KiB) Viewed 18210 times
zfs_dedup.zip
(2.64 KiB) Downloaded 271 times

stoatwblr
Posts: 1
Joined: Fri Nov 30, 2012 11:41 pm

Re: ZFS Templates

#9 Post by stoatwblr » Sat Dec 01, 2012 11:36 am

b0fh wrote:I created a template for ZFS deduplication ratio.
Add

Code: Select all

extend zfs_dedup_ratio /bin/bash -c "zpool list | grep <your pool name> | cut -d \" \" -f 13 | tr -cd '[:digit:]'"
to your snmpd.conf.
You seem to be missing an OID... :)

Has anyone tried converting these to linux/BSDese? (Linux at least doesn't have kstat)

b0fh
Posts: 32
Joined: Fri Jul 18, 2003 3:46 am

Re: ZFS Templates

#10 Post by b0fh » Sat Dec 01, 2012 12:55 pm

stoatwblr wrote: You seem to be missing an OID... :)
No! With net-snmp it is not necessary to provide an OID (in fact it is recommended to leave it blank). The script is accessible at "NET-SNMP-EXTEND-MIB::nsExtendInput."zfs_dedup_ratio"

scotia70
Posts: 3
Joined: Sat Feb 14, 2009 8:21 am

Re: ZFS Templates

#11 Post by scotia70 » Wed Dec 26, 2012 11:14 pm

DavidTomic wrote:
X505 wrote:
Did you tried to execute each command line from the .1.3.6.1.4.1.2021.88 extend manually to check if something go wrong ?
If I run the commands directly, then zpool_snap and zpool_data_used seem to just hang forever. The others seem to be working OK.

I don't really do much work with scripts, so apologies if I'm missing something simple.
Not sure if you're still having problems. I found when running the command under csh it should hang (completely - I'd have to kill the session). Under sh or bash they run fine.

HTH,
Scott

gentoouser
Posts: 5
Joined: Tue Aug 23, 2011 8:46 am

Re: ZFS Templates

#12 Post by gentoouser » Tue Aug 27, 2013 10:24 pm

Here is an update for zfsonlinux users.

Use the following for snmpd.conf:

Code: Select all

#Zpool Usage
extend .1.3.6.1.4.1.2021.88 zpool_name /bin/bash -c "sudo zpool list -H -o name"
extend .1.3.6.1.4.1.2021.88 zpool_snap /bin/bash -c "sudo zpool list -Ho name|for strzpool in `xargs`;do sudo zfs get -rHp -o value usedbysnapshots $strzpool|awk -F: '{sum+=$1} END{print sum}';done"
extend .1.3.6.1.4.1.2021.88 zpool_used /bin/bash -c "sudo zpool list -Ho name|xargs sudo zfs get -Hp -o value used"
extend .1.3.6.1.4.1.2021.88 zpool_data_used /bin/bash -c "sudo zpool list -Ho name|for strzpool in `xargs`;do snap=`sudo zfs get -rHp -o value usedbysnapshots $strzpool|awk -F: '{sum+=$1} END{print sum}'`;pool=`sudo zfs get -Hp -o value used $zpool`; echo $pool $snap|awk '{print (\$1-\$2);}';done"
extend .1.3.6.1.4.1.2021.88 zpool_available /bin/bash -c "sudo zpool list -Ho name|xargs sudo zfs get -Hp -o value available"
extend .1.3.6.1.4.1.2021.88 zpool_capacity /bin/bash -c "sudo zpool list -H -o capacity"
#ARC Usage
extend .1.3.6.1.4.1.2021.85 arc_meta_max /bin/bash -c "sudo grep arc_meta_max /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
extend .1.3.6.1.4.1.2021.85 arc_meta_used /bin/bash -c "sudo grep arc_meta_used /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
extend .1.3.6.1.4.1.2021.85 arc_size /bin/bash -c "sudo grep -m 1 size /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
extend .1.3.6.1.4.1.2021.85 arc_meta_limit /bin/bash -c "sudo grep arc_meta_limit /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
extend .1.3.6.1.4.1.2021.85 arc_meta_c_max /bin/bash -c "sudo grep c_max /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
#ARC Stats
extend .1.3.6.1.4.1.2021.89 arc_hits /bin/bash -c "sudo grep -m 1 hits /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
extend .1.3.6.1.4.1.2021.89 arc_misses /bin/bash -c "sudo grep misses /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
#L2ARC Stats
extend .1.3.6.1.4.1.2021.89 arc_l2_hits /bin/bash -c "sudo grep l2_hits /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"
extend .1.3.6.1.4.1.2021.89 arc_l2_misses /bin/bash -c "sudo grep l2_misses /proc/spl/kstat/zfs/arcstats |awk -F' ' '{print $(NF)}'"#VFS IOPS
#VFS IOPS
extend .1.3.6.1.4.1.2021.90 vopstats_zfs_nread /bin/bash -c "humnum=$(sudo zpool iostat | tail -n 1 | awk -F' ' '{print $4}') ; case ${humnum: -1:1} in [0-9]) echo $humnum | tr -d ' ' ;; 'K') calc -p ${humnum: 0:-1}*1024**1 | tr -d ' ' ;; 'M') calc -p ${humnum: 0:-1}*1024**2 | tr -d ' ' ;; 'G') calc -p ${humnum: 0:-1}*1024**3 | tr -d ' ' ;; 'T') calc -p ${humnum: 0:-1}*1024**4 | tr -d ' ' ;; 'P') calc -p ${humnum: 0:-1}*1024**5 ;; esac"
extend .1.3.6.1.4.1.2021.90 vopstats_zfs_nwrite /bin/bash -c "humnum=$(sudo zpool iostat | tail -n 1 | awk -F' ' '{print $5}') ; case ${humnum: -1:1} in [0-9]) echo $humnum | tr -d ' ' ;; 'K') calc -p ${humnum: 0:-1}*1024**1 | tr -d ' ' ;; 'M') calc -p ${humnum: 0:-1}*1024**2 | tr -d ' ' ;; 'G') calc -p ${humnum: 0:-1}*1024**3 | tr -d ' ' ;; 'T') calc -p ${humnum: 0:-1}*1024**4 | tr -d ' ' ;; 'P') calc -p ${humnum: 0:-1}*1024**5 ;; esac"
#VFS Throughput
extend .1.3.6.1.4.1.2021.90 vopstats_zfs_read_bytes /bin/bash -c "humnum=$(sudo zpool iostat | tail -n 1 | awk -F' ' '{print $6}') ; case ${humnum: -1:1} in [0-9]) echo $humnum | tr -d ' ' ;; 'K') calc -p ${humnum: 0:-1}*1024**1 | tr -d ' ' ;; 'M') calc -p ${humnum: 0:-1}*1024**2 | tr -d ' ' ;; 'G') calc -p ${humnum: 0:-1}*1024**3 | tr -d ' ' ;; 'T') calc -p ${humnum: 0:-1}*1024**4 | tr -d ' ' ;; 'P') calc -p ${humnum: 0:-1}*1024**5 ;; esac"
extend .1.3.6.1.4.1.2021.90 vopstats_zfs_write_bytes /bin/bash -c "humnum=$(sudo zpool iostat | tail -n 1 | awk -F' ' '{print $7}') ; case ${humnum: -1:1} in [0-9]) echo $humnum | tr -d ' ' ;; 'K') calc -p ${humnum: 0:-1}*1024**1 | tr -d ' ' ;; 'M') calc -p ${humnum: 0:-1}*1024**2 | tr -d ' ' ;; 'G') calc -p ${humnum: 0:-1}*1024**3 | tr -d ' ' ;; 'T') calc -p ${humnum: 0:-1}*1024**4 | tr -d ' ' ;; 'P') calc -p ${humnum: 0:-1}*1024**5 ;; esac"

Edit /etc/sudoers.d/zfs

Code: Select all

## Cmnd alias specification
Cmnd_Alias C_ZFS = \
  /sbin/zfs "", /sbin/zfs help *, \
  /sbin/zfs get, /sbin/zfs get *, \
  /sbin/zfs list, /sbin/zfs list *, \
  /sbin/zpool "", /sbin/zpool help *, \
  /sbin/zpool iostat, /sbin/zpool iostat *, \
  /sbin/zpool list, /sbin/zpool list *, \
  /sbin/zpool status, /sbin/zpool status *, \
  /sbin/zpool upgrade, /sbin/zpool upgrade -v, \
  /bin/grep * /proc/spl/kstat/zfs/*
#
Runas_Alias R_ROOT = root
## allow any user to use basic read-only ZFS commands
ALL ALL = (R_ROOT) NOPASSWD: C_ZFS


zyx
Posts: 3
Joined: Fri Jan 17, 2014 7:16 am

Re: ZFS Templates

#13 Post by zyx » Tue Aug 05, 2014 4:44 am

Hi,

Is this still the latest version of the template?
Under 3.1.5 i get error messages parsing the snmpd.conf

Aug 5 11:34:27 host-1 snmpd[2283]: [ID 702911 daemon.error] duplicate registration: MIB modules nsExtendNumEntries and nsExtendNumEntries (oid .1.3.6.1.4.1.2021.87.1).
Aug 5 11:34:27 host-1 snmpd[2283]: [ID 702911 daemon.error] duplicate registration: MIB modules nsExtendNumEntries and nsExtendNumEntries (oid .1.3.6.1.4.1.2021.88.1).
Aug 5 11:34:27 host-1 snmpd[2283]: [ID 702911 daemon.error] duplicate registration: MIB modules nsExtendNumEntries and nsExtendNumEntries (oid .1.3.6.1.4.1.2021.85.1).
Aug 5 11:34:27 host-1 snmpd[2283]: [ID 702911 daemon.error] duplicate registration: MIB modules nsExtendNumEntries and nsExtendNumEntries (oid .1.3.6.1.4.1.2021.89.1).
Aug 5 11:34:27 host-1 snmpd[2283]: [ID 702911 daemon.error] duplicate registration: MIB modules nsExtendNumEntries and nsExtendNumEntries (oid .1.3.6.1.4.1.2021.90.1).

Could you post an update?

Thanks,
Z

Post Reply