Cacti (home)ForumsRepositoryDocumentation
Cacti: offical forums and support  

 FAQFAQ   SearchSearch   MemberlistMemberlist    RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in    


[HOWTO] Use a remote script via SSH as Data Input, no SNMP

 
Post new topic   Reply to topic    Cacti Forum Index -> Informational/HOWTO's
Author Message
white-jeroen



Joined: 22 Mar 2008
Posts: 13
PostPosted: Sun Mar 23, 2008 2:26 pm    Post subject: [HOWTO] Use a remote script via SSH as Data Input, no SNMP Reply with quote
I like to create scripts that process input for Cacti graphs on several servers, but I don't like to use SNMP to query them. I am using SSH to execute remote scripts for a year now, and I take the time to explain it here, so others can use it too.

Let say we have the following situation:

  • cactihost.your.dom = Server that runs cacti
  • queryme.your.dom = Server that you want to make a graph of
  • On cactihost.your.dom the Cacti cron job runs as user 'cacti'.
  • The Cacti home dir on cactihost.your.dom is /var/www/cacti

To make it possible for the Cacti cron job to execute a query script on queryme.your.dom, it needs to be able to log in using SSH without having to enter a password. We are going to realize this using a public key.

On cactihost.your.dom you become root and execute the following:
Code:
# mkdir /var/www/cacti/.ssh
# chown cacti:cacti /var/www/cacti/.ssh
# chmod 700 /var/www/cacti/.ssh
# su cacti
$ ssh-keygen
Enter file in which to save the key (/var/www/cacti/.ssh/id_rsa): [Return]
Enter passphrase (empty for no passphrase): [Return]
Enter same passphrase again: [Return]
$ cat /var/www/cacti/.ssh/id_rsa.pub

Copy the complete output of this last command.

Log in as root on queryme.your.dom and execute the following:
Code:
# adduser cactiquery
# su cactiquery
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ echo [Paste the copied text] >>~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys

Now let's test it. Execute the following on cactihost.your.dom.
I assume you are still user 'cacti', otherwise do # su cacti
Code:
$ ssh cactiquery@queryme.your.dom  -p22 'df -h'

The first time it ask if you are sure you want to continue, answer 'yes'.
The command should output the disk space status of the queryied server.
If you execute the command again, it should just output the info, and not ask anything.

So now let's create a test script that counts the number of httpd processes running on queryme.your.dom. Make sure you are logged in as 'cactiquery' on queryme.your.dom. Execute:
Code:
$ echo 'echo -n "httpds:"`ps h -C httpd |wc -l`' >~/httpds.sh
$ chmod 700 ~/httpds.sh
$ ~/httpds.sh

The last command executes the script, and shout output something like:
Quote:
httpds:8

Now lets do a last test on cactihost.your.dom, execute as user 'cacti':
Code:
$ ssh cactiquery@queryme.your.dom  -p22 '~/httpds.sh'

It should give the same output.

Now it's time to create the Data Input Method in the Cacti Console.
  • Click the 'Console' tab on the top of the cacti web interface
  • Click 'Data Input Methods'
  • Click 'Add' on the top right of the page
  • Name: httpd process count (ssh)
  • Input type: Script/Command
  • Input string:
    Code:
    ssh cactiquery@<hostname> -p22 '~/httpds.sh'

  • Click the create button
  • Click the 'Add' link in the Input Fields box
  • Field[Input]: hostname
  • Friendly name: Hostname
  • Special Type Code: hostname
  • Click 'create'
  • Click the 'Add' link in the Output Fields box
  • Field [Output]: httpds
  • Friendly Name: Number of httpd processes
  • Update RRD File: checked
  • Click 'create'
  • Click 'save'

It should look like this:


Now you can create a Data Template that uses this Data Input Method (as GAUGE), and a Graph Template using that Data Source.

I hope this howto helps someone to build his/her own remote query script and use it in Cacti, without the need of SMTP. Please let me know.

Last edited by white-jeroen on Tue Mar 25, 2008 8:58 am; edited 6 times in total
Back to top
white-jeroen



Joined: 22 Mar 2008
Posts: 13
PostPosted: Mon Mar 24, 2008 3:55 am    Post subject: If you followed this howto and your graph doesn't plot Reply with quote
Of course you need to wait at least 15 minutes before checking the results in the graph.
If your graph still doesn't plot anything:

  • In the Cacti web interface click the "Console" tab
  • Click 'System Utilities'
  • Click 'View Cacti Log File'
  • If you see yellow error lines, click on the DS[x] number

If it is the datasource you created yourself, there might be a problem with the Data Input Method

  • Click 'System Utilities'
  • Click 'View Poller Cache'
  • Search for the Data Source you created to connect to the Data Input method from the HowTo
  • Copy the line after 'Script:' from that item
  • Log in as root on cactihost.your.dom and:
    Code:
    # su cacti
    $ [paste the line you copied]


Maybe it asks again if you are sure you want to connect because the hostname is not exactly the same (for example an IP) as the hostname when you tested during the HowTo.
If that is the case, enter 'Yes' and test the command agian. Now it should not ask anything and from then your graph should start working.

If you get an other error, maybe that gives you an hint of what could be the problem.
Back to top
kramsay



Joined: 21 Jan 2005
Posts: 14
PostPosted: Fri Sep 26, 2008 6:56 pm    Post subject: Reply with quote
nice description! As you know SSH has a powerful remote use and I do lots of remote ssh exec cron jobs in my servers. But I never thought about using Cacti script/command usage before. I will now ....

You've got to LOVE this forum

Thanks white-jeroen
Back to top
white-jeroen



Joined: 22 Mar 2008
Posts: 13
PostPosted: Sat Sep 27, 2008 3:34 pm    Post subject: Reply with quote
You're welcome.
Well, I don't spend my days clicking around this forum and reading every new post. What I do love is Cacti itself! It is a very nice piece of software to get insight into what happens on your servers, routers, switches, etc.

What I love the most is Jesus Christ.

Kind regards, Jeroen.
Back to top
hackerkatt



Joined: 30 Sep 2008
Posts: 19
PostPosted: Tue Sep 30, 2008 10:39 pm    Post subject: Reply with quote
Great info. I like this approach to getting remote info. How do you deal with getting info on the remote server that "cactiuser" wouldn't have rights to? Say it's admin or part of wheel group, or some other group. I am trying to get info from my QMAIL servers. qmail-stat is owned by root with the user as qmail.

HK
Back to top
white-jeroen



Joined: 22 Mar 2008
Posts: 13
PostPosted: Wed Oct 01, 2008 3:53 pm    Post subject: Reply with quote
Well... actually I use it with qmail.
On the qmail server I have a user 'scripts' which has remote access.

I have created the following script on /home/scripts/qmailqueue.php
Code:
#!/usr/bin/php -q
<?
$sData = shell_exec('/var/qmail/bin/qmail-qstat');
$aMatch = array();
if (preg_match('#queue:\s*(\d+)#i',$sData,$aMatch))
{
  print ' Queue:'.$aMatch[1];
}
if (preg_match('#preprocessed:\s*(\d+)#i',$sData,$aMatch))
{
  print ' NotPreproc:'.$aMatch[1];
}
print "\n";
?>


I have to admit I don't exactly know if I did change the chmod or ownership of the file /var/qmail/bin/qmail-qstat, but here is what it looks like:
Code:
-r-xr-xr-x  1 root qmail 375 Aug 22 14:56 /var/qmail/bin/qmail-qstat

I edited /etc/group to add the user 'scripts' to the group 'qmail':
Code:
qmail:x:2520:scripts


If you don't want to change file permissions or the group file I have an other solution:
edit /etc/sudoers as root and add the following line:
Code:
scripts   ALL = NOPASSWD: /var/qmail/bin/qmail-qstat

Now replace the line with the 'shell_exec' from the above script to:
Code:
$sData = shell_exec('/usr/bin/sudo /var/qmail/bin/qmail-qstat');


It is some time ago I installed this stuff, so try if it works for you, and post here what goes wrong, maybe I can help you.
Back to top
white-jeroen



Joined: 22 Mar 2008
Posts: 13
PostPosted: Wed Oct 01, 2008 4:27 pm    Post subject: Creating Qmail delivery and spam graph based on log Reply with quote
hackerkatt, I have an extra present for you:
I use my log parsing and counting script 'logstats.inc.php' from the tgz I attached to the post http://forums.cacti.net/viewtopic.php?p=130395#130395 alse to get some qmail stats with this script:

Code:
#!/usr/bin/php -q
<?
require_once('logstats.inc.php');

### SETTINGS
$sScriptName = basename($argv[0]);
$sLogFile = '/usr/local/psa/var/log/maillog';
$sDataFile = 'data/'.$sScriptName.'.txt';
$aData = array();
$aGrep['#spamd: processing message#']                            = 'SpamCheck';
$aGrep['#spamd: identified spam#']                               = 'Spam';
$aGrep['#qmail:.*?starting delivery.*?to remote#']               = 'DeliveryAttemptRemote';
$aGrep['#qmail:.*?starting delivery.*?to local#']                = 'DeliveryAttemptLocal';
$aGrep['#qmail: .*? delivery .*? success: .*?accepted_message#'] = 'DeliverySuccessRemote';
$aGrep['#qmail: .*? delivery .*? success: did_#']                = 'DeliverySuccessLocal';

$aData = &ReadLog($sLogFile,$sDataFile,$aGrep);
$aData['Ham'] = $aData['SpamCheck'] - $aData['Spam'];
$sOutput = OutputData($aData);
print $sOutput;
exit;
?>

I placed the above script in a file called '/home/scripts/maillogstats.php' and made it executable. I placed the file from the tgz at '/home/scripts/logstats.inc.php'.
I did the following command to give the user 'scripts' from group 'scripts' read-only access to the qmail maillog:
Code:

#chgrp scripts /usr/local/psa/var/log/maillog
#chmod g+r /usr/local/psa/var/log/maillog

So this is what it looks like:
Code:

 -rw-r-----  1 root scripts 28561971 Oct  1 23:08 /usr/local/psa/var/log/maillog


I use the data to plot this graph


Have fun...
To understand the log processing script you might want to read the full explanation on http://forums.cacti.net/viewtopic.php?p=130395#130395.
Back to top
hackerkatt



Joined: 30 Sep 2008
Posts: 19
PostPosted: Thu Oct 02, 2008 8:49 pm    Post subject: Reply with quote
You rock! The first reply was awesome. Your present was, what can I say, just that. Thanks so much. I've put together a few scripts and graphs for monitoring Mailfoundry device. I will post this for any interested. I'm new to Cacti, up to my knees right now. Trying to understand the various graphing setups, etc. The scripts are easy, aside from gathering from remote servers via scripts. That's where you helped me a lot. Thanks again.

Hackerkatt
aka Tony
Back to top
hackerkatt



Joined: 30 Sep 2008
Posts: 19
PostPosted: Fri Oct 03, 2008 1:34 am    Post subject: Reply with quote
Jeroen,

I'm not much of a PHP coder, perl, c, pascal. I'm trying to determine what you are doing with the aGrep array. And what is being pulled in from logstats.inc.php. Is ReadLog a function in logstat.inc.php

Hackerkatt
Back to top
white-jeroen



Joined: 22 Mar 2008
Posts: 13
PostPosted: Tue Oct 07, 2008 12:44 pm    Post subject: Reply with quote
Hackerkatt,
When you are not used to PHP I have to explain this:
Whey you would use a Hash in Perl, you can just use an array in PHP. PHP allows arrays to have text-values as key instead of only numbers like Perl. So:
Code:
### Perl: 
my %Hash;
$Hash['key'] = 'value';

### PHP:
$aArray = array();
$aArray['key'] = 'value';


The $aGrep is such an array. The keys are regular expressions that are tried on every log line by the ReadLog function which is indeed in logstats.inc.php.
The ReadLog functions returns the array $aData.
When one of the keys from $aGrep matches a line, the 'value' from the array is used as key for $aData, to increase $aData[key] by one.

Example of $aData returned:

$aData['SpamCheck'] = 15;
$aData['Spam'] = 12;

Have fun, Jeroen.
Back to top
Display posts from previous:   
Post new topic   Reply to topic    Cacti Forum Index -> Informational/HOWTO's All times are GMT - 5 Hours
Page 1 of 1

 



Powered by phpBB © 2001, 2005 phpBB Group