Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Sat May 25, 2019 1:12 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: [HOWTO] Use externally updated rrd files + BONUS TRACK
PostPosted: Sun Feb 26, 2006 7:57 am 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
This document is not updated any more. Last version, see http://docs.cacti.net/manual:087:8_rrdtool.05_external_rrds#externally_updated_rrds
Preface
Due to a proposal of BSOD2600 I try to compile a HowTo for Using externally updated rrd files. Cacti itself provides a very flexible and fast data gatherer, but situations may occur, where this is not what you want. Assuming, that there already is some rrd file filled by other means, question arises how to use this with cacti's flexible rrdtool front-end administration facilities.
Sometimes people call this feature Importing external rrds to cacti. But what I'm going to explain is not an automated function. It will require some manual interaction.
Of course, the webserver must have at least read access to the required rrd file(s). For sake of easiness, I'll assume the file to be located in cacti's default ./rra/ directory. In my examples, this file is called example.rrd.

As always: Use this on your own risk

Table of Contents
Chapter I: Get rrdtool info
Chapter II: Create the Data Template
Chapter III. Create the Graph Template
Chapter IV: Prepare the Host
Chapter V: Creating the Data Sources
Chapter VI: Create the new Graph
Bonus Track: Updating RRD Files from Remote

Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Last edited by gandalf on Sat Dec 22, 2007 10:09 am, edited 3 times in total.

Top
 Profile  
 
 Post subject: Chapter I: Get rrdtool info
PostPosted: Sun Feb 26, 2006 10:24 am 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Chapter I: Get rrdtool info
First task is to get information about the data sources used in the rrd file. This is done using rrdtool info <rrd file>:
Code:
rrdtool info ./rra/external.rrd
filename = "/var/www/html/cacti/rra/external.rrd"
rrd_version = "0003"
step = 300
last_update = 1140957748
ds[external_ds1].type = "GAUGE"
ds[external_ds1].minimal_heartbeat = 600
ds[external_ds1].min = 0.0000000000e+00
ds[external_ds1].max = 5.0000000000e+02
ds[external_ds1].last_ds = "UNKN"
ds[external_ds1].value = 4.4510209500e+02
ds[external_ds1].unknown_sec = 0
ds[external_ds2].type = "GAUGE"
ds[external_ds2].minimal_heartbeat = 600
ds[external_ds2].min = 0.0000000000e+00
ds[external_ds2].max = 5.0000000000e+02
ds[external_ds2].last_ds = "UNKN"
ds[external_ds2].value = 7.4183682500e+02
ds[external_ds2].unknown_sec = 0
ds[external_ds3].type = "GAUGE"
ds[external_ds3].minimal_heartbeat = 600
ds[external_ds3].min = 0.0000000000e+00
ds[external_ds3].max = 5.0000000000e+02
ds[external_ds3].last_ds = "UNKN"
ds[external_ds3].value = 1.0385715550e+03
ds[external_ds3].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
(... more to follow ...)
From the ds[.....] statements, the names of the ds' are taken. In this case the data sources are named
  • external_ds1,
  • external_ds2,
  • external_ds3
respectively. While this is not that a meaningful name, it should show you the principles when dealing with multi-ds rrds. Alongside with this, it is good to know the ds[...].type, ds[...].min and ds[...].max for correct definition of the data sources. In this case, all data sources are of type GAUGE. This will not affect data gathering nor graphing, but to me it seems to be advantageous to create the correct data sources.
All other paramaters (step, heartbeat, xff, rra[..].rows) are assumed to be standard settings.

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Last edited by gandalf on Sun Feb 26, 2006 11:13 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Chapter II: Create the Data Template
PostPosted: Sun Feb 26, 2006 11:12 am 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Chapter II: Create the Data Template
The Data Template will tell cacti, how the data is stored within the rrd file. This is the way to tell cacti about all data sources and their properties. The purpose is twofold:
  • tell cacti, how to create the correct rrd file with all parameters
  • store the name of the data sources for later use with Graph Templates
While the first goal is not needed in this context, the second one is crucial.
So lets define a new Data Template. Goto Data Templates and Add a new one:

Image


Fill in the usual header data:

Image


If you want to associate this to a certain host, you may use |host_description| as a placeholder as usual. As this external.rrd file is updated externally, you must set the Data Input Method to None.
Select the Associated RRA's as they will define the Detailed Graph Views (usually Daily, Weekly, Monthly, Yearly). And you'll have to uncheck the Data Source Active checkbox. This will prevent cacti from actually gathering data for this Data Template. Now add the first Data Source:

Image



Internal Data Source Name
You must use the existing data source name of the rrd as retrieved in Chapter I. In this example, use external_ds1.

Minimum Value
Fill in the ds[...].min value from rrdtool info above. In this case, use 0. This is not really needed, but for sake of consistency I recommend this.

Maximum Value
Fill in the ds[...].max value from rrdtool info above. In this case, use 500. This is not really needed, but for sake of consistency I recommend this.

Data Source Type
Fill in the ds[...].type value from rrdtool info above. In this case, use GAUGE. This is not really needed, but for sake of consistency I recommend this.

Heartbeat
Fill in the ds[...].minimal_heartbeat value from rrdtool info above. In this case, use 600. This is not really needed, but for sake of consistency I recommend this.

Now Create to see:

Image


Repeat this for all other data sources of the example.rrd (use the New function of Data Source Items):


Image


You will have noticed, that no Custom Data is given as defined by the Data Input Method set to NONE. You'll see the result as:

Image


Now you're done.

Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Chapter III. Create the Graph Template
PostPosted: Sun Feb 26, 2006 11:42 am 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Chapter III. Create the Graph Template
The Graph template will tell cacti, what Data Sources should be shown on the Graph. This is very straightforward; simply use the Data Sources as defined in Chapter II and use all the Graph magic cacti provides. So Add a new Graph Template

Image


and fill in the usual header data:

Image


Now add the first Graph Item:

Image


and fill in the usual Data:

Image


Use the Legend Option as a time saver:

Image


And add all other Graph Items like that:

Image


Image


to end up in


Image


Of course you will use more meaningful input for Text Format. That's all for now.

Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Chapter IV: Prepare the Host
PostPosted: Sun Feb 26, 2006 12:10 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Chapter IV: Prepare the Host
Now its time for the Host to be prepared. You have the choice between two different approaches
  • the Host already exists (perhaps you're polling some other data from this host) and the status is up
  • the Host does not yet exist in cacti's tables and shall never been polled for other data by cacti's own poller
The first approach does not need any additional changes to the Devices list.
The second approch will be more common. You will need a Host entry in the Devices list even for this host. So we will create kind of a dummy entry. Please goto the Devices list and Add a new one:

Image


Fill in Description and Name as usual. To deactivate all checks, please check Disable Host and leave SNMP Community empty.

Image

Create

Image


Please proceed to the next chapter now

Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Chapter V: Creating the Data Sources
PostPosted: Sun Feb 26, 2006 12:28 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Chapter V: Creating the Data Sources
Normally, you would create the Data Sources and Graphs automagically using Create Graphs for this Host. But using this approach, cacti would enumerate and generate the needed rrd file(s) on its own. That's not wanted now.
Our task is: make cacti generate all Data Sources stuff but accept our external.rrd file name. This is accomplished by adding the Data Sources manually from the Data Sources list. In this case, there's no difference whether you're going to use an already existing host or the kind of host we generated in the previous chapter.

Image


Now select the Data Template we generated in Chapter II:

Image


and Create. You will be prompted to fill in the full path to your external.rrd file. If this resides in cacti's default ./rra directory, you may use <path_rra> for this. Remember, that the web server must have at least read access to that file.:

Image


The result is shown in the next image

Image


Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Chapter VI: Create the new Graph
PostPosted: Sun Feb 26, 2006 12:40 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Chapter VI: Create the new Graph
Now we're nearly done! The last Step now!
Let's create a new Graph from Graph Management now.

Image


Select the Selected Graph Template be defined in Chapter III:

Image


and Create. Now select all needed Data Source [...] and Save:

Image


The result is shown like:

Image


Please select this Graph again and Turn on Graph Debug Mode to see

Image


Isn't it nice, this Graph :D :D :lol: :roll:

have fun, use cacti
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject: Bonus Track: Updating RRD Files from Remote
PostPosted: Sat Mar 04, 2006 3:31 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Bonus Track: Updating RRD Files from Remote
It is known, that rrd files are architecture-dependant. So, if you're updating external rrd files and graphing them via cacti, there's a need to make cacti access these rrd files, e.g. by nfs. This will not succeed when mixing architectures.
And nfs may not be the best choice for all cases. But with rrdtool 1.2.x there's a new feature, rrd server. Pasted from rrdtool homepage:
Quote:
RRD Server

If you want to create a RRD-Server, you must choose a TCP/IP Service number and add them to /etc/services like this:

rrdsrv 13900/tcp # RRD server

Attention: the TCP port 13900 isn't officially registered for rrdsrv. You can use any unused port in your services file, but the server and the client system must use the same port, of course.

With this configuration you can add RRDtool as meta-server to /etc/inetd.conf. For example:

rrdsrv stream tcp nowait root /opt/rrd/bin/rrdtool rrdtool - /var/rrd

Don't forget to create the database directory /var/rrd and reinitialize your inetd.

If all was setup correctly, you can access the server with perl sockets, tools like netcat, or in a quick interactive test by using 'telnet localhost rrdsrv'.

NOTE: that there is no authentication with this feature! Do not setup such a port unless you are sure what you are doing.
For my local setup (RHEL 4.0), I had to modify this a bit. My /etc/services
Code:
rrdsrv         13900/tcp                       # RRD server
is standard. And I put
Code:
# default: off
# description: RRDTool as a service
service rrdsrv
{
        disable         = no
        socket_type     = stream
        protocol        = tcp
        wait            = no
        user            = cactiuser
        server          = /usr/bin/rrdtool
        server_args     = - /var/www/html/cacti/rra
}
as /etc/xinetd.d/rrdsrv. Then xinetd is updated to start rrdsrv.
user should be set to the user defined in include/config.php.
server_args should be set to cacti's rra directory.
server contains the full path to rrdtool's binary. To verify this, try
Code:
telnet localhost 13900
info external.rrd
assuming, that the rrd file "external.rrd" used in this howto is located in the ./rra directory.
Now its time for some remote script to use this new feature. As an example, see
Code:
#!/usr/bin/perl
use IO::Socket;

my $host = shift @ARGV;
my $port = shift @ARGV;
my $rrd  = shift @ARGV;

my $socket = IO::Socket::INET->new(PeerAddr=> $host,
                                PeerPort=> $port,
                                Proto=> 'tcp',
                                Type=> SOCK_STREAM)
                                or die "Can't talk to $host at $port";

my $_cmd = "update " . $rrd . " N:" . int(rand(10)) . ":" . int(rand(10)) . ":" . int(rand(10));
print $socket $_cmd . "\n";
close $socket;
Of course,
Code:
my $_cmd = "update " . $rrd . " N:" . int(rand(10)) . ":" . int(rand(10)) . ":" . int(rand(10));
is only an example prepared for the "external.rrd" of our example. To use this for updating your own rrd files, this must fit to the data source definitions of your special rrd file. In our example, I put
Code:
*/5 * * * *     cactiuser       /usr/bin/perl /var/www/html/cacti/scripts/rrd-remote-update.pl localhost 13900 external.rrd
into crontab to perform regular updates.

Disadvantages
This handling has the great disadvantage, that you must configure the rrd file name to each single updating script. This rrd file name on the remote system must match the one on cacti's host. There's a good chance to mess things up when used for lots of rrds. But for some few files this may be appropriate.

Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 11:12 am 
Offline

Joined: Sun Feb 18, 2007 11:09 am
Posts: 17
Hi , i am trying to import a rrd file in to cacti but i am having a lot of troubles ...

see this :

# rrdtool info imq1-tc.rrd
filename = "imq1-tc.rrd"
rrd_version = "0003"
step = 10
last_update = 1171814910
ds[101-].type = "DERIVE"
ds[101-].minimal_heartbeat = 60
ds[101-].min = 0.0000000000e+00
ds[101-].max = NaN
ds[101-].last_ds = "0"
ds[101-].value = 0.0000000000e+00
ds[101-].unknown_sec = 0
ds[102-].type = "DERIVE"
ds[102-].minimal_heartbeat = 60
ds[102-].min = 0.0000000000e+00
ds[102-].max = NaN
ds[102-].last_ds = "0"
ds[102-].value = 0.0000000000e+00
ds[102-].unknown_sec = 0
ds[103-].type = "DERIVE"
ds[103-].minimal_heartbeat = 60
ds[103-].min = 0.0000000000e+00
ds[103-].max = NaN
ds[103-].last_ds = "0"
ds[103-].value = 0.0000000000e+00
ds[103-].unknown_sec = 0
ds[104-].type = "DERIVE"
ds[104-].minimal_heartbeat = 60
ds[104-].min = 0.0000000000e+00
ds[104-].max = NaN
ds[104-].last_ds = "0"
ds[104-].value = 0.0000000000e+00
ds[104-].unknown_sec = 0
ds[105-].type = "DERIVE"
ds[105-].minimal_heartbeat = 60
ds[105-].min = 0.0000000000e+00
ds[105-].max = NaN
ds[105-].last_ds = "0"
ds[105-].value = 0.0000000000e+00
ds[105-].unknown_sec = 0
ds[106-].type = "DERIVE"
ds[106-].minimal_heartbeat = 60
ds[106-].min = 0.0000000000e+00
ds[106-].max = NaN
ds[106-].last_ds = "0"
ds[106-].value = 0.0000000000e+00
ds[106-].unknown_sec = 0
ds[191-].type = "DERIVE"
ds[191-].minimal_heartbeat = 60
ds[191-].min = 0.0000000000e+00
ds[191-].max = NaN
ds[191-].last_ds = "1702761427"
ds[191-].value = 3.1231833343e+02
ds[191-].unknown_sec = 0
ds[192-].type = "DERIVE"
ds[192-].minimal_heartbeat = 60
ds[192-].min = 0.0000000000e+00
ds[192-].max = NaN
ds[192-].last_ds = "0"
ds[192-].value = 0.0000000000e+00
ds[192-].unknown_sec = 0
ds[193-].type = "DERIVE"
ds[193-].minimal_heartbeat = 60
ds[193-].min = 0.0000000000e+00
ds[193-].max = NaN
ds[193-].last_ds = "43731333"
ds[193-].value = 0.0000000000e+00
ds[193-].unknown_sec = 0
ds[194-].type = "DERIVE"
ds[194-].minimal_heartbeat = 60
ds[194-].min = 0.0000000000e+00
ds[194-].max = NaN
ds[194-].last_ds = "402684"
ds[194-].value = 0.0000000000e+00
ds[194-].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 8640
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[0].cdp_prep[2].value = NaN
rra[0].cdp_prep[2].unknown_datapoints = 0
rra[0].cdp_prep[3].value = NaN
rra[0].cdp_prep[3].unknown_datapoints = 0
rra[0].cdp_prep[4].value = NaN
rra[0].cdp_prep[4].unknown_datapoints = 0
rra[0].cdp_prep[5].value = NaN
rra[0].cdp_prep[5].unknown_datapoints = 0
rra[0].cdp_prep[6].value = NaN
rra[0].cdp_prep[6].unknown_datapoints = 0
rra[0].cdp_prep[7].value = NaN
rra[0].cdp_prep[7].unknown_datapoints = 0
rra[0].cdp_prep[8].value = NaN
rra[0].cdp_prep[8].unknown_datapoints = 0
rra[0].cdp_prep[9].value = NaN
rra[0].cdp_prep[9].unknown_datapoints = 0


data source names ends with a `-` and cacti shows a error when i try create data template using those names.

So i tried to select use per data source name but after all the passes when i try to build the graphs i get :


RRDTool Command:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="Localhost - Traffic_Control_GRAPH_Template" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="" \
--slope-mode \
GPRINT::LAST:"Current\:%8.2lf %s" \
GPRINT::AVERAGE:"Average\:%8.2lf %s" \
GPRINT::MAX:"Maximum\:%8.2lf %s\n" \
GPRINT::LAST:"Current\:%8.2lf %s" \
GPRINT::AVERAGE:"Average\:%8.2lf %s" \
GPRINT::MAX:"Maximum\:%8.2lf %s\n"

RRDTool Says:

ERROR: Could not parse line 'GPRINT::LAST:Current\:%8.2lf %s'





Any help ???


Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 1:29 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
Hmm, I'm not sure when it comes to "-" in ds names. Perhaps this screws everything up. All I can say now, is that the graph is definitively broken. But let's please start at your Data Template (a screenshot would do).
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 2:01 pm 
Offline

Joined: Sun Feb 18, 2007 11:09 am
Posts: 17
OK.
Starting by data template ...


Attachments:
File comment: data template ...
gkrellShoot_02-18-07_155320.jpg
gkrellShoot_02-18-07_155320.jpg [ 91.33 KiB | Viewed 92212 times ]
File comment: The error with `-` in the end ...
gkrellShoot_02-18-07_155049.jpg
gkrellShoot_02-18-07_155049.jpg [ 111.07 KiB | Viewed 92212 times ]
Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 2:02 pm 
Offline

Joined: Sun Feb 18, 2007 11:09 am
Posts: 17
after graph template ...


Attachments:
gkrellShoot_02-18-07_155820.jpg
gkrellShoot_02-18-07_155820.jpg [ 133.12 KiB | Viewed 92211 times ]
gkrellShoot_02-18-07_155650.jpg
gkrellShoot_02-18-07_155650.jpg [ 68.58 KiB | Viewed 92211 times ]
Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 2:02 pm 
Offline

Joined: Sun Feb 18, 2007 11:09 am
Posts: 17
anything else ?


Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 2:48 pm 
Offline
Developer
User avatar

Joined: Thu Dec 02, 2004 2:46 am
Posts: 22376
Location: Muenster, Germany
colchaodemola wrote:
OK.
Starting by data template ...
Ok, as you said; cacti is not accepting the "-" sign. From current rrdtool man pages (man rrdcreate), I've found
Quote:
DS:ds-name:DST:dst arguments
...
ds-name is the name you will use to reference this particular data source from an RRD. A ds-name
must be 1 to 19 characters long in the characters [a-zA-Z0-9_].
This states, that a "-" sign is not allowed (although in your case, the rrd files accepted this char). So I'm afraid: the solution requires to change the ds names (this is supported by "rrdtool tune ..."). It will require to change your data sampling script(s) as well.
Reinhard

_________________
Official Cacti Documentation
Official Debugging Help
Central Plugin Repository
Central Templates Repository


Top
 Profile  
 
 Post subject:
PostPosted: Sun Feb 18, 2007 3:30 pm 
Offline

Joined: Sun Feb 18, 2007 11:09 am
Posts: 17
If you know perl could you give me a help ?

I am trying to change the script but i always get some syntax error ...

The script is here : http://edseek.com/~jasonb/code/polltc-1.05.tar.gz

It would be a big help :D, it is a very small script ...


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group