Transporter, now at v0.2

Announcements concerning Plugins for Cacti

Moderators: Moderators, Developers

Author
Message
User avatar
johnrembo
Cacti User
Posts: 216
Joined: Mon Apr 24, 2006 3:33 am

Transporter, now at v0.2

#1 Post by johnrembo » Thu Jul 31, 2008 7:15 am

Hi All,

Transporter plugin allows you to simply save poller-output data in cacti's database, so that other applications could access output data without touching rrd-files.

table fragment:
Image

transporter settings:
Image

In current release I've added some sample functions, so you could see what we can do with the archived data (truth, sample calculations currently works only on interface-traffic data templates):

Image

some statistical data:
Image

and charts:
Image
Attachments
node_stats.php.diff.gz
apply this patch if stats/chart page isn't displayed correctly on fresh cacti releazes
(562 Bytes) Downloaded 451 times
transporter_v02.tar.gz
(83.82 KiB) Downloaded 861 times
transporter_v02.zip
(83.13 KiB) Downloaded 697 times
Last edited by johnrembo on Thu Jul 29, 2010 9:33 am, edited 1 time in total.
[url=http://forums.cacti.net/viewtopic.php?t=24934&highlight=zond]zond plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?t=28384]memorize plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?p=141322#141322]transporter plugin[/url]

dononeil
Cacti User
Posts: 194
Joined: Wed Aug 06, 2008 4:45 pm

#2 Post by dononeil » Tue Aug 12, 2008 12:12 pm

I downloaded and isntalled the plugin and it's putting the data in the DB ok, but when I click on the "Chart without RRD" I just get a blank window. Do I need to wait until enough data is stored, or is there something else going on?

User avatar
johnrembo
Cacti User
Posts: 216
Joined: Mon Apr 24, 2006 3:33 am

#3 Post by johnrembo » Wed Aug 13, 2008 2:18 am

dononeil wrote:I downloaded and isntalled the plugin and it's putting the data in the DB ok, but when I click on the "Chart without RRD" I just get a blank window. Do I need to wait until enough data is stored, or is there something else going on?
well,hmm....for data to be displayed on the chart it is enough to have 3 records for the same data-source in transporter output table (poller should run at least 3 times).
If required data is not yet available - it should display error message.
If you don't see any errors - it's time to check your web-server's logs for errors.

ps: have you enabled "Transporter statistics" under "User management"?
[url=http://forums.cacti.net/viewtopic.php?t=24934&highlight=zond]zond plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?t=28384]memorize plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?p=141322#141322]transporter plugin[/url]

FGor
Posts: 1
Joined: Fri Jul 25, 2008 1:28 am
Location: Moscow
Contact:

poller interval??

#4 Post by FGor » Tue Aug 19, 2008 1:33 am

This is plugin work only with update interval = 5min ??

User avatar
johnrembo
Cacti User
Posts: 216
Joined: Mon Apr 24, 2006 3:33 am

Re: poller interval??

#5 Post by johnrembo » Tue Aug 19, 2008 2:02 am

FGor wrote:This is plugin work only with update interval = 5min ??
nope...it should work with any poller interval

Code: Select all

$pi=read_config_option("poller_interval");
[url=http://forums.cacti.net/viewtopic.php?t=24934&highlight=zond]zond plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?t=28384]memorize plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?p=141322#141322]transporter plugin[/url]

dononeil
Cacti User
Posts: 194
Joined: Wed Aug 06, 2008 4:45 pm

#6 Post by dononeil » Thu Oct 23, 2008 3:29 pm

How do I 'unravel' the 'local_data_id' to the host table 'id' value? I don't see where the local_data_id matches any other table values that can eventually get me to where I can figure out what host the data is associated with.

User avatar
johnrembo
Cacti User
Posts: 216
Joined: Mon Apr 24, 2006 3:33 am

#7 Post by johnrembo » Fri Oct 24, 2008 2:11 am

dononeil wrote:How do I 'unravel' the 'local_data_id' to the host table 'id' value? I don't see where the local_data_id matches any other table values that can eventually get me to where I can figure out what host the data is associated with.
Hi,
first part of an answer is here:
http://forums.cacti.net/viewtopic.php?t=24959

and second part is at graph_local table (graph_local.id is actualy local_graph_id from 1st part).
[url=http://forums.cacti.net/viewtopic.php?t=24934&highlight=zond]zond plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?t=28384]memorize plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?p=141322#141322]transporter plugin[/url]

User avatar
TheWitness
Developer
Posts: 14804
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#8 Post by TheWitness » Fri Oct 24, 2008 8:07 am

johnrembo,

Very nice, I have been wanting to do something like this for a while. However, you may want to implement some form of partitioning to allow the data to be pruned rapidly. Since MySQL does not currently support partitioning (5.0 anyway), you might want to consider a workaround, or we should start to look at MySQL 5.1 to resolve table size and management issues.

I will likely be adding this type of functionality to the base, but for now, this is your domain :)

Regards,

Larry
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

User avatar
johnrembo
Cacti User
Posts: 216
Joined: Mon Apr 24, 2006 3:33 am

#9 Post by johnrembo » Fri Oct 24, 2008 9:22 am

speaking of sql partitioning - i've seen "handmade" solutions in mysql versions prior to 5.1 (watch4net APG - EMC SMARTS frontend), but I had no interest in it, because when you run "transporter" on 8 cpu machine with 16Gb's or ram and 250 days uptime - you simply do not care of how big your heap-table is and how many "slow queries" it produces :)

adopting the plugin for "mortal" users requires some time, currently I don't have it (at least until end of 2008). And, if we'll proceed with complete rrdtool replacement (otherwise i do not see any reasons why we should need endless partitioned sql storage) - this plugin should contain sub-plugin - "rrdtool emulator" - that's a challenge you know :)

ps: we had some tests a while ago - ~2500 DSL customers whose snr's, iptv/internet traffic and errors/discards were monitored during the weekend. 3 days of pooling every 1min resulted in ~60Gb "transporter" sql table. Of course plugin is far not perfect and there are many ways to optimize it, but stillt's more complicated that it might seem.
TheWitness wrote:johnrembo,

Very nice, I have been wanting to do something like this for a while. However, you may want to implement some form of partitioning to allow the data to be pruned rapidly. Since MySQL does not currently support partitioning (5.0 anyway), you might want to consider a workaround, or we should start to look at MySQL 5.1 to resolve table size and management issues.

I will likely be adding this type of functionality to the base, but for now, this is your domain :)

Regards,

Larry
[url=http://forums.cacti.net/viewtopic.php?t=24934&highlight=zond]zond plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?t=28384]memorize plugin[/url]
[url=http://forums.cacti.net/viewtopic.php?p=141322#141322]transporter plugin[/url]

User avatar
TheWitness
Developer
Posts: 14804
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#10 Post by TheWitness » Wed Oct 29, 2008 7:21 pm

My goal is not to replace RRDtool, but to use something like www.fusionio.com, combined with something like the CSV exported data to feed a Flash Chart type application like www.fusioncharts.com (no relationship to FusionIO). Then, you can build any Flash type graph you want from the CSV data. However, I need to change that CSV function not to use the PHP XML functions as they are absurd in their use of memory.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

dononeil
Cacti User
Posts: 194
Joined: Wed Aug 06, 2008 4:45 pm

#11 Post by dononeil » Mon Nov 03, 2008 12:10 pm

I'm still having a hard time figuring out what device in my cacti setup is being referenced by the "local_data_id" field in the transporter table. The post before telling me to look at someone elses code didn't get me far, it wasn't what I was looking for.

I need to basically be able to re-map the "local_data_id" field in the transporter table to the "id' in the host table.

Any suggestions on how to get from point a to point b, even if multiple queries, would be appreciated.

Thanks!

dononeil
Cacti User
Posts: 194
Joined: Wed Aug 06, 2008 4:45 pm

#12 Post by dononeil » Thu Nov 06, 2008 12:24 pm

I've run into a new 'bug' of sorts with the transporter plugin... when I try to use the CLI commands I get the following:

php /www/cacti/cli/add_graphs.php --list-graph-templates | grep -i "Interface - Traffic (bits/sec, 64 bit counters)"

PHP Warning: Call-time pass-by-reference has been deprecated; If you would like to pass it by reference, modify the declaration of [runtime function name](). If you would like to enable call-time pass-by-reference, you can set allow_call_time_pass_reference to true in your INI file in /data/cacti/plugins/transporter/setup.php on line 119

83 Interface - Traffic (bits/sec, 64 bit counters)

Now, you can see that I get the proper output on the last line, but of course this is causing my perl script to bomb, because I'm expecting just the last line of output. If I disable the transporter plugin I don't get the Warning.

Looking at the setup.php in the transporter plugin yields:

transporter_process($dbtable,$dbhistcycles,&$rrd_update_array);

I have no idea on how to fix this, to get the warnings to go away so my perl code works again. Any ideas?

User avatar
TheWitness
Developer
Posts: 14804
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#13 Post by TheWitness » Fri Nov 07, 2008 8:03 pm

Edit the php.ini file and locate:

allow_call_time_pass_reference

Set it to "Yes" or "On" (not sure which it is) and restart Apache. Although the restart is not required for the CLI.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

User avatar
zorrosam
Cacti User
Posts: 244
Joined: Thu May 03, 2007 3:17 pm
Location: Italy

problem with plugin

#14 Post by zorrosam » Wed Nov 19, 2008 8:43 am

Hi,

i can't obtain any data in the window after that i clicked on the transporte icon on the right of the graph.

The output is:

no traffic_in or traffic_out data on current graph. Transporter must run at least three times(3 polling cycles) before calculations are available.

Why?

The plugin is enable .... and i think the works with the cacti poller. is correct?

any help is appreciate.

zorro

sangayya.alagundimath
Posts: 20
Joined: Thu Aug 20, 2009 7:41 am

#15 Post by sangayya.alagundimath » Thu Aug 20, 2009 11:21 pm

Hi All,

i have downloaded the plugin and extracted it, but can any one tell me how to install this.

Post Reply