Cacti (home)ForumsRepositoryDocumentation
Cacti: offical forums and support  

 FAQFAQ   SearchSearch   MemberlistMemberlist    RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in    


rrd_use_poller_output not working

 
Post new topic   Reply to topic    Cacti Forum Index -> Plugin: (Network Weather Map)
Author Message
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Mon Mar 24, 2008 4:07 pm    Post subject: rrd_use_poller_output not working Reply with quote

I am running boost, so I am trying to get PHP Weathermap 0.95b working with cacti 0.8.7b and the latest boost out of SVN. However, the rrd_use_poller_output option in my config file doesn't appear to be working. As long as the RRD files are up to date my weathermap updates properly, if boost hasn't flushed them then I just see "0" for all of my links.

Here is the only thing I see in my log that appears to be related:
03/24/2008 02:01:15 PM - WEATHERMAP: Poller[0] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output: traffic_in is not a valid RRD filename within this Cacti install.
03/24/2008 02:01:15 PM - WEATHERMAP: Poller[0] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output: traffic_out is not a valid RRD filename within this Cacti install.
03/24/2008 02:01:15 PM - WEATHERMAP: Poller[0] [Map 1] rwc-desktop.conf: ReadData: LINK rwcdc86-1, target: /var/log/cacti/rra/rwc_desktop_switch_rwcdc86_traffic_in_1242.rrd on config line 124 had no valid data, according to WeatherMapDataSource_rrd

And I will see those errors for every link that doesn't get a value.

Any thoughts?
Back to top
Howie
Cacti Guru User


Joined: 16 Sep 2004
Posts: 1989
Location: United Kingdom

PostPosted: Mon Mar 24, 2008 4:37 pm    Post subject: Reply with quote

How about if you run with DEBUG logging for a poller cycle? You should get some more information then. It is taking notice of your SET line, because those warnings are from the poller_output part of the deal.

It does take a few cycles to get up to speed with poller_output though - one pass to find the required rrds, and then two more before it has a real value (it needs two values to get a difference). You can see if it's working at all by looking in the weathermap_data table to see if it has anything.

One other thing is that it only works if the path you use to your rrds is the same one Cacti uses (the /var/log looks a little funny). It expects to be able to convert the TARGET back into a <rra_path>/blah.rrd, just like Cacti itself uses.

I am using poller_output with 0.95b OK at the moment (although not with Boost), so I don't think it's a general thing.
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Mon Mar 24, 2008 4:43 pm    Post subject: Reply with quote

I wonder if it's the symlink from the cacti install to /var/log/cacti/rra that is causing it trouble. I'll try changing the path and see if that helps. Here is the debug info, I was just running it with the intention of posting when I saw you post.

Code:

03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@WeatherMapDataSource_rrd.php:383] [Map 1] rwc-desktop.conf: RRD ReadData: Returning (610321156.92,610321156.92,1206394200)
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@Weathermap.class.php:606] [Map 1] rwc-desktop.conf: ReadData: New Target: /var/log/cacti/rra/rwc_core_router_1_traffic_in_654.rrd
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@Weathermap.class.php:637] [Map 1] rwc-desktop.conf: ReadData: Matched for WeatherMapDataSource_rrd. Calling WeatherMapDataSource_rrd->ReadData()
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@WeatherMapDataSource_rrd.php:332] [Map 1] rwc-desktop.conf: SCALING result by 8
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:64] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output style
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:70] [Map 1] rwc-desktop.conf: ******************************************************************
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:73] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - looking for 0 value
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:76] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - DS name is traffic_in
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:85] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - Adding new weathermap_data row for /var/log/cacti/rra/rwc_core_router_1_traffic_in_654.rrd:traffic_in
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output: traffic_in is not a valid RRD filename within this Cacti install.
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:73] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - looking for 1 value
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:76] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - DS name is traffic_out
03/24/2008 02:36:25 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:85] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - Adding new weathermap_data row for /var/log/cacti/rra/rwc_core_router_1_traffic_in_654.rrd:traffic_out
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output: traffic_out is not a valid RRD filename within this Cacti install.
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:142] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - result is ,
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:143] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - ended
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@WeatherMapDataSource_rrd.php:360] [Map 1] rwc-desktop.conf: RRD ReadData: Target DS names are traffic_in and traffic_out
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:173] [Map 1] rwc-desktop.conf: RRD ReadData: traditional style
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:188] [Map 1] rwc-desktop.conf: RRD ReadData: Running: /usr/bin/rrdtool fetch /var/log/cacti/rra/rwc_core_router_1_traffic_in_654.rrd AVERAGE --start now-800 --end now
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:208] [Map 1] rwc-desktop.conf: > 1206393900: 2.0913231166e+07 1.5850227363e+07
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:208] [Map 1] rwc-desktop.conf: > 1206394200: 3.1450352714e+07 1.6741332200e+07
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:208] [Map 1] rwc-desktop.conf: > 1206394500: nan nan
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:208] [Map 1] rwc-desktop.conf: > 1206394800: nan nan
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:208] [Map 1] rwc-desktop.conf: >
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:215] [Map 1] rwc-desktop.conf: RRD ReadData: Read 5 lines from rrdtool
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:216] [Map 1] rwc-desktop.conf: RRD ReadData: Headings are:                      traffic_in         traffic_out
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:225] [Map 1] rwc-desktop.conf: --
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:225] [Map 1] rwc-desktop.conf: --1206394800: nan nan
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:225] [Map 1] rwc-desktop.conf: --1206394500: nan nan
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:225] [Map 1] rwc-desktop.conf: --1206394200: 3.1450352714e+07 1.6741332200e+07
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:246] [Map 1] rwc-desktop.conf: 3.1450352714e+07 is OK value for traffic_in
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:246] [Map 1] rwc-desktop.conf: 1.6741332200e+07 is OK value for traffic_out
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_real_rrdtool@WeatherMapDataSource_rrd.php:280] [Map 1] rwc-desktop.conf: RRD ReadDataFromRealRRD: Returning (3.1450352714e+07,3.1450352714e+07,1206394200)
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@WeatherMapDataSource_rrd.php:383] [Map 1] rwc-desktop.conf: RRD ReadData: Returning (251602821.712,251602821.712,1206394200)
03/24/2008 02:36:26 PM - WEATHERMAP: Poller[0] DEBUG: [ReadData@Weathermap.class.php:606] [Map 1] rwc-desktop.conf: ReadData: New Target: /var/log/cacti/rra/rwc_core_router_1_traffic_in_657.rrd
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Mon Mar 24, 2008 4:47 pm    Post subject: Reply with quote

The the other possibility -- the SVN version of boost (the only one that works at this point) uses a different poller_output table. It reads everything into poller_output_boost, and it looks like poller_output stays empty. If I could find he code that looking at poller_output I'll point it to the new table and see if that makes any difference.
Back to top
Howie
Cacti Guru User


Joined: 16 Sep 2004
Posts: 1989
Location: United Kingdom

PostPosted: Mon Mar 24, 2008 5:00 pm    Post subject: Reply with quote

wwwdrich wrote:
The the other possibility -- the SVN version of boost (the only one that works at this point) uses a different poller_output table. It reads everything into poller_output_boost, and it looks like poller_output stays empty. If I could find he code that looking at poller_output I'll point it to the new table and see if that makes any difference.


No, it's not that. Weathermap doesn't look into the tables at all, it hooks into the poller process just as the poller is putting data into the poller_output table, and saves a copy of the relevant things for itself.

I have had Boost running on my test server with weathermap in the (recent) past.
Back to top
Howie
Cacti Guru User


Joined: 16 Sep 2004
Posts: 1989
Location: United Kingdom

PostPosted: Mon Mar 24, 2008 5:04 pm    Post subject: Reply with quote

wwwdrich wrote:
I wonder if it's the symlink from the cacti install to /var/log/cacti/rra that is causing it trouble. I'll try changing the path and see if that helps. Here is the debug info, I was just running it with the intention of posting when I saw you post.


If Cacti thinks the data is somewhere else (/var/www/html/cacti/rra say), then that is the path you must use in your weathermap configs. Weathermap needs to know how to match up the TARGET with the Cacti updates, and they take the form "<rra_path>/bla.rrd", so it must assume that the Cacti rra_path can be replaced with "<rra_path>". If you have a symlink in there, then there are two paths to the same thing, and Cacti (and therefore weathermap) only know about one of them.

This is only an issue with using poller_output - weathermap will follow symlinks for files with no trouble. It will definitely get confused trying to match up poller_output updates to a symlinked directory though.
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Mon Mar 24, 2008 5:13 pm    Post subject: Reply with quote

I just tweaked all of the paths in my config file to match what should be the path in the database - i.e. /var/www/html/cacti/rra instead of where the files really live. Where is it getting rra_path from now? Does cacti calculate it from PHP_SELF? I have also noticed that it is rewriting the URL to ignore the symlink we have at our cacti base -- which is going to make upgrades a nightmare in the future (i.e. .../cacti is a symlink to .../cacti-0.8.7b). This is new behaviour in 0.8.7b, and causes other headaches like making me run the boost tools with full paths instead of as ./blahblah.php.
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Mon Mar 24, 2008 5:24 pm    Post subject: Reply with quote

I think if I can figure out what it wants in path_rra I will have this licked. As an example:

Code:
mysql> select * from data_template_data where data_source_path like '%rwc_desktop_switch_rwcdc86_traffic_in_1242.rrd';
+------+-----------------------------+---------------+------------------+---------------+--------+-----------------------------------------------------------------+--------------------------------------------------------------------------+-----------------------------------------------------------+----------+--------+------------+----------+----------+
| id   | local_data_template_data_id | local_data_id | data_template_id | data_input_id | t_name | name    | name_cache                                                               | data_source_path                                          | t_active | active | t_rrd_step | rrd_step | t_rra_id |
+------+-----------------------------+---------------+------------------+---------------+--------+-----------------------------------------------------------------+--------------------------------------------------------------------------+-----------------------------------------------------------+----------+--------+------------+----------+----------+
| 1347 |                          41 |          1242 |               41 |       2 | NULL   |   |host_description| - Traffic - |query_ifAlias| |query_ifName| |   RWC Desktop Switch rwcdc86 - Traffic - rwc-core-1-uplink-1 ethernet1/1 | <path_rra>/rwc_desktop_switch_rwcdc86_traffic_in_1242.rrd | NULL     | on     | NULL       |      300 | NULL     |
+------+-----------------------------+---------------+------------------+---------------+--------+-----------------------------------------------------------------+--------------------------------------------------------------------------+-----------------------------------------------------------+----------+--------+------------+----------+----------+
1 row in set (0.02 sec)


So I know the RRD data is in the correct table. But the part that doesn't look good:
Code:

mysql> select count(*) from weathermap_data;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.00 sec)


So even after three polling cycles weathermap still hasn't found it.

I'm about to head out (I came in at 6am this morning -- all of the code is starting to look the same), so I'll dig a little deeper tomorrow.

Thanks for the help!
Back to top
Howie
Cacti Guru User


Joined: 16 Sep 2004
Posts: 1989
Location: United Kingdom

PostPosted: Mon Mar 24, 2008 6:12 pm    Post subject: Reply with quote

Looks like it will always be $base_path/rra...
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Tue Mar 25, 2008 9:58 am    Post subject: Reply with quote

Ok, I think I see the problem. It looks like weathermap is using the absolute path for the file -- i.e. following the symlink. Whereas, Cacti has the relative path to the file, i.e. /var/www/html/cacti-0.8.7b/rra. So, I see messages like:


Code:

03/25/2008 07:51:25 AM - WEATHERMAP: Poller[0] DEBUG: [wmrrd_read_from_poller_output@WeatherMapDataSource_rrd.php:85] [Map 1] rwc-desktop.conf: RRD ReadData: poller_output - Adding new weathermap_data row for /var/log/cacti/rra/rwc_desktop_switch_rwcidf4b_traffic_in_2137.rrd:traffic_in


in the debug log, even though I have:
Code:

        TARGET /var/www/html/cacti-0.8.7b/rra/rwc_desktop_switch_rwcidf4b_traffic_in_2137.rrd

in my weathermap config file.

Any thoughts as to a workaround for this? Since weathermap is following the link, there is nothing I can put in my config file to fix this as far as I know (and I have tried all of the various combinations of paths for this one). I think this is due to the change in 0.8.7 for cacti to "calculate" it's own base path instead of allowing me to specify a location for rra_path. I suppose I could force a path in global.php, but then I have to remember to do that every time we upgrade.
Back to top
Howie
Cacti Guru User


Joined: 16 Sep 2004
Posts: 1989
Location: United Kingdom

PostPosted: Tue Mar 25, 2008 11:45 am    Post subject: Reply with quote

Weathermap doesn't follow the link. It just tries to replace a chunk of the path with $cacti_root/rra. It doesn't actually look at the FS at all.

So if you use whatever Cacti thinks it's root directory is, in the TARGET, it should all work. This is all it does (line 65 or so, WeathermapDataSource_rrd.php):
Code:

$db_rrdname = realpath($rrdfile);
$db_rrdname = str_replace($config["base_path"]."/rra","<path_rra>",$db_rrdname);


since all your paths are already absolute, you might want to try commenting out that first line, which may actually look at the FS, and follow links.

base_path is calculated by Cacti, but it's basically the directory that holds the include directory.
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Tue Mar 25, 2008 1:38 pm    Post subject: Reply with quote

I haven't tried it yet (trying to get a different map cleaned up), but it looks like that is the culprit. From the PHP docs:
Quote:

realpath() expands all symbolic links and resolves references to '/./', '/../' and extra '/' characters in the input path . and return the canonicalized absolute pathname.
Back to top
wwwdrich
Cacti User


Joined: 03 Feb 2005
Posts: 54
Location: San Jose, CA

PostPosted: Tue Mar 25, 2008 4:10 pm    Post subject: Reply with quote

I can't just comment it out, otherwise db_rrdname isn't set. However, changing it as below fixes it!

Code:

$db_rrdname = $rrdfile;


Thanks again for the help -- I owe you a beer (or at least something off of your wishlist). BTW, I prefer the bongo ties from MarkerTek over the cable ties that were on your Amazon Wishlist this morning -- and someone beat me to buying the Rosco swatch book! )
Back to top
Howie
Cacti Guru User


Joined: 16 Sep 2004
Posts: 1989
Location: United Kingdom

PostPosted: Tue Mar 25, 2008 4:40 pm    Post subject: Reply with quote

wwwdrich wrote:
I can't just comment it out, otherwise db_rrdname isn't set. However, changing it as below fixes it!

Code:

$db_rrdname = $rrdfile;


Thanks again for the help -- I owe you a beer (or at least something off of your wishlist). BTW, I prefer the bongo ties from MarkerTek over the cable ties that were on your Amazon Wishlist this morning -- and someone beat me to buying the Rosco swatch book! )


lol Actually I did a little tidy-up. I have a different set of ball bungees, and the Lee swatch book already... I figured that one book of 200+ gels would be enough

Thanks for the thought though.
Back to top
Display posts from previous:   
Post new topic   Reply to topic    Cacti Forum Index -> Plugin: (Network Weather Map) All times are GMT - 5 Hours
Page 1 of 1

 



Powered by phpBB © 2001, 2005 phpBB Group