Handy Repopulate Poller Cache Script

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Post Reply
Author
Message
User avatar
TheWitness
Developer
Posts: 14817
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Handy Repopulate Poller Cache Script

#1 Post by TheWitness » Thu Nov 10, 2005 8:59 pm

Hi,

I was asked to write a quick and dirty rebuild poller cache script and I tought I would share it for the rest of the community.

To use, extract to the <cacti_webroot> directory

type "php rebuild_poller_cache.php"

Watch it go.

TheWitness
Attachments
rebuild_poller_cache.zip
(1.19 KiB) Downloaded 12429 times
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

drose12
Posts: 31
Joined: Thu Mar 10, 2005 11:02 am

#2 Post by drose12 » Thu Nov 10, 2005 9:02 pm

Thanks a bunch!
You do great work!

User avatar
N3NCY
Cacti User
Posts: 242
Joined: Thu Sep 22, 2005 7:50 pm
Location: Landenberg, PA
Contact:

#3 Post by N3NCY » Sat Nov 12, 2005 5:06 pm

Ok, I have been waiting, now is the time to ask:

What is the poller cache?
When would you want to clear your poller cache?
What is the down side to clearing poller cache?
Doesn't poller cache rebuild automagically next run?
Why would a person want to rebuild poller cache manually via this script?

Sorry, as you can see, poller cache is a complete poller mystery to me!
:D
Thank you,

Ernie
http://www.NMSWorld.com
[b]Dual Zeon Dual Core 2.6Ghz / 8GB RAM / 4x15k RPM SATA RAID5[/b]
[b]Cacti Version[/b] - 0.8.7b
[b]Poller Type[/b] - cactid 0.8.7 with Boost v1.7
[b]Server Info[/b] - FreeBSD 7.0-RELEASE
[b]Web Server[/b] - Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.8g DAV/2 PHP/5.2.5 mod_perl/2.0.3 Perl/v5.8.8
[b]PHP[/b] - 5.2.6
[b]MySQL[/b] - 5.0.51b Mod: poller_output ENGINE = MEMORY
[b]RRDTool[/b] - 1.3.0
[b]SNMP[/b] - 5.4.1
[b]Plugins[/b] - Host Info (hostinfo - v0.2), Update Checker (update - v0.3), Network Tools (tools - v0.2), FlowView (flowview - v0.3), Read-only Devices Tab (devices - v0.4), Network Discovery (discovery - v0.8.3), Syslog Monitoring (syslog - v0.5.2), Thresholds (thold - v0.3.9), Device Monitoring (monitor - v0.8.2), PHP Network Weathermap (weathermap - v0.941), SuperLinks (superlinks - v0.72), Report Creator (reports - v0.1b)

User avatar
TheWitness
Developer
Posts: 14817
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#4 Post by TheWitness » Sat Nov 12, 2005 9:12 pm

What is the poller cache?
The poller_cache is (aka poller_item table) a intermediary location for all data source items. Since a data source is made from several database components in 0.8.6x, it made sense to speed up the polling to place that information in a static file to increase poller performance.
When would you want to clear your poller cache?
There are a few things that don't have inheritance when you change them, like "Data Input Methods". If you were to say change one and then want all of your Data Sources (aka Poller_items aka Poller_cache) to be updated with the changes, you would have to "update"/"reload" we are going to change the name, the poller_cache.
What is the down side to clearing poller cache?
It just takes lot's of time when you have 20k data sources. You should always attempt to clear between polling cycles to avoid gaps in your graphs. Another downside is that if you have damaged a template and then subsequentyly cleared your poller cache, items may no longer graph.

Best advice is good advice, back up your environment regularly.
Doesn't poller cache rebuild automagically next run?
No, it updates on a host by host basis when you edit it, a data template/data source/graph/graph template, when you edit them.
Why would a person want to rebuild poller cache manually via this script?
So you can watch it prgress instead of looking at your blank web page. We should do something more animated in the web space, I know. Besides, a user asked me to write it an I thought the community might benefit.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

User avatar
rony
Developer/Forum Admin
Posts: 6016
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

#5 Post by rony » Wed Jan 04, 2006 3:27 pm

Now included in 0.8.6h.
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]

BelgianViking
Cacti User
Posts: 97
Joined: Thu Mar 24, 2005 4:59 am
Location: Brussels, Belgium

#6 Post by BelgianViking » Wed Aug 02, 2006 1:45 am

Witness,
Due to the fact that I have a device on which interfaces appear and dissapear (virtual interfaces on a Nortel VPN Router), I have to 'Reload Data Query' regularly to get the graphs going. Would this script do the same thing automatically ? I guess it would be easily adapted to only reload the cache for this device without touching that of all other devices...
Thanks in advance !
[size=75][color=#EE5019]| Cacti 0.8.6g | MySQL 4.1.14 w Query Cache | Net-SNMP 5.2.1 | IIS 6 | fast-cgi | PHP 5.0.3 | RRDtool 1.2.9 | Windows 2003 Server SP1 | Cactid 0.8.6f |
| Dell 2450 - 2x P3 733 MHz, 1GB RAM |[/size][/color]

User avatar
TheWitness
Developer
Posts: 14817
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

#7 Post by TheWitness » Fri Nov 10, 2006 8:10 am

Cron it and it then becomes automatic.

TheWitnes
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

crasis_prefect
Posts: 1
Joined: Mon Oct 08, 2007 11:10 am

#8 Post by crasis_prefect » Thu Oct 18, 2007 2:38 pm

Would this script query all the routers I'm monitoring to see if any new interfaces have been added to device? I've been trying to find out how cacti handles that.

Regards,

fmerrill
Posts: 12
Joined: Tue Oct 28, 2008 7:56 pm
Location: North Carolina

#9 Post by fmerrill » Fri Dec 26, 2008 12:20 am

crasis_prefect wrote:Would this script query all the routers I'm monitoring to see if any new interfaces have been added to device? I've been trying to find out how cacti handles that.
There is a cli script that can recheck a host for changes/new interfaces on polled devices: poller_reindex_hosts.php
That is in the <cacti_root>/cli/ directory.

If interface indexes have changed as the result of a change on the device itself, that will pick them up, but, that does not automatically add any new graphs for those new interfaces.

If you only have one or two devices that you know have changed, then it's almost easier to just go to the device form, go the 'create graphs for this device page' click the green circle to have it poll and re-index the device, and the new interfaces will show up, and then you can add any additional graphs needed.

At some point I was hoping I could get around to setting up and catching device change SNMP traps, and then have that launch a script that reindexed the device, then automatically added graphs if new interfaces met some specific requirements (I don't always graph all interfaces on a device), but, I just have never gotten around to it.

I'd really like to see Cacti do re-indexing automatically - maybe a certain number of devices per hour/day/whatever time-frame, then having it tell me somewhere in the management interface of 'new' or 'changed' interfaces on devices, and then giving me the option of creating/deleting/changing graphs at that time.

pathebest15
Posts: 9
Joined: Mon Jul 05, 2010 10:17 am

#10 Post by pathebest15 » Wed Jul 07, 2010 10:15 am

hello guys i know it is an old post but i have the same issue i coudn't graph in cacti after figure it out with help i know it is my poller is quite massive i tried the rebuild command in cli but i got a message error Cannot connect to Mysql server on localhost ... specify a valid Mysql database in include/config.php. I looked everywhere how to do it without success

thank you for your help

User avatar
TFC
Cacti Pro User
Posts: 722
Joined: Wed Apr 09, 2003 2:17 am
Location: Izmir/Turkey

Re: Handy Repopulate Poller Cache Script

#11 Post by TFC » Mon Aug 22, 2011 5:27 am

I have 4 hosts
3 of them is Windows. One of them linux localhost.
I try to monitor CPU,MEM and Disk Usages of windows server.
So I haveatleast 3x3=12 rrd filesfow windows server (dependshow many drives such as C, D and E) and 5 linux localhost rrd files.
But altough I rebuild poller cache manually or by using CLI script, still i cant see my Disk usage rrd files.

What is the problem?
I mean:
08/22/2011 01:35:08 PM - SPINE: Poller[0] NOTE: Spine is behaving in a 0.8.7g manner
08/22/2011 01:35:08 PM - SPINE: Poller[0] Host[2] TH[1] NOTE: There are '2' Polling Items for this Host
08/22/2011 01:35:08 PM - SPINE: Poller[0] Host[2] TH[1] DS[8] SCRIPT: /usr/bin/perl -wU /var/www/cacti/scripts/nt_cpu.pl 10.1.1.24, output: 1
Name "main::UID" used only once: possible typo at /var/www/cacti/scripts/nt_memuse.pl line 3.
Name "main::EUID" used only once: possible typo at /var/www/cacti/scripts/nt_memuse.pl line 3.
08/22/2011 01:35:08 PM - SPINE: Poller[0] Host[2] TH[1] DS[16] SCRIPT: /usr/bin/perl -wU /var/www/cacti/scripts/nt_memuse.pl 10.1.1.24, output: total:869.98 used:469.38
08/22/2011 01:35:08 PM - SPINE: Poller[0] Time: 0.2587 s, Threads: 1, Hosts: 2
But actually 3 polling item for this host. Why spien cant see third items?

Sorry for next posts.. Acidentall pushing. I couldnt delete them
Last edited by TFC on Mon Aug 22, 2011 5:38 am, edited 2 times in total.

User avatar
TFC
Cacti Pro User
Posts: 722
Joined: Wed Apr 09, 2003 2:17 am
Location: Izmir/Turkey

Re: Handy Repopulate Poller Cache Script

#12 Post by TFC » Mon Aug 22, 2011 5:29 am

I have 4 hosts
3 of them is Windows. One of them linux localhost.
I try to monitor CPU,MEM and Disk Usages of windows server.
So I haveatleast 3x3=12 rrd filesfow windows server (dependshow many drives such as C, D and E) and 5 linux localhost rrd files.
But altough I rebuild poller cache manually or by using CLI script, still i cant see my Disk usage rrd files.

What is the problem?

User avatar
TFC
Cacti Pro User
Posts: 722
Joined: Wed Apr 09, 2003 2:17 am
Location: Izmir/Turkey

Re: Handy Repopulate Poller Cache Script

#13 Post by TFC » Mon Aug 22, 2011 5:30 am

I have 4 hosts
3 of them is Windows. One of them linux localhost.
I try to monitor CPU,MEM and Disk Usages of windows server.
So I haveatleast 3x3=12 rrd filesfow windows server (dependshow many drives such as C, D and E) and 5 linux localhost rrd files.
But altough I rebuild poller cache manually or by using CLI script, still i cant see my Disk usage rrd files.

What is the problem?

ealcaniz
Posts: 40
Joined: Fri Sep 30, 2011 3:37 am

Re: Handy Repopulate Poller Cache Script

#14 Post by ealcaniz » Thu Apr 18, 2013 9:02 am

bash-3.2$ php rebuild_poller_cache.php
PHP Fatal error: Call to undefined function db_execute() in /var/www/cacti/rebuild_poller_cache.php on line 44

I got this issue. any ideas?
Thanks

Edu

surajitpal2007
Posts: 35
Joined: Tue Feb 15, 2011 4:51 am

Re: Handy Repopulate Poller Cache Script

#15 Post by surajitpal2007 » Thu Sep 26, 2013 3:46 pm

Hi,

I am running cacti in windows 2003. "Rebuild Poller Cache", "View SNMP Cache" and "View Poller Cache" option showing in systemutility however, "clear poller cache" is not showing.

Please advise.

Thanks a ton in advance.

--Jit

Post Reply