Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Sun Apr 23, 2017 4:52 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 2 posts ] 
Author Message
 Post subject: 0.8.8h upgrade to 1.0.4 some graphs not updating
PostPosted: Thu Mar 16, 2017 10:41 am 
Offline

Joined: Thu Mar 16, 2017 4:05 am
Posts: 2
After the upgrade, we have graphs for certain devices no longer updating.

Things I've tried to fix this:

Upgrade SPINE from 0.8.8h to 1.0.4
Clear poller cache
Change RRD from Simple to Advanced mode.
Enable debug for affected devices, and system wide too (not seeing errors).
Change poller from SPINE to cmd.php.
Chmod'ed all RRDs and directory to 777.
Chown'ed all RRDs and directory to cacti user and group.
Installed Boost (since disabled).
Enabled RRDfile Auto Clean
Made sure timezone info is correct at OS, PHP and MariaDB.
Changed poller and cron intervals from 5 mins to 1 min (alternating and at the same time), reverted back to 5 mins.
Increased number of threads from 30 to 60, reverted.
Increased number of poller processes from 4 to 6.
Increased number of vCPUs from 2 to 4.
Script and server timeout value from 25 to 180 (and in php.ini).
Manually running SPINE and poller from command line.

I'm not seeing errors. The RRD files are being updated, but there's nothing in the graphs in the web interface.

If I move an RRD to something.rrd.old, a new file is created with permissions 644 and the graph starts to be populated.
If I modify a different RRD file that's not graphing to 644, it still doesn't start graphing.
If I move the .old file back to the original file name and ensure 644 permissions - it doesn't work.

CentOS 7.3
Cacti 1.0.4
SPINE 1.0.4
RRDTool 1.4.8
MariaDB 5.5.52-MariaDB
PHP 5.4.16

Has anyone else come across this? I've been trawling through the forums, and it seems others have/had similar issues after upgrade, but their resolution steps never seem to work for me...


Top
 Profile  
 
 Post subject: Re: 0.8.8h upgrade to 1.0.4 some graphs not updating
PostPosted: Fri Mar 17, 2017 8:32 am 
Offline

Joined: Thu Mar 16, 2017 4:05 am
Posts: 2
OK, so found the issue and the answer.

After the update, I had rebooted the VM (it's on Hyper-V, but has Time Synch guest tools disabled - we have NTP configured, so thought it enough).

When it came back online, it thought it was Apr 20th, and I didn't notice that at first.

Corrected the time.

But, it had updated 811 graphs out of 6308 by the time I noticed.

Code:
rrdtool info graphname.rrd | grep last_update
shows the unix timestamp, and just by chance I noticed that it was in the future.

Tried to update graphs with 'rrdtool update N:1:2:3' or whatever, and it refused to update them due to the timestamp.

From everything I've read online, rrdtool will not allow that to be overwritten. So people export to XML, then back again, etc.

Luckily, stumbled upon this nice one-liner: https://stackoverflow.com/a/11645890

Threw that into a script called 'rrd-time-correction.sh'

Exported all rrd filenames to a new file:
Code:
ls >> test.txt


Created another script:
Code:
#!/bin/bash
while read p; do
  lasttime=`rrdtool info $p | grep last_update | awk '{print $3}'`
   if [ $lasttime -gt `date +%s` ]
   then
     echo $p,$lasttime
     cp $p $p.bkp
     ( exec /usr/share/cacti/rra/rrd-time-correction.sh $p )
   fi
done <test.txt


So it will echo the RRD name and unix timestamp of every RRD with a last_update timestamp in the future, backup the RRD file, then run the first script (which will modify the last_update time to right now)

It took 15 minutes to do 811 graphs for me.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group