Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Sat Jun 23, 2018 6:35 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: Broken graphs
PostPosted: Mon May 28, 2018 7:38 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Hi Guys,

We have setup cacti for sometimes now on one of our server 2008 R2 to monitor the network. untill last two weeks everything was working great. But now it looks like that we get some empty spaces in the graphs. Nothing has been changed on the Cacti server and the logs show no error at all. Any ide why we suddenly see broken graphs.

This is one of the graphs and we see the same for all of the servers that we monitor (see attachment)

Thaks


Attachments:
Demosrv.JPG
Demosrv.JPG [ 40.89 KiB | Viewed 226 times ]
Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Mon May 28, 2018 12:03 pm 
Offline
Developer

Joined: Thu Apr 07, 2005 3:29 pm
Posts: 2708
Location: B/CS Texas
Without seeing the logs for that time period, its difficult to tell. You really don't give us anything to go off of.

_________________
Report a bug
Download Releases
1.X Compatible Plugins


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Tue May 29, 2018 2:35 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Thaks for the tpdate,
Do you mean the IIS logs or logs of Cacti itself?
in the cacti logs only see this warning:
05/29/2018 05:10:01 AM - POLLER: Poller[0] WARNING: Scheduled Task is out of sync with the Poller Interval! The Poller Interval is '300' seconds, with a maximum of a '300' second Scheduled Task, but 516 seconds have passed since the last poll!

If you want I can also upload the portion of the IIS logs as well

Thanks


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Tue May 29, 2018 11:23 pm 
Offline
Developer

Joined: Thu Apr 07, 2005 3:29 pm
Posts: 2708
Location: B/CS Texas
Just the Cacti logs is all that is needed. That one particular warning you have would most likely be the problem. Either your scheduled task is not kicking off in time, or your clock time is out of sync. Could be a few other issues, but those are the 1st 2 I would check.

_________________
Report a bug
Download Releases
1.X Compatible Plugins


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Wed May 30, 2018 4:12 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Thanks for your update,

I did check the graphs and it looks like that we again see an break in the graphs between 00:00 AM and 02:00 AM.
at the same time see some warnings in the cacti logs.
Also ren the php c:/cacti_web_root/cacti/poller.php and it shows everything is ok.
can you please tell us how can we first correct these issues?

Thanks


Attachments:
cactipoller.JPG
cactipoller.JPG [ 37.99 KiB | Viewed 198 times ]
cactilog1.JPG
cactilog1.JPG [ 261.24 KiB | Viewed 203 times ]
graph1.JPG
graph1.JPG [ 42.12 KiB | Viewed 203 times ]
Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Wed May 30, 2018 10:01 am 
Offline
Developer

Joined: Thu Apr 07, 2005 3:29 pm
Posts: 2708
Location: B/CS Texas
Your maximum run time is going over. If it always happens during a specific time frame, do you have backups or something else running at that time? You are going from normally taking 10 seconds to taking 5 minutes, so something is definitely slowing your server down during that period.

First step is to go into Settings and up the process count of cmd.php. Having just 1 process makes it really slow. cmd.php is super slow compared to spine, but you can deal with that later. I would up it to say, 10 for now.

_________________
Report a bug
Download Releases
1.X Compatible Plugins


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Thu May 31, 2018 4:19 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Thank you for the update,

sure at the time that we see the blanck space in graphs mostlly is back-up is running
I did change the Number of PHP Script Servers under settings-Poller to 10.
I will give it a try and because this does happen randomly, when I see the issue again I will let you know.

Thanks


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Mon Jun 11, 2018 3:46 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Hi,

It looks like that increasing the Poller to 10 did not solve the issue.

Any sugestion?

Thanks


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Mon Jun 11, 2018 9:44 am 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 1114
There is an error saying that your poller task is out of sync with the scheduled task. This suggests that your tasks are still overrunning above or becoming hung for some reason (more than likely when trying to communicate with other systems).

Try turning up the logging levels to see if any more information is produced that might make it clearer.

_________________
Cacti Resources:
Cacti Website
Cacti Issues
Cacti Releases

My resources:
How to submit Pull Requests
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Tue Jun 12, 2018 5:11 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Hi,

I will enable the DeBug, statistics,Errors,results and I/O and will post the result here.

Thanks again.


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Mon Jun 18, 2018 6:32 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Hi,

I did enable the debuge logs but did not find any usefull information, or I dont know what I should look for!
Any suggestion on how to get rid of the wranning regarding the poller interval?


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Tue Jun 19, 2018 2:34 am 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 1114
There is an option within the Task Scheduler's task properties to say only run one instance (which is used by default). You can change that to be run another copy (or the language equivalent) but I think that would mean you have multiple processes running and polling at the same time.

_________________
Cacti Resources:
Cacti Website
Cacti Issues
Cacti Releases

My resources:
How to submit Pull Requests
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Tue Jun 19, 2018 3:18 am 
Offline

Joined: Mon May 28, 2018 7:28 am
Posts: 8
Hi,

This is what I could find in the logs:
Host[51] ERROR: HOST EVENT: Host is DOWN Message: Host did not respond to SNMP
06/16/2018 10:10:11 PM - CMDPHP: Poller[0] Host[54] RECACHE DQ[1] OID: .1.3.6.1.2.1.1.3.0, output: 23505264
06/16/2018 10:10:11 PM - CMDPHP: Poller[0] Host[54] DS[52] SNMP: v2: 10.0.10.232, dsname: traffic_in, oid: .1.3.6.1.2.1.2.2.1.10.12, output: 2042039928
06/16/2018 10:10:11 PM - CMDPHP: Poller[0] Host[54] DS[52] SNMP: v2: 10.0.10.232, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.12, output: 1045884375
06/16/2018 10:10:11 PM - CMDPHP: Poller[0] Time: 1.1433


CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (69, 'traffic_out', '2018-06-13 01:47:26', '1753281276')'
06/13/2018 01:55:01 AM - POLLER: Poller[0] WARNING: Scheduled Task is out of sync with the Poller Interval! The Poller Interval is '300' seconds, with a maximum of a '300' second Scheduled Task, but 481 seconds have passed since the last poll!
06/13/2018 01:55:01 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 6, Data Sources: traffic_in(DS[55]), traffic_out(DS[55]), traffic_in(DS[68]), traffic_out(DS[68]), traffic_in(DS[69]), traffic_out(DS[69])
06/13/2018 01:55:10 AM - SYSTEM STATS: Time:8.4062 Method:cmd.php Processes:10 Threads:N/A Hosts:34 HostsPerProcess:4 DataSources:68 RRDsProcessed:34
06/13/2018 02:00:19 AM - POLLER: Poller[0] WARNING: Scheduled Task is out of sync with the Poller Interval! The Poller Interval is '300' seconds, with a maximum of a '300' second Scheduled Task, but 318 seconds have passed since the last poll!
06/13/2018 02:00:30 AM


Top
 Profile  
 
 Post subject: Re: Broken graphs
PostPosted: Fri Jun 22, 2018 2:50 am 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 1114
shahin wrote:
CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (69, 'traffic_out', '2018-06-13 01:47:26', '1753281276')'


This bit worries me. It would suggest to me a problem with your MySQL installation. Unless I'm missing something there, the command looks perfectly valid.

_________________
Cacti Resources:
Cacti Website
Cacti Issues
Cacti Releases

My resources:
How to submit Pull Requests
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group