Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Sat Dec 15, 2018 9:53 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Wed Mar 07, 2018 8:05 pm 
Offline

Joined: Mon Jul 10, 2017 3:31 pm
Posts: 8
Hello there folks.

I am having a strange issue with syslog msgs not showing up under the syslog tab. I can see the msgs in the MySQL database and I see the Cacti logs showing the msgs as XFer which makes it seem like they are being moved somewhere else. I know I must be missing something simple :oops:.

I have researched this extensively through all of the available documentation.

Any thoughts would be greatly appreciated :)

Here is my config from a clean install:
Quote:
Ubuntu 16.04 LTS
Cacti 1.1.36
syslog plugin 2.1
syslog-ng 3.5
Using the cactidb for syslog storage


Here is my syslog config.php
Code:
<?php
/*
+-------------------------------------------------------------------------+
| Copyright (C) 2007-2014 The Cacti Group                                 |
|                                                                         |
| This program is free software; you can redistribute it and/or           |
| modify it under the terms of the GNU General Public License             |
| as published by the Free Software Foundation; either version 2          |
| of the License, or (at your option) any later version.                  |
|                                                                         |
| This program is distributed in the hope that it will be useful,         |
| but WITHOUT ANY WARRANTY; without even the implied warranty of          |
| MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the           |
| GNU General Public License for more details.                            |
+-------------------------------------------------------------------------+
| Cacti: The Complete RRDTool-based Graphing Solution                     |
+-------------------------------------------------------------------------+
| This code is designed, written, and maintained by the Cacti Group. See  |
| about.php and/or the AUTHORS file for specific developer information.   |
+-------------------------------------------------------------------------+
| http://www.cacti.net/                                                   |
+-------------------------------------------------------------------------+
*/

global $config, $database_type, $database_default, $database_hostname;
global $database_username, $database_password, $database_port;

/* revert if you dont use the Cacti database */
$use_cacti_db = true;

if (!$use_cacti_db) {
        $syslogdb_type     = 'mysql';
        $syslogdb_default  = 'cactidb';
        $syslogdb_hostname = 'localhost';
        $syslogdb_username = 'someuser';
        $syslogdb_password = 'somepassword';
        $syslogdb_port     = 3306;
}else{
        $syslogdb_type     = $database_type;
        $syslogdb_default  = $database_default;
        $syslogdb_hostname = $database_hostname;
        $syslogdb_username = $database_username;
        $syslogdb_password = $database_password;
        $syslogdb_port     = $database_port;
}

/* field in the incomming table */
$syslog_incoming_config['dateField']          = 'date';
$syslog_incoming_config['timeField']          = 'time';
$syslog_incoming_config['priorityField']      = 'priority_id';
$syslog_incoming_config['facilityField']      = 'facility_id';
$syslog_incoming_config['hostField']          = 'host_id';
$syslog_incoming_config['textField']          = 'message';
$syslog_incoming_config['id']                 = 'seq';

?>


Here is my syslog-ng.conf file:
Code:
@version: 3.5
#Bare minimum syslog-ng configuration

# First, set some global options.
options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);
          owner(root); group(adm); perm(0640); stats_freq(0);
          bad_hostname(^gconfd);
};

# we are using udp, and this is a collector for net traffic only
source s_all { udp(); };

destination d_mysql {
          sql(type(mysql)
          host("localhost") username("someuser") password("somepassword")
          database("cactidb") table("syslog_incoming")
          columns("facility", "priority", "date", "time", "host", "message")
          values("$FACILITY","","$YEAR-$MONTH-$DAY", "$HOUR:$MIN:$SEC","$HOST_FROM","$MSG")
          indexes("facility", "priority", "date", "time", "host", "msg"));
};

log { source(s_all); destination(d_mysql); };


Here is what the Cacti log is showing:
Code:
03/07/2018 18:55:03 - SYSTEM SYSLOG STATS:Time:0.02 Deletes:0 Incoming:0 Removes:0 XFers:221 Alerts:0 Alarms:0 Reports:0
03/07/2018 18:55:03 - SYSTEM STATS: Time:1.2432 Method:cmd.php Processes:1 Threads:N/A Hosts:1 HostsPerProcess:1 DataSources:5 RRDsProcessed:5


Top
 Profile  
 
 Post subject: Re: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Thu Mar 08, 2018 8:14 pm 
Offline
Cacti User
User avatar

Joined: Wed Jul 20, 2016 8:00 pm
Posts: 177
I get the same thing under STATS. XFers might be related to the fact that it moves msgs from syslog_incoming to syslog.

Quote:
09/Mar/2018 14:03:16 - SYSTEM SYSLOG STATS:Time:0.72 Deletes:0 Incoming:0 Removes:3 XFers:0 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 14:02:34 - SYSTEM SYSLOG STATS:Time:0.73 Deletes:0 Incoming:0 Removes:0 XFers:7 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 14:01:52 - SYSTEM SYSLOG STATS:Time:0.74 Deletes:0 Incoming:0 Removes:0 XFers:17 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 14:00:47 - SYSTEM SYSLOG STATS:Time:0.71 Deletes:0 Incoming:0 Removes:0 XFers:33 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 13:59:17 - SYSTEM SYSLOG STATS:Time:0.61 Deletes:0 Incoming:0 Removes:0 XFers:10 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 13:58:34 - SYSTEM SYSLOG STATS:Time:0.72 Deletes:0 Incoming:0 Removes:0 XFers:21 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 13:57:32 - SYSTEM SYSLOG STATS:Time:0.77 Deletes:0 Incoming:0 Removes:0 XFers:8 Alerts:18 Alarms:0 Reports:3
09/Mar/2018 13:56:50 - SYSTEM SYSLOG STATS:Time:0.69 Deletes:0 Incoming:0 Removes:0 XFers:11 Alerts:18 Alarms:0 Reports:3


You're only inserting 6 of 7 values, I'm surprised it's not failing.

This is mine rsyslog config. yours is missing "program". I'd expect the mysql query to fail but you say you're getting messages on your database. Any errors on clog?

Quote:
$template cacti_syslog,"INSERT INTO syslog_incoming(facility_id, priority_id, program, date, time, host, message) \
values (%syslogfacility%, %syslogpriority%, '%programname%', '%timereported:::date-mysql%', '%timereported:::date-mysql%', '%HOSTNAME%', TRIM('%msg%'))", SQL


If you look at Configuration/Plugin Management/Syslog, is it Syslog enabled? Not sure if the Syslog TAB looks at syslog table or both syslog and syslog_incoming.

Code:
Syslog Enabled
If this checkbox is set, records will be transferred from the Syslog Incoming table to the main syslog table and Alerts and Reports will be enabled. Please keep in mind that if the system is disabled log entries will still accumulate into the Syslog Incoming table as this is defined by the rsyslog or syslog-ng process.

_________________
Cacti - 1.1.37
Poller Type - Spine
Weathermap 0.98a
Server Info - Linux 3.10.0 - Centos 7
Web Server - Apache/2.4.6 PHP 5.4.16
MySQL - 5.5 ;RRDTool - 1.4.8 ;SNMP - 5.7.2
Religion - Anti forum pets
Code:
SYSTEM STATS: Time:17.7383 Method:spine Processes:1 Threads:10 Hosts:279 HostsPerProcess:279 DataSources:7309 RRDsProcessed:3718

Code:
WEATHERMAP STATS: Weathermap 0.98 run complete - Thu, 16 Nov 17 08:42:56 +1300: 39 maps were run in 23.22 seconds with 0 warnings.


Top
 Profile  
 
 Post subject: Re: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Fri Mar 09, 2018 11:50 am 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 1818
Basically, that option means that you will only see things in the syslog table if it's enabled. Without that, no reporting etc will be triggered but your incoming table will still grow even though you see nothing.

_________________
Official Cacti Developer

Cacti Resources:
Cacti Website (including releases)
Cacti Issues
Cacti Development Releases
Cacti Development Documentation

My resources:
How to submit Pull Requests
Development Wiki and How To's
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Tue Mar 27, 2018 5:00 am 
Offline

Joined: Mon Jun 13, 2011 11:20 pm
Posts: 34
Do not forget to set the crontab for syslog poller to run ..every interval time you decide to run .. This is the part I miss and took a lot of time to search which I don't recall anymore where I found it :D

Here is mine :

*/20 * * * * root /usr/bin/php /var/www/cactisyslog/plugins/syslog/syslog_process.php

Initially I run it every 5 minutes , but everytime it took longer to run and could take more than 10 minutes to finish .. Database could grow to 20 GB and more. So I decide to run it every 20 minutes and create removal rules to delete unnecesary log.

Hope it helps.


Top
 Profile  
 
 Post subject: Re: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Tue Mar 27, 2018 6:21 pm 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 1818
I'm not sure that having a separate cron file should be in place. The poller should kick off the syslog's pollers (which is what occurs in most circumstances).

_________________
Official Cacti Developer

Cacti Resources:
Cacti Website (including releases)
Cacti Issues
Cacti Development Releases
Cacti Development Documentation

My resources:
How to submit Pull Requests
Development Wiki and How To's
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Tue Mar 27, 2018 10:37 pm 
Offline

Joined: Mon Jun 13, 2011 11:20 pm
Posts: 34
I'm not really sure either.

In my case it works. I try to run cacti poller and clog not show syslog stats, only cacti poller stats. I don't run any graph poller , the unit just run for syslog :

Code:
2018/03/28 10:30:00 - SYSTEM STATS: Time:298.4407 Method:cmd.php Processes:1 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:0 RRDsProcessed:0
2018/03/28 10:30:00 - POLLER: Poller[1] Maximum runtime of 298 seconds exceeded. Exiting.


But syslog is updating . I can see it in syslog tabs with correct latest time.

With syslog process run as crontab , I got :

Code:
2018/03/28 10:20:19 - SYSTEM SYSLOG STATS:Time:1218.14 Deletes:0 Incoming:38053 Removes:57 XFers:0 Alerts:22 Alarms:131 Reports:0


So basically both cacti poller and syslog poller run in crontab works . Only issue with cacti poller , syslog stats not showed in clog.

Update :
I change poller to spine.
Now it works.

Code:
2018/03/28 11:15:01 - SYSTEM STATS: Time:0.1127 Method:spine Processes:1 Threads:1 Hosts:0 HostsPerProcess:0 DataSources:0 RRDsProcessed:0
2018/03/28 11:11:53 - SYSTEM SYSLOG STATS:Time:2512.59 Deletes:0 Incoming:9524 Removes:17 XFers:0 Alerts:22 Alarms:36 Reports:0
2018/03/28 11:11:50 - SYSTEM SYSLOG STATS:Time:2209.9 Deletes:0 Incoming:2178 Removes:2 XFers:0 Alerts:22 Alarms:5 Reports:0


Now I have to deal with syslog stats issue run not enough time to finish because syslog_process.php filled up the linux process little by little if default to run for 5 minutes.


Top
 Profile  
 
 Post subject: Re: Cacti 1.1.36 with syslog 2.1 and syslog-ng 3.5
PostPosted: Wed Apr 11, 2018 5:59 pm 
Offline
Cacti User
User avatar

Joined: Wed Jul 20, 2016 8:00 pm
Posts: 177
In the future, don't add separated crontab jobs for other pollers within cacti because Cacti invokes plugin's poller every time it runs.

Add to your rsyslog cacti config the following line to prevent linux messages from getting to syslog database.

:fromhost , isequal , "yourserverhostname" stop

_________________
Cacti - 1.1.37
Poller Type - Spine
Weathermap 0.98a
Server Info - Linux 3.10.0 - Centos 7
Web Server - Apache/2.4.6 PHP 5.4.16
MySQL - 5.5 ;RRDTool - 1.4.8 ;SNMP - 5.7.2
Religion - Anti forum pets
Code:
SYSTEM STATS: Time:17.7383 Method:spine Processes:1 Threads:10 Hosts:279 HostsPerProcess:279 DataSources:7309 RRDsProcessed:3718

Code:
WEATHERMAP STATS: Weathermap 0.98 run complete - Thu, 16 Nov 17 08:42:56 +1300: 39 maps were run in 23.22 seconds with 0 warnings.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group