Syslog 1.0 Released

Announcements concerning Plugins for Cacti

Moderators: Moderators, Developers

Post Reply
Author
Message
philipz
Cacti User
Posts: 100
Joined: Tue Sep 22, 2009 7:46 am

Re: Syslog 1.0 Released

#136 Post by philipz » Tue Oct 26, 2010 1:51 am

TheWitness wrote:[

Post your feature request and bug on Mantis http://bugs.cacti.net

TheWitness
Done; 1 bug and 1 feature request posted.

Tx

fata51
Posts: 17
Joined: Wed Nov 16, 2005 8:34 am

Re: Syslog 1.0 Released

#137 Post by fata51 » Wed Nov 10, 2010 9:45 am

Hello,

I'm trying to use Syslog (version 1.05) with a remote syslog-ng/database server.

I copied the "syslog" directory into the plugins cacti directory

First, I tried to modify the syslog/config.php before install the plugin into the plugin management console as following :

/* revert if you dont use the Cacti database */
$use_cacti_db = false;

if (!$use_cacti_db) {
$syslogdb_type = 'mysql';
$syslogdb_default = 'syslog';
$syslogdb_hostname = '192.168.57.49';
$syslogdb_username = 'syslog';
$syslogdb_password = '****';
$syslogdb_port = 3306;

I tested this configuration by connecting with the command "mysql -h 192.168.57.49 -u syslog -p syslog" and it's OK.

In the plugin management console, I choose "install" and then the following settings : MyISAM storage, Traditional Table (because of my MySQL server version) and 1 Year. After clicking on the "install" button, the page is loading for a while and finally ending with a blank page. When, I go back to the cacti console, the left menu has been modified, the plugin appeared as "Not Installed" in the plugin management and there is no syslog tab.

Secondly, I tried to install the plugin for a cacti database use and then modify the config.php. In this case, the plugin has been correctly installed and enabled. I changed the config.php as posted above but didn't manage to have anything displayed in the syslog tab. I checked the syslog_incoming table, it's correctly populated from syslog-ng daemon. I manually ran the command "php -q syslog_process.php --debug" and get the following :

cacti:/var/www/cacti/plugins/syslog# sudo -u syslog php -q syslog_process.php --debug
SYSLOG: Syslog Table is NOT Partitioned
SYSLOG: Deleted 0, Syslog Message(s) (older than 2009-11-10 days)
SYSLOG: Unique ID = 77
SYSLOG: Found 156749, New Message(s) to process
SYSLOG: Found 0, Removal Rule(s) to process
SYSLOG: Found 0, Alert Rules to process
SYSLOG: Moved , Message(s) to the 'syslog' table
SYSLOG: Deleted 156749, Already Processed Message(s) from incoming
SYSLOG: Updated , Hosts in the syslog hosts table
SYSLOG: Processing Reports...
SYSLOG: We have 0 Reports in the database
SYSLOG: Finished processing Reports...
11/10/2010 03:42:19 PM - SYSTEM SYSLOG STATS:Time:11.05 Deletes:0 Incoming:156749 Removes:0 XFers:0 Alerts:0 Alarms:0 Reports:0

No messages are moved to the syslog table but I don't understand why...

I also have some logs in the cacti.log file :

11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) (SELECT DISTINCT host FROM `syslog`.`syslog_incoming`) ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog` (logtime, priority_id, facility_id, host_id, message) SELECT TIMESTAMP(`date`, `time`), priority_id, facility_id, host_id, message FROM (SELECT date, time, priority_id, facility_id, host_id, message FROM syslog_incoming AS si INNER JOIN syslog_facilities AS sf ON sf.facility=si.facility INNER JOIN syslog_priorities AS sp ON sp.priority=si.priority INNER JOIN syslog_hosts AS sh ON sh.host=si.host WHERE status=3) AS merge'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_host_facilities` (host_id, facility_id) SELECT host_id, facility_id FROM ((SELECT DISTINCT host, facility FROM `syslog`.`syslog_incoming`) AS s INNER JOIN `syslog`.`syslog_hosts` AS sh ON s.host=sh.host INNER JOIN `syslog`.`syslog_facilities` AS sf ON sf.facility=s.facility) ON DUPLICATE KEY UPDATE host_id=VALUES(host_id)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) SELECT DISTINCT host FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_priorities` (priority) SELECT DISTINCT priority FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE priority=VALUES(priority)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_facilities` (facility) SELECT DISTINCT facility FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE facility=VALUES(facility)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"DELETE FROM `syslog`.`syslog_removed` WHERE logtime < '2009-11-10''
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1054', SQL:"DELETE FROM `syslog`.`syslog` WHERE logtime < '2009-11-10''

Any tips would be very appreciated ;-)

Thanks in advance,

Vincent

Palermo
Cacti User
Posts: 87
Joined: Thu Sep 11, 2008 11:48 pm

Re: Syslog 1.0 Released

#138 Post by Palermo » Tue Nov 23, 2010 4:35 am

Hi,
I have an question.
I`f followed the manual according:
http://docs.cacti.net/plugin:syslog.config

all the install procedure went well. But i don`t think its working. I don`t see syslog rolling in.
i think its more a rsyslog problem. be course the database is not filing.

What i did.
yum install rsyslog rsyslog-mysql

then updatetd the config file: "/etc/rsyslog.conf"
$ModLoad ommysql
$template cacti_syslog,"INSERT INTO syslog_incoming(facility, priority, date, time, host, message) values (%syslogfacility%, %syslogpriority%, '%timereported:::date-mysql%', '%timereported:::date-mysql%', '%HOSTNAME%', '%msg%')", SQL
*.* >localhost,syslog,cactiuser,password;cacti_syslog
then restarted rsyslogd.

Code: Select all

service rsyslogd restart
i have an error :

Code: Select all

rsyslogd: unrecognized service
So i used commando:

Code: Select all

service rsyslog restart
Shutting down system logger:                               [  OK  ]
Starting system logger:                                         [  OK  ]
Updated the IP tables:

Code: Select all

-A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 514 -j ACCEPT
restarted the iptables.

No updates in the database tables.

Anyone maybe an idea?

Thx in advanced.

fata51
Posts: 17
Joined: Wed Nov 16, 2005 8:34 am

Re: Syslog 1.0 Released

#139 Post by fata51 » Wed Nov 24, 2010 4:56 am

fata51 wrote:Hello,

I'm trying to use Syslog (version 1.05) with a remote syslog-ng/database server.

I copied the "syslog" directory into the plugins cacti directory

First, I tried to modify the syslog/config.php before install the plugin into the plugin management console as following :

/* revert if you dont use the Cacti database */
$use_cacti_db = false;

if (!$use_cacti_db) {
$syslogdb_type = 'mysql';
$syslogdb_default = 'syslog';
$syslogdb_hostname = '192.168.57.49';
$syslogdb_username = 'syslog';
$syslogdb_password = '****';
$syslogdb_port = 3306;

I tested this configuration by connecting with the command "mysql -h 192.168.57.49 -u syslog -p syslog" and it's OK.

In the plugin management console, I choose "install" and then the following settings : MyISAM storage, Traditional Table (because of my MySQL server version) and 1 Year. After clicking on the "install" button, the page is loading for a while and finally ending with a blank page. When, I go back to the cacti console, the left menu has been modified, the plugin appeared as "Not Installed" in the plugin management and there is no syslog tab.

Secondly, I tried to install the plugin for a cacti database use and then modify the config.php. In this case, the plugin has been correctly installed and enabled. I changed the config.php as posted above but didn't manage to have anything displayed in the syslog tab. I checked the syslog_incoming table, it's correctly populated from syslog-ng daemon. I manually ran the command "php -q syslog_process.php --debug" and get the following :

cacti:/var/www/cacti/plugins/syslog# sudo -u syslog php -q syslog_process.php --debug
SYSLOG: Syslog Table is NOT Partitioned
SYSLOG: Deleted 0, Syslog Message(s) (older than 2009-11-10 days)
SYSLOG: Unique ID = 77
SYSLOG: Found 156749, New Message(s) to process
SYSLOG: Found 0, Removal Rule(s) to process
SYSLOG: Found 0, Alert Rules to process
SYSLOG: Moved , Message(s) to the 'syslog' table
SYSLOG: Deleted 156749, Already Processed Message(s) from incoming
SYSLOG: Updated , Hosts in the syslog hosts table
SYSLOG: Processing Reports...
SYSLOG: We have 0 Reports in the database
SYSLOG: Finished processing Reports...
11/10/2010 03:42:19 PM - SYSTEM SYSLOG STATS:Time:11.05 Deletes:0 Incoming:156749 Removes:0 XFers:0 Alerts:0 Alarms:0 Reports:0

No messages are moved to the syslog table but I don't understand why...

I also have some logs in the cacti.log file :

11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) (SELECT DISTINCT host FROM `syslog`.`syslog_incoming`) ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog` (logtime, priority_id, facility_id, host_id, message) SELECT TIMESTAMP(`date`, `time`), priority_id, facility_id, host_id, message FROM (SELECT date, time, priority_id, facility_id, host_id, message FROM syslog_incoming AS si INNER JOIN syslog_facilities AS sf ON sf.facility=si.facility INNER JOIN syslog_priorities AS sp ON sp.priority=si.priority INNER JOIN syslog_hosts AS sh ON sh.host=si.host WHERE status=3) AS merge'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_host_facilities` (host_id, facility_id) SELECT host_id, facility_id FROM ((SELECT DISTINCT host, facility FROM `syslog`.`syslog_incoming`) AS s INNER JOIN `syslog`.`syslog_hosts` AS sh ON s.host=sh.host INNER JOIN `syslog`.`syslog_facilities` AS sf ON sf.facility=s.facility) ON DUPLICATE KEY UPDATE host_id=VALUES(host_id)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) SELECT DISTINCT host FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_priorities` (priority) SELECT DISTINCT priority FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE priority=VALUES(priority)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_facilities` (facility) SELECT DISTINCT facility FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE facility=VALUES(facility)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"DELETE FROM `syslog`.`syslog_removed` WHERE logtime < '2009-11-10''
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1054', SQL:"DELETE FROM `syslog`.`syslog` WHERE logtime < '2009-11-10''

Any tips would be very appreciated ;-)

Thanks in advance,

Vincent
Hello,

I'm kind of stuck with this thing, any clue?

Thanks,

Vincent

trungtano
Cacti User
Posts: 90
Joined: Fri Apr 10, 2009 1:57 am
Contact:

syslog 1.04 Level error

#140 Post by trungtano » Mon Dec 13, 2010 5:46 am

Hi,

I have just upgrade my Syslog to 1.04 and Setting to 0.7 running on cacti 0.8.7g but my plugin Syslog seem not running some function:

- It does not display the colour as level define, in tab Alert Log it just have colour of Notice Level other Level have the same colour :o



Do you have any Idea?

Thanks for help
Attachments
Untitled.jpg
Untitled.jpg (471.2 KiB) Viewed 11667 times

armstrongzhao
Posts: 28
Joined: Mon Feb 02, 2009 10:21 am
Location: CHINA

Re: Syslog 1.0 Released

#141 Post by armstrongzhao » Wed Jan 26, 2011 10:17 pm

Does Mysql 5.5 support syslog 1.05?

hpsmartyz
Posts: 34
Joined: Tue Oct 30, 2007 5:23 am

Re: Syslog 1.0 Released

#142 Post by hpsmartyz » Sat Feb 19, 2011 9:07 am

Hello

thanks for this new version!
I have an issue related to the warning priority level.
I have logs with such priority level, but none of them has the color associated with it. Further, when I filter the logs using either Debug, Info++, Notice++, Warning++ the warning logs do not show up in the list.
I am using syslog-ng v2.0.10-2

Thanks for your help.
If needed I can dig in the code if you tell me were to look at

armstrongzhao
Posts: 28
Joined: Mon Feb 02, 2009 10:21 am
Location: CHINA

Re: Syslog 1.0 Released

#143 Post by armstrongzhao » Mon Feb 21, 2011 9:48 am

My OS enviroment is centos 5.5 (64bit) cacti 0.8.7 mysql 5.5 syslog 1.05 rsyslog 3.22
The latest syslog version is cool with mysql 5.5.

The table partition of mysql 5.5 is much quicker than 5.1.

The amount of the syslog file is 19G. Hardware CPU model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz, Memory 2G. The server works well.

I disabled rsyslog host name resolution, rsyslog -x. Some of hosts can display IP in the GUI, but others still display host names. How to deal with it?

User avatar
TheWitness
Developer
Posts: 14817
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Syslog 1.0 Released

#144 Post by TheWitness » Sat Sep 17, 2011 8:50 am

Syslog 1.21 will be released approximately the same time as Cacti 0.8.7h.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

JohnYYC
Cacti User
Posts: 58
Joined: Thu Apr 22, 2010 1:23 pm

Re: Syslog 1.0 Released

#145 Post by JohnYYC » Wed Dec 21, 2011 2:07 pm

I am currently using Syslog version 1.22 on Cacti 0.8.7g. I was wondering if it was possible to disable the "Statistics" I have no need for them and it seems to fill up my database at a pretty steady rate. I looked under Settings but couldn't find anything.

If it is not currently possible could I make a feature request for the option to enable/disable Statistics?

User avatar
TheWitness
Developer
Posts: 14817
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Syslog 1.0 Released

#146 Post by TheWitness » Wed Dec 21, 2011 5:11 pm

JohnYYC wrote:I am currently using Syslog version 1.22 on Cacti 0.8.7g. I was wondering if it was possible to disable the "Statistics" I have no need for them and it seems to fill up my database at a pretty steady rate. I looked under Settings but couldn't find anything.

If it is not currently possible could I make a feature request for the option to enable/disable Statistics?
Yea, it's not a bad idea, the performance is horrible right now anyway.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of MacTrack, Boost, CLog, SpikeKill, Platform RTM, DSStats, maintainer of Spine, lot's of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Central Plugin Repository
Central Templates Repository


I'm still out there people. Getting excited for Cacti 1.2. I think it will be a great release.

mastrboy
Posts: 26
Joined: Wed Aug 02, 2006 2:40 am

Re: Syslog 1.0 Released

#147 Post by mastrboy » Tue Feb 14, 2012 7:42 am

Since I'm unable to create a account on the dokuwiki at http://docs.cacti.net/ I'll just post this stuff here:

Rsyslog tips,

Tip1, howto only log messages from a subnet/external host into the database:

Code: Select all

if $fromhost-ip startswith '192.168.0.' then :ommysql:localhost,cacti,DB_USER,DB_PASSWORD;cacti_syslog
& ~
The '& ~' discards the message after putting it into SQL, meaning your normal /var/log files won't get flooded.

Tip2, Strip timestamps from cisco syslog messages before inserting them, messages that don't match the rule will get inserted as is.

Code: Select all

# Cacti template with cisco regexp stripping
$Template cacti_syslog,"INSERT INTO syslog_incoming(facility, priority, date, time, host, message) VALUES ('%syslogfacility-text%', '%syslogpriority-text%',  '%timereported:::date-mysql%', '%timereported:::date-mysql%', '%HOSTNAME%', '%msg:R,ERE,2,FIELD:(.*[0-2][0-9]:[0-5][0-9]:[0-5][0-9]: )(.*)--end%')", SQL
Tip3, async logging for very busy systems with file backlog in case the database goes down.

Code: Select all

# Async logging
$WorkDirectory /var/log/rsyslog
$ActionQueueType LinkedList   # use asynchronous processing
$ActionQueueFileName srvrfwd  # set file name, also enables disk mode
$ActionResumeRetryCount -1    # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
Syslog plugin tip about colors, we've been having problems with the colors randomly appear in syslog, commenting out some lines i setup.php solves this:
Line 1336, file setup.php:

Code: Select all

// changing colours should not need a re-initialize of the session, and sometimes $_SESSION["syslog_* is set, but empty...
//      if (!isset($_SESSION["syslog_colors"])) {
                foreach($syslog_levels as $level) {
                        $syslog_colors[$level] = db_fetch_cell("SELECT hex FROM colors WHERE id=" . read_config_option("syslog_" . $level . "_bg"));
                        $syslog_text_colors[$level] = db_fetch_cell("SELECT hex FROM colors WHERE id=" . read_config_option("syslog_" . $level . "_fg"));
                }

                $_SESSION["syslog_colors"] = $syslog_colors;
                $_SESSION["syslog_text_colors"] = $syslog_text_colors;
//      }else{
//              $syslog_colors = $_SESSION["syslog_colors"];
//              $syslog_text_colors = $_SESSION["syslog_text_colors"];
//      }

User avatar
ltitusl
Posts: 18
Joined: Wed Sep 19, 2012 9:00 am

Re: Syslog 1.0 Released

#148 Post by ltitusl » Fri Jan 04, 2013 1:39 pm

Thanks for all the great info, super helpful!
information wants to be free

gurulee
Posts: 14
Joined: Fri Nov 08, 2013 1:23 pm
Location: CT

Re: Syslog 1.0 Released

#149 Post by gurulee » Mon Nov 18, 2013 11:56 am

I am not seeing any remote hosts in my syslog, only localhost messages like the one below:

Code: Select all

(root) CMD (php /var/www/html/poller.php > /dev/null 2>&1)
Can someone help me?
_________________
Lee
"knowledge breeds confidence, confidence breeds success"

gurulee
Posts: 14
Joined: Fri Nov 08, 2013 1:23 pm
Location: CT

Re: Syslog 1.0 Released

#150 Post by gurulee » Thu Jan 02, 2014 8:51 am

mastrboy, Thank you for the tips!

But I am seeing hosts in my Syslog screen and verified that data is populating in the mysql syslog tables, BUT there's no messages showing in the Cacti Syslog GUI...

Any ideas, anyone?
_________________
Lee
"knowledge breeds confidence, confidence breeds success"

Post Reply