Syslog Plugin 2.1 backup(traditional) / restore(partitioned)

General discussion about Plugins for Cacti

Moderators: Moderators, Developers

Post Reply
Author
Message
chrisgsmith40
Posts: 7
Joined: Fri Jun 01, 2018 11:02 pm

Syslog Plugin 2.1 backup(traditional) / restore(partitioned)

#1 Post by chrisgsmith40 » Wed Jun 13, 2018 3:46 pm

Thanks again netniV for your help on thread viewtopic.php?f=14&t=58812 - unfortunately, need more direction, specifically ...

[*] WORKING: Syslog Plugin 2.1 backup (traditional) / restore (traditional) [my install, can see both files "syslog.MYI" and "syslog.MYD" in /var/lib/mysql/syslog GROWING mysql redirect-in]
[*] WORKING: Syslog Plugin 2.1 backup (partitioned) / restore (partitioned) [via mariabackup @ https://mariadb.com/kb/en/library/full- ... db-backup/ ]
[*] LOST: Syslog Plugin 2.1 backup (traditional) / restore (partitioned) :(

In that same /var/lib/mysql/syslog I can see all the "waiting" partitioned files like ... syslog_removed#P#d20180421.ibd, syslog_removed#P#d20180420.ibd, syslog_removed#P#d20180419.ibd, etc...

And can see through text searching the plugin's PHP source, file functions.php has functions for managing "in process" partitions , etc...

Am guessing / assuming a massage of something like the following link is needed / required? @ http://www.dbastuff.net/2014/09/mysql-d ... tions.html - but grateful for "checking in with you / others" before going off into the weeds if there's an easier way?

Regards,
Christopher

netniV
Cacti Guru User
Posts: 2682
Joined: Sun Aug 27, 2017 12:05 am

Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio

#2 Post by netniV » Fri Jun 15, 2018 11:00 am

I'll be honest, I'm not sure as I'm not really up to speed on MySQL partitioning. I would suspect however, that in order to take a single partition and make it multiple there would be some work. I'm not sure you would be able to do that once data is populated as that would surely cause confusion between where the data should and is sitting. If the dump and reload works, it's probably a safer option anyway.

chrisgsmith40
Posts: 7
Joined: Fri Jun 01, 2018 11:02 pm

Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio

#3 Post by chrisgsmith40 » Fri Jun 15, 2018 5:37 pm

Thanks netniV - this is telling me it hasn’t been approached/doco’d yet - I did have a “waking dream”, of how it can be done, am going to try and re-use/reference what’s already in the plugins *.php - #NoobOnAMission

chrisgsmith40
Posts: 7
Joined: Fri Jun 01, 2018 11:02 pm

Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio

#4 Post by chrisgsmith40 » Mon Jul 02, 2018 4:06 pm

I think I got it - see attached file! Works for my circumstances, - USE AT YOUR OWN RISK - obviously, needs a serious code review!

Regards
syslog-migration.zip
One-way migration from 'traditional' to 'partitioned' SYSLOG
(4.03 KiB) Downloaded 24 times

chrisgsmith40
Posts: 7
Joined: Fri Jun 01, 2018 11:02 pm

Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio

#5 Post by chrisgsmith40 » Tue Aug 07, 2018 3:29 am

For those who care, small "post script"

BEFORE running my noob script, I would suggest TRUNCATE of the following tables from the database you're COPYING FROM ...

TRUNCATE TABLE syslog_facilities;
TRUNCATE TABLE syslog_host_facilities;
TRUNCATE TABLE syslog_hosts;
TRUNCATE TABLE syslog_programs;
TRUNCATE TABLE syslog_priorities;
TRUNCATE TABLE syslog_remove;
TRUNCATE TABLE syslog_removed;

After that, the assumption is that database you're COPYING TO has been set up, BUT the Cacti SYSLOG Plug-IN has not been enabled yet

Run my noob script - even though the script finishes within about 30min, my MariaDB-server was still "processing the partitions" for another 1hr - I just watched "top" until CPUs were idle.

Then, while in the Cacti SYSLOG Plugin directory ...

php ./syslog-process.php

(This is to primarily to process any large number of waiting rows in syslog_incoming)

Obviously, if you have your syslog / rsyslog daemon set to dump into syslog_incoming, you won't get this "completely" emptied

Then, go ahead and enable the Plug-In - and you're off

Post Reply