Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Thu Jan 24, 2019 12:07 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: Syslog Plugin 2.1 backup(traditional) / restore(partitioned)
PostPosted: Wed Jun 13, 2018 3:46 pm 
Offline

Joined: Fri Jun 01, 2018 11:02 pm
Posts: 7
Thanks again netniV for your help on thread https://forums.cacti.net/viewtopic.php?f=14&t=58812 - unfortunately, need more direction, specifically ...

[*] WORKING: Syslog Plugin 2.1 backup (traditional) / restore (traditional) [my install, can see both files "syslog.MYI" and "syslog.MYD" in /var/lib/mysql/syslog GROWING mysql redirect-in]
[*] WORKING: Syslog Plugin 2.1 backup (partitioned) / restore (partitioned) [via mariabackup @ https://mariadb.com/kb/en/library/full-backup-and-restore-with-mariadb-backup/ ]
[*] LOST: Syslog Plugin 2.1 backup (traditional) / restore (partitioned) :(

In that same /var/lib/mysql/syslog I can see all the "waiting" partitioned files like ... syslog_removed#P#d20180421.ibd, syslog_removed#P#d20180420.ibd, syslog_removed#P#d20180419.ibd, etc...

And can see through text searching the plugin's PHP source, file functions.php has functions for managing "in process" partitions , etc...

Am guessing / assuming a massage of something like the following link is needed / required? @ http://www.dbastuff.net/2014/09/mysql-dump-and-partitions.html - but grateful for "checking in with you / others" before going off into the weeds if there's an easier way?

Regards,
Christopher


Top
 Profile  
 
 Post subject: Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio
PostPosted: Fri Jun 15, 2018 11:00 am 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 1946
I'll be honest, I'm not sure as I'm not really up to speed on MySQL partitioning. I would suspect however, that in order to take a single partition and make it multiple there would be some work. I'm not sure you would be able to do that once data is populated as that would surely cause confusion between where the data should and is sitting. If the dump and reload works, it's probably a safer option anyway.

_________________
Official Cacti Developer

Cacti Resources:
Cacti Website (including releases)
Cacti Issues
Cacti Development Releases
Cacti Development Documentation

My resources:
How to submit Pull Requests
Development Wiki and How To's
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio
PostPosted: Fri Jun 15, 2018 5:37 pm 
Offline

Joined: Fri Jun 01, 2018 11:02 pm
Posts: 7
Thanks netniV - this is telling me it hasn’t been approached/doco’d yet - I did have a “waking dream”, of how it can be done, am going to try and re-use/reference what’s already in the plugins *.php - #NoobOnAMission


Top
 Profile  
 
 Post subject: Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio
PostPosted: Mon Jul 02, 2018 4:06 pm 
Offline

Joined: Fri Jun 01, 2018 11:02 pm
Posts: 7
I think I got it - see attached file! Works for my circumstances, - USE AT YOUR OWN RISK - obviously, needs a serious code review!

Regards
Attachment:
File comment: One-way migration from 'traditional' to 'partitioned' SYSLOG
syslog-migration.zip [4.03 KiB]
Downloaded 12 times


Top
 Profile  
 
 Post subject: Re: Syslog Plugin 2.1 backup(traditional) / restore(partitio
PostPosted: Tue Aug 07, 2018 3:29 am 
Offline

Joined: Fri Jun 01, 2018 11:02 pm
Posts: 7
For those who care, small "post script"

BEFORE running my noob script, I would suggest TRUNCATE of the following tables from the database you're COPYING FROM ...

TRUNCATE TABLE syslog_facilities;
TRUNCATE TABLE syslog_host_facilities;
TRUNCATE TABLE syslog_hosts;
TRUNCATE TABLE syslog_programs;
TRUNCATE TABLE syslog_priorities;
TRUNCATE TABLE syslog_remove;
TRUNCATE TABLE syslog_removed;

After that, the assumption is that database you're COPYING TO has been set up, BUT the Cacti SYSLOG Plug-IN has not been enabled yet

Run my noob script - even though the script finishes within about 30min, my MariaDB-server was still "processing the partitions" for another 1hr - I just watched "top" until CPUs were idle.

Then, while in the Cacti SYSLOG Plugin directory ...

php ./syslog-process.php

(This is to primarily to process any large number of waiting rows in syslog_incoming)

Obviously, if you have your syslog / rsyslog daemon set to dump into syslog_incoming, you won't get this "completely" emptied

Then, go ahead and enable the Plug-In - and you're off


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: Google [Bot] and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group