High-Availability Cacti service

crevette · Joined: 08 Nov 2007 Posts: 11

Hello,

I need to design a HA cacti service. with a high SLA (99,999%) but with a limited amount of money (2 servers and a shared storage ~8000€)

I was thinking running cacti over a Red Hat Cluster Service (in Active-Passive mode with a shared storage); One server will gather the data on the shared storage, and the other server will be inactive; if the primary goes down, the secondary will act as the primary. Did someone tested this design ?
it is the best way to go ?

Do you have other tested designs ? The solution should have an automatic failover; and the failback should easiest as possible.

Thanks for sharing your experience.

gandalf · Posted: Fri Jan 04, 2008 3:37 pm Post subject:

We simply run two cacti installations in parallel. Each day, the primary MySQL table is dumped and restored to the secondary. And all rrd files are rsynced. That's enough for us, perhaps not for you ...
Reinhard

crevette · Joined: 08 Nov 2007 Posts: 11

Actually we already running 2 servers in parallels for the actual software we run.

I was thinking to do it again with Cacti; but I wanted also to try another architecture. Running 2 servers in parallels means you have 2 instances of Cacti to administer, and you need to have to think about how to handle failover and failback between two servers if something goes wrong.

Gandalft, did you already run into such problems with your installation ?

Having only instance sounded something harder to setup but easier to maintain afterward.

I still waiting some feedback from people running these kind of architecture.

Thanks

gandalf · Posted: Wed Jan 09, 2008 2:48 pm Post subject:

rony · Posted: Wed Jan 09, 2008 3:25 pm Post subject:

I suggest mysql replication from the master host the secondary (Disaster Recovery Host).

Do not replicate the following tables:

poller_output
poller_command
poller_time
poller_reindex
user_log

Run the poller on both machines. Make sure all admin changes are done on primary node.

khufure · Posted: Thu Jan 10, 2008 8:50 pm Post subject:

don't use shared storage. if you want the filesystem auto-syncd, use DRBD instead. (http://www.drbd.org).

cvcrud · Joined: 04 Oct 2005 Posts: 45 Location: Los Dallos, TX

i use shared storage. If my primary goes down, I make sure its down, then just run a simple little "cactiup" script that starts all nessecary services and mounts the nfs shares. The data base uses mysql replication, master/slave. I have tested a fail over a few times (havent had a real failure yet), and it works fine.

just curious rony, why not those tables... merely a i/o caution?

SoftDux · Posted: Tue Jul 22, 2008 9:11 am Post subject:

TheWitness · Posted: Thu Jul 24, 2008 11:36 pm Post subject:

The reason for this is that their contents changes very often and they are very transient (aka empty after each poll). This will drive network I/O that is not needed and stuff your binlog's full of crap.

I prefer Linux Clustering with a shared file system (not GFS cause it's too slow) and failover scripts for the Database, and Load Ballancing using the boost server for the Web Server. Oh yes, and two servers is the way to go for the big installations > 4k hosts and 40k graphs.

TheWitness

SoftDux · Posted: Fri Jul 25, 2008 4:53 am Post subject:

Cool, thanx for the explenation. I'm planning on clustering the MySQL DB over a slow ADSL link - i.e. with a server on either side of the switch. So I need something that's quick, and I think the network overhead of just the MySQL cluster will be less than a whole file system cluster?

TheWitness · Posted: Fri Jul 25, 2008 11:47 am Post subject:

Correct.

Howie · Posted: Fri Jul 25, 2008 11:59 am Post subject:

TheWitness · Posted: Fri Jul 25, 2008 12:08 pm Post subject:

Good point.

SoftDux · Posted: Thu Aug 07, 2008 11:51 am Post subject:

Sure, but that you can sync with rsync, right? And what about plugins? If I run srync to sync the whole /var/www/html/cacti folder, will it sync the plugins as well?