New Cacti Architecture (0.8.8) - RFC Resonse Location

[rony]

I know from past discussions that this is an issue we have identified, but I'm glad someone brought it up in this thread.

[psyber]

I can see that you put some thought into this. I'm attempting to as well.
I work in what is essentially a NOC environment and for us alerting and prediction is everything
Couple of questions/comments/observations about the proposed architecture:

Alerting: Where would be the best pace to tap into to do alerting in your prescribed layout?? I would imagine a tap into the rrd data pipeline could provide threshold and aberrant behavior triggers.

Do you have any specific existing protocols in mind for the rrd update service? XML/RPC SOAP ....

I'm running the scenario of patch day in my head where each machine is taken out of the equation one at a time and everything looks ok until we get to the RRD update servers. I'm curious, does the db cache the polled data until the update server comes online then serilize the data to the RRD server when its back up to fill out the rrd? How could alerting work in this scenario???? Maybe tee some of the data in a sort of raid scenario where you could lose one rrd server and still have updates and alerting continue. you would need a sync mechanism to bring the short rrds up to par and a redistribution mechanism to enable adding another server to the cluster. On the httpd side you would need a way to determine which of the two rrds to display based on last update.

[bbice]

I'm glad this topic is coming up for discussion. I definitely like the idea of having multiple pollers because we would like to be able to have multiple pollers in multiple geographic locations to keep as much of the polling traffic "local" as possible (less snmp traffic going over WAN/VPN connections and/or crossing high-latency paths).

One thing we'll want to be careful about, particularly with as many separate DB instances as I see in the diagram is that SQL database performance (at least in my experience) suffers horribly over high-latency links. This is especially true for apps doing lots of small DB queries instead of few big queries. (the last time I tried testing this it didn't matter what flavor of DB server I used either - accessing a server over a WAN circuit with a 30 ms latency dropped the number of transactions per second I could do from tens of thousands to hundreds)

I'd initially thought the way to spread my polling load out over multiple areas would be to have separate stand-alone cacti instances (say, one in each major office monitoring nearby hosts), but with hooks in the code (maybe a new plugin?) to be able to add references to graphs, hosts, and trees from remote cacti server.

But provided having poller groups separated from the rest of the architecture by medium to high latency links doesn't cause problems, this new design looks even better.

[andrew2]

Wow, I'm late to the party on this one.

[ldrolez]

Hi !

Here's my small patch for Cacti 0.8.7b that implements a master/slaves arch:

http://forums.cacti.net/viewtopic.php?p=127122

It modifies only poller.php. The slaves send data by inserting poller_output lines in the master database using TCP port 3306.

Cheers,

Ludo.

[driden91]

hello

I would like your opinion on this architecture.

for you it's possible to realise this architecture ?




for the master / slave poller i want use this patch : http://forums.cacti.net/viewtopic.php?p=127122

[killshoot]

Well... performance for RRD update through NFS are very poor (tried with netapp 3050)

@All

My spec : 2200 appliances, 50'000 ds and 40000 graphs
Polling time with spine ~130 second
dual quad core Xeon 3ghz, 4gb ram

A distributed architecture is a good thing BUT FIRST we need to optimize Cacti. In fact I use a dual quad core xeon and my problem is that one core is always at 100% and the other sleeps.

Spine is multithreaded, but other process (and plugin) are only monothreaded. That would say that mysql is always accessed by one php process thus mysql allow maximum one CPU per process thus 1 core do the job and other sleep.

Vince

[cigamit]

Such is the problem with PHP. PHP is supposedly thread safe, but most modules with it are not.


http://www.php.net/manual/en/install.unix.apache2.php

[gandalf]