Cacti (home)ForumsDocumentation
Cacti: offical forums and support
It is currently Fri Mar 22, 2019 11:34 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: Spine performance expectations
PostPosted: Thu Feb 28, 2019 12:23 am 
Offline

Joined: Tue Jul 21, 2015 11:05 pm
Posts: 10
Hi folks,

I've read the manual online and that Spine can process like 20'000 data sources in under 60 seconds, but I'm having trouble getting it to process even 2000 within 60 seconds.

I have 2100 routers and switches that I'd like to poll with a 1-minute interval to graph traffic statistics of just one interface per device.
The most I've ever been able to get out of cacti is 800-ish devices/1600-ish data sources before it runs over the 60 second poll interval.

Have I misinterpreted what spine can do? Should it be able to poll all 2100 devices within 60 seconds or will that require multiple remote pollers?
I've tried threads, processes and reducing timeout values but I just can't get near the 20'000 data sources that has been quoted.

At the moment I'm running 10 processes and 30 threads for 519 hosts, 1280 data sources and it's taking between 30 to 50 seconds to get through them.
I've tried both the old 0.8 version of cacti with php5.5 and the new 1.2.1 version of cacti with php7.1.3 and Spine 1.2.1 via cygwin. Edit: both windows servers.

Is anyone successfully polling tens of thousands of data sources within 60 seconds? If so -- how did you do it?


Top
 Profile  
 
 Post subject: Re: Spine performance expectations
PostPosted: Thu Feb 28, 2019 12:07 pm 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 2285
It all comes down to the hardware you are utilising. When you start talking about a lot of devices, you need a physical, not virtual machine. You need fast disks and copious memory. You may also need to implement Boost (that batches up RRD file updates).

_________________
Official Cacti Developer

Cacti Resources:
Cacti Website (including releases)
Cacti Issues
Cacti Development Releases
Cacti Development Documentation

My resources:
How to submit Pull Requests
Development Wiki and How To's
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Spine performance expectations
PostPosted: Mon Mar 04, 2019 11:41 pm 
Offline

Joined: Tue Jul 21, 2015 11:05 pm
Posts: 10
Thanks netniV, I think I'll split it up over a few remote pollers.
I've been googling and searching through the cacti documentation but can't find much info on that feature.
Are you able to tell me if devices are added to the different pollers individually? Or do you add all of your devices to the main poller and then the load is split up automatically across the available pollers?

EG: If I have 3000 hosts and 5 remote pollers. Do I have to add a portion to poller1, poller2, poller3/etc. Or does the main poller go "okay, I'll split those 3000 hosts over 5 pollers so poller1 you get <x>, poller2 you get <y>/etc"

I feel like the answer to that question is probably the former - each remote poller is responsible for whatever devices you add to that individual poller.
If that's true... are you able to tell me if each device's data ends up in the same database table? Or does each remote poller maintain its own database records?

At the moment I pull some statistics straight out of the database for use in other applications but if I'm wondering if I'll lose that ability if I move to a remote poller setup?
I'd have to query all 5 remote poller database and collate the data for use in the other application?


Top
 Profile  
 
 Post subject: Re: Spine performance expectations
PostPosted: Tue Mar 05, 2019 7:07 am 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 2285
You assign devices to individual pollers yourself. We don't currently have an automatic option as the poller could sit anywhere in the network.

_________________
Official Cacti Developer

Cacti Resources:
Cacti Website (including releases)
Cacti Issues
Cacti Development Releases
Cacti Development Documentation

My resources:
How to submit Pull Requests
Development Wiki and How To's
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
 Post subject: Re: Spine performance expectations
PostPosted: Mon Mar 11, 2019 7:57 pm 
Offline

Joined: Tue Jul 21, 2015 11:05 pm
Posts: 10
Thanks netniV.
I ended up installing cacti on a few servers (not in remote poller mode) and wrote my own script to dish out all my hosts across the available servers.
Seems to be working well so far.


Top
 Profile  
 
 Post subject: Re: Spine performance expectations
PostPosted: Mon Mar 11, 2019 8:07 pm 
Offline
Cacti Guru User

Joined: Sun Aug 27, 2017 12:05 am
Posts: 2285
If you neaten it up, we could always have it posted here for others to use

_________________
Official Cacti Developer

Cacti Resources:
Cacti Website (including releases)
Cacti Issues
Cacti Development Releases
Cacti Development Documentation

My resources:
How to submit Pull Requests
Development Wiki and How To's
Updated NetSNMP Memory template for Cacti 1.x
Cisco SFP template for Cacti 0.8.8


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 15 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group