InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

Addons for Cacti and discussion about those addons

Moderators: Moderators, Developers

Post Reply
Author
Message
User avatar
phalek
Cacti Guru User
Posts: 2710
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#106 Post by phalek » Thu Apr 06, 2017 5:05 am

geoffg, the plugin does some debug logging when increasing the cacti log settings. It should show what data it found. Did you check this ?

It should be something like

Code: Select all

DEBUG: Adding the following data to INFLUXB ....
DEBUG: Finished adding the data to INFLUXB ....
....
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
[/size]

mapickett77
Posts: 1
Joined: Tue Apr 18, 2017 1:30 pm

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#107 Post by mapickett77 » Tue Apr 18, 2017 2:23 pm

Hello,

I seem to be running into similar index_type errors on 0.66.
CereusTransporter version 0.66
Cacti version 1.1.3

2017-04-18 12:20:04 - ERROR PHP NOTICE in Plugin 'CereusTransporter': Undefined index: index_type in file: /var/www/html/cacti-1.1.3/plugins/CereusTransporter/setup.php on line: 179
2017-04-18 12:20:04 - CMDPHP PHP ERROR NOTICE Backtrace: (/poller.php: 717 api_plugin_hook)(/lib/plugins.php: 68 api_plugin_run_plugin_hook)(/lib/plugins.php: 161 CereusTransporter_poller_bottom)(/plugins/CereusTransporter/setup.php: 179 CactiErrorHandler)(/lib/functions.php: 4262 cacti_debug_backtrace)
2017-04-18 12:20:04 - ERROR PHP NOTICE in Plugin 'CereusTransporter': Undefined index: scheme in file: /var/www/html/cacti-1.1.3/plugins/CereusTransporter/vendor/influxdb/influxdb-php/src/InfluxDB/Client.php on line: 256
2017-04-18 12:20:04 - CMDPHP PHP ERROR NOTICE Backtrace: (/poller.php: 717 api_plugin_hook)(/lib/plugins.php: 68 api_plugin_run_plugin_hook)(/lib/plugins.php: 161 CereusTransporter_poller_bottom)(/plugins/CereusTransporter/setup.php: 254 CereusTransporter_send_data)(/plugins/CereusTransporter/setup.php: 83 CereusTransporter_influxdb_send_data)(/plugins/CereusTransporter/includes/influxdb.php: 27 fromDSN)(/plugins/CereusTransporter/vendor/influxdb/influxdb-php/src/InfluxDB/Client.php: 256 CactiErrorHandler)(/lib/functions.php: 4262 cacti_debug_backtrace)
2017-04-18 12:20:04 - CereusTransporter ERROR: is not a valid scheme

Any help would be appreciated.

Thanks!
McLean

Axel90
Cacti User
Posts: 125
Joined: Fri Feb 10, 2017 12:41 pm

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#108 Post by Axel90 » Thu Apr 20, 2017 7:02 am

Hey phalek,

I have a cacti server 0.8.8f and the CereusTransporter Plugin 0.64.

In my opinion, everything is working fine!

The data is exported to the InfluxDB.

my question now:

How can I setup single graphs in grafana: For example the cpu utilization of a specific host?

I read all the posts in this topic, you talked about a manual for that. Are you already done with that?

And is there already a custom dashboard you could share?

Thanks in advance!!

and btw. great work!

rbarreto
Posts: 5
Joined: Thu Jul 21, 2016 8:34 pm

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#109 Post by rbarreto » Tue May 30, 2017 9:49 am

Hi phalek

First of all, thanks for sharing!

I'm testing this plugin with Cacti / InfluxDB / Grafana.

The data is exported from cacti into influxdb with success and I got some graphs with grafana.

I'd like to know if is there a way to get the total bandwidth in GB/TB used per month from the data exported. From the measurement "interface.traffic" it just has the type: traffic_in and traffic_out.

Thank you

Regards

michael067
Posts: 6
Joined: Thu Jul 20, 2017 12:53 pm

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#110 Post by michael067 » Tue Jan 09, 2018 8:44 am

hi,

I have cacti 1.1.30 and cereustransport 0.66, i have this error :

2018/01/09 14:40:16 - CMDPHP PHP ERROR NOTICE Backtrace: (/poller.php: 722 api_plugin_hook)(/lib/plugins.php: 73 api_plugin_run_plugin_hook)(/lib/plugins.php: 170 CereusTransporter_poller_bottom)(/plugins/CereusTransporter/setup.php: 179 CactiErrorHandler)(/lib/functions.php: 4434 cacti_debug_backtrace)
2018/01/09 14:40:16 - ERROR PHP NOTICE in Plugin 'CereusTransporter': Undefined index: index_type in file: /opt/cacti/plugins/CereusTransporter/setup.php on line: 179

thanks for your help.

Michael

User avatar
phalek
Cacti Guru User
Posts: 2710
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#111 Post by phalek » Tue Jan 09, 2018 9:00 am

Hi

Try the attached setup.php file
Attachments
setup_CereusTransporter_Update.zip
(5.54 KiB) Downloaded 131 times
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
[/size]

michael067
Posts: 6
Joined: Thu Jul 20, 2017 12:53 pm

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#112 Post by michael067 » Tue Jan 09, 2018 9:52 am

Perfect :-)

Thanks for your fast response.

Michael

nguyentan1980
Posts: 1
Joined: Tue Oct 27, 2015 1:22 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#113 Post by nguyentan1980 » Tue Jan 23, 2018 2:37 am

Hi Phalek,

Thanks for sharing.

I got this error message when I install CereusTransport.
2018/01/23 14:36:07 - CereusTransporter INFO: Adding data to influxdb
2018/01/23 14:36:07 - CereusTransporter ERROR: Writing has failed, exception: [curl] 7: Failed connect to 172.17.12.45:8086; Connection refused [url] http://172.17.12.45:8086/write?db=cacti&precision=s
2018/01/23 14:36:07 - CereusTransporter ERROR: Writing has failed, exception: [curl] 7: Failed connect to 172.17.12.45:8086; Connection refused [url] http://172.17.12.45:8086/write?db=cacti&precision=s
2018/01/23 14:36:07 - CereusTransporter STATS: Time:0.15 LookupTableSize:32 LookupUnitsTableSize:29 MetricsCount:45 AverageHostTime:0.05 SlowestHostnames:[127.0.0.1: 0.136s, 172.17.12.1: 0.000s, : 0.000s]

Please help me .

User avatar
phalek
Cacti Guru User
Posts: 2710
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#114 Post by phalek » Tue Jan 23, 2018 3:46 am

Doesn't it state in the description for the url to use the following ?

Code: Select all

Target URL/System
Full Database URL. Example: influxdb://user:[email protected]:8086/dbname for InfluxDB, https://user:[email protected]:8070/ for Bosun
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
[/size]

User avatar
Joon
Posts: 1
Joined: Mon Jan 22, 2018 8:21 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#115 Post by Joon » Tue Jan 23, 2018 9:25 am

Hello Phalek,

Thanks for this great plugin.

Sometimes I have wrong networks metrics displayed on Grafana, different than what Cacti shows on it's own graphs.

Cereus v0.64 is running on Cacti 0.8.8a, with spine poller running every minute on nearly 400 devices.
The data are exported to Influxdb and viewed with Grafana.

Is it because of the polling frequency being too high ?

Examples :
Attachments
Comparison-monthly-B.PNG
Monthly comparison, device B, peak
Comparison-monthly-B.PNG (115.75 KiB) Viewed 1536 times
Comparison-weekly.PNG
Weekly comparison, device A
Comparison-weekly.PNG (139.87 KiB) Viewed 1536 times
Comparison-monthly.PNG
Monthly comparison, device A
Comparison-monthly.PNG (119.47 KiB) Viewed 1536 times

chrispin2
Posts: 5
Joined: Wed Mar 21, 2018 9:52 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#116 Post by chrispin2 » Wed Mar 21, 2018 9:55 am

Hi

I have tried the latest setup attached here. It seems to be running however, i dont see any data in my InfluxDB

2018/03/22 11:15:03 - CereusTransporter STATS: Time:1.01 LookupTableSize:4548 LookupUnitsTableSize:4478 MetricsCount:0 AverageHostTime:0.00 SlowestHostnames:[: 0.000s]
2018/03/22 11:15:02 - CereusTransporter INFO: Adding data to influxdb
2018/03/22 11:15:01 - SYSTEM STATS: Time:298.8106 Method:spine Processes:6 Threads:6 Hosts:0 HostsPerProcess:0 DataSources:0 RRDsProcessed:0
2018/03/22 11:15:01 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.

chrispin2
Posts: 5
Joined: Wed Mar 21, 2018 9:52 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#117 Post by chrispin2 » Thu Mar 22, 2018 12:30 pm

Hi,

I have a main poller and a report poller, both file structures are the same, however from the looks of it, the remote poller seems to run the transporter plugin less frequently than the main poller. Any idea how do i make the remote poller update every 5 mins.

Code: Select all

2018/03/23 00:50:03 - CereusTransporter STATS: Time:1.00 LookupTableSize:4548 LookupUnitsTableSize:4478 MetricsCount:0 AverageHostTime:0.00 SlowestHostnames:[: 0.000s]
2018/03/23 00:50:02 - CereusTransporter INFO: Adding data to influxdb
2018/03/23 00:50:01 - SYSTEM STATS: Time:298.9260 Method:spine Processes:7 Threads:14 Hosts:0 HostsPerProcess:0 DataSources:0 RRDsProcessed:0
2018/03/23 00:50:01 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.
2018/03/23 00:45:03 - CereusTransporter STATS: Time:1.55 LookupTableSize:4548 LookupUnitsTableSize:4478 MetricsCount:0 AverageHostTime:0.00 SlowestHostnames:[: 0.000s]
2018/03/23 00:45:01 - CereusTransporter INFO: Adding data to influxdb
2018/03/23 00:45:01 - SYSTEM STATS: Time:298.7341 Method:spine Processes:7 Threads:14 Hosts:0 HostsPerProcess:0 DataSources:0 RRDsProcessed:0
2018/03/23 00:45:01 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.
2018/03/23 00:40:33 - CereusTransporter STATS: Time:32.78 LookupTableSize:4548 LookupUnitsTableSize:4478 MetricsCount:7597 AverageHostTime:0.11 SlowestHostnames:[xx: 0.373s, xx: 0.355s, xx: 0.278s, xx: 0.210s]
2018/03/23 00:40:00 - CereusTransporter INFO: Adding data to influxdb
2018/03/23 00:40:00 - SYSTEM STATS: Time:298.7034 Method:spine Processes:7 Threads:14 Hosts:0 HostsPerProcess:0 DataSources:0 RRDsProcessed:0
2018/03/23 00:40:00 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.
2018/03/23 00:35:02 - CereusTransporter STATS: Time:1.00 LookupTableSize:4548 LookupUnitsTableSize:4478 MetricsCount:0 AverageHostTime:0.00 SlowestHostnames:[: 0.000s]
2018/03/23 00:35:01 - CereusTransporter INFO: Adding data to influxdb 

netniV
Cacti Guru User
Posts: 2670
Joined: Sun Aug 27, 2017 12:05 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#118 Post by netniV » Thu Mar 22, 2018 3:58 pm

Check that you have the same Data Collection profiles and the same cron tabs on both machines. If they are the same, see if the transport has settings that define it's cycle and that they are the same if there is any.

chrispin2
Posts: 5
Joined: Wed Mar 21, 2018 9:52 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#119 Post by chrispin2 » Mon Mar 26, 2018 1:09 am

netniV wrote:Check that you have the same Data Collection profiles and the same cron tabs on both machines. If they are the same, see if the transport has settings that define it's cycle and that they are the same if there is any.
Hi, Thanks for the reply.

The data collection profiles between master and poller are synced as both of them capture the settings from the same master database.
Checked the transport, there is no other settings.

Would anyone know if there is a hardcoded value for the reporting time?

User avatar
phalek
Cacti Guru User
Posts: 2710
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#120 Post by phalek » Mon Mar 26, 2018 1:32 am

Why is there actually a cut in the monthly Grafana graph at around 1/4 ? The data before that date looks fine, afterwards it is "different"
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
[/size]

Post Reply