InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

Addons for Cacti and discussion about those addons

Moderators: Moderators, Developers

Post Reply
Author
Message
lucajet
Posts: 19
Joined: Wed Oct 27, 2010 1:37 pm

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Upda

#136 Post by lucajet » Wed Jun 05, 2019 4:03 am

Could be possible to pass to influxdb just a subset of the data sources instead of everything?

User avatar
phalek
Cacti Guru User
Posts: 2739
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#137 Post by phalek » Tue Aug 27, 2019 2:17 am

Hi

Currently the plugin doesn't allow this, it just graps everything and sends it off to influxdb
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#138 Post by bigblue » Thu Jan 16, 2020 5:09 am

Hi,

I have a problem getting the plugin to work:
cacti logs:
"ERROR: A DB Exec Failed!, Error: '1114', SQL: "INSERT INTO `plugin_CereusTransporter_data
"
log mysql:
"mysqld: The table 'plugin_CereusTransporter_data' is full
"

I tried to increase the @@max_heap_table_size to 1Gb, but nothing worked.
The influxdb database does not receive any data.

Can you help me?

Cacti version: 0.8.8h
CereusTransporter version: 0.64
Devices > 1200 Hosts
Graph Management > 70K Graphs

Regards

User avatar
phalek
Cacti Guru User
Posts: 2739
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#139 Post by phalek » Thu Jan 16, 2020 6:02 am

Drop the table and re-create it:

Code: Select all

mysql> drop table plugin_CereusTransporter_data;

mysql> CREATE TABLE `plugin_CereusTransporter_data` (`timestamp` VARCHAR(1024) NOT NULL DEFAULT '0',`local_data_id` INT(11) NOT NULL DEFAULT '0',`key` VARCHAR(1024) NOT NULL DEFAULT '0',`value` VARCHAR(1024) NOT NULL DEFAULT '0',INDEX `local_data_id` (`local_data_id`),INDEX `key` (`key`)) COMMENT='NMID CereusTransporter Data' COLLATE='latin1_swedish_ci' ENGINE=MyISAM;
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#140 Post by bigblue » Thu Jan 16, 2020 7:43 am

phalek wrote:
Thu Jan 16, 2020 6:02 am
Drop the table and re-create it:

Code: Select all

mysql> drop table plugin_CereusTransporter_data;

mysql> CREATE TABLE `plugin_CereusTransporter_data` (`timestamp` VARCHAR(1024) NOT NULL DEFAULT '0',`local_data_id` INT(11) NOT NULL DEFAULT '0',`key` VARCHAR(1024) NOT NULL DEFAULT '0',`value` VARCHAR(1024) NOT NULL DEFAULT '0',INDEX `local_data_id` (`local_data_id`),INDEX `key` (`key`)) COMMENT='NMID CereusTransporter Data' COLLATE='latin1_swedish_ci' ENGINE=MyISAM;
great!
Thank you, I have no more errors in cacti log and mysqldb.
Cacti log:
"01/16/2020 01:28:34 PM - CereusTransporter: Poller [0] INFO: Adding data to influxdb"

The plugin_CereusTransporter_data table is well filled, but my InfluxDB database is not receiving any packets:

# tcpdump -i enp0s3 port 8086
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
2 packets received by filter
0 packets dropped by kernel

# influx -precision rfc3339
Connected to http://localhost:8086 version 1.7.9
InfluxDB shell version: 1.7.9
> SHOW DATABASES
name: databases
name
----
_internal
> quit

However, I configured the plugin like that:
influxdb://xxx.xxx.185.222:8086/cacti

User avatar
phalek
Cacti Guru User
Posts: 2739
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#141 Post by phalek » Thu Jan 16, 2020 7:45 am

you're using localhost but referr to a different IP for the URL

Can you open the Influx Webpage using the IP ? It may just be a firewall problem.
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#142 Post by bigblue » Thu Jan 16, 2020 8:03 am

Sorry... I didn't understand influxdb:// means local connection...

I configured the plugin in http, but still nothing in influxdb.

Is there a special configuration to do for influxdb?
Is it necessary to create the DB manually?
Last edited by bigblue on Thu Jan 16, 2020 9:31 am, edited 1 time in total.

User avatar
phalek
Cacti Guru User
Posts: 2739
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#143 Post by phalek » Thu Jan 16, 2020 8:09 am

Where is influxdb running, on the same host as cacti ( then use influxdb://localhost ) or on a different server ( then use influxdb://<remote ip> ... and make sure the firewall actually allows the connection to the influxdb port ) ?

Also if you're using PHP 7 then this will probably not work
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

User avatar
phalek
Cacti Guru User
Posts: 2739
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#144 Post by phalek » Thu Jan 16, 2020 8:13 am

Oh, and i didn't see the cacti database in influxdb. did you create it at all ?
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#145 Post by bigblue » Thu Jan 16, 2020 9:30 am

Yes, I'm using another server, so I changed it to "influxdb://<remote ip>:8086/cacti".
The firerwall is disabled.
The telnet <remote ip> 8086 works fine.

But still nothing comes to influxdb.

My php version:
$ php -v
PHP 5.4.16 (cli) (built: Nov 1 2019 16:04:20)

I didn't create a table in influxdb, now I have.
Is there anything else we can do?

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#146 Post by bigblue » Fri Jan 17, 2020 8:36 am

I did some php debugging, the program doesn't pass the next command:
$polling_data = db_fetch_assoc( "SELECT `timestamp`, `local_data_id`, `key`, `value` FROM plugin_CereusTransporter_data ORDER BY `timestamp`,`local_data_id`,`key` );

Do you know why?

User avatar
phalek
Cacti Guru User
Posts: 2739
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#147 Post by phalek » Fri Jan 17, 2020 8:41 am

What error message do you get when calling this SQL statement from within mysql ?
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#148 Post by bigblue » Fri Jan 17, 2020 8:55 am

There isn't one... it runs and apparently finishes normally:
MariaDB [cacti]> SELECT `timestamp`, `local_data_id`, `key`, `value` FROM plugin_CereusTransporter_data ORDER BY `timestamp`,`local_data_id`,`key`
-> ;
+------------+---------------+---------------------+------------------+
| timestamp | local_data_id | key | value |
+------------+---------------+---------------------+------------------+
| 1579269002 | 19093 | 5min_cpu | 22 |
| 1579269002 | 19094 | cisco_memfree | 4737352 |
| 1579269002 | 19095 | cisco_memused | 17571344 |
| 1579269002 | 23640 | 5min_cpu | 23 |
...
...
| 1579269040 | 1330015 | 5min_cpu | 8 |
| 1579269040 | 1330016 | cisco_memfree | 25587620 |
| 1579269040 | 1330017 | cisco_memused | 11599784 |
| 1579269040 | 1330018 | arp_available | 5936 |
| 1579269040 | 1330018 | arp_used | 1434 |
+------------+---------------+---------------------+------------------+
209110 rows in set (3.72 sec)

bigblue
Posts: 8
Joined: Thu Feb 04, 2016 5:15 am

Re: InfluxDB/Grafana/Bosun + Cacti = CereusTransport - Updated!

#149 Post by bigblue » Fri Jan 17, 2020 10:35 am

I replaced db_fetch_assoc with mysql_fetch_assoc and the function works.
But I'm doing this without knowing anything about php :-?
I hope I'm not doing too much wrong...
:cry:

Post Reply