Enhance Structured path setting

Anything that you think should be in Cacti.

Moderators: Moderators, Developers

Post Reply
Author
Message
User avatar
phalek
Cacti Guru User
Posts: 2710
Joined: Thu Jan 31, 2008 6:39 am
Location: Kressbronn, Germany
Contact:

Enhance Structured path setting

#1 Post by phalek » Tue Oct 20, 2015 3:52 am

Can we change the structured path setting to be more flexible i.e. by providing a template ?

Or at least add the poller_id to the structured path ( example shown below ).

This would allow us to have different storages (iSCSI, harddisks, NFS) for each poller so it scales a lot better.


lib/functions.php ( around line 2237 ):

Code: Select all

/* generate_data_source_path - creates a new data source path from scratch using the first data source
     item name and updates the database with the new value
   @arg $local_data_id - (int) the ID of the data source to generate a new path for
   @returns - the new generated path */
function generate_data_source_path($local_data_id) {
        global $config;

        $host_part = ""; $ds_part = "";

        $extended_paths = read_config_option("extended_paths");

        /* try any prepend the name with the host description */
        $host = db_fetch_row("SELECT
                host.id,
                host.poller_id,
                host.description
                FROM (host, data_local)
                WHERE data_local.host_id=host.id
                AND data_local.id=$local_data_id
                LIMIT 1");

        $host_name = $host["description"];
        $host_id   = $host["id"];
        $host_poller_id = $host["poller_id"];

        /* put it all together using the local_data_id at the end */
        if ($extended_paths == "on") {
                $new_path = "<path_rra>/$host_poller_id/$host_id/$local_data_id.rrd";
        }else{
                if (!empty($host_name)) {
                        $host_part = strtolower(clean_up_file_name($host_name)) . "_";
                }

                /* then try and use the internal DS name to identify it */
                $data_source_rrd_name = db_fetch_cell("SELECT data_source_name
                        FROM data_template_rrd
                        WHERE local_data_id=$local_data_id
                        ORDER BY id");

                if (!empty($data_source_rrd_name)) {
                        $ds_part = strtolower(clean_up_file_name($data_source_rrd_name));
                }else{
                        $ds_part = "ds";
                }

                $new_path = "<path_rra>/$host_part$ds_part" . "_" . "$local_data_id.rrd";
        }

        /* update our changes to the db */
        db_execute("UPDATE data_template_data SET data_source_path='$new_path' WHERE local_data_id=$local_data_id");

        return $new_path;
}
cli/structure_rra_paths.php ( from line 92: )

Code: Select all

/* fetch all DS having wrong path */
$data_sources = db_fetch_assoc("SELECT
                                local_data_id,
                                host_id,
                                poller_id,
                                data_source_path,
                                CONCAT('<path_rra>/', poller_id, '/', host_id,'/', local_data_id, '.rrd') AS new_data_source_path,
                                REPLACE(data_source_path, '<path_rra>', '$base_rra_path') AS rrd_path,
                                REPLACE(CONCAT('<path_rra>/', poller_id, '/', host_id, '/', local_data_id, '.rrd'), '<path_rra>', '$base_rra_path') AS new_rrd_path
                                        FROM data_template_data
                                        INNER JOIN data_local ON data_local.id=data_template_data.local_data_id
                                        INNER JOIN host ON host.id=data_local.host_id
                                WHERE data_source_path != CONCAT('<path_rra>/', poller_id, '/', host_id, '/', local_data_id, '.rrd')"
                                 . ($hostId === NULL ? "" : " AND host_id=$hostId"));

/* setup some counters */
$done_count   = 0;
$warn_count   = 0;

/* scan all data sources */
foreach ($data_sources as $info) {
        $new_base_path = "$base_rra_path" . "/" . $info["poller_id"] . '/' . $info["host_id"];
        $new_rrd_path  = $info["new_rrd_path"];
        $old_rrd_path  = $info["rrd_path"];

        /* create one subfolder for every host */
        if (!is_dir($new_base_path)) {
                /* see if we can create the dirctory for the new file */
                if (mkdir($new_base_path, 0775, true)) {
                        echo "NOTE: New Directory '$new_base_path' Created for RRD Files\n";
Greetings,
Phalek
---
Need more help ? Read the Cacti documentation or my new Cacti 1.x Book
Need on-site support ? Look here Cacti Workshop
Need professional Cacti support ? Look here CereusService
---
Plugins : CereusTransporter | CereusReporting | nmidWebService | nmidSmokeping | nmidWeb2 |

Code: Select all

CereusServer Master:  SYSTEM STATS: Time:2.5621 Method:spine Processes:1 Threads:16 Hosts:446 HostsPerProcess:446 DataSources:14683 RRDsProcessed:7573
CereusServer Agent:   SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
[/size]

der_doc
Cacti User
Posts: 51
Joined: Mon Jul 24, 2006 7:20 am
Location: Braunschweig, Germany

Re: Enhance Structured path setting

#2 Post by der_doc » Wed Nov 18, 2015 3:32 am

Hi phalek,

now is winter in germany and I will have a look to my plugin. :D
The idea with the structured path setting for the rrd files is good, so i think. I will test to build this in the plugin as an option to use. The idea with the plugin was basic a feature only for local cluster so the rrd files were good reversed to only one NFS storage :roll:
But the other idea is to connect the poller over VPN so the location for the rrd files were better on local poller.

Unhappily I´am not longer works as network admin in a big network infrastructur (switches with snmp), in this time, the plugin work on one real server with 3-4 virtual server and a snmp simulation over java. :oops: In this case the development is not so system near as in last times.

But i will have a look to integrate the structured path setting for the rrd files in one of the next updates.

Best Regards,
Der_Doc
Image
ImageImage

Post Reply