Cacti THOLD alerts to Slack

Announcements concerning Plugins for Cacti

Moderators: Moderators, Developers

Author
Message
jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Cacti THOLD alerts to Slack

#1 Post by jamesb » Mon Jul 04, 2016 6:41 pm

I've written a Python script that will turn THOLD email alerts into Slack rich messages with graph.
The script leverages Sendmail aliases and the sendmail restricted shell (smrsh) to pipe the default THOLD email message in.

You will need to edit set some global variables (within the script). Please let me know if you have any questions.

Download here: https://github.com/jamesboswell/thold-slack

Features:
* Sends Cacti THOLD message to a Slack channel of your choice
* Allows hyperlink to URL of your choice (plugins/thold/thold_graph.php)
* Hyperlink to graph of threshold breached
* Optionally includes graph directly in Slack
* RED (Alert), YELLOW (Warning), BLUE (Restored) alerts

Screenshot examples:
Attachments
cacti-thold-slack-restored.png
Restored example
cacti-thold-slack-restored.png (157.38 KiB) Viewed 7077 times
cacti-thold-slack-ALERT-baseline.png
ALERT example
cacti-thold-slack-ALERT-baseline.png (197.7 KiB) Viewed 7077 times

MissouriSpartan
Posts: 34
Joined: Fri Dec 16, 2016 12:52 pm

Re: Cacti THOLD alerts to Slack

#2 Post by MissouriSpartan » Wed Apr 12, 2017 4:08 pm

I know this is a couple years old, but your script works perfectly!!!!

However, I'm trying to figure out how to implement putting graphs directly into the THOLD messages in Slack. I'm a little confused as to what you mean by mapping a directory to Apache.

N00b at this, I guess.

Help?

User avatar
Osiris
Cacti Pro User
Posts: 835
Joined: Mon Jan 05, 2015 10:10 am

Re: Cacti THOLD alerts to Slack

#3 Post by Osiris » Sun Apr 23, 2017 6:35 pm

Thats pretty cool.
Before history, there was a paradise, now dust.

User avatar
MoonDogg
Posts: 32
Joined: Thu Mar 16, 2017 8:43 am

Re: Cacti THOLD alerts to Slack

#4 Post by MoonDogg » Mon Apr 24, 2017 3:07 pm

This is great!! anyone know if you can use sendmail and SMTP notification? I am using SMTP right now, but would have to switch to sendmail to implement this.

Update: never mind I got SendMail installed and working with all my notifications with no problem. Now working on getting the graph's in the Slack notifications.
Ubuntu 16.04.2 LTS
Cacti 1.1.17
Spine 1.1.17
RRDtool 1.5.5
PHP 7.0.18
mysql 14.14 (5.7.19)
Cycle 4.0
Monitor 2.1
Gexport 1.0
Thold 1.02
PHP Weathermap 0.98a

User avatar
MoonDogg
Posts: 32
Joined: Thu Mar 16, 2017 8:43 am

Re: Cacti THOLD alerts to Slack

#5 Post by MoonDogg » Wed Apr 26, 2017 9:51 am

MissouriSpartan wrote:I know this is a couple years old, but your script works perfectly!!!!

However, I'm trying to figure out how to implement putting graphs directly into the THOLD messages in Slack. I'm a little confused as to what you mean by mapping a directory to Apache.

N00b at this, I guess.

Help?
What it is saying is that :

Code: Select all

slack_image_url     =   "http://cacti/tmp/img/"
Needs to go to the same folder as :

Code: Select all

image_path           = '/var/www/html/tmp/img/'
This is my path's and I am using ubuntu server 16.04. I also have "cacti" map to the ip address of my server in DNS.

I still have not got this to work yet. I think the script is looking for an image attachment and the new system puts inline images into the e-mail. I have tried to change it to PNG and JPEG attached images and it still does not work. The images are not getting copied to that directory. I am at a loss at this point. It also does not look like the e-mails are changing image formats when I change the settings, so I am not sure whats going on.
Ubuntu 16.04.2 LTS
Cacti 1.1.17
Spine 1.1.17
RRDtool 1.5.5
PHP 7.0.18
mysql 14.14 (5.7.19)
Cycle 4.0
Monitor 2.1
Gexport 1.0
Thold 1.02
PHP Weathermap 0.98a

jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Re: Cacti THOLD alerts to Slack

#6 Post by jamesb » Mon May 01, 2017 1:40 pm

MissouriSpartan wrote:I know this is a couple years old, but your script works perfectly!!!!

However, I'm trying to figure out how to implement putting graphs directly into the THOLD messages in Slack. I'm a little confused as to what you mean by mapping a directory to Apache.

N00b at this, I guess.

Help?
You need to configure your web server to map your image directory to a reachable URL for to embed graphs in Slack attachments. Slack posts don't actually upload the PNG per THOLD alert. The Slack notification includes a URL to the PNG file, that the Slack image proxy accesses.

Here is an example from my server, where my script outputs PNG files to /data/tmp/img in line 58. You can change line 58 to any directory you like.
https://github.com/jamesboswell/thold-s ... ack.py#L58

Code: Select all

/data/tmp/img/f6bd35eda2f34bb7824c8fbc297c1abe.png
is accessible at

Code: Select all

http://example.com/img/f6bd35eda2f34bb7824c8fbc297c1abe.png
This is done in Apache with a simple Alias and Location directives

Code: Select all

Alias /img /data/tmp/img  # this maps URL to disk location
<Location /img>
    Allow from All
    Satisfy any
</Location>
You should see something like this in your logs

Code: Select all

54.152.5.26 - - [28/Apr/2017:17:21:25 -0400] "GET /ed94329640844ae5a5458ee80c542c4f.png HTTP/1.1" 200 50572 "-" "Slack-ImgProxy (+https://api.slack.com/robots)"
You can make your URL whatever you like, you just need to map the URL to the location of the PNG files on disk.

jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Re: Cacti THOLD alerts to Slack

#7 Post by jamesb » Mon May 01, 2017 1:42 pm

MoonDogg wrote:This is great!! anyone know if you can use sendmail and SMTP notification? I am using SMTP right now, but would have to switch to sendmail to implement this.

Update: never mind I got SendMail installed and working with all my notifications with no problem. Now working on getting the graph's in the Slack notifications.
Glad you got it working. As you discovered it has to be Sendmail. SMTP is using PHP to directly send the email, where as Sendmail is processing the email (and in this case piping it to my Python script)

User avatar
MoonDogg
Posts: 32
Joined: Thu Mar 16, 2017 8:43 am

Re: Cacti THOLD alerts to Slack

#8 Post by MoonDogg » Mon May 01, 2017 1:51 pm

jamesb wrote:
MoonDogg wrote:This is great!! anyone know if you can use sendmail and SMTP notification? I am using SMTP right now, but would have to switch to sendmail to implement this.

Update: never mind I got SendMail installed and working with all my notifications with no problem. Now working on getting the graph's in the Slack notifications.
Glad you got it working. As you discovered it has to be Sendmail. SMTP is using PHP to directly send the email, where as Sendmail is processing the email (and in this case piping it to my Python script)
I am still having issues with the graph's.. as you may have noticed I submitted an issue on your github page. On my server the images are not even showing up in the directory, so I don't know what the issue is there. And from what I just read you wrote it just links to the graph.. so does that link have to be accessible from the internet or just the network you are looking at slack on? Because my cacti server is internal only.
Ubuntu 16.04.2 LTS
Cacti 1.1.17
Spine 1.1.17
RRDtool 1.5.5
PHP 7.0.18
mysql 14.14 (5.7.19)
Cycle 4.0
Monitor 2.1
Gexport 1.0
Thold 1.02
PHP Weathermap 0.98a

jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Re: Cacti THOLD alerts to Slack

#9 Post by jamesb » Mon May 01, 2017 2:15 pm

MoonDogg wrote: I am still having issues with the graph's.. as you may have noticed I submitted an issue on your github page. On my server the images are not even showing up in the directory, so I don't know what the issue is there. And from what I just read you wrote it just links to the graph.. so does that link have to be accessible from the internet or just the network you are looking at slack on? Because my cacti server is internal only.
Ok, firstly, if the images aren't even showing up, check file permissions. The script is run as the mail user on my server for example

Code: Select all

# ps -ef | grep thold-slack
mail     13636 13634  0  2015 ?        00:00:00 /usr/bin/python /etc/smrsh/thold-slack.py
So my image path is writeable by "mail"

Code: Select all

image_path           = '/data/tmp/img/'
I actually have it owned by root, and set to 1777 (sticky bit)

Code: Select all

stat -c "%a %n" /data/tmp/img/
1777 /data/tmp/img/
The image files are 0644 when written by smrsh

Code: Select all

 File: ‘/data/tmp/img/fe217580bd90436b9fdd633352886fdf.png’
  Size: 50355     	Blocks: 112        IO Block: 32768  regular file
Device: 13h/19d	Inode: 166115      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    8/    mail)   Gid: (   12/    mail)

MissouriSpartan
Posts: 34
Joined: Fri Dec 16, 2016 12:52 pm

Re: Cacti THOLD alerts to Slack

#10 Post by MissouriSpartan » Thu May 04, 2017 9:08 am

Well, I did the following:

Set an Alias as the following:

Code: Select all

Alias /img /data/tmp/img
<Directory /img>
        Allow from all
        Satisfy any
</Directory>
Restarted Apache2 service. Then went to check on my permissions for the /data/tmp/img. This was the result of the check:

Code: Select all

drwxrwxrwx   3 root smmsp  4096 Apr 10 12:55 data

1777 /data/tmp/img/
Then checked on the thold-slack.py owner:

Code: Select all

root      5498 18452  0 09:07 pts/1    00:00:00 grep --color=auto thold-slack

Still no graphs appearing on the alerts. There are URLs showing up, which we can click on, but the /data/tmp/img directory is not being written to with graph PNG images.

jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Re: Cacti THOLD alerts to Slack

#11 Post by jamesb » Tue May 09, 2017 2:59 pm

MissouriSpartan wrote: Still no graphs appearing on the alerts. There are URLs showing up, which we can click on, but the /data/tmp/img directory is not being written to with graph PNG images.
Can you navigate to your Settings tab, Thresholds, and scroll down and confirm Send Alerts as Text is NOT checked?

MissouriSpartan
Posts: 34
Joined: Fri Dec 16, 2016 12:52 pm

Re: Cacti THOLD alerts to Slack

#12 Post by MissouriSpartan » Tue May 09, 2017 3:11 pm

Yes. This is unchecked.

jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Re: Cacti THOLD alerts to Slack

#13 Post by jamesb » Tue May 09, 2017 3:33 pm

@MissouriSpartan, can you forward one of your THOLD alert emails to yourself as an attachment, then open the attachment in a text editor?

You should see some lines that have "image" content type. I just want to make sure it's in there somewhere, my guess is it's A) not in there or B) it is, but my script is looking in the wrong index of the get_message_payload list returned. It works for me, but perhaps something is different with your generated emails.

Code: Select all

[email protected]'><br>

----ALT__MAILER_9014_1428530199_2768360--
----REL__MAILER_9014_1428530199_2768360
Content-ID: <[email protected]>
Content-Type: image/jpg
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="12753.jpg"

MissouriSpartan
Posts: 34
Joined: Fri Dec 16, 2016 12:52 pm

Re: Cacti THOLD alerts to Slack

#14 Post by MissouriSpartan » Tue May 09, 2017 4:01 pm

I may be in over my head here..............how do you forward as an attachment with Sendmail? I'm sorry, I'm a n00b with Sendmail.

jamesb
Posts: 7
Joined: Mon Jul 04, 2016 6:21 pm

Re: Cacti THOLD alerts to Slack

#15 Post by jamesb » Fri May 12, 2017 4:43 pm

MissouriSpartan wrote:I may be in over my head here..............how do you forward as an attachment with Sendmail? I'm sorry, I'm a n00b with Sendmail.
Not in Sendmail, are you sending THOLD alerts to your email now? Many programs such as Microsoft Outlook allow you to forward as an attachment.

If you can save an existing THOLD email alert that has the graphs, as an EML mail file, then we can see what's going on from there.

Post Reply