ApacheStats 0.8.2 (multiple RRD, Low hdd, Script Server)

Templates, scripts for templates, scripts and requests for templates.

Moderators: Moderators, Developers

Post Reply
Author
Message
User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

ApacheStats 0.8.2 (multiple RRD, Low hdd, Script Server)

#1 Post by RCK » Thu Jan 10, 2008 5:45 pm

Dear community,

I released a new version of ApacheStats with one goal in mind: reduce disk usage.
The previous versions were using only one "Data Template", the results was each graph was using one 733K RRD file with same data inside.
In order to change that, I have updated the php script to filter each graph data, and rebuild the whole templates to use it 8)
The result is 6 small RRD files (24k) to graph one host, it's a total of 350K instead of 5400K!

Warning:
This template will only work with ONE consolidation function (AVERAGE) into your RRA definition.
(Cacti default is using TWO consolidation functions).
To check how many consolidation function your have, go to "Management" -> "Data Sources" -> "RRAs", then edit the 4 archives and see how many lines are selected. You only need the "AVERAGE" line here.
note: you can also create alternate RRA functions with only "AVERAGE" line selected, then link those alternate RRA from ApacheStats08 data template. ("Templates" -> "Data Templates")

Upgrade step
  1. update to cacti 0.8.7e
  2. upload / update your file /scripts/ss_apache_stats.php
  3. Delete all Apache08 CDEF in "Management" -> "Graph Management" -> "CDEFs"
  4. Delete all Apache08 graph templates in "Templates" -> "Graph Templates"
  5. Import the XML "cacti_host_template_webserver_-_apache.xml" into your cacti website
Install step
  1. update to cacti 0.8.7e
  2. upload the "ss_apache_stats.php" file to your cacti/scripts/ directory
  3. import the XML "cacti_host_template_webserver_-_apache.xml" into your cacti website
  4. If you don't care about 1min polling system, go into (Data Templates), edit the 5 "ApacheStats08" and remove "Hourly (1 Minute Average)" associated RRA's
  5. add one device with the "WebServer - Apache" host template, the 7 graphs templates will show up in the (Associated Graph Templates) list
  6. click (Create Graphs for this Host) and choose all graphs EXCEPT number (D) and (F) (to share common data source later), click create
  7. choose the last two graphs (D) and (F) and click create
If you want to save 100K, you can remove the duplicate RRD by sharing common data source (*)
  1. go to (Graph Management) and edit "<host> ApacheStats08 - D - Bytes / Hit - same data source..."
  2. - change the data source "<host> ApacheStats08 (apache_total_hits)" to the other "<host> ApacheStats08 (apache_total_hits)"
  3. - change the data source "<host> ApacheStats08 (apache_total_kbytes)" to the other "<host> ApacheStats08 (apache_total_kbytes)", and save
  4. go to (Graph Management) and edit "<host> ApacheStats08 - F - Thread Details (%) - same data source..."
  5. - change the data source "<host> ApacheStats08 (apache_busy_workers)" to the other "<host> ApacheStats08 (apache_busy_workers)"
  6. - change the data source "<host> ApacheStats08 (apache_idle_workers)" to the other "<host> ApacheStats08 (apache_idle_workers)", and save
  7. go to (Data Source) and look at (<host> ApacheStats08 - Total Hits), (<host> ApacheStats08 - Total kBytes), (<host> ApacheStats08 - Workers)
  8. you will have to locate the 3 biggest <ID> and delete them. (look at the second column)
  9. If you have correctly migrated the date source at step 2,3,5,6, Cacti won't propose you to delete one associated graph.
  10. if cacti had time to create them, delete the 3 RRD files that have those <ID>, or use the RRDCLEANER plugin to perform the job.
(*) I don't think it is currently possible to automatically create multiple graphs with common data source in cacti, dear cacti masters, please add this option in futur ! :)

Best regards,
RCK

---------------
Changelog 0.8.2
---------------
- export was done with cacti 0.8.7e
- ss_apache_stats.php: add support for apache 1.x - BusyServers,IdleServers (thanks helzerr)
- ss_apache_stats.php: no more floor() on CPU query to have better values (thanks helzerr)
- fixed the CDEFs function for graph D and F (thanks kilos / spkane)
- fixed some multiple graph comments after import.

---------------
Changelog 0.8.1
---------------
- export was done with cacti 0.8.7b
- CDEF were updated to correctly works with one consolidation function (AVERAGE)
- fixed some graph template that were still using the (MAX) consolidation.

---------------
Changelog 0.8.0
---------------
- data splitted into multiple RRD files to save 87% disk space (708K instead of 5400K)
- no more thread_O useless RRA
- no more redondant RRD data
- fix some useless "Graph Item Inputs"
- cacti 0.8.7 support
- this version of ss_apache_stats.php is compatible with ApacheStats 0.64

previous version credits
Attachments
ApacheStats_0.8.2.zip
ss_apache_stats.php + cacti_host_template_webserver_-_apache.xml
(12.78 KiB) Downloaded 14604 times
Last edited by RCK on Thu Oct 15, 2009 6:07 am, edited 36 times in total.
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

#2 Post by RCK » Thu Jan 10, 2008 5:48 pm

Graph output
Attachments
apachestats08_E.png
apachestats08_E.png (50.52 KiB) Viewed 187783 times
apachestats08_D.png
apachestats08_D.png (38.27 KiB) Viewed 187783 times
apachestats08_C.png
apachestats08_C.png (34.75 KiB) Viewed 187783 times
apachestats08_B.png
apachestats08_B.png (34.06 KiB) Viewed 187783 times
apachestats08_A.png
apachestats08_A.png (31.45 KiB) Viewed 187783 times
Last edited by RCK on Fri Jan 11, 2008 5:54 am, edited 1 time in total.
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

#3 Post by RCK » Thu Jan 10, 2008 5:49 pm

Graph output
Attachments
apachestats08_G.png
apachestats08_G.png (65.58 KiB) Viewed 187781 times
apachestats08_F.png
apachestats08_F.png (51.54 KiB) Viewed 187781 times
Last edited by RCK on Fri Jan 11, 2008 5:54 am, edited 1 time in total.
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

User avatar
fmangeant
Cacti Guru User
Posts: 2326
Joined: Fri Sep 19, 2003 8:36 am
Location: Sophia-Antipolis, France
Contact:

#4 Post by fmangeant » Fri Jan 11, 2008 2:51 am

Many many thanks :)
[size=84]
[color=green]HOWTOs[/color] :
[list][*][url=http://forums.cacti.net/viewtopic.php?t=15353]Install and configure the Net-SNMP agent for Unix[/url]
[*][url=http://forums.cacti.net/viewtopic.php?t=26151]Install and configure the Net-SNMP agent for Windows[/url]
[*][url=http://forums.cacti.net/viewtopic.php?t=28175]Graph multiple servers using an SNMP proxy[/url][/list]
[color=green]Templates[/color] :
[list][*][url=http://forums.cacti.net/viewtopic.php?t=15412]Multiple CPU usage for Linux[/url]
[*][url=http://forums.cacti.net/viewtopic.php?p=125152]Memory & swap usage for Unix[/url][/list][/size]

phi00611
Posts: 19
Joined: Thu Dec 20, 2007 4:40 am

#5 Post by phi00611 » Fri Jan 11, 2008 3:36 am

Hello,
This is exactly what I was looking for.

Thanks.

phi00611
Posts: 19
Joined: Thu Dec 20, 2007 4:40 am

#6 Post by phi00611 » Fri Jan 11, 2008 5:18 am

Hello,

I have the following error when I try to import the host_template "cacti_host_template_webserver_-_apache.xml" :

Error: XML: Hash version does not exist.

Could you help me?[/b]
[size=75]O.S.: [b]DEBIAN [/b][color=red]v4[/color] Or [b]CENTOS [/b][color=red]v4.4[/color]
WebServer : Apache [color=red]v2.2.x[/color]
[color=blue]cacti[/color] : [color=red]0.8.7a[/color][/size]

User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

#7 Post by RCK » Fri Jan 11, 2008 5:42 am

phi00611 wrote:Hello,

I have the following error when I try to import the host_template "cacti_host_template_webserver_-_apache.xml" :

Error: XML: Hash version does not exist.

Could you help me?[/b]
Be sure to have at least cacti version 0.8.7a
If this is the case, update your php version to 5.2.5
I tested my package under a fresh install without problem, can you also do the same if the cacti & php version are up-to-date ?
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

phi00611
Posts: 19
Joined: Thu Dec 20, 2007 4:40 am

#8 Post by phi00611 » Fri Jan 11, 2008 6:00 am

I updated cacti to 0.8.7a, and now the import works fine.

Thanks for you help.
[size=75]O.S.: [b]DEBIAN [/b][color=red]v4[/color] Or [b]CENTOS [/b][color=red]v4.4[/color]
WebServer : Apache [color=red]v2.2.x[/color]
[color=blue]cacti[/color] : [color=red]0.8.7a[/color][/size]

phi00611
Posts: 19
Joined: Thu Dec 20, 2007 4:40 am

#9 Post by phi00611 » Fri Jan 11, 2008 8:02 am

I updated my version of php 5.2.0-8 to 5.2.5. But all the graphes remain empty?
[size=75]O.S.: [b]DEBIAN [/b][color=red]v4[/color] Or [b]CENTOS [/b][color=red]v4.4[/color]
WebServer : Apache [color=red]v2.2.x[/color]
[color=blue]cacti[/color] : [color=red]0.8.7a[/color][/size]

User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

#10 Post by RCK » Fri Jan 11, 2008 8:57 am

phi00611 wrote:I updated my version of php 5.2.0-8 to 5.2.5. But all the graphes remain empty?
1- import the templates again
2- delete the data source and graph that are empty
3- be sure to have uploaded ss_apache_stats.php to scripts folder
4- test the script with "php ss_apache_stats.php <host>" command
5- create your graph again
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

User avatar
fmangeant
Cacti Guru User
Posts: 2326
Joined: Fri Sep 19, 2003 8:36 am
Location: Sophia-Antipolis, France
Contact:

#11 Post by fmangeant » Mon Jan 21, 2008 9:52 am

Hi

the only change that I'd make would be to graph bits/s instead of kbits/s, and let Cacti do the maths.
[size=84]
[color=green]HOWTOs[/color] :
[list][*][url=http://forums.cacti.net/viewtopic.php?t=15353]Install and configure the Net-SNMP agent for Unix[/url]
[*][url=http://forums.cacti.net/viewtopic.php?t=26151]Install and configure the Net-SNMP agent for Windows[/url]
[*][url=http://forums.cacti.net/viewtopic.php?t=28175]Graph multiple servers using an SNMP proxy[/url][/list]
[color=green]Templates[/color] :
[list][*][url=http://forums.cacti.net/viewtopic.php?t=15412]Multiple CPU usage for Linux[/url]
[*][url=http://forums.cacti.net/viewtopic.php?p=125152]Memory & swap usage for Unix[/url][/list][/size]

User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

#12 Post by RCK » Mon Jan 21, 2008 1:03 pm

It's already like this no ?
I thought only graph title was displaying "kbit/s"

What would you exactly change ? :D
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

User avatar
fmangeant
Cacti Guru User
Posts: 2326
Joined: Fri Sep 19, 2003 8:36 am
Location: Sophia-Antipolis, France
Contact:

#13 Post by fmangeant » Mon Jan 21, 2008 3:54 pm

RCK wrote:It's already like this no ?
I thought only graph title was displaying "kbit/s"

What would you exactly change ? :D
So maybe the graph title & legend should be renamed to "bits/s" ?

I'm also wondering something : do you know how the stats for a "big" web server will be handled ?

http://www.apache.org/server-status returns that :

Code: Select all

Server uptime: 6 hours 38 minutes 44 seconds
Total accesses: 2239679 - Total Traffic: 206.5 GB
:o
[size=84]
[color=green]HOWTOs[/color] :
[list][*][url=http://forums.cacti.net/viewtopic.php?t=15353]Install and configure the Net-SNMP agent for Unix[/url]
[*][url=http://forums.cacti.net/viewtopic.php?t=26151]Install and configure the Net-SNMP agent for Windows[/url]
[*][url=http://forums.cacti.net/viewtopic.php?t=28175]Graph multiple servers using an SNMP proxy[/url][/list]
[color=green]Templates[/color] :
[list][*][url=http://forums.cacti.net/viewtopic.php?t=15412]Multiple CPU usage for Linux[/url]
[*][url=http://forums.cacti.net/viewtopic.php?p=125152]Memory & swap usage for Unix[/url][/list][/size]

User avatar
RCK
Cacti User
Posts: 50
Joined: Fri Jan 26, 2007 12:38 pm

#14 Post by RCK » Mon Jan 21, 2008 4:35 pm

You know, I only take previous template and update the rrd processing :)
* Rolf Poser for apachestats 0.4 - http://forums.cacti.net/viewtopic.php?t=9861
* Mahuani for apachestats 0.6 SS - http://forums.cacti.net/viewtopic.php?t=17995

kbit/s title is from there, they should have a good reason for that :p

LOL !
I will try to monitor apache.org, just to see ;)
Cacti: 0.8.7g | Spine: 0.8.7g
Plugins: RRDclean - Weathermap - Thold - Realtime
My Template: ApacheStats 0.8.2 (multiple RRD, Low disk usage, Script Server)

Weedy
Posts: 23
Joined: Thu Jan 26, 2006 7:07 pm

#15 Post by Weedy » Wed Jan 23, 2008 10:04 am

I dont know what the kbit/s graph is ACTUALLY plotting but I'm guessing it is supposed to plot apache traffic per second.

Code: Select all

 # lynx --dump http://127.0.0.1/server-status?auto
Total Accesses: 692641
Total kBytes: 10627606
CPULoad: .173695
Uptime: 49754
ReqPerSec: 13.9213
BytesPerSec: 218730
BytesPerReq: 15711.8
BusyWorkers: 18
IdleWorkers: 57
Scoreboard: ................................................................___W
_K____K___K____W_____.......................................___CK_____K__W_KW___
_K___...........................................................................
............................_______KK___W__K____K___K...........................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
............................................................................

So why do we use "Total kBytes" when we have "BytesPerSec"? right now "Total kBytes" shows about 10 gigs and "BytesPerSec" something like 213.6 kB/second. But cacti is graphing like "1.3 kBits/s" (no way in hell thats right), anyway I'm going to make the edit and see how it works.

Post Reply