Larry,
In my testing of the spine I downloaded and compiled I ran into a issue that seems really odd.. It's going to be a bit hard to explain so I'll do my best. Please let me know if you have any questions.
Some of my graphs when using spine fail 100% of the time to plot data. Some have portions of them fail 100% of the time(I.E one item out of 4 on a graph)
They are always the same graphs, and it is only a problem with Spine (CMD.PHP works fine)
Here are some examples of problem graphs. I have a few dozen that are having this issue. The oddest part is, these same types of graphs on other devices work just fine with spine, not a single miss poll of blimp in the data
This device works fine (same data sources, same graphs)
Code:
[cactiuser@ossccacti01 cacti]$ spine -V 5 313 313
SPINE: Using spine config file [/etc/spine.conf]
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_webroot''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_cactilog''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The path_php_server variable is <removed>cacti/script_server.php
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The path_cactilog variable is <removed>cacti/log/cacti.log
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_destination''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The log_destination variable is 1 (FILE)
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_php_binary''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The path_php variable is /usr/bin/php
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'availability_method''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The availability_method variable is 2
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_recovery_count''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The ping_recovery_count variable is 2
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_failure_count''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The ping_failure_count variable is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_method''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The ping_method variable is 2
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_retries''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The ping_retries variable is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_timeout''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The ping_timeout variable is 400
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_perror''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The log_perror variable is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_pwarn''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The log_pwarn variable is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_pstats''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The log_pstats variable is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'max_threads''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The threads variable is 50
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'poller_interval''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The polling interval is 300 seconds
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'concurrent_processes''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The number of concurrent processes is 60
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'script_timeout''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The script timeout is 600
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'php_servers''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The number of php script servers to run is 10
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action FROM poller_item WHERE action=2 AND host_id BETWEEN 313 AND 313 LIMIT 1'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: StartHost='313', EndHost='313', TotalPHPScripts='0'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The PHP Script Server is Not Required
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'max_get_size''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The Maximum SNMP OID Get Size is 60
09/24/2008 04:16:15 PM - SPINE: Poller[0] Version 0.8.7c starting
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: MySQL is Thread Safe!
09/24/2008 04:16:15 PM - SPINE: Poller[0] SPINE: Initializing Net-SNMP API
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SNMP Header Version is 5.3.1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SNMP Library Version is 5.3.1
09/24/2008 04:16:15 PM - SPINE: Poller[0] SPINE: Initializing PHP Script Server(s)
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT id FROM host WHERE disabled='' AND id BETWEEN 313 AND 313 ORDER BY id'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: Initial Value of Active Threads is 0
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: Valid Thread to be Created
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: In Poller, About to Start Polling of Host
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=0 AND rrd_next_step < 0 GROUP BY snmp_port'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context FROM poller_item WHERE host_id=0 and rrd_next_step <=0 ORDER by snmp_port'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_next_step-300 WHERE host_id=0'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_step-300 WHERE rrd_next_step < 0 and host_id=0'
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[0] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: Valid Thread to be Created
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 2
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: In Poller, About to Start Polling of Host
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT id, hostname, snmp_community, snmp_version, snmp_username, snmp_password, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_port, snmp_timeout, max_oids, availability_method, ping_method, ping_port, ping_timeout, ping_retries, status, status_event_count, status_fail_date, status_rec_date, status_last_error, min_time, max_time, cur_time, avg_time, total_polls, failed_polls, availability FROM host WHERE id=313'
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] SNMP Result: Host responded to SNMP
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE host SET status='3', status_event_count='0', status_fail_date='2008-09-11 10:50:00', status_rec_date='2008-09-11 13:30:00', status_last_error='Host did not respond to SNMP', min_time='2.434020', max_time='6036.345010', cur_time='2.995010', avg_time='9.067302', total_polls='56318', failed_polls='147', availability='99.7390' WHERE id='313''
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT data_query_id, action, op, assert_value, arg1 FROM poller_reindex WHERE host_id=313'
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] Host has no information for recache.
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=313 AND rrd_next_step < 0 GROUP BY snmp_port'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context FROM poller_item WHERE host_id=313 and rrd_next_step <=0 ORDER by snmp_port'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_next_step-300 WHERE host_id=313'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_step-300 WHERE rrd_next_step < 0 and host_id=313'
09/24/2008 04:16:15 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 1
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13635] SNMP: v1: 10.14.56.44, dsname: star_aaa_curr_auth, oid: 1.3.6.1.4.1.8164.1.14.6.0, value: 0
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13636] SNMP: v1: 10.14.56.44, dsname: star_data_in_17, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.17.1, value: 10044027
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13637] SNMP: v1: 10.14.56.44, dsname: star_data_out_17, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.17.1, value: 47514706
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13638] SNMP: v1: 10.14.56.44, dsname: star_data_in_18, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.18.1, value: 11136093
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13639] SNMP: v1: 10.14.56.44, dsname: star_data_out_18, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.18.1, value: 46977695
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13640] SNMP: v1: 10.14.56.44, dsname: star_data_in_19, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.19.1, value: 23822002
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13641] SNMP: v1: 10.14.56.44, dsname: star_data_out_19, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.19.1, value: 6012434
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13642] SNMP: v1: 10.14.56.44, dsname: star_data_in_20, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.20.1, value: 25727589
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13643] SNMP: v1: 10.14.56.44, dsname: star_data_out_20, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.20.1, value: 6059373
09/24/2008 04:16:15 PM - SPINE: Poller[0] Host[313] DS[13644] SNMP: v1: 10.14.56.44, dsname: star_data_in_21_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.21.1, value: 3079833
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13645] SNMP: v1: 10.14.56.44, dsname: star_data_out_21_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.21.1, value: 8082061
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13646] SNMP: v1: 10.14.56.44, dsname: star_data_in_22_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.22.1, value: 1846496
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13647] SNMP: v1: 10.14.56.44, dsname: star_data_out_22_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.22.1, value: 7143563
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13648] SNMP: v1: 10.14.56.44, dsname: star_data_in_23_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.23.1, value: 27037183
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13649] SNMP: v1: 10.14.56.44, dsname: star_data_out_23_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.23.1, value: 6097434
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13650] SNMP: v1: 10.14.56.44, dsname: star_data_in_26_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.26.1, value: 28971925
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13651] SNMP: v1: 10.14.56.44, dsname: star_data_out_26_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.26.1, value: 6022455
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13652] SNMP: v1: 10.14.56.44, dsname: star_hareg_iacc_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.14.99.100.110.45.105.110.103.114.58.104.97.49, value: 25985080
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13653] SNMP: v1: 10.14.56.44, dsname: star_hareg_iacc_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.14.99.100.110.45.105.110.103.114.58.104.97.50, value: 279097
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13654] SNMP: v1: 10.14.56.44, dsname: star_hareg_id_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.18.99.100.110.45.105.110.103.114.58.104.97.49, value: 1354157
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13655] SNMP: v1: 10.14.56.44, dsname: star_hareg_id_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.18.99.100.110.45.105.110.103.114.58.104.97.50, value: 16373
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13656] SNMP: v1: 10.14.56.44, dsname: star_hareg_init_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.10.99.100.110.45.105.110.103.114.58.104.97.49, value: 27339284
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13657] SNMP: v1: 10.14.56.44, dsname: star_hareg_init_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.10.99.100.110.45.105.110.103.114.58.104.97.50, value: 295470
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13658] SNMP: v1: 10.14.56.44, dsname: star_hareg_racc_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.15.99.100.110.45.105.110.103.114.58.104.97.49, value: 202130021
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13659] SNMP: v1: 10.14.56.44, dsname: star_hareg_racc_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.15.99.100.110.45.105.110.103.114.58.104.97.50, value: 2963749
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13660] SNMP: v1: 10.14.56.44, dsname: star_hareg_rden_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.19.99.100.110.45.105.110.103.114.58.104.97.49, value: 274364
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13661] SNMP: v1: 10.14.56.44, dsname: star_hareg_rden_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.19.99.100.110.45.105.110.103.114.58.104.97.50, value: 795
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13662] SNMP: v1: 10.14.56.44, dsname: star_hareg_ren_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.11.99.100.110.45.105.110.103.114.58.104.97.49, value: 202419863
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13663] SNMP: v1: 10.14.56.44, dsname: star_hareg_ren_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.11.99.100.110.45.105.110.103.114.58.104.97.50, value: 2964629
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13667] SNMP: v1: 10.14.56.44, dsname: star_haerr_admn_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.27.99.100.110.45.105.110.103.114.58.104.97.49, value: 1348417
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13668] SNMP: v1: 10.14.56.44, dsname: star_haerr_bad_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.25.99.100.110.45.105.110.103.114.58.104.97.49, value: 270
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13669] SNMP: v1: 10.14.56.44, dsname: star_haerr_faut_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.31.99.100.110.45.105.110.103.114.58.104.97.49, value: 2294
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13670] SNMP: v1: 10.14.56.44, dsname: star_haerr_idm_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.26.99.100.110.45.105.110.103.114.58.104.97.49, value: 47458
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13671] SNMP: v1: 10.14.56.44, dsname: star_haerr_mnau_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.30.99.100.110.45.105.110.103.114.58.104.97.49, value: 178806
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13672] SNMP: v1: 10.14.56.44, dsname: star_haerr_nore_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.29.99.100.110.45.105.110.103.114.58.104.97.49, value: 462
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13673] SNMP: v1: 10.14.56.44, dsname: star_haerr_unsp_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.28.99.100.110.45.105.110.103.114.58.104.97.49, value: 0
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13681] SNMP: v1: 10.14.56.44, dsname: star_sess_mobile_tx, oid: 1.3.6.1.4.1.8164.1.13.15.0, value: 103123430
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[13682] SNMP: v1: 10.14.56.44, dsname: star_sess_mobile_rx, oid: 1.3.6.1.4.1.8164.1.13.16.0, value: 22221735
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16176] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16176] SNMP: v1: 10.14.56.44, dsname: star_data_in_27_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.27.1, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16177] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16177] SNMP: v1: 10.14.56.44, dsname: star_data_out_27_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.27.1, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DEBUG: The POPEN returned the following File Descriptor 7
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[17588] SCRIPT: python <removed>cacti/scripts/id_counters.py 10.14.56.44, output: 17011
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DEBUG: The POPEN returned the following File Descriptor 7
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[17589] SCRIPT: python <removed>cacti/scripts/HA_TOTAL_IP_counters.py 10.14.56.44, output: 360400
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DEBUG: The POPEN returned the following File Descriptor 7
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[17590] SCRIPT: python <removed>cacti/scripts/HA_USED_IP_counters.py 10.14.56.44, output: 343389
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16178] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16178] SNMP: v1: 10.14.56.44, dsname: star_data_in_37_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.37.1, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16179] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16179] SNMP: v1: 10.14.56.44, dsname: star_data_out_37_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.37.1, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16180] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16180] SNMP: v1: 10.14.56.44, dsname: star_data_in_38_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.38.1, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16181] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[16181] SNMP: v1: 10.14.56.44, dsname: star_data_out_38_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.38.1, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[17587] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DS[17587] SNMP: v1: 10.14.56.44, dsname: star_sess_curr, oid: 1.3.6.1.4.1.8164.1.12.1.0, value: U
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: SQL:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (13635,'star_aaa_curr_auth','2008-09-24 16:16:15','0'),(13636,'star_data_in_17','2008-09-24 16:16:15','10044027'),(13637,'star_data_out_17','2008-09-24 16:16:15','47514706'),(13638,'star_data_in_18','2008-09-24 16:16:15','11136093'),(13639,'star_data_out_18','2008-09-24 16:16:15','46977695'),(13640,'star_data_in_19','2008-09-24 16:16:15','23822002'),(13641,'star_data_out_19','2008-09-24 16:16:15','6012434'),(13642,'star_data_in_20','2008-09-24 16:16:15','25727589'),(13643,'star_data_out_20','2008-09-24 16:16:15','6059373'),(13644,'star_data_in_21_1','2008-09-24 16:16:15','3079833'),(13645,'star_data_out_21_1','2008-09-24 16:16:15','8082061'),(13646,'star_data_in_22_1','2008-09-24 16:16:15','1846496'),(13647,'star_data_out_22_1','2008-09-24 16:16:15','7143563'),(13648,'star_data_in_23_1','2008-09-24 16:16:15','27037183'),(13649,'star_data_out_23_1','2008-09-24 16:16:15','6097434'),(13650,'star_data_in_26_1','2008-09-24 16:16:15','2897192'
09/24/2008 04:16:17 PM - SPINE: Poller[0] Host[313] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 0
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: SQL:'replace into settings (name,value) values ('date',NOW())'
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: SQL:'insert into poller_time (poller_id, start_time, end_time) values (0, NOW(), NOW())'
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: Thread Cleanup Complete
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: PHP Script Server Pipes Closed
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: Allocated Variable Memory Freed
09/24/2008 04:16:17 PM - SPINE: Poller[0] DEBUG: MYSQL Free & Close Completed
09/24/2008 04:16:17 PM - SPINE: Poller[0] Time: 2.1426 s, Threads: 50, Hosts: 2
Totally fails with spine, again same device same graphs/data sources:
Quote:
[cactiuser@ossccacti01 cacti]$ spine -V 5 312 312
SPINE: Using spine config file [/etc/spine.conf]
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_webroot''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_cactilog''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The path_php_server variable is <removed>cacti/script_server.php
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The path_cactilog variable is <removed>cacti/log/cacti.log
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_destination''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The log_destination variable is 1 (FILE)
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_php_binary''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The path_php variable is /usr/bin/php
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'availability_method''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The availability_method variable is 2
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_recovery_count''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The ping_recovery_count variable is 2
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_failure_count''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The ping_failure_count variable is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_method''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The ping_method variable is 2
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_retries''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The ping_retries variable is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_timeout''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The ping_timeout variable is 400
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_perror''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The log_perror variable is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_pwarn''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The log_pwarn variable is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_pstats''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The log_pstats variable is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'max_threads''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The threads variable is 50
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'poller_interval''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The polling interval is 300 seconds
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'concurrent_processes''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The number of concurrent processes is 60
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'script_timeout''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The script timeout is 600
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'php_servers''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The number of php script servers to run is 10
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action FROM poller_item WHERE action=2 AND host_id BETWEEN 312 AND 312 LIMIT 1'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: StartHost='312', EndHost='312', TotalPHPScripts='0'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The PHP Script Server is Not Required
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'max_get_size''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The Maximum SNMP OID Get Size is 60
09/24/2008 04:12:33 PM - SPINE: Poller[0] Version 0.8.7c starting
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: MySQL is Thread Safe!
09/24/2008 04:12:33 PM - SPINE: Poller[0] SPINE: Initializing Net-SNMP API
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SNMP Header Version is 5.3.1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SNMP Library Version is 5.3.1
09/24/2008 04:12:33 PM - SPINE: Poller[0] SPINE: Initializing PHP Script Server(s)
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT id FROM host WHERE disabled='' AND id BETWEEN 312 AND 312 ORDER BY id'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: Initial Value of Active Threads is 0
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: Valid Thread to be Created
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: In Poller, About to Start Polling of Host
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=0 AND rrd_next_step < 0 GROUP BY snmp_port'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context FROM poller_item WHERE host_id=0 and rrd_next_step <=0 ORDER by snmp_port'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_next_step-300 WHERE host_id=0'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_step-300 WHERE rrd_next_step < 0 and host_id=0'
09/24/2008 04:12:33 PM - SPINE: Poller[0] Host[0] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: Valid Thread to be Created
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 2
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: In Poller, About to Start Polling of Host
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT id, hostname, snmp_community, snmp_version, snmp_username, snmp_password, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_port, snmp_timeout, max_oids, availability_method, ping_method, ping_port, ping_timeout, ping_retries, status, status_event_count, status_fail_date, status_rec_date, status_last_error, min_time, max_time, cur_time, avg_time, total_polls, failed_polls, availability FROM host WHERE id=312'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 1
09/24/2008 04:12:33 PM - SPINE: Poller[0] Host[312] SNMP Result: Host responded to SNMP
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE host SET status='3', status_event_count='0', status_fail_date='2008-07-23 01:40:00', status_rec_date='2008-07-23 01:45:00', status_last_error='Host did not respond to SNMP', min_time='9.999990', max_time='3721.486090', cur_time='15.656950', avg_time='19.648889', total_polls='56318', failed_polls='113', availability='99.7994' WHERE id='312''
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT data_query_id, action, op, assert_value, arg1 FROM poller_reindex WHERE host_id=312'
09/24/2008 04:12:33 PM - SPINE: Poller[0] Host[312] Host has no information for recache.
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=312 AND rrd_next_step < 0 GROUP BY snmp_port'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context FROM poller_item WHERE host_id=312 and rrd_next_step <=0 ORDER by snmp_port'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_next_step-300 WHERE host_id=312'
09/24/2008 04:12:33 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_step-300 WHERE rrd_next_step < 0 and host_id=312'
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13880] SNMP: v1: 10.14.120.44, dsname: star_aaa_curr_auth, oid: 1.3.6.1.4.1.8164.1.14.6.0, value: 0
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13881] SNMP: v1: 10.14.120.44, dsname: star_data_in_17, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.17.1, value: 53157304
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13882] SNMP: v1: 10.14.120.44, dsname: star_data_out_17, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.17.1, value: 215226783
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13883] SNMP: v1: 10.14.120.44, dsname: star_data_in_18, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.18.1, value: 50616449
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13884] SNMP: v1: 10.14.120.44, dsname: star_data_out_18, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.18.1, value: 213399163
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13885] SNMP: v1: 10.14.120.44, dsname: star_data_in_19, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.19.1, value: 93509199
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13886] SNMP: v1: 10.14.120.44, dsname: star_data_out_19, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.19.1, value: 29164633
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13887] SNMP: v1: 10.14.120.44, dsname: star_data_in_20, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.20.1, value: 134857021
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13888] SNMP: v1: 10.14.120.44, dsname: star_data_out_20, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.20.1, value: 29139881
09/24/2008 04:12:34 PM - SPINE: Poller[0] Host[312] DS[13889] SNMP: v1: 10.14.120.44, dsname: star_data_in_21_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.21.1, value: 11717160
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13890] SNMP: v1: 10.14.120.44, dsname: star_data_out_21_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.21.1, value: 39586119
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13891] SNMP: v1: 10.14.120.44, dsname: star_data_in_22_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.22.1, value: 10197174
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13892] SNMP: v1: 10.14.120.44, dsname: star_data_out_22_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.22.1, value: 38530654
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13893] SNMP: v1: 10.14.120.44, dsname: star_data_in_23_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.23.1, value: 106254824
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13894] SNMP: v1: 10.14.120.44, dsname: star_data_out_23_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.23.1, value: 29158883
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13895] SNMP: v1: 10.14.120.44, dsname: star_data_in_26_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.11.26.1, value: 153163054
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13896] SNMP: v1: 10.14.120.44, dsname: star_data_out_26_1, oid: 1.3.6.1.4.1.8164.1.29.1.1.12.26.1, value: 29261770
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13897] SNMP: v1: 10.14.120.44, dsname: star_hareg_iacc_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.14.99.100.110.45.105.110.103.114.58.104.97.49, value: 91141446
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13898] SNMP: v1: 10.14.120.44, dsname: star_hareg_iacc_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.14.99.100.110.45.105.110.103.114.58.104.97.50, value: 2738330
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13899] SNMP: v1: 10.14.120.44, dsname: star_hareg_id_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.18.99.100.110.45.105.110.103.114.58.104.97.49, value: 1850085
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13900] SNMP: v1: 10.14.120.44, dsname: star_hareg_id_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.18.99.100.110.45.105.110.103.114.58.104.97.50, value: 61730
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13901] SNMP: v1: 10.14.120.44, dsname: star_hareg_init_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.10.99.100.110.45.105.110.103.114.58.104.97.49, value: 92991605
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13902] SNMP: v1: 10.14.120.44, dsname: star_hareg_init_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.10.99.100.110.45.105.110.103.114.58.104.97.50, value: 2800066
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13903] SNMP: v1: 10.14.120.44, dsname: star_hareg_racc_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.15.99.100.110.45.105.110.103.114.58.104.97.49, value: 684956314
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13904] SNMP: v1: 10.14.120.44, dsname: star_hareg_racc_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.15.99.100.110.45.105.110.103.114.58.104.97.50, value: 23096242
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13905] SNMP: v1: 10.14.120.44, dsname: star_hareg_rden_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.19.99.100.110.45.105.110.103.114.58.104.97.49, value: 868357
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13906] SNMP: v1: 10.14.120.44, dsname: star_hareg_rden_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.19.99.100.110.45.105.110.103.114.58.104.97.50, value: 9851
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13907] SNMP: v1: 10.14.120.44, dsname: star_hareg_ren_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.11.99.100.110.45.105.110.103.114.58.104.97.49, value: 685879896
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13908] SNMP: v1: 10.14.120.44, dsname: star_hareg_ren_ha2, oid: 1.3.6.1.4.1.8164.1.22.1.1.11.99.100.110.45.105.110.103.114.58.104.97.50, value: 23107442
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DS[13912] SNMP: v1: 10.14.120.44, dsname: star_haerr_admn_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.27.99.100.110.45.105.110.103.114.58.104.97.49, value: 1620555
09/24/2008 04:12:35 PM - SPINE: Poller[0] Host[312] DEBUG: The POPEN returned the following File Descriptor 7
09/24/2008 04:12:36 PM - SPINE: Poller[0] Host[312] DS[17604] SCRIPT: python <removed>cacti/scripts/id_counters.py 10.14.120.44, output: 18872
09/24/2008 04:12:36 PM - SPINE: Poller[0] Host[312] DEBUG: The POPEN returned the following File Descriptor 7
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[17605] SCRIPT: python <removed>cacti/scripts/HA_TOTAL_IP_counters.py 10.14.120.44, output: 286686
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DEBUG: The POPEN returned the following File Descriptor 7
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[17606] SCRIPT: python <removed>cacti/scripts/HA_USED_IP_counters.py 10.14.120.44, output: 267821
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13913] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13913] SNMP: v1: 10.14.120.44, dsname: star_haerr_bad_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.25.99.100.110.45.105.110.103.114.58.104.97.49, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13914] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13914] SNMP: v1: 10.14.120.44, dsname: star_haerr_faut_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.31.99.100.110.45.105.110.103.114.58.104.97.49, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13915] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13915] SNMP: v1: 10.14.120.44, dsname: star_haerr_idm_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.26.99.100.110.45.105.110.103.114.58.104.97.49, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13916] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13916] SNMP: v1: 10.14.120.44, dsname: star_haerr_mnau_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.30.99.100.110.45.105.110.103.114.58.104.97.49, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13917] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13917] SNMP: v1: 10.14.120.44, dsname: star_haerr_nore_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.29.99.100.110.45.105.110.103.114.58.104.97.49, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13918] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13918] SNMP: v1: 10.14.120.44, dsname: star_haerr_unsp_ha1, oid: 1.3.6.1.4.1.8164.1.22.1.1.28.99.100.110.45.105.110.103.114.58.104.97.49, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13926] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13926] SNMP: v1: 10.14.120.44, dsname: star_sess_mobile_tx, oid: 1.3.6.1.4.1.8164.1.13.15.0, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13927] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[13927] SNMP: v1: 10.14.120.44, dsname: star_sess_mobile_rx, oid: 1.3.6.1.4.1.8164.1.13.16.0, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[17603] WARNING: Result from SNMP not valid. Partial Result: ...
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DS[17603] SNMP: v1: 10.14.120.44, dsname: star_sess_curr, oid: 1.3.6.1.4.1.8164.1.12.1.0, value: U
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: SQL:'INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES (13880,'star_aaa_curr_auth','2008-09-24 16:12:33','0'),(13881,'star_data_in_17','2008-09-24 16:12:33','53157304'),(13882,'star_data_out_17','2008-09-24 16:12:33','215226783'),(13883,'star_data_in_18','2008-09-24 16:12:33','50616449'),(13884,'star_data_out_18','2008-09-24 16:12:33','213399163'),(13885,'star_data_in_19','2008-09-24 16:12:33','93509199'),(13886,'star_data_out_19','2008-09-24 16:12:33','29164633'),(13887,'star_data_in_20','2008-09-24 16:12:33','134857021'),(13888,'star_data_out_20','2008-09-24 16:12:33','29139881'),(13889,'star_data_in_21_1','2008-09-24 16:12:33','11717160'),(13890,'star_data_out_21_1','2008-09-24 16:12:33','39586119'),(13891,'star_data_in_22_1','2008-09-24 16:12:33','10197174'),(13892,'star_data_out_22_1','2008-09-24 16:12:33','38530654'),(13893,'star_data_in_23_1','2008-09-24 16:12:33','106254824'),(13894,'star_data_out_23_1','2008-09-24 16:12:33','29158883'),(13895,'star_data_in_26_1','2008-09-24 16:12:3'
09/24/2008 04:12:37 PM - SPINE: Poller[0] Host[312] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 0
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: SQL:'replace into settings (name,value) values ('date',NOW())'
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: SQL:'insert into poller_time (poller_id, start_time, end_time) values (0, NOW(), NOW())'
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: Thread Cleanup Complete
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: PHP Script Server Pipes Closed
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: Allocated Variable Memory Freed
09/24/2008 04:12:37 PM - SPINE: Poller[0] DEBUG: MYSQL Free & Close Completed
09/24/2008 04:12:37 PM - SPINE: Poller[0] Time: 4.0226 s, Threads: 50, Hosts: 2