Upgrade trouble - no visible output

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

Hi guys,
after a long deep breath I took the chance to try to upgrade my (very) old Mactrack plugin.
After strugglin with svn and stuff I finally had a new look in the mactrack tab (something has worked).
It looks like my devices and sites are still there, I can export them and it's saying "Showing Rows 1 to 30 of 66 [1,2,3]" but nothing is shown ??

I will paste a shot of my upgrade process since there are some erors - perhaps someone can decipher if I'm in trouble

And even better: Help me how to get it work again.

-----------snip--------------------------
[root@netwatch mactrack]# mysql -u cactiuser -p cacti < mactrack.sql --force
Enter password:
ERROR 1050 (42S01) at line 5: Table 'mac_track_approved_macs' already exists
ERROR 1050 (42S01) at line 16: Table 'mac_track_device_types' already exists
ERROR 1050 (42S01) at line 35: Table 'mac_track_devices' already exists
ERROR 1050 (42S01) at line 78: Table 'mac_track_ip_ranges' already exists
ERROR 1050 (42S01) at line 93: Table 'mac_track_ips' already exists
ERROR 1050 (42S01) at line 117: Table 'mac_track_macauth' already exists
ERROR 1050 (42S01) at line 129: Table 'mac_track_macwatch' already exists
ERROR 1050 (42S01) at line 146: Table 'mac_track_oui_database' already exists
ERROR 1050 (42S01) at line 159: Table 'mac_track_ports' already exists
ERROR 1050 (42S01) at line 193: Table 'mac_track_processes' already exists
ERROR 1050 (42S01) at line 205: Table 'mac_track_scan_dates' already exists
ERROR 1050 (42S01) at line 214: Table 'mac_track_scanning_functions' already exists
ERROR 1050 (42S01) at line 225: Table 'mac_track_sites' already exists
ERROR 1050 (42S01) at line 242: Table 'mac_track_temp_ports' already exists
[root@netwatch mactrack]# php database_upgrade.php
Content-type: text/html; charset=iso-8859-15
X-Powered-By: PHP/4.3.9

SUCCESS: Modify Column, Table -> mac_track_devices, Column -> snmp_timeout, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> snmp_retries, Ok
SUCCESS: Add Column, Table -> mac_track_ports, Column -> dns_hostname, Already Exists!
SUCCESS: Add Column, Table -> mac_track_temp_ports, Column -> dns_hostname, Already Exists!
SUCCESS: Add Column, Table -> mac_track_ips, Column -> dns_hostname, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> snmp_port, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> macs_active, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> snmp_sysName, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> snmp_sysLocation, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> snmp_sysContact, Already Exists!
SUCCESS: Create Table, Table -> mac_track_scanning_functions, Already Exists!
SUCCESS: Execute SQL, Change Primary Key For 'mac_track_devices', Ok
SUCCESS: Add Index, Table -> mac_track_devices, Index -> device_id, Already Exists!
SUCCESS: Add Column, Table -> mac_track_sites, Column -> total_oper_ports, Already Exists!
SUCCESS: Execute SQL, Change Primary Key For 'mac_track_device_types', Ok
SUCCESS: Add Index, Table -> mac_track_device_types, Index -> device_type_id, Already Exists!
SUCCESS: Modify Column, Table -> mac_track_scanning_functions, Column -> scanning_function, Ok
SUCCESS: Modify Column, Table -> mac_track_scanning_functions, Column -> description, Ok
SUCCESS: Create Table, Table -> mac_track_scan_dates, Already Exists!
SUCCESS: Execute SQL, Addition of Scan Dates, Ok
SUCCESS: Add Index, Table -> mac_track_devices, Index -> snmp_sysDescr, Already Exists!
SUCCESS: Add Index, Table -> mac_track_devices, Index -> snmp_sysObjectID, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> device_type_id, Already Exists!
SUCCESS: Add Index, Table -> mac_track_devices, Index -> device_type_id, Already Exists!
SUCCESS: Add Index, Table -> mac_track_ports, Index -> port_name, Already Exists!
SUCCESS: Add Index, Table -> mac_track_ports, Index -> dns_hostname, Already Exists!
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> ips_total, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> vlans_total, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> ports_total, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> ports_active, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> ports_trunk, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> macs_active, Ok
SUCCESS: Add Column, Table -> mac_track_temp_ports, Column -> updated, Already Exists!
SUCCESS: Add Index, Table -> mac_track_temp_ports, Index -> updated, Already Exists!
SUCCESS: Add Index, Table -> mac_track_temp_ports, Index -> ip_address, Already Exists!
SUCCESS: Create Table, Table -> mac_track_ip_ranges, Already Exists!
SUCCESS: Add Column, Table -> mac_track_ip_ranges, Column -> ips_max_date, Already Exists!
SUCCESS: Add Column, Table -> mac_track_ip_ranges, Column -> ips_current_date, Already Exists!
SUCCESS: Add Column, Table -> mac_track_processes, Column -> device_id, Already Exists!
SUCCESS: Add Column, Table -> mac_track_processes, Column -> process_id, Already Exists!
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> snmp_readstring, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> snmp_readstrings, Ok
SUCCESS: Execute SQL, Change mac_track_temp_ports to Memory Table, Ok
SUCCESS: Execute SQL, Change mac_track_ips to Memory Table, Ok
SUCCESS: Create Table, Table -> mac_track_approved_macs, Already Exists!
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> ignorePorts, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> snmp_readstring, Ok
SUCCESS: Modify Column, Table -> mac_track_devices, Column -> snmp_readstrings, Ok
SUCCESS: Create Table, Table -> mac_track_oui_database, Already Exists!
SUCCESS: Add Column, Table -> mac_track_ports, Column -> vendor_mac, Already Exists!
SUCCESS: Add Index, Table -> mac_track_ports, Index -> vendor_mac, Already Exists!
SUCCESS: Add Column, Table -> mac_track_temp_ports, Column -> vendor_mac, Already Exists!
SUCCESS: Add Index, Table -> mac_track_temp_ports, Index -> vendor_mac, Already Exists!
SUCCESS: Execute SQL, Add Vendor Macs To 'mac_track_ports', Ok
SUCCESS: Execute SQL, Add Vendor Macs To 'mac_track_temp_ports', Ok
SUCCESS: Add Column, Table -> mac_track_temp_ports, Column -> authorized, Already Exists!
SUCCESS: Add Index, Table -> mac_track_temp_ports, Index -> authorized, Already Exists!
SUCCESS: Add Column, Table -> mac_track_ports, Column -> authorized, Already Exists!
SUCCESS: Add Index, Table -> mac_track_ports, Index -> authorized, Already Exists!
SUCCESS: Create Table, Table -> mac_track_macwatch, Already Exists!
SUCCESS: Create Table, Table -> mac_track_macauth, Already Exists!
SUCCESS: Create Table, Table -> mac_track_vlans, Already Exists!
SUCCESS: Execute SQL, Add VLANS to VLAN Table, Ok
ERROR: Modify Column, Table -> mac_track_devices, Column -> description, Column Does NOT Exist!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> device_name, Already Exists!
SUCCESS: Add Index, Table -> mac_track_devices, Index -> device_name, Already Exists!
SUCCESS: Add Column, Table -> mac_track_sites, Column -> customer_contact, Already Exists!
SUCCESS: Add Column, Table -> mac_track_sites, Column -> netops_contact, Already Exists!
SUCCESS: Add Column, Table -> mac_track_sites, Column -> facilities_contact, Already Exists!
SUCCESS: Add Column, Table -> mac_track_sites, Column -> site_info, Already Exists!
SUCCESS: Add Column, Table -> mac_track_device_types, Column -> serial_number_oid, Already Exists!
ERROR: Execute SQL, Move Device Names from the 'description' field to the 'device_name' field., Failed!
ERROR: Execute SQL, Blank out the 'description' field as it will now be used for something else, Failed!
SUCCESS: Add Column, Table -> mac_track_macwatch, Column -> email_addresses, Already Exists!
SUCCESS: Add Column, Table -> mac_track_macwatch, Column -> mac_id, Already Exists!
SUCCESS: Add Column, Table -> mac_track_macwatch, Column -> date_last_seen, Already Exists!
SUCCESS: Add Column, Table -> mac_track_macauth, Column -> mac_id, Already Exists!
SUCCESS: Add Column, Table -> mac_track_ports, Column -> device_name, Already Exists!
SUCCESS: Add Column, Table -> mac_track_temp_ports, Column -> device_name, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> notes, Already Exists!
ERROR: Modify Column, Table -> mac_track_ips, Column -> description, Column Does NOT Exist!
SUCCESS: Delete Column, Table -> mac_track_devices, Column -> serial_number, Column Does NOT Exist!
SUCCESS: Delete Column, Table -> mac_track_devices, Column -> asset_id, Column Does NOT Exist!
SUCCESS: Create Table, Table -> mac_track_interfaces, Already Exists!
SUCCESS: Add Column, Table -> mac_track_scanning_functions, Column -> type, Already Exists!
SUCCESS: Add Column, Table -> mac_track_device_types, Column -> ip_scanning_function, Already Exists!
SUCCESS: Execute SQL, Update the Scanning Function Type to 'Mac' for undefined types, Ok
SUCCESS: Execute SQL, Set the IP Scanning function to N/A for Device Type 1, Ok
SUCCESS: Execute SQL, Set the IP Scanning function to 'get_standard_arp_table' for Routers and L3 Switches, Ok
SUCCESS: Add Column, Table -> mac_track_interfaces, Column -> vlan_trunk, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> user_name, Already Exists!
SUCCESS: Add Column, Table -> mac_track_devices, Column -> user_password, Already Exists!

Updating Device Types in Devices Table. Please be patient.
......

Device Types Updated, You have '66' Good Device Type Mapping and '0' Bad Device Type Mapping.
If the Bad Device type mapping is greater than '0', you should inspect your devices for unmapped device types.

Importing the Vendor MAC Address Table from the IEEE

PHP Notice: Undefined variable: oui_file in /var/www/html/public_html/cacti-0.8.6h/plugins/mactrack/mactrack_import_ouidb.php on line 67
Getting OUI Database from the IEEE
OUI Database Download from IEEE Complete
..........................................................................................................
There were '10526' Entries Added/Updated in the database.
There were '0' Records Removed from the database.

Database Upgrade Complete
[root@netwatch mactrack]#

-----------snap------------------------------------

TheWitness · Posted: Fri Jul 20, 2007 11:58 am Post subject:

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

Hello, thanks for your always prompt reply!

OK - I ran: # php -q poller_mactrack.php - I got 390 errors:

PHP Notice: Undefined index: 1 in /var/www/html/public_html/cacti-0.8.6h/plugins/mactrack/lib/mactrack_functions.php on line 413

.... to ...
PHP Notice: Undefined index: 390 in /var/www/html/public_html/cacti-0.8.6h/plugins/mactrack/lib/mactrack_functions.php on line 413
07/21/2007 07:30:44 AM - SYSTEM MACTRACK STATS: Time:1051.4571 ConcurrentProcesses:7 Devices:66

...... but I haven't been able to reproduce the error!
- still no output.

- Dritz

TheWitness · Posted: Sat Jul 21, 2007 10:36 am Post subject:

Well, I corrected the error's in SVN. But it is unlikely that this is the source of the problem as it's indicative of a missing ifType in your device, which is unlikely.

Grap the updated macktrack_functions.php file and see what happens now.

TheWitness

You might as well dump your DB and send it my way. If it's too large for e-mail, contact me again and I will PM you and upload site.

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

Hi again,
I've sent you a mail with a link to download a dump of my DB.
I hope you can use it.
- Dritz

TheWitness · Posted: Mon Jul 23, 2007 6:54 pm Post subject:

I never received it. Can you PM me a link?

TheWitness

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

I have PM'ed you.
I can't see my msg in 'sent box', only in 'outbox' - did you recieve this?

- Dritz

TheWitness · Posted: Tue Jul 24, 2007 6:35 am Post subject:

Ok, I reviewed your structure as compared to the standard structure and what I can see is that aside from your's assigning defaults of '' or '0' instead of NULL, the only other change is that most of your tables are MEMORY type, which of course is good.

However, if you have lot's of MAC/IP's, you "may" have to increase your "max_heap_table_size" in your my.cnf to something like 128mbytes of something larger. The default is 16MBytes. Once you make that change, restart MySQL.

Let me know what you find. Oh, if that does not get it, start disabling other plugins until MacTrack is the only one left. Also, please post your Cacti log file.

Also, what is your version of MySQL?

TheWitness

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

OK.
Increased max_heap_table_size and tmp_table_size to 128M
Restarted MySql.
Still no visual

Current running parameters:
[root@netwatch etc]# mysqladmin -u cactiuser -p variables
Enter password:
+---------------------------------+--------------------------------------------------------+
| Variable_name | Value |
+---------------------------------+--------------------------------------------------------+
| back_log | 50 |
| basedir | /usr/ |
| bdb_cache_size | 8388600 |
| bdb_home | /var/lib/mysql/ |
| bdb_log_buffer_size | 32768 |
| bdb_logdir | |
| bdb_max_lock | 10000 |
| bdb_shared_data | OFF |
| bdb_tmpdir | /tmp/ |
| binlog_cache_size | 32768 |
| bulk_insert_buffer_size | 8388608 |
| character_set_client | latin1 |
| character_set_connection | latin1 |
| character_set_database | latin1 |
| character_set_results | latin1 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
| collation_connection | latin1_swedish_ci |
| collation_database | latin1_swedish_ci |
| collation_server | latin1_swedish_ci |
| concurrent_insert | ON |
| connect_timeout | 5 |
| datadir | /var/lib/mysql/ |
| date_format | %Y-%m-%d |
| datetime_format | %Y-%m-%d %H:%i:%s |
| default_week_format | 0 |
| delay_key_write | ON |
| delayed_insert_limit | 100 |
| delayed_insert_timeout | 300 |
| delayed_queue_size | 1000 |
| expire_logs_days | 0 |
| flush | OFF |
| flush_time | 0 |
| ft_boolean_syntax | + -><()~*:""&| |
| ft_max_word_len | 84 |
| ft_min_word_len | 4 |
| ft_query_expansion_limit | 20 |
| ft_stopword_file | (built-in) |
| group_concat_max_len | 1024 |
| have_archive | NO |
| have_bdb | YES |
| have_blackhole_engine | NO |
| have_compress | YES |
| have_crypt | YES |
| have_csv | NO |
| have_example_engine | NO |
| have_geometry | YES |
| have_innodb | YES |
| have_isam | YES |
| have_ndbcluster | NO |
| have_openssl | YES |
| have_query_cache | YES |
| have_raid | NO |
| have_rtree_keys | YES |
| have_symlink | YES |
| init_connect | |
| init_file | |
| init_slave | |
| innodb_additional_mem_pool_size | 1048576 |
| innodb_autoextend_increment | 8 |
| innodb_buffer_pool_awe_mem_mb | 0 |
| innodb_buffer_pool_size | 8388608 |
| innodb_data_file_path | ibdata1:10M:autoextend |
| innodb_data_home_dir | |
| innodb_fast_shutdown | ON |
| innodb_file_io_threads | 4 |
| innodb_file_per_table | OFF |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_flush_method | |
| innodb_force_recovery | 0 |
| innodb_lock_wait_timeout | 50 |
| innodb_locks_unsafe_for_binlog | OFF |
| innodb_log_arch_dir | |
| innodb_log_archive | OFF |
| innodb_log_buffer_size | 1048576 |
| innodb_log_file_size | 5242880 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | ./ |
| innodb_max_dirty_pages_pct | 90 |
| innodb_max_purge_lag | 0 |
| innodb_mirrored_log_groups | 1 |
| innodb_open_files | 300 |
| innodb_table_locks | ON |
| innodb_thread_concurrency | 8 |
| interactive_timeout | 28800 |
| join_buffer_size | 131072 |
| key_buffer_size | 8388600 |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| language | /usr/share/mysql/english/ |
| large_files_support | ON |
| license | GPL |
| local_infile | ON |
| locked_in_memory | OFF |
| log | OFF |
| log_bin | OFF |
| log_error | |
| log_slave_updates | OFF |
| log_slow_queries | OFF |
| log_update | OFF |
| log_warnings | 1 |
| long_query_time | 10 |
| low_priority_updates | OFF |
| lower_case_file_system | OFF |
| lower_case_table_names | 0 |
| max_allowed_packet | 1048576 |
| max_binlog_cache_size | 4294967295 |
| max_binlog_size | 1073741824 |
| max_connect_errors | 10 |
| max_connections | 100 |
| max_delayed_threads | 20 |
| max_error_count | 64 |
| max_heap_table_size | 134216704 |
| max_insert_delayed_threads | 20 |
| max_join_size | 4294967295 |
| max_length_for_sort_data | 1024 |
| max_prepared_stmt_count | 16382 |
| max_relay_log_size | 0 |
| max_seeks_for_key | 4294967295 |
| max_sort_length | 1024 |
| max_tmp_tables | 32 |
| max_user_connections | 0 |
| max_write_lock_count | 4294967295 |
| myisam_data_pointer_size | 4 |
| myisam_max_extra_sort_file_size | 2147483648 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_recover_options | OFF |
| myisam_repair_threads | 1 |
| myisam_sort_buffer_size | 8388608 |
| myisam_stats_method | nulls_unequal |
| net_buffer_length | 16384 |
| net_read_timeout | 30 |
| net_retry_count | 10 |
| net_write_timeout | 60 |
| new | OFF |
| old_passwords | ON |
| open_files_limit | 1024 |
| pid_file | /var/run/mysqld/mysqld.pid |
| port | 3306 |
| preload_buffer_size | 32768 |
| prepared_stmt_count | 0 |
| protocol_version | 10 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 0 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
| range_alloc_block_size | 2048 |
| read_buffer_size | 131072 |
| read_only | OFF |
| read_rnd_buffer_size | 262144 |
| relay_log_purge | ON |
| relay_log_space_limit | 0 |
| rpl_recovery_rank | 0 |
| secure_auth | OFF |
| server_id | 0 |
| skip_external_locking | ON |
| skip_networking | OFF |
| skip_show_database | OFF |
| slave_net_timeout | 3600 |
| slave_transaction_retries | 0 |
| slow_launch_time | 2 |
| socket | /var/lib/mysql/mysql.sock |
| sort_buffer_size | 2097144 |
| sql_mode | |
| sql_notes | ON |
| sql_warnings | ON |
| storage_engine | MyISAM |
| sync_binlog | 0 |
| sync_frm | ON |
| sync_replication | 0 |
| sync_replication_slave_id | 0 |
| sync_replication_timeout | 0 |
| system_time_zone | CEST |
| table_cache | 64 |
| table_type | MyISAM |
| thread_cache_size | 0 |
| thread_stack | 196608 |
| time_format | %H:%i:%s |
| time_zone | SYSTEM |
| tmp_table_size | 134217728 |
| tmpdir | |
| transaction_alloc_block_size | 8192 |
| transaction_prealloc_size | 4096 |
| tx_isolation | REPEATABLE-READ |
| version | 4.1.20 |
| version_bdb | Sleepycat Software: Berkeley DB 4.1.24: (May 24, 2006) |
| version_comment | Source distribution |
| version_compile_machine | i686 |
| version_compile_os | redhat-linux-gnu |
| wait_timeout | 28800 |
+---------------------------------+--------------------------------------------------------+

Going to disable other plugins, one by one.
Stay tuned!

TheWitness · Posted: Wed Jul 25, 2007 7:08 am Post subject:

Ok, good. The next step will be to add some additional logging and disable some code sections to bring forward the issue. What MySQL version?

TheWitness

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

TheWitness · Posted: Wed Jul 25, 2007 10:10 pm Post subject:

Hmm. Look at your log and look for SQL Errors. If you get any, post them.

Larry

TheWitness · Posted: Wed Jul 25, 2007 10:12 pm Post subject:

This issue may be related to "HOW" you MySQL installation buggered up the NULL's that I had setup in the database. You system converted them to ''

So, on my system a column is defined with a defult of NULL on yours the default was ''.

Sounds like MySQL is hosing things up. So you may not get an error message. Look into the MySQL variable "sql_mode".

TheWitness

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

Sounds fair to me, but:
---------------------------------------------
mysql> set @@SQL_MODE=NULL;
ERROR 1231 (42000): Variable 'sql_mode' can't be set to the value of 'NULL'
mysql> SELECT @@global.sql_mode;
+-------------------+
| @@global.sql_mode |
+-------------------+
| |
+-------------------+
1 row in set (0.00 sec)

mysql> SELECT @@session.sql_mode;
+--------------------+
| @@session.sql_mode |
+--------------------+
| |
+--------------------+
1 row in set (0.00 sec)

mysql>

..... what does your system say when you issue the command (in mysql)?:

SELECT @@global.sql_mode;
SELECT @@session.sql_mode;

- Dritz

Dr.Dritz · Joined: 02 Apr 2003 Posts: 42 Location: Southern Denmark

Hi again,
My problem is solved !

I have upgraded to
Cacti Version 0.8.6j, Plugin Architecture 1.1 and relevant patches.

I don