Developing Information Schema Plugins

Developing INFORMATION_SCHEMA Plugins Mark Leith Support Manager, Americas [email_address]

Overview INFORMATION_SCHEMA overview Plugin Overview Developing a simple Hello World I_S table Building and installing Interacting with another program Interacting with another library

INFORMATION_SCHEMA Specified within the SQL Standard (ISO/IEC 9075-11:2003) ‏ Virtual tables which give database metadata MySQL implements a subset of the Standard http://dev.mysql.com/doc/refman/5.1/en/information-schema.html http://www. xcdsql .org/MySQL/information_schema/5.1/MySQL_5_1_INFORMATION_SCHEMA.html

Plugins A means to load shared libraries in to a running MySQL instance Developed in C/C++ http://dev.mysql.com/doc/refman/5.1/en/plugin-api.html User Defined Functions Storage Engines Fulltext parsers Daemons INFORMATION_SCHEMA Tables!

Located in include/mysql/plugin.h The Plugin Interface struct st_mysql_plugin { int type; /* the plugin type (a MYSQL_XXX_PLUGIN value) */ void *info; /* pointer to type-specific plugin descriptor */ const char *name; /* plugin name */ const char *author; /* plugin author (for SHOW PLUGINS) */ const char *descr; /* general descriptive text (for SHOW PLUGINS ) */ int license; /* the plugin license (PLUGIN_LICENSE_XXX) */ int (*init)( void *); /* the function to invoke when plugin is loaded */ int (*deinit)( void *) /* the function to invoke when plugin is unloaded */ unsigned int version; /* plugin version (for SHOW PLUGINS) */ struct st_mysql_show_var *status_vars; struct st_mysql_sys_var **system_vars; void * __reserved1; /* reserved for dependency checking */ };

Plugin Types Define the plugin type (types listed below) Used within plugin functions such as plugin_foreach() MYSQL_UDF_PLUGIN 0 /* User-defined function */ MYSQL_STORAGE_ENGINE_PLUGIN 1 /* Storage Engine */ MYSQL_FTPARSER_PLUGIN 2 /* Full-text parser plugin */ MYSQL_DAEMON_PLUGIN 3 /* The daemon/raw plugin type */ MYSQL_INFORMATION_SCHEMA_PLUGIN 4 /* The I_S plugin type */ I_S plugins loaded within sql/sql_show.cc int schema_tables_add(THD *thd, List<LEX_STRING> *files, const char *wild) { … if (plugin_foreach(thd, add_schema_table, MYSQL_INFORMATION_SCHEMA_PLUGIN, &add_data)) DBUG_RETURN( 1 );

Starting a HELLO_WORLD Table Must have MySQL source tree Create a new plugin/hello_world/hello_world.cc file plugin.h and mysql_priv.h includes required Add a prototype for schema_table_store_record() /* Required for schema_table_store_record() * / #include "mysql_priv.h" #include <mysql/plugin.h> bool schema_table_store_record(THD *thd, TABLE *table);

Define the Table Structure Uses an array of ST_FIELD_INFO structs field_name Column name field_length Column length or display length field_type Column datatype value Not used within I_S plugins field_flags Set NULL / UNSIGNED attributes old_name Internal mapping for I_S tables to SHOW output open_method Open table using supplied method Last entry in the array is an end marker ST_FIELD_INFO hello_world_fields[]= { { "HELLO" , 10 , MYSQL_TYPE_STRING, 0 , 0 , "Hello" , 0 }, { "WORLD" , 10 , MYSQL_TYPE_STRING, 0 , 0 , "World" , 0 }, { 0 , 0 , MYSQL_TYPE_NULL, 0 , 0 , 0 , 0 } };

A Closer Look Field_type Field_flags Open_method VARCHAR INT BIGINT DECIMAL DATETIME MYSQL_TYPE_STRING MYSQL_TYPE_LONG MYSQL_TYPE_LONGLONG MYSQL_TYPE_DECIMAL MYSQL_TYPE_DATETIME MY_I_S_MAYBE_NULL MY_I_S_UNSIGNED SKIP_OPEN_TABLE OPEN_FRM_ONLY OPEN_FULL_TABLE

Define Function to Fill Table Called every time the table is accessed Fills the virtual table with data int fill_hello_world(THD *thd, TABLE_LIST *tables, COND *cond) { DBUG_ENTER( "fill_hello_world_is_plugin" ); CHARSET_INFO *scs= system_charset_info; TABLE *table= tables->table; int rc= 0 ; table->field[ 0 ]->store( "Hello" , strlen( "Hello" ), scs); table->field[ 1 ]->store( "World" , strlen( "World" ), scs); if (schema_table_store_record(thd, table)) rc= 1 ; DBUG_RETURN(rc); }

Create the Init Function Runs when the plugin is installed/loaded ST_SCHEMA_TABLE is an internal table representation Points to the ST_FIELD_INFO struct / table definition Points to the function to fill the table Manages any other init needed for the plugin int hello_world_plugin_init( void *p) { DBUG_ENTER( "init_hello_world_is_plugin" ); ST_SCHEMA_TABLE *schema= (ST_SCHEMA_TABLE*) p; schema->fields_info= hello_world_fields; schema->fill_table= fill_hello_world; DBUG_RETURN( 0 ); }

Create the Deinit Function Run when the plugin is unloaded Does nothing for HELLO_WORLD Use this function to do any clean up in your plugin int hello_world_plugin_deinit( void *p) { DBUG_ENTER( "deinit_info_schema_example_plugin" ); DBUG_RETURN( 0 ); }

Create the Plugin Definition Info struct points to the interface version built against struct st_mysql_information_schema hello_world_plugin_info= { MYSQL_INFORMATION_SCHEMA_INTERFACE_VERSION }; mysql_declare_plugin(hello_world_plugin) { MYSQL_INFORMATION_SCHEMA_PLUGIN, &hello_world_plugin_info, /* Pointer to info struct */ "HELLO_WORLD" , /* Plugin Name (used in INSTALL PLUGIN) */ "Mark Leith, MySQL AB" , /* Plugin Author */ "HELLO_WORLD example plugin" , /* Plugin Description */ PLUGIN_LICENSE_GPL, /* _GPL, _BSD or _PROPRIETARY */ hello_world_plugin_init, /* Pointer to plugin init function */ hello_world_plugin_deinit, /* Pointer to plugin deinit function */ 0x0100 , /* 1.0 */ NULL , /* status variables */ NULL , /* system variables */ NULL /* config options */ } mysql_declare_plugin_end;

Build the Plugin Define MYSQL_DYNAMIC_PLUGIN Make shared Include the sql and include source directories g++ -DMYSQL_DYNAMIC_PLUGIN -shared > -I/home/leithal/mysql/mysql-5.1/include > -I/home/leithal/mysql/mysql-5.1/sql > -o is_hello_world.so hello_world.cc ls -l total 16 -rw-r--r-- 1 leithal leithal 1712 2008-04-11 14:20 hello_world.cc -rwxr-xr-x 1 leithal leithal 8226 2008-04-11 14:27 is_hello_world.so cp is_hello_world.so /usr/local/mysql/lib/mysql/plugin

Install and Use! mysql> INSTALL PLUGIN HELLO_WORLD SONAME 'is_hello_world.so'; Query OK, 0 rows affected (0.01 sec) mysql> USE INFORMATION_SCHEMA; Database changed mysql> SHOW TABLES LIKE 'HELL%'; +--------------------------------------+ | Tables_in_information_schema (HELL%) | +--------------------------------------+ | HELLO_WORLD | +--------------------------------------+ 1 row in set (0.00 sec) mysql> SELECT * FROM HELLO_WORLD; +-------+-------+ | HELLO | WORLD | +-------+-------+ | Hello | World | +-------+-------+ 1 row in set (0.00 sec)

So We'll Do Something Interesting How's the disk space doing on my db server? root@achilles:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/hda1 4.6G 3.5G 886M 81% / varrun 126M 84K 125M 1% /var/run varlock 126M 0 126M 0% /var/lock procbususb 10M 76K 10M 1% /proc/bus/usb udev 10M 76K 10M 1% /dev devshm 126M 0 126M 0% /dev/shm lrm 126M 18M 108M 14% /lib/modules/2.6.17-12-generic/volatile If I can get this in a table I can track it over time I can also report on it easily How about an I_S table with events to catalog?

Create Template & Table Structure Add stdio.h,string.h and stdlib.h includes Mirror init, deinit and info functions Fill in the plugin declaration scruct appropriately Define the new table: ST_FIELD_INFO fs_info_schema_fields[]= { { "FILESYSTEM" , 120 , MYSQL_TYPE_STRING, 0 , 0 , "Filesystem" , 0 }, { "SIZE" , 8 , MYSQL_TYPE_LONGLONG, 0 , 0 , "Mountpoint Size" , 0 }, { "USED" , 8 , MYSQL_TYPE_LONGLONG, 0 , 0 , "Used Space" , 0 }, { "AVAILABLE" , 8 , MYSQL_TYPE_LONGLONG, 0 , 0 , "Available Space" , 0 }, { "CAPACITY" , 4 , MYSQL_TYPE_STRING, 0 , 0 , "Percent Used" , 0 }, { "MOUNTED_ON" , 120 , MYSQL_TYPE_STRING, 0 , 0 , "Filesystem Mounted On" , 0 }, { 0 , 0 , MYSQL_TYPE_STRING, 0 , 0 , 0 , 0 } };

Running Other Programs Open with popen() and read in the results /* get filesystem information from df on linux like systems */ FILE *f; char buf[ 128 ]; char *c, *s, *size; const char delim[]= " " ; unsigned long long uli; double d; if ( NULL != (f= popen( "/bin/df -h" , "r" ))) { int ln= 0 ; while (!feof(f)) { fgets(buf, sizeof (buf), f); switch (ln++) {

Tokenize and Push Results Can do this how you like /* skip the header line of df */ case 0 : break ; default : c= buf; /* hack to stop processing when falling off the end of output */ if (strchr(c, ' ' ) == NULL ) break ; /* Filesystem */ s= strtok(c, delim); table->field[ 0 ]->store(s, strlen(s), scs); /* Size */ s= strtok( NULL , delim); d= strtod(s, &size); uli= get_bytes(d, size); table->field[ 1 ]->store(uli, TRUE ); … .

Store the Row and Finish up /* Use% */ s= strtok( NULL , delim); table->field[ 4 ]->store(s, strlen(s), scs); /* Mountpoint */ s= strtok( NULL , delim); stripnl(s); table->field[ 5 ]->store(s, strlen(s), scs); /* store the row */ if (schema_table_store_record(thd, table)) rc= 1 ; break ; } } pclose(f); } else rc= 1 ; DBUG_RETURN(rc); }

Build, Install, Try.. Build as before and try it out! mysql> select * from file_system_mountpoints; +------------+------------+------------+-----------+----------+-----------------------------------------+ | FILESYSTEM | SIZE | USED | AVAILABLE | CAPACITY | MOUNTED_ON | +------------+------------+------------+-----------+----------+-----------------------------------------+ | /dev/hda1 | 4939212390 | 3758096384 | 930086912 | 81% | / | | varrun | 132120576 | 86016 | 131072000 | 1% | /var/run | | varlock | 132120576 | 0 | 132120576 | 0% | /var/lock | | procbususb | 10485760 | 77824 | 10485760 | 1% | /proc/bus/usb | | udev | 10485760 | 77824 | 10485760 | 1% | /dev | | devshm | 132120576 | 0 | 132120576 | 0% | /dev/shm | | lrm | 132120576 | 18874368 | 113246208 | 14% | /lib/modules/2.6.17-12-generic/volatile | +------------+------------+------------+-----------+----------+-----------------------------------------+ 7 rows in set (0.06 sec) mysql> desc file_system_mountpoints; +------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------+--------------+------+-----+---------+-------+ | FILESYSTEM | varchar(120) | NO | | | | | SIZE | bigint(11) | NO | | 0 | | | USED | bigint(11) | NO | | 0 | | | AVAILABLE | bigint(11) | NO | | 0 | | | CAPACITY | varchar(4) | NO | | | | | MOUNTED_ON | varchar(120) | NO | | | | +------------+--------------+------+-----+---------+-------+ 6 rows in set (0.01 sec)

Now Let's Use another Library The SIGAR library from Hyperic Lets us do cross platform OS stats monitoring Released under GPL Again, start with the same base functions etc. as before

Include Lib Header, Use Lib in Fill sigar_t *t; sigar_open(&t); sigar_file_system_list_t fslist; sigar_file_system_list_get(t, &fslist); for (uint i = 0 ; i < fslist.number; i++) { sigar_file_system_t fs = fslist.data[i]; sigar_file_system_usage_t fsusage; rc= sigar_file_system_usage_get(t, fs.dir_name, &fsusage); if (fs.type == 2 || fs.type == 3 ) { table->field[ 0 ]->store(fs.dir_name, strlen(fs.dir_name), scs); table->field[ 1 ]->store(fsusage.total, TRUE ); table->field[ 2 ]->store(fsusage.used, TRUE ); table->field[ 3 ]->store(fsusage.free, TRUE ); table->field[ 4 ]->store(fsusage.files, TRUE ); if (schema_table_store_record(thd, table)) rc= 1 ; } } sigar_file_system_list_destroy(t, &fslist); sigar_close(t);

Building and Including the Library -lsigar links the library Use -rpath to pass in library location This is for runtime Can also use LD_LIBRARY_PATH -L (linker) and -I (include) paths -L points to the directory with the built library g++ -DMYSQL_DYNAMIC_PLUGIN -Wall -shared -lsigar -Wl,-rpath -Wl,/home/leithal/os_stats_info_schema/sigar/ -L/home/leithal/os_stats_info_schema/sigar/ -I/home/leithal/os_stats_info_schema/sigar/include/ -I/home/leithal/mysql/mysql-5.1/include -I/home/leithal/mysql/mysql-5.1/sql -o os_stats_info_schema.so os_stats_info_schema.cc

Try it out! mysql> INSTALL PLUGIN os_disk_usage SONAME 'os_stats_info_schema.so'; Query OK, 0 rows affected (0.00 sec) mysql> SHOW TABLES LIKE 'OS%'; +------------------------------------+ | Tables_in_information_schema (OS%) | +------------------------------------+ | OS_DISK_USAGE | +------------------------------------+ 1 row in set (0.00 sec) mysql> SELECT * FROM OS_DISK_USAGE; +------------+---------+---------+---------+--------+ | FILESYSTEM | SIZE | USED | FREE | FILES | +------------+---------+---------+---------+--------+ | / | 4799024 | 3656800 | 1142224 | 610432 | +------------+---------+---------+---------+--------+ 1 row in set (0.00 sec)

Resources and Questions! http://www.markleith.co.uk/?p=18 http://rpbouman.blogspot.com/2008/02/mysql-information-schema-plugins-best.html http://rpbouman.blogspot.com/2008/02/reporting-mysql-internals-with.html (Great resource for monitoring server internals) Questions?

Developing Information Schema Plugins

More Related Content

What's hot

What's hot (20)

Similar to Developing Information Schema Plugins

Similar to Developing Information Schema Plugins (20)

Recently uploaded

Recently uploaded (20)

Developing Information Schema Plugins