|
|
| Applications |
| Mailing Lists |
| Miscellaneous |
| Operating Systems |
| Programming |
bk commit into 5.2 tree (cbell:1.2610) BUG#31383
Date: Mon Oct 29 2007 - 09:46:54 EDT
Below is the list of changes that have just been committed into a local
5.2 repository of cbell. When cbell does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html
ChangeSet@1.2610, 2007-10-29 09:46:18-04:00, cbell@mysql_cab_desk. +15 -0 BUG#31383 : Consistent Snapshot driver not consistent
This patch changes the default driver to use a separate thread to open and lock tables. The default driver opens tables on the prelock() call from the kernel. The snapshot driver initiates the CS read on the lock() call from the kernel and opens tables in the first call to get_data() after the lock is taken. This is due to the fact that the default driver's validity point is at open_and_lock_tables() while the snapshot driver's validity point is at the start of the transaction.
mysql-test/r/backup_snapshot.result@1.3, 2007-10-29 09:46:04-04:00, cbell@mysql_cab_desk. +75 -10 BUG#31383 : Consistent Snapshot driver not consistent
New result file.
mysql-test/t/backup_snapshot.test@1.3, 2007-10-29 09:46:04-04:00, cbell@mysql_cab_desk. +109 -9 BUG#31383 : Consistent Snapshot driver not consistent
Test modified to use the get_lock, release_lock functions to stop the CS driver in progress allowing simultaneous inserts from the second connection. This patch makes this test accurately test the CS driver.
sql/backup/CMakeLists.txt@1.2, 2007-10-29 09:46:07-04:00, cbell@mysql_cab_desk. +4 -1 BUG#31383 : Consistent Snapshot driver not consistent
Added the new be_thread source file and dependency for backup.
sql/backup/Makefile.am@1.10, 2007-10-29 09:46:07-04:00, cbell@mysql_cab_desk. +4 -2 BUG#31383 : Consistent Snapshot driver not consistent
Added the new be_thread source file and dependency for backup.
sql/backup/backup_engine.h@1.3, 2007-10-29 09:46:08-04:00, cbell@mysql_cab_desk. +2 -0 BUG#31383 : Consistent Snapshot driver not consistent
Added the tables_in_backup and lock_thd variables so that it can be accessed from external threads given a pointer to the default or consistent snapshot driver.
sql/backup/be_default.cc@1.9, 2007-10-29 09:46:09-04:00, cbell@mysql_cab_desk. +48 -0 BUG#31383 : Consistent Snapshot driver not consistent
Added new methods to support using a separate thread to open and lock tables for backup.
sql/backup/be_default.h@1.7, 2007-10-29 09:46:09-04:00, cbell@mysql_cab_desk. +26 -28 BUG#31383 : Consistent Snapshot driver not consistent
Added new methods to support using a separate thread to open and lock tables for backup.
sql/backup/be_snapshot.cc@1.4, 2007-10-29 09:46:10-04:00, cbell@mysql_cab_desk. +25 -17 BUG#31383 : Consistent Snapshot driver not consistent
Modifies the CS driver to open and close its own tables while executing in the kernel's thread.
sql/backup/be_snapshot.h@1.4, 2007-10-29 09:46:11-04:00, cbell@mysql_cab_desk. +6 -3 BUG#31383 : Consistent Snapshot driver not consistent
Added methods and variables for opening and closing tables.
sql/backup/be_thread.cc@1.1, 2007-10-29 09:46:13-04:00, cbell@mysql_cab_desk. +176 -0 BUG#31383 : Consistent Snapshot driver not consistent
Added new error message for error handling in threads in default and snapshot drivers.
sql/backup/be_thread.cc@1.0, 2007-10-29 09:46:13-04:00, cbell@mysql_cab_desk. +0 -0
sql/backup/be_thread.h@1.1, 2007-10-29 09:46:13-04:00, cbell@mysql_cab_desk. +39 -0 BUG#31383 : Consistent Snapshot driver not consistent
New source file for mutex initialization and helper methods for using a thread to open and lock tables in default and snapshot drivers.
sql/backup/be_thread.h@1.0, 2007-10-29 09:46:13-04:00, cbell@mysql_cab_desk. +0 -0
sql/backup/data_backup.cc@1.11, 2007-10-29 09:46:11-04:00, cbell@mysql_cab_desk. +0 -31 BUG#31383 : Consistent Snapshot driver not consistent
Removed code to call open and lock tables from kernel.
sql/mysqld.cc@1.660, 2007-10-29 09:46:05-04:00, cbell@mysql_cab_desk. +9 -0 BUG#31383 : Consistent Snapshot driver not consistent
Adds new thread mutex and condition variables for backup.
sql/share/errmsg.txt@1.112, 2007-10-29 09:46:12-04:00, cbell@mysql_cab_desk. +3 -0 BUG#31383 : Consistent Snapshot driver not consistent
Added new error message for error handling in threads in the default driver.
sql/sql_class.h@1.396, 2007-10-29 09:46:06-04:00, cbell@mysql_cab_desk. +2 -1 BUG#31383 : Consistent Snapshot driver not consistent
Added a new thread for backup.
diff -Nrup a/mysql-test/r/backup_snapshot.result b/mysql-test/r/backup_snapshot.result
--- a/mysql-test/r/backup_snapshot.result 2007-07-02 13:42:57 -04:00@@ -11,21 +11,73 @@ INSERT INTO bup_snapshot.t1 VALUES ("07
+++ b/mysql-test/r/backup_snapshot.result 2007-10-29 09:46:04 -04:00
INSERT INTO bup_snapshot.t1 VALUES ("08 Some data to test");
INSERT INTO bup_snapshot.t1 VALUES ("09 Some data to test");
INSERT INTO bup_snapshot.t1 VALUES ("10 Some data to test");
+CREATE TABLE bup_snapshot.t2 (a int) ENGINE=CSV;
+INSERT INTO bup_snapshot.t2 VALUES (1), (2), (3), (4), (5);
con1: Show that the new data doesn't exist before backup. SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%'; word
SELECT COUNT(*) FROM bup_snapshot.t1;
COUNT(*)
10
-con1: Backing up database.
+SELECT COUNT(*) FROM bup_snapshot.t2;
+COUNT(*)
+5
+con2: Getting lock on driver.
+SELECT get_lock("backup_cs_locked", 100);
+get_lock("backup_cs_locked", 100)
+1
+con1: Backing up database. Spawn this and continue...
BACKUP DATABASE bup_snapshot TO "bup_snapshot.bak";
+con2: Wait until backup pauses then insert new data.
+INSERT INTO bup_snapshot.t1 VALUES("- Dave Mathews");
+INSERT INTO bup_snapshot.t1 VALUES("- Yes");
+INSERT INTO bup_snapshot.t1 VALUES("- Jethro Tull");
+DELETE FROM bup_snapshot.t1 WHERE word LIKE '10%';
+con2: Showing the data after inserts.
+SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
+word
+- Dave Mathews
+- Yes
+- Jethro Tull
+SELECT COUNT(*) FROM bup_snapshot.t1;
+COUNT(*)
+12
+con2: Release lock on driver.
+SELECT release_lock("backup_cs_locked");
+release_lock("backup_cs_locked")
+1
Backup Summary
- header = 22 bytes - meta-data = 95 bytes - data = 260 bytes+con1: Dropping the database
+ header = 29 bytes
+ meta-data = 181 bytes
+ data = 310 bytes
-------------- - total 377 bytes -con2: Inserting new data.
+ total 520 bytes
+DROP TABLE bup_snapshot.t1;
+con1: Restoring the database
+RESTORE FROM "bup_snapshot.bak";
+Restore Summary
+ header = 29 bytes
+ meta-data = 181 bytes
+ data = 310 bytes
+ --------------
+ total 520 bytes
+con1: Showing the data (no new data should be here).
+SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
+word
+SELECT COUNT(*) FROM bup_snapshot.t1;
+COUNT(*)
+10
+SELECT COUNT(*) FROM bup_snapshot.t2;
+COUNT(*)
+5
+con2: Getting lock on driver.
+SELECT get_lock("backup_cs_reading", 100);
+get_lock("backup_cs_reading", 100)
+1
+con1: Backing up database. Spawn this and continue...
+BACKUP DATABASE bup_snapshot TO "bup_snapshot.bak";
+con2: Wait until backup pauses then insert new data.
INSERT INTO bup_snapshot.t1 VALUES("- Dave Mathews");
INSERT INTO bup_snapshot.t1 VALUES("- Yes");
INSERT INTO bup_snapshot.t1 VALUES("- Jethro Tull");
@@ -39,20 +91,33 @@ word
SELECT COUNT(*) FROM bup_snapshot.t1;
COUNT(*)
12
+con2: Release lock on driver.
+SELECT release_lock("backup_cs_reading");
+release_lock("backup_cs_reading")
+1
+Backup Summary
+ header = 29 bytes
+ meta-data = 181 bytes
+ data = 310 bytes
+ --------------
+ total 520 bytes
con1: Dropping the database
DROP TABLE bup_snapshot.t1;
con1: Restoring the database
RESTORE FROM "bup_snapshot.bak";
Restore Summary
- header = 22 bytes - meta-data = 95 bytes - data = 260 bytes
+ header = 29 bytes
+ meta-data = 181 bytes
+ data = 310 bytes
-------------- - total 377 bytes
+ total 520 bytes
con1: Showing the data (no new data should be here). SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%'; word
SELECT COUNT(*) FROM bup_snapshot.t1;
COUNT(*)
10
+SELECT COUNT(*) FROM bup_snapshot.t2;
+COUNT(*)
+5
DROP DATABASE bup_snapshot;
diff -Nrup a/mysql-test/t/backup_snapshot.test b/mysql-test/t/backup_snapshot.test
--- a/mysql-test/t/backup_snapshot.test 2007-07-02 13:42:57 -04:00@@ -3,13 +3,23 @@
+++ b/mysql-test/t/backup_snapshot.test 2007-10-29 09:46:04 -04:00
# The test is designed to show that a consistent snapshot # backup can be taken while data is being inserted and deleted. # -# TODO -# - Make the test run the insert statements in parallel with the backup -# command using --send and --reap and signals from backup code.+# The following tests test these conditions.
+# The test is testing the driver to ensure it is entering a
+# consistent read state during the backup. There are several
+# breakpoints in the code that can be used. The two most
+# useful ones are:
+#
+# backup_cs_unlock - occurs after the lock and before
+# the open and lock tables.
+#
+# backup_cs_reading - occurs after the open and lock
+# tables during the read tables portion.
+#
#
--source include/have_innodb.inc --source include/have_debug.inc
+--source.include/have_csv.inc
--disable_warnings
DROP DATABASE IF EXISTS bup_snapshot;
@@ -22,8 +32,11 @@ connect (con2,localhost,root,,);
connection con1;
-# Create a table and load it with lots of data.+# Setup for tests.
+#
+#
+# Create a table and load it with lots of data.
CREATE TABLE bup_snapshot.t1 (word CHAR(20)) ENGINE=INNODB;
INSERT INTO bup_snapshot.t1 VALUES ("01 Some data to test"); @@ -37,19 +50,42 @@ INSERT INTO bup_snapshot.t1 VALUES ("08 INSERT INTO bup_snapshot.t1 VALUES ("09 Some data to test"); INSERT INTO bup_snapshot.t1 VALUES ("10 Some data to test");
+# Use a non-CS supported table to show driver can coexist with default driver
+CREATE TABLE bup_snapshot.t2 (a int) ENGINE=CSV;
+INSERT INTO bup_snapshot.t2 VALUES (1), (2), (3), (4), (5);
+
--echo con1: Show that the new data doesn't exist before backup.
SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
SELECT COUNT(*) FROM bup_snapshot.t1;
+SELECT COUNT(*) FROM bup_snapshot.t2;
+
+connection con2;
+
+#
+# Test 1: Check for consistent read prior to open and lock tables
+#
+
+--echo con2: Getting lock on driver.
+SELECT get_lock("backup_cs_locked", 100);
# While a consistent snapshot backup is executed, # no external inserts should be visible to the transaction.
---echo con1: Backing up database.
-BACKUP DATABASE bup_snapshot TO "bup_snapshot.bak";
+connection con1;
+
+--echo con1: Backing up database. Spawn this and continue...
+send BACKUP DATABASE bup_snapshot TO "bup_snapshot.bak";
connection con2;
---echo con2: Inserting new data.
+--echo con2: Wait until backup pauses then insert new data.
+
+# Must wait to know when backup has entered lock.
+let $wait_condition = SELECT state = "debug_sync_point: backup_cs_locked"
+ FROM INFORMATION_SCHEMA.PROCESSLIST
+ WHERE info LIKE "backup database %";
+--source include/wait_condition.inc
+
INSERT INTO bup_snapshot.t1 VALUES("- Dave Mathews");
INSERT INTO bup_snapshot.t1 VALUES("- Yes");
INSERT INTO bup_snapshot.t1 VALUES("- Jethro Tull");
@@ -59,8 +95,13 @@ DELETE FROM bup_snapshot.t1 WHERE word L
SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
SELECT COUNT(*) FROM bup_snapshot.t1;
+--echo con2: Release lock on driver.
+SELECT release_lock("backup_cs_locked");
+
connection con1;
+reap;
+
# Now restore the database and then check to make sure the new rows
# were not backed up.
@@ -73,9 +114,68 @@ RESTORE FROM "bup_snapshot.bak";
--echo con1: Showing the data (no new data should be here).
SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
SELECT COUNT(*) FROM bup_snapshot.t1;
+SELECT COUNT(*) FROM bup_snapshot.t2;
-DROP DATABASE bup_snapshot;
+remove_file $MYSQLTEST_VARDIR/master-data/bup_snapshot.bak;
+
+#
+# Test 2: Check for consistent read after open and lock tables
+#
+
+connection con2;
+
+--echo con2: Getting lock on driver.
+SELECT get_lock("backup_cs_reading", 100);
+
+# While a consistent snapshot backup is executed,
+# no external inserts should be visible to the transaction.
+
+connection con1;
---exec rm $MYSQLTEST_VARDIR/master-data/bup_snapshot.bak
+--echo con1: Backing up database. Spawn this and continue...
+send BACKUP DATABASE bup_snapshot TO "bup_snapshot.bak";
+
+connection con2;
+
+--echo con2: Wait until backup pauses then insert new data.
+
+# Must wait to know when backup has entered lock.
+let $wait_condition = SELECT state = "debug_sync_point: backup_cs_reading"
+ FROM INFORMATION_SCHEMA.PROCESSLIST
+ WHERE info LIKE "backup database %";
+--source include/wait_condition.inc
+
+INSERT INTO bup_snapshot.t1 VALUES("- Dave Mathews");
+INSERT INTO bup_snapshot.t1 VALUES("- Yes");
+INSERT INTO bup_snapshot.t1 VALUES("- Jethro Tull");
+DELETE FROM bup_snapshot.t1 WHERE word LIKE '10%';
+
+--echo con2: Showing the data after inserts.
+SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
+SELECT COUNT(*) FROM bup_snapshot.t1;
+
+--echo con2: Release lock on driver.
+SELECT release_lock("backup_cs_reading");
+
+connection con1;
+
+reap;
+
+# Now restore the database and then check to make sure the new rows
+# were not backed up.
+
+--echo con1: Dropping the database
+DROP TABLE bup_snapshot.t1;
+
+--echo con1: Restoring the database
+RESTORE FROM "bup_snapshot.bak";
+
+--echo con1: Showing the data (no new data should be here).
+SELECT * FROM bup_snapshot.t1 WHERE word LIKE '-%';
+SELECT COUNT(*) FROM bup_snapshot.t1;
+SELECT COUNT(*) FROM bup_snapshot.t2;
+
+DROP DATABASE bup_snapshot;
+remove_file $MYSQLTEST_VARDIR/master-data/bup_snapshot.bak;
diff -Nrup a/sql/backup/CMakeLists.txt b/sql/backup/CMakeLists.txt --- a/sql/backup/CMakeLists.txt 2007-10-03 12:56:57 -04:00
+++ b/sql/backup/CMakeLists.txt 2007-10-29 09:46:07 -04:00
@@ -25,8 +25,11 @@ INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/ SET(BACKUP_SOURCES stream.cc logger.cc string_pool.cc archive.cc meta_backup.cc data_backup.cc sql_backup.cc be_default.cc buffer_iterator.cc - be_snapshot.cc)
+ be_snapshot.cc be_thread.cc)
IF(NOT SOURCE_SUBLIBS)
ADD_LIBRARY(backup ${BACKUP_SOURCES})
ENDIF(NOT SOURCE_SUBLIBS)
+
+ADD_DEPENDENCIES(backup mysys)
+
diff -Nrup a/sql/backup/Makefile.am b/sql/backup/Makefile.am --- a/sql/backup/Makefile.am 2007-10-05 09:12:28 -04:00@@ -440,6 +440,8 @@ class Backup_driver: public Driver
+++ b/sql/backup/Makefile.am 2007-10-29 09:46:07 -04:00
@@ -33,7 +33,8 @@ libbackup_la_SOURCES = \ sql_backup.cc \ be_default.cc \ be_snapshot.cc \ - buffer_iterator.cc
+ buffer_iterator.cc \
+ be_thread.cc
noinst_HEADERS = \ api_types.h \ @@ -50,7 +51,8 @@ noinst_HEADERS = \ meta_backup.h \ be_default.h \ be_snapshot.h \ - buffer_iterator.h
+ buffer_iterator.h \
+ be_thread.h
DEFS = \ -DMYSQL_SERVER \ diff -Nrup a/sql/backup/backup_engine.h b/sql/backup/backup_engine.h --- a/sql/backup/backup_engine.h 2007-06-27 08:14:17 -04:00
+++ b/sql/backup/backup_engine.h 2007-10-29 09:46:08 -04:00
virtual size_t init_size() =0;
+ TABLE_LIST *tables_in_backup; ///< List of tables used in backup.
+ THD *lock_thd; ///< Locking thread pointer
}; // Backup_driver class
/**
diff -Nrup a/sql/backup/be_default.cc b/sql/backup/be_default.cc --- a/sql/backup/be_default.cc 2007-07-02 13:42:58 -04:00@@ -126,6 +127,24 @@ Backup::Backup(const Table_list &tables, }
+++ b/sql/backup/be_default.cc 2007-10-29 09:46:09 -04:00
@@ -116,6 +116,7 @@ Backup::Backup(const Table_list &tables, tbl_num= 0; /* set table number to 0 */ mode= INITIALIZE; /* initialize read */ lock_called= FALSE; /* lock has not been called */
+ lock_thd= 0; /* set lock thread to 0 */
/* Create a TABLE_LIST * list for iterating through the tables.
/**+ * prelock() (given that thr_lock() is blocking).
+ * @brief Prelock call to setup locking.
+ *
+ * Launches a separate thread ("locking thread") which will lock
+ * tables. Locking in a separate thread is needed to have a non-blocking
+ */
+result_t Backup::prelock()
+{
+ DBUG_ENTER("Default_backup::open_tables");
+
+ pthread_t th;
+ if (pthread_create(&th, &connection_attrib,
+ backup_thread_for_locking, this))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+/**
- @brief Start table read. *
- This method saves the handler for the table and initializes the
@@ -239,6 +258,20 @@ result_t Backup::get_data(Buffer &buf)
buf.size= 0;
buf.table_no= 0;
buf.last= TRUE;
+ if (lock_thd)
+ {
+ /*
+ Block until lock thread is complete.
+ */
+ pthread_mutex_lock(&THR_LOCK_driver_thread);
+ m_thd->enter_cond(&COND_driver_lock, &THR_LOCK_driver_thread,
+ "Default driver: waiting for table locks");
+ while (!backup_thread_locked())
+ pthread_cond_wait(&COND_driver_lock, &THR_LOCK_driver_thread);
+ m_thd->exit_cond("Default driver: got table locks");
+
+ lock_called= TRUE;
+ }
DBUG_RETURN(READY); }
@@ -304,6 +337,21 @@ result_t Backup::get_data(Buffer &buf)
buf.size= 0;
buf.last= TRUE;
mode= GET_NEXT_TABLE;
+
+ /*
+ Optimization: If this is the last table to read, close the tables and
+ kill the lock thread. This only applies iff we are using the thread.
+ */
+ if (tables_in_backup->next_global == NULL)
+ {
+ if (lock_thd)
+ {
+ lock_thd->awake(THD::KILL_CONNECTION);
+ pthread_cond_broadcast(&COND_driver_thread_wait);
+ }
+ cur_table= NULL;
+ lock_thd= 0;
+ }
}
else if (last_read_res != 0)
DBUG_RETURN(ERROR);
diff -Nrup a/sql/backup/be_default.h b/sql/backup/be_default.h
--- a/sql/backup/be_default.h 2007-07-02 13:42:58 -04:00
+++ b/sql/backup/be_default.h 2007-10-29 09:46:09 -04:00
@@ -6,6 +6,8 @@
#include "archive.h" #include "buffer_iterator.h" #include "backup_aux.h"
+#include "mysql_priv.h"
+#include "be_thread.h"
namespace default_backup {
@@ -85,25 +87,23 @@ class Backup: public Backup_driver
public:
enum has_data_info { YES, WAIT, EOD };
Backup(const Table_list &tables, THD *t_thd, thr_lock_type lock_type);
- virtual ~Backup() { backup::free_table_list(all_tables); };
+ virtual ~Backup() { backup::free_table_list(all_tables); };
size_t size() { return UNKNOWN_SIZE; };
size_t init_size() { return 0; };
- result_t begin(const size_t) { return backup::OK; };
+ result_t begin(const size_t) { return backup::OK; };
result_t end() { return backup::OK; };
result_t get_data(Buffer &buf);
- result_t lock()
- {
- lock_called= TRUE;
- return backup::OK;
- };
+ result_t lock() { return backup::OK; };
result_t unlock() { return backup::OK; };
result_t cancel() { return backup::OK; };
TABLE_LIST *get_table_list() { return all_tables; }
void free() { delete this; };
+ result_t prelock();
protected:
- my_bool lock_called; ///< Checks to see if locks have been reached. - THD *m_thd; ///< Pointer to current thread struct.};
+ my_bool lock_called; ///< Checks to see if locks have been reached.
+ THD *m_thd; ///< Pointer to current thread struct.
+ TABLE *cur_table; ///< The table currently being read.
private: /* @@ -112,30 +112,28 @@ class Backup: public Backup_driver set in the code as a response to conditions or flow of data. */ typedef enum { - INITIALIZE, ///< Indicates time to initialize read - GET_NEXT_TABLE, ///< Open next table in the list - READ_RCD, ///< Reading rows from table mode - READ_RCD_BUFFER, ///< Buffer records mode - CHECK_BLOBS, ///< See if record has blobs - READ_BLOB, ///< Reading blobs from record mode - READ_BLOB_BUFFER ///< Buffer blobs mode
+ INITIALIZE, ///< Indicates time to initialize read
+ GET_NEXT_TABLE, ///< Open next table in the list
+ READ_RCD, ///< Reading rows from table mode
+ READ_RCD_BUFFER, ///< Buffer records mode
+ CHECK_BLOBS, ///< See if record has blobs
+ READ_BLOB, ///< Reading blobs from record mode
+ READ_BLOB_BUFFER ///< Buffer blobs mode
} BACKUP_MODE; result_t start_tbl_read(TABLE *tbl); result_t end_tbl_read(); int next_table(); - BACKUP_MODE mode; ///< Indicates which mode the code is in - int tbl_num; ///< The index of the current table. - TABLE *cur_table; ///< The table currently being read. - handler *hdl; ///< Pointer to table handler. - uint *cur_blob; ///< The current blob field. - uint *last_blob_ptr; ///< Position of last blob field. - MY_BITMAP *read_set; ///< The file read set. - Buffer_iterator rec_buffer; ///< Buffer iterator for windowing records - Buffer_iterator blob_buffer; ///< Buffer iterator for windowing BLOB fields - byte *ptr; ///< Pointer to blob data from record. - TABLE_LIST *all_tables; ///< Reference to list of tables used. - TABLE_LIST *tables_in_backup; ///< List of tables used in backup.
+ BACKUP_MODE mode; ///< Indicates which mode the code is in
+ int tbl_num; ///< The index of the current table.
+ handler *hdl; ///< Pointer to table handler.
+ uint *cur_blob; ///< The current blob field.
+ uint *last_blob_ptr; ///< Position of last blob field.
+ MY_BITMAP *read_set; ///< The file read set.
+ Buffer_iterator rec_buffer; ///< Buffer iterator for windowing records
+ Buffer_iterator blob_buffer; ///< Buffer iterator for windowing BLOB fields
+ byte *ptr; ///< Pointer to blob data from record.
+ TABLE_LIST *all_tables; ///< Reference to list of tables used.
/**
diff -Nrup a/sql/backup/be_snapshot.cc b/sql/backup/be_snapshot.cc --- a/sql/backup/be_snapshot.cc 2007-07-02 13:42:59 -04:00
+++ b/sql/backup/be_snapshot.cc 2007-10-29 09:46:10 -04:00
@@ -77,20 +77,6 @@ result_t Engine::get_backup(const uint32
DBUG_RETURN(OK);
}
-/**
- * @brief End backup process.
- *
- * This method unlocks all of the tables.
- *
- * @retval backup::OK all tables unlocked.
- */
-result_t Backup::end()
-{
- DBUG_ENTER("Snapshot_backup::end");
- end_active_trans(m_thd);
- DBUG_RETURN(OK);
-}
-
result_t Backup::lock()
{
DBUG_ENTER("Snapshot_backup::lock()"); @@ -105,13 +91,35 @@ result_t Backup::lock()
if (res)
DBUG_RETURN(ERROR);
lock_called= TRUE;
+ BACKUP_SYNC("backup_cs_locked");
DBUG_RETURN(OK);
}
-result_t Backup::unlock()
+result_t Backup::get_data(Buffer &buf)
{
- DBUG_ENTER("Snapshot_backup::unlock()");
- DBUG_RETURN(OK);
+ result_t res;
+
+ if (!tables_open && lock_called)
+ {
+ BACKUP_SYNC("backup_cs_open_tables");
+ open_and_lock_tables(m_thd, tables_in_backup);
+ tables_open= TRUE;
+ }
+ if (lock_called)
+ BACKUP_SYNC("backup_cs_reading");
+
+ res= default_backup::Backup::get_data(buf);
+
+ /*
+ If this is the last table to be read, close the transaction
+ and unlock the tables.
+ */
+ if (cur_table == NULL)
+ {
+ end_active_trans(m_thd);
+ close_thread_tables(m_thd);
+ }
+ return(res);
}
/**
diff -Nrup a/sql/backup/be_snapshot.h b/sql/backup/be_snapshot.h --- a/sql/backup/be_snapshot.h 2007-07-02 13:42:59 -04:00{
+++ b/sql/backup/be_snapshot.h 2007-10-29 09:46:11 -04:00
@@ -59,13 +59,16 @@ class Backup: public default_backup::Bac
public:
Backup(const Table_list &tables, THD *t_thd):
- default_backup::Backup(tables, t_thd, TL_READ) {};
+ default_backup::Backup(tables, t_thd, TL_READ) { tables_open= FALSE; };
virtual ~Backup() {};
result_t begin(const size_t) { return backup::OK; };
- result_t end();
+ result_t end() { return backup::OK; };
+ result_t get_data(Buffer &buf);
result_t prelock() { return backup::READY; }
result_t lock();
- result_t unlock();
+ result_t unlock() { return backup::OK; };
+ private:
+ my_bool tables_open; ///< Indicates if tables are open
};
/**
diff -Nrup a/sql/backup/be_thread.cc b/sql/backup/be_thread.cc
--- /dev/null Wed Dec 31 16:00:00 196900@@ -0,0 +1,176 @@
+++ b/sql/backup/be_thread.cc 2007-10-29 09:46:13 -04:00
+/* Copyright (C) 2004-2007 MySQL AB
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; version 2 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+
+/**
+ * @file
+ *
+ * @brief Contains the thread methods for online backup.
+ *
+ * The methods in this class are used to initialize the mutexes
+ * for the backup threads. Helper methods are included to make thread
+ * calls easier for the driver code.
+ */
+
+#include "be_thread.h"
+
+my_bool backup_thread_is_locked= FALSE;
+
+/**
+ @brief Lock tables in driver.
+
+ This method creates a new THD for use in the new thread. It calls
+ the method to open and lock the tables.
+ */
+pthread_handler_t backup_thread_for_locking(void *arg)
+{
+ TABLE_LIST *tables_in_backup= 0;
+
+ DBUG_PRINT("info", ("Default_backup - lock_tables_in_separate_thread"));
+
+ /*
+ Turn off condition variable check for lock.
+ */
+ pthread_mutex_lock(&THR_LOCK_driver_thread);
+ backup_thread_is_locked= FALSE;
+ pthread_mutex_unlock(&THR_LOCK_driver_thread);
+
+ my_thread_init();
+ pthread_detach_this_thread();
+
+ /*
+ First, create a new THD object.
+ */
+ DBUG_PRINT("info",("Online backup creating THD struct for thread"));
+ THD *thd= new THD;
+ if (unlikely(!thd))
+ {
+ delete thd;
+ return (0);
+ }
+ (static_cast<Backup_driver *>(arg))->lock_thd= thd;
+
+ thd->thread_stack = (char*)&thd; // remember where our stack is
+ pthread_mutex_lock(&LOCK_thread_count);
+ thd->thread_id= thread_id++;
+ pthread_mutex_unlock(&LOCK_thread_count);
+ if (unlikely(thd->store_globals())) // for a proper MEM_ROOT
+ {
+ delete thd;
+ return (0);
+ }
+
+ thd->init_for_queries(); // opening tables needs a proper LEX
+ thd->command= COM_DAEMON;
+ thd->system_thread= SYSTEM_THREAD_BACKUP;
+ thd->version= refresh_version;
+ thd->set_time();
+ thd->main_security_ctx.host_or_ip= "";
+ thd->client_capabilities= 0;
+ my_net_init(&thd->net, 0);
+ thd->main_security_ctx.master_access= ~0;
+ thd->main_security_ctx.priv_user= 0;
+ thd->real_id= pthread_self();
+ /*
+ Making this thread visible to SHOW PROCESSLIST is useful for
+ troubleshooting a backup job (why does it stall etc).
+ */
+ pthread_mutex_lock(&LOCK_thread_count);
+ threads.append(thd);
+ pthread_mutex_unlock(&LOCK_thread_count);
+ lex_start(thd);
+ mysql_reset_thd_for_next_command(thd);
+
+ /*
+ Now open and lock the tables.
+ */
+ DBUG_PRINT("info",("Online backup open tables in thread"));
+ tables_in_backup= (static_cast<Backup_driver *>(arg))->tables_in_backup;
+ /*
+ Make sure we have tables to open.
+ */
+ if (!tables_in_backup)
+ {
+ DBUG_PRINT("info",("Online backup locking error no tables to lock"));
+ return (0);
+ }
+
+ /*
+ As locking tables can be a long operation, we need to support
+ cancellability during that time. So we publish our THD to the thread which
+ created us.
+ */
+ if (open_and_lock_tables(thd, tables_in_backup))
+ {
+ DBUG_PRINT("info",("Online backup locking thread dying"));
+ close_thread_tables(thd);
+ return (0);
+ }
+
+ /*
+ Turn on condition variable check for lock.
+ */
+ pthread_mutex_lock(&THR_LOCK_driver_thread);
+ backup_thread_is_locked= TRUE;
+ pthread_mutex_unlock(&THR_LOCK_driver_thread);
+
+ /*
+ Signal waiting driver that condition for lock is obtained.
+ */
+ pthread_cond_broadcast(&COND_driver_lock);
+
+ /*
+ Part of work is done. Rest until woken up.
+ */
+ pthread_mutex_lock(&THR_LOCK_driver_thread);
+ thd->enter_cond(&COND_driver_thread_wait, &THR_LOCK_driver_thread,
+ "Online backup driver thread: holding table locks");
+ while (!thd->killed)
+ pthread_cond_wait(&COND_driver_thread_wait, &THR_LOCK_driver_thread);
+ thd->exit_cond("Online backup driver thread: terminating");
+
+ DBUG_PRINT("info",("Online backup driver thread locking thread terminating"));
+
+ /*
+ Cleanup and return.
+ */
+ close_thread_tables(thd);
+ net_end(&thd->net);
+
+ /*
+ Turn off condition variable check for lock.
+ */
+ pthread_mutex_lock(&THR_LOCK_driver_thread);
+ backup_thread_is_locked= FALSE;
+ pthread_mutex_unlock(&THR_LOCK_driver_thread);
+
+ my_thread_end();
+ delete thd;
+ pthread_exit(0);
+ return (0);
+}
+
+/**
+ backup_thread_locked
+
+ Checks to see if lock thread has obtained the lock.
+
+ @returns TRUE if locked, FALSE if not locked yet.
+ */
+my_bool backup_thread_locked()
+{
+ DBUG_ENTER("Default_backup - Checking for lock");
+ DBUG_RETURN(backup_thread_is_locked);
+}
diff -Nrup a/sql/backup/be_thread.h b/sql/backup/be_thread.h
--- /dev/null Wed Dec 31 16:00:00 196900@@ -0,0 +1,39 @@
+++ b/sql/backup/be_thread.h 2007-10-29 09:46:13 -04:00
+#ifndef _BACKUP_THREAD_H
+#define _BACKUP_THREAD_H
+
+#include "../mysql_priv.h"
+#include "backup_engine.h"
+
+/**
+ Locking of tables goes through several states.
+*/
+typedef enum {
+ LOCK_NOT_STARTED,
+ LOCK_IN_PROGRESS,
+ LOCK_ACQUIRED,
+ LOCK_ERROR
+} LOCK_STATE;
+
+extern pthread_mutex_t THR_LOCK_driver_thread;
+extern pthread_cond_t COND_driver_lock;
+extern pthread_cond_t COND_driver_thread_wait;
+
+/**
+ Macro for error handling.
+*/
+#define SET_STATE_TO_ERROR_AND_DBUG_RETURN { \
+ int state= ERROR; \
+ DBUG_PRINT("error",("driver got an error at %s:%d",__FILE__,__LINE__)); \
+ DBUG_RETURN(backup::ERROR); }
+
+/**
+ backup_thread_for_locking
+
+ This method creates a new thread and opens and locks the tables.
+*/
+pthread_handler_t backup_thread_for_locking(void *arg);
+
+my_bool backup_thread_locked();
+
+#endif
+
diff -Nrup a/sql/backup/data_backup.cc b/sql/backup/data_backup.cc --- a/sql/backup/data_backup.cc 2007-09-11 05:29:29 -04:00
+++ b/sql/backup/data_backup.cc 2007-10-29 09:46:11 -04:00
@@ -384,9 +384,6 @@ int write_table_data(THD*, Backup_info &
DBUG_PRINT("backup/data",("initializing scheduler"));
- TABLE_LIST *table_list= 0;
- TABLE_LIST *table_list_last= 0; - // add unknown "at end" drivers to scheduler, rest to inactive list
for (uint no=0; no < info.img_count; ++no) @@ -418,12 +415,6 @@ int write_table_data(THD*, Backup_info &
inactive.push_back(p);
}
- if (!def_or_snap_used)
- def_or_snap_used= ((i->type() == Image_info::DEFAULT_IMAGE) ||
- (i->type() == Image_info::SNAPSHOT_IMAGE));
- if (def_or_snap_used)
- get_default_snapshot_tables(&p->drv(), NULL,
- &table_list, &table_list_last);
}
/*
@@ -508,22 +499,6 @@ int write_table_data(THD*, Backup_info &
if (sch.prepare())
goto error;
- /*
- Open tables for default and snapshot drivers.
- */
- if (table_list)
- {
- if (open_and_lock_tables(::current_thd, table_list))
- {
- DBUG_PRINT("backup",
- ( "error on open tables for default and snapshot drivers!" ));
- info.report_error(ER_BACKUP_OPEN_TABLES, "backup");
- DBUG_RETURN(ERROR);
- }
- if (table_list_last)
- table_list_last->next_global= NULL; // break lists
- }
-
while (sch.prepare_count > 0)
if (sch.step())
goto error;
@@ -548,12 +523,6 @@ int write_table_data(THD*, Backup_info &
DBUG_PRINT("backup/data",("-- DONE --")); }
- - /* - If the default or snapshot drivers are used, close the tables. - */ - if (def_or_snap_used) - close_thread_tables(::current_thd);
info.data_size= s.bytes - start_bytes;
diff -Nrup a/sql/mysqld.cc b/sql/mysqld.cc
--- a/sql/mysqld.cc 2007-10-17 09:57:33 -04:00+pthread_cond_t COND_driver_lock;
+++ b/sql/mysqld.cc 2007-10-29 09:46:05 -04:00
@@ -577,6 +577,9 @@ pthread_mutex_t LOCK_mysql_create_db, LO LOCK_crypt, LOCK_bytes_sent, LOCK_bytes_received, LOCK_global_system_variables, LOCK_user_conn, LOCK_slave_list, LOCK_active_mi;
+pthread_mutex_t THR_LOCK_driver_thread;
+pthread_cond_t COND_driver_thread_wait;
/*
The below lock protects access to two global server variables: max_prepared_stmt_count and prepared_stmt_count. These variables @@ -1335,6 +1338,9 @@ static void clean_up_mutexes()
(void) pthread_mutex_destroy(&LOCK_bytes_sent); (void) pthread_mutex_destroy(&LOCK_bytes_received); (void) pthread_mutex_destroy(&LOCK_user_conn);+ (void) pthread_cond_destroy(&COND_driver_lock);
+ (void) pthread_mutex_destroy(&THR_LOCK_driver_thread);
+ (void) pthread_cond_destroy(&COND_driver_thread_wait);
Events::destroy_mutexes();
#ifdef HAVE_OPENSSL
(void) pthread_mutex_destroy(&LOCK_des_key_file);
@@ -3074,6 +3080,9 @@ static int init_thread_environment()
(void) pthread_mutex_init(&LOCK_global_read_lock, MY_MUTEX_INIT_FAST); (void) pthread_mutex_init(&LOCK_prepared_stmt_count, MY_MUTEX_INIT_FAST); (void) pthread_mutex_init(&LOCK_uuid_generator, MY_MUTEX_INIT_FAST);+ (void) pthread_cond_init(&COND_driver_lock, NULL);
+ (void) pthread_mutex_init(&THR_LOCK_driver_thread, MY_MUTEX_INIT_FAST);
+ (void) pthread_cond_init(&COND_driver_thread_wait, NULL);
#ifdef HAVE_OPENSSL
(void) pthread_mutex_init(&LOCK_des_key_file,MY_MUTEX_INIT_FAST); #ifndef HAVE_YASSL
diff -Nrup a/sql/share/errmsg.txt b/sql/share/errmsg.txt
--- a/sql/share/errmsg.txt 2007-10-17 09:57:35 -04:00@@ -6195,6 +6195,9 @@ ER_BACKUP_SEND_DATA_RETRY ER_BACKUP_OPEN_TABLES
+++ b/sql/share/errmsg.txt 2007-10-29 09:46:12 -04:00
eng "Open and lock tables failed in %-.64s"
+ER_BACKUP_THREAD_INIT
+ eng "Cannot initialize backup threads."
+
ER_VIEW_NO_CREATION_CTX
eng "View `%-.64s`.`%-.64s` has no creation context"
ER_VIEW_INVALID_CREATION_CTX
diff -Nrup a/sql/sql_class.h b/sql/sql_class.h
--- a/sql/sql_class.h 2007-09-13 04:56:30 -04:00@@ -936,7 +936,8 @@ enum enum_thread_type
+++ b/sql/sql_class.h 2007-10-29 09:46:06 -04:00
SYSTEM_THREAD_SLAVE_SQL= 4, SYSTEM_THREAD_NDBCLUSTER_BINLOG= 8, SYSTEM_THREAD_EVENT_SCHEDULER= 16, - SYSTEM_THREAD_EVENT_WORKER= 32
+ SYSTEM_THREAD_EVENT_WORKER= 32,
+ SYSTEM_THREAD_BACKUP= 64
};
-- MySQL Code Commits Mailing List For list archives: http://lists.mysql.com/commits To unsubscribe: http://lists.mysql.com/commits?unsub=lists@pantek.comReceived on Mon Oct 29 09:47:11 2007
This archive was generated by hypermail 2.1.8 : Thu Jul 03 2008 - 10:54:36 EDT
|
Contact Us Legal Notices Order Services Online Pantek Home Privacy Policy IT news Site Map Pantek Library |