You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

6687 lines
182 KiB

MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M; Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for encrypted pages even in system datafiles should contain key_version except very first page (0:0) is after encryption overwritten with flush lsn. Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1 The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during InnoDB startup. At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION from the first page of each file in the InnoDB system tablespace. If there are multiple files, the minimum and maximum LSN can differ. These numbers are passed to InnoDB startup. Having the number in other files than the first file of the InnoDB system tablespace is not providing much additional value. It is conflicting with other use of the field, such as on InnoDB R-tree index pages and encryption key_version. This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to other files than the first file of the InnoDB system tablespace (page number 0:0) when system tablespace is encrypted. If tablespace is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to all first pages of system tablespace to avoid unnecessary warnings on downgrade. open_or_create_data_files(): pass only one flushed_lsn parameter xb_load_tablespaces(): pass only one flushed_lsn parameter. buf_page_create(): Improve comment about where FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set. fil_write_flushed_lsn(): A new function, merged from fil_write_lsn_and_arch_no_to_file() and fil_write_flushed_lsn_to_data_files(). Only write to the first page of the system tablespace (page 0:0) if tablespace is encrypted, or write all first pages of system tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE) afterwards. fil_read_first_page(): read flush_lsn and crypt_data only from first datafile. fil_open_single_table_tablespace(): Remove output of LSN, because it was only valid for the system tablespace and the undo tablespaces, not user tablespaces. fil_validate_single_table_tablespace(): Remove output of LSN. checkpoint_now_set(): Use fil_write_flushed_lsn and output a error if operation fails. Remove lsn variable from fsp_open_info. recv_recovery_from_checkpoint_start(): Remove unnecessary second flush_lsn parameter. log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn and output error if it fails. open_or_create_data_files(): Pass only one flushed_lsn variable.
9 years ago
MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M; Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for encrypted pages even in system datafiles should contain key_version except very first page (0:0) is after encryption overwritten with flush lsn. Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1 The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during InnoDB startup. At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION from the first page of each file in the InnoDB system tablespace. If there are multiple files, the minimum and maximum LSN can differ. These numbers are passed to InnoDB startup. Having the number in other files than the first file of the InnoDB system tablespace is not providing much additional value. It is conflicting with other use of the field, such as on InnoDB R-tree index pages and encryption key_version. This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to other files than the first file of the InnoDB system tablespace (page number 0:0) when system tablespace is encrypted. If tablespace is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to all first pages of system tablespace to avoid unnecessary warnings on downgrade. open_or_create_data_files(): pass only one flushed_lsn parameter xb_load_tablespaces(): pass only one flushed_lsn parameter. buf_page_create(): Improve comment about where FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set. fil_write_flushed_lsn(): A new function, merged from fil_write_lsn_and_arch_no_to_file() and fil_write_flushed_lsn_to_data_files(). Only write to the first page of the system tablespace (page 0:0) if tablespace is encrypted, or write all first pages of system tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE) afterwards. fil_read_first_page(): read flush_lsn and crypt_data only from first datafile. fil_open_single_table_tablespace(): Remove output of LSN, because it was only valid for the system tablespace and the undo tablespaces, not user tablespaces. fil_validate_single_table_tablespace(): Remove output of LSN. checkpoint_now_set(): Use fil_write_flushed_lsn and output a error if operation fails. Remove lsn variable from fsp_open_info. recv_recovery_from_checkpoint_start(): Remove unnecessary second flush_lsn parameter. log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn and output error if it fails. open_or_create_data_files(): Pass only one flushed_lsn variable.
9 years ago
MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M; Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for encrypted pages even in system datafiles should contain key_version except very first page (0:0) is after encryption overwritten with flush lsn. Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1 The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during InnoDB startup. At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION from the first page of each file in the InnoDB system tablespace. If there are multiple files, the minimum and maximum LSN can differ. These numbers are passed to InnoDB startup. Having the number in other files than the first file of the InnoDB system tablespace is not providing much additional value. It is conflicting with other use of the field, such as on InnoDB R-tree index pages and encryption key_version. This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to other files than the first file of the InnoDB system tablespace (page number 0:0) when system tablespace is encrypted. If tablespace is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to all first pages of system tablespace to avoid unnecessary warnings on downgrade. open_or_create_data_files(): pass only one flushed_lsn parameter xb_load_tablespaces(): pass only one flushed_lsn parameter. buf_page_create(): Improve comment about where FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set. fil_write_flushed_lsn(): A new function, merged from fil_write_lsn_and_arch_no_to_file() and fil_write_flushed_lsn_to_data_files(). Only write to the first page of the system tablespace (page 0:0) if tablespace is encrypted, or write all first pages of system tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE) afterwards. fil_read_first_page(): read flush_lsn and crypt_data only from first datafile. fil_open_single_table_tablespace(): Remove output of LSN, because it was only valid for the system tablespace and the undo tablespaces, not user tablespaces. fil_validate_single_table_tablespace(): Remove output of LSN. checkpoint_now_set(): Use fil_write_flushed_lsn and output a error if operation fails. Remove lsn variable from fsp_open_info. recv_recovery_from_checkpoint_start(): Remove unnecessary second flush_lsn parameter. log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn and output error if it fails. open_or_create_data_files(): Pass only one flushed_lsn variable.
9 years ago
  1. /******************************************************
  2. XtraBackup: hot backup tool for InnoDB
  3. (c) 2009-2017 Percona LLC and/or its affiliates
  4. Originally Created 3/3/2009 Yasufumi Kinoshita
  5. Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
  6. Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
  7. This program is free software; you can redistribute it and/or modify
  8. it under the terms of the GNU General Public License as published by
  9. the Free Software Foundation; version 2 of the License.
  10. This program is distributed in the hope that it will be useful,
  11. but WITHOUT ANY WARRANTY; without even the implied warranty of
  12. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  13. GNU General Public License for more details.
  14. You should have received a copy of the GNU General Public License
  15. along with this program; if not, write to the Free Software
  16. Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
  17. *******************************************************
  18. This file incorporates work covered by the following copyright and
  19. permission notice:
  20. Copyright (c) 2000, 2011, MySQL AB & Innobase Oy. All Rights Reserved.
  21. This program is free software; you can redistribute it and/or modify it under
  22. the terms of the GNU General Public License as published by the Free Software
  23. Foundation; version 2 of the License.
  24. This program is distributed in the hope that it will be useful, but WITHOUT
  25. ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
  26. FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
  27. You should have received a copy of the GNU General Public License along with
  28. this program; if not, write to the Free Software Foundation, Inc., 59 Temple
  29. Place, Suite 330, Boston, MA 02111-1307 USA
  30. *******************************************************/
  31. //#define XTRABACKUP_TARGET_IS_PLUGIN
  32. #include <mysql_version.h>
  33. #include <my_base.h>
  34. #include <my_getopt.h>
  35. #include <mysql_com.h>
  36. #include <my_default.h>
  37. #include <mysqld.h>
  38. #include <fcntl.h>
  39. #include <string.h>
  40. #ifdef __linux__
  41. # include <sys/prctl.h>
  42. #include <sys/resource.h>
  43. #endif
  44. #include <btr0sea.h>
  45. #include <dict0priv.h>
  46. #include <lock0lock.h>
  47. #include <log0recv.h>
  48. #include <row0mysql.h>
  49. #include <row0quiesce.h>
  50. #include <srv0start.h>
  51. #include <buf0dblwr.h>
  52. #include <list>
  53. #include <sstream>
  54. #include <set>
  55. #include <mysql.h>
  56. #define G_PTR uchar*
  57. #include "common.h"
  58. #include "datasink.h"
  59. #include "xb_regex.h"
  60. #include "fil_cur.h"
  61. #include "write_filt.h"
  62. #include "xtrabackup.h"
  63. #include "ds_buffer.h"
  64. #include "ds_tmpfile.h"
  65. #include "xbstream.h"
  66. #include "changed_page_bitmap.h"
  67. #include "read_filt.h"
  68. #include "wsrep.h"
  69. #include "innobackupex.h"
  70. #include "backup_mysql.h"
  71. #include "backup_copy.h"
  72. #include "backup_mysql.h"
  73. #include "xb0xb.h"
  74. #include "encryption_plugin.h"
  75. #include <sql_plugin.h>
  76. #include <srv0srv.h>
  77. #include <crc_glue.h>
  78. /* TODO: replace with appropriate macros used in InnoDB 5.6 */
  79. #define PAGE_ZIP_MIN_SIZE_SHIFT 10
  80. #define DICT_TF_ZSSIZE_SHIFT 1
  81. #define DICT_TF_FORMAT_ZIP 1
  82. #define DICT_TF_FORMAT_SHIFT 5
  83. int sys_var_init();
  84. my_bool innodb_inited= 0;
  85. /* === xtrabackup specific options === */
  86. char xtrabackup_real_target_dir[FN_REFLEN] = "./xtrabackup_backupfiles/";
  87. char *xtrabackup_target_dir= xtrabackup_real_target_dir;
  88. my_bool xtrabackup_version = FALSE;
  89. my_bool xtrabackup_backup = FALSE;
  90. my_bool xtrabackup_prepare = FALSE;
  91. my_bool xtrabackup_copy_back = FALSE;
  92. my_bool xtrabackup_move_back = FALSE;
  93. my_bool xtrabackup_decrypt_decompress = FALSE;
  94. my_bool xtrabackup_print_param = FALSE;
  95. my_bool xtrabackup_export = FALSE;
  96. my_bool xtrabackup_apply_log_only = FALSE;
  97. longlong xtrabackup_use_memory = 100*1024*1024L;
  98. my_bool xtrabackup_create_ib_logfile = FALSE;
  99. long xtrabackup_throttle = 0; /* 0:unlimited */
  100. lint io_ticket;
  101. os_event_t wait_throttle = NULL;
  102. os_event_t log_copying_stop = NULL;
  103. char *xtrabackup_incremental = NULL;
  104. lsn_t incremental_lsn;
  105. lsn_t incremental_to_lsn;
  106. lsn_t incremental_last_lsn;
  107. xb_page_bitmap *changed_page_bitmap = NULL;
  108. char *xtrabackup_incremental_basedir = NULL; /* for --backup */
  109. char *xtrabackup_extra_lsndir = NULL; /* for --backup with --extra-lsndir */
  110. char *xtrabackup_incremental_dir = NULL; /* for --prepare */
  111. char xtrabackup_real_incremental_basedir[FN_REFLEN];
  112. char xtrabackup_real_extra_lsndir[FN_REFLEN];
  113. char xtrabackup_real_incremental_dir[FN_REFLEN];
  114. char *xtrabackup_tmpdir;
  115. char *xtrabackup_tables = NULL;
  116. char *xtrabackup_tables_file = NULL;
  117. char *xtrabackup_tables_exclude = NULL;
  118. typedef std::list<regex_t> regex_list_t;
  119. static regex_list_t regex_include_list;
  120. static regex_list_t regex_exclude_list;
  121. static hash_table_t* tables_include_hash = NULL;
  122. static hash_table_t* tables_exclude_hash = NULL;
  123. char *xtrabackup_databases = NULL;
  124. char *xtrabackup_databases_file = NULL;
  125. char *xtrabackup_databases_exclude = NULL;
  126. static hash_table_t* databases_include_hash = NULL;
  127. static hash_table_t* databases_exclude_hash = NULL;
  128. static hash_table_t* inc_dir_tables_hash;
  129. struct xb_filter_entry_struct{
  130. char* name;
  131. ibool has_tables;
  132. hash_node_t name_hash;
  133. };
  134. typedef struct xb_filter_entry_struct xb_filter_entry_t;
  135. static ulint thread_nr[SRV_MAX_N_IO_THREADS + 6];
  136. static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 6];
  137. lsn_t checkpoint_lsn_start;
  138. lsn_t checkpoint_no_start;
  139. lsn_t log_copy_scanned_lsn;
  140. ibool log_copying = TRUE;
  141. ibool log_copying_running = FALSE;
  142. ibool io_watching_thread_running = FALSE;
  143. ibool xtrabackup_logfile_is_renamed = FALSE;
  144. int xtrabackup_parallel;
  145. char *xtrabackup_stream_str = NULL;
  146. xb_stream_fmt_t xtrabackup_stream_fmt = XB_STREAM_FMT_NONE;
  147. ibool xtrabackup_stream = FALSE;
  148. const char *xtrabackup_compress_alg = NULL;
  149. ibool xtrabackup_compress = FALSE;
  150. uint xtrabackup_compress_threads;
  151. ulonglong xtrabackup_compress_chunk_size = 0;
  152. /* sleep interval beetween log copy iterations in log copying thread
  153. in milliseconds (default is 1 second) */
  154. ulint xtrabackup_log_copy_interval = 1000;
  155. static ulong max_buf_pool_modified_pct;
  156. /* Ignored option (--log) for MySQL option compatibility */
  157. char* log_ignored_opt = NULL;
  158. /* === metadata of backup === */
  159. #define XTRABACKUP_METADATA_FILENAME "xtrabackup_checkpoints"
  160. char metadata_type[30] = ""; /*[full-backuped|log-applied|
  161. full-prepared|incremental]*/
  162. lsn_t metadata_from_lsn = 0;
  163. lsn_t metadata_to_lsn = 0;
  164. lsn_t metadata_last_lsn = 0;
  165. #define XB_LOG_FILENAME "xtrabackup_logfile"
  166. ds_file_t *dst_log_file = NULL;
  167. static char mysql_data_home_buff[2];
  168. const char *defaults_group = "mysqld";
  169. /* === static parameters in ha_innodb.cc */
  170. #define HA_INNOBASE_ROWS_IN_TABLE 10000 /* to get optimization right */
  171. #define HA_INNOBASE_RANGE_COUNT 100
  172. ulong innobase_large_page_size = 0;
  173. /* The default values for the following, type long or longlong, start-up
  174. parameters are declared in mysqld.cc: */
  175. long innobase_additional_mem_pool_size = 1*1024*1024L;
  176. long innobase_buffer_pool_awe_mem_mb = 0;
  177. long innobase_file_io_threads = 4;
  178. long innobase_read_io_threads = 4;
  179. long innobase_write_io_threads = 4;
  180. long innobase_force_recovery = 0;
  181. long innobase_log_buffer_size = 1024*1024L;
  182. long innobase_log_files_in_group = 2;
  183. long innobase_open_files = 300L;
  184. longlong innobase_page_size = (1LL << 14); /* 16KB */
  185. static ulong innobase_log_block_size = 512;
  186. char* innobase_doublewrite_file = NULL;
  187. char* innobase_buffer_pool_filename = NULL;
  188. longlong innobase_buffer_pool_size = 8*1024*1024L;
  189. longlong innobase_log_file_size = 48*1024*1024L;
  190. /* The default values for the following char* start-up parameters
  191. are determined in innobase_init below: */
  192. char* innobase_ignored_opt = NULL;
  193. char* innobase_data_home_dir = NULL;
  194. char* innobase_data_file_path = NULL;
  195. /* The following has a misleading name: starting from 4.0.5, this also
  196. affects Windows: */
  197. char* innobase_unix_file_flush_method = NULL;
  198. /* Below we have boolean-valued start-up parameters, and their default
  199. values */
  200. ulong innobase_fast_shutdown = 1;
  201. my_bool innobase_use_doublewrite = TRUE;
  202. my_bool innobase_use_checksums = TRUE;
  203. my_bool innobase_use_large_pages = FALSE;
  204. my_bool innobase_file_per_table = FALSE;
  205. my_bool innobase_locks_unsafe_for_binlog = FALSE;
  206. my_bool innobase_rollback_on_timeout = FALSE;
  207. my_bool innobase_create_status_file = FALSE;
  208. my_bool innobase_adaptive_hash_index = TRUE;
  209. static char *internal_innobase_data_file_path = NULL;
  210. /* The following counter is used to convey information to InnoDB
  211. about server activity: in selects it is not sensible to call
  212. srv_active_wake_master_thread after each fetch or search, we only do
  213. it every INNOBASE_WAKE_INTERVAL'th step. */
  214. #define INNOBASE_WAKE_INTERVAL 32
  215. ulong innobase_active_counter = 0;
  216. static char *xtrabackup_debug_sync = NULL;
  217. my_bool xtrabackup_incremental_force_scan = FALSE;
  218. /* The flushed lsn which is read from data files */
  219. lsn_t flushed_lsn= 0;
  220. ulong xb_open_files_limit= 0;
  221. char *xb_plugin_dir;
  222. char *xb_plugin_load;
  223. my_bool xb_close_files= FALSE;
  224. /* Datasinks */
  225. ds_ctxt_t *ds_data = NULL;
  226. ds_ctxt_t *ds_meta = NULL;
  227. ds_ctxt_t *ds_redo = NULL;
  228. static bool innobackupex_mode = false;
  229. static long innobase_log_files_in_group_save;
  230. static char *srv_log_group_home_dir_save;
  231. static longlong innobase_log_file_size_save;
  232. /* String buffer used by --print-param to accumulate server options as they are
  233. parsed from the defaults file */
  234. static std::ostringstream print_param_str;
  235. /* Set of specified parameters */
  236. std::set<std::string> param_set;
  237. static ulonglong global_max_value;
  238. extern "C" sig_handler handle_fatal_signal(int sig);
  239. my_bool opt_galera_info = FALSE;
  240. my_bool opt_slave_info = FALSE;
  241. my_bool opt_no_lock = FALSE;
  242. my_bool opt_safe_slave_backup = FALSE;
  243. my_bool opt_rsync = FALSE;
  244. my_bool opt_force_non_empty_dirs = FALSE;
  245. my_bool opt_noversioncheck = FALSE;
  246. my_bool opt_no_backup_locks = FALSE;
  247. my_bool opt_decompress = FALSE;
  248. my_bool opt_remove_original = FALSE;
  249. static const char *binlog_info_values[] = {"off", "lockless", "on", "auto",
  250. NullS};
  251. static TYPELIB binlog_info_typelib = {array_elements(binlog_info_values)-1, "",
  252. binlog_info_values, NULL};
  253. ulong opt_binlog_info;
  254. char *opt_incremental_history_name = NULL;
  255. char *opt_incremental_history_uuid = NULL;
  256. char *opt_user = NULL;
  257. char *opt_password = NULL;
  258. char *opt_host = NULL;
  259. char *opt_defaults_group = NULL;
  260. char *opt_socket = NULL;
  261. uint opt_port = 0;
  262. char *opt_login_path = NULL;
  263. char *opt_log_bin = NULL;
  264. const char *query_type_names[] = { "ALL", "UPDATE", "SELECT", NullS};
  265. TYPELIB query_type_typelib= {array_elements(query_type_names) - 1, "",
  266. query_type_names, NULL};
  267. ulong opt_lock_wait_query_type;
  268. ulong opt_kill_long_query_type;
  269. uint opt_kill_long_queries_timeout = 0;
  270. uint opt_lock_wait_timeout = 0;
  271. uint opt_lock_wait_threshold = 0;
  272. uint opt_debug_sleep_before_unlock = 0;
  273. uint opt_safe_slave_backup_timeout = 0;
  274. const char *opt_history = NULL;
  275. #if defined(HAVE_OPENSSL)
  276. my_bool opt_ssl_verify_server_cert = FALSE;
  277. #endif
  278. /* Whether xtrabackup_binlog_info should be created on recovery */
  279. static bool recover_binlog_info;
  280. /* Simple datasink creation tracking...add datasinks in the reverse order you
  281. want them destroyed. */
  282. #define XTRABACKUP_MAX_DATASINKS 10
  283. static ds_ctxt_t *datasinks[XTRABACKUP_MAX_DATASINKS];
  284. static uint actual_datasinks = 0;
  285. static inline
  286. void
  287. xtrabackup_add_datasink(ds_ctxt_t *ds)
  288. {
  289. xb_ad(actual_datasinks < XTRABACKUP_MAX_DATASINKS);
  290. datasinks[actual_datasinks] = ds; actual_datasinks++;
  291. }
  292. /* ======== Datafiles iterator ======== */
  293. datafiles_iter_t *
  294. datafiles_iter_new(fil_system_t *f_system)
  295. {
  296. datafiles_iter_t *it;
  297. it = static_cast<datafiles_iter_t *>
  298. (ut_malloc(sizeof(datafiles_iter_t)));
  299. it->mutex = os_mutex_create();
  300. it->system = f_system;
  301. it->space = NULL;
  302. it->node = NULL;
  303. it->started = FALSE;
  304. return it;
  305. }
  306. fil_node_t *
  307. datafiles_iter_next(datafiles_iter_t *it)
  308. {
  309. fil_node_t *new_node;
  310. os_mutex_enter(it->mutex);
  311. if (it->node == NULL) {
  312. if (it->started)
  313. goto end;
  314. it->started = TRUE;
  315. } else {
  316. it->node = UT_LIST_GET_NEXT(chain, it->node);
  317. if (it->node != NULL)
  318. goto end;
  319. }
  320. it->space = (it->space == NULL) ?
  321. UT_LIST_GET_FIRST(it->system->space_list) :
  322. UT_LIST_GET_NEXT(space_list, it->space);
  323. while (it->space != NULL &&
  324. (it->space->purpose != FIL_TABLESPACE ||
  325. UT_LIST_GET_LEN(it->space->chain) == 0))
  326. it->space = UT_LIST_GET_NEXT(space_list, it->space);
  327. if (it->space == NULL)
  328. goto end;
  329. it->node = UT_LIST_GET_FIRST(it->space->chain);
  330. end:
  331. new_node = it->node;
  332. os_mutex_exit(it->mutex);
  333. return new_node;
  334. }
  335. void
  336. datafiles_iter_free(datafiles_iter_t *it)
  337. {
  338. os_mutex_free(it->mutex);
  339. ut_free(it);
  340. }
  341. /* ======== Date copying thread context ======== */
  342. typedef struct {
  343. datafiles_iter_t *it;
  344. uint num;
  345. uint *count;
  346. os_ib_mutex_t count_mutex;
  347. os_thread_id_t id;
  348. } data_thread_ctxt_t;
  349. /* ======== for option and variables ======== */
  350. enum options_xtrabackup
  351. {
  352. OPT_XTRA_TARGET_DIR = 1000, /* make sure it is larger
  353. than OPT_MAX_CLIENT_OPTION */
  354. OPT_XTRA_BACKUP,
  355. OPT_XTRA_PREPARE,
  356. OPT_XTRA_EXPORT,
  357. OPT_XTRA_APPLY_LOG_ONLY,
  358. OPT_XTRA_PRINT_PARAM,
  359. OPT_XTRA_USE_MEMORY,
  360. OPT_XTRA_THROTTLE,
  361. OPT_XTRA_LOG_COPY_INTERVAL,
  362. OPT_XTRA_INCREMENTAL,
  363. OPT_XTRA_INCREMENTAL_BASEDIR,
  364. OPT_XTRA_EXTRA_LSNDIR,
  365. OPT_XTRA_INCREMENTAL_DIR,
  366. OPT_XTRA_TABLES,
  367. OPT_XTRA_TABLES_FILE,
  368. OPT_XTRA_DATABASES,
  369. OPT_XTRA_DATABASES_FILE,
  370. OPT_XTRA_CREATE_IB_LOGFILE,
  371. OPT_XTRA_PARALLEL,
  372. OPT_XTRA_STREAM,
  373. OPT_XTRA_COMPRESS,
  374. OPT_XTRA_COMPRESS_THREADS,
  375. OPT_XTRA_COMPRESS_CHUNK_SIZE,
  376. OPT_LOG,
  377. OPT_INNODB,
  378. OPT_INNODB_CHECKSUMS,
  379. OPT_INNODB_DATA_FILE_PATH,
  380. OPT_INNODB_DATA_HOME_DIR,
  381. OPT_INNODB_ADAPTIVE_HASH_INDEX,
  382. OPT_INNODB_DOUBLEWRITE,
  383. OPT_INNODB_FAST_SHUTDOWN,
  384. OPT_INNODB_FILE_PER_TABLE,
  385. OPT_INNODB_FLUSH_LOG_AT_TRX_COMMIT,
  386. OPT_INNODB_FLUSH_METHOD,
  387. OPT_INNODB_LOCKS_UNSAFE_FOR_BINLOG,
  388. OPT_INNODB_LOG_GROUP_HOME_DIR,
  389. OPT_INNODB_MAX_DIRTY_PAGES_PCT,
  390. OPT_INNODB_MAX_PURGE_LAG,
  391. OPT_INNODB_ROLLBACK_ON_TIMEOUT,
  392. OPT_INNODB_STATUS_FILE,
  393. OPT_INNODB_ADDITIONAL_MEM_POOL_SIZE,
  394. OPT_INNODB_AUTOEXTEND_INCREMENT,
  395. OPT_INNODB_BUFFER_POOL_SIZE,
  396. OPT_INNODB_COMMIT_CONCURRENCY,
  397. OPT_INNODB_CONCURRENCY_TICKETS,
  398. OPT_INNODB_FILE_IO_THREADS,
  399. OPT_INNODB_IO_CAPACITY,
  400. OPT_INNODB_READ_IO_THREADS,
  401. OPT_INNODB_WRITE_IO_THREADS,
  402. OPT_INNODB_USE_NATIVE_AIO,
  403. OPT_INNODB_PAGE_SIZE,
  404. OPT_INNODB_LOG_BLOCK_SIZE,
  405. OPT_INNODB_DOUBLEWRITE_FILE,
  406. OPT_INNODB_BUFFER_POOL_FILENAME,
  407. OPT_INNODB_FORCE_RECOVERY,
  408. OPT_INNODB_LOCK_WAIT_TIMEOUT,
  409. OPT_INNODB_LOG_BUFFER_SIZE,
  410. OPT_INNODB_LOG_FILE_SIZE,
  411. OPT_INNODB_LOG_FILES_IN_GROUP,
  412. OPT_INNODB_MIRRORED_LOG_GROUPS,
  413. OPT_INNODB_OPEN_FILES,
  414. OPT_INNODB_SYNC_SPIN_LOOPS,
  415. OPT_INNODB_THREAD_CONCURRENCY,
  416. OPT_INNODB_THREAD_SLEEP_DELAY,
  417. OPT_XTRA_DEBUG_SYNC,
  418. OPT_INNODB_CHECKSUM_ALGORITHM,
  419. OPT_INNODB_UNDO_DIRECTORY,
  420. OPT_INNODB_UNDO_TABLESPACES,
  421. OPT_INNODB_LOG_CHECKSUM_ALGORITHM,
  422. OPT_XTRA_INCREMENTAL_FORCE_SCAN,
  423. OPT_DEFAULTS_GROUP,
  424. OPT_OPEN_FILES_LIMIT,
  425. OPT_PLUGIN_DIR,
  426. OPT_PLUGIN_LOAD,
  427. OPT_INNODB_ENCRYPT_LOG,
  428. OPT_CLOSE_FILES,
  429. OPT_CORE_FILE,
  430. OPT_COPY_BACK,
  431. OPT_MOVE_BACK,
  432. OPT_GALERA_INFO,
  433. OPT_SLAVE_INFO,
  434. OPT_NO_LOCK,
  435. OPT_SAFE_SLAVE_BACKUP,
  436. OPT_RSYNC,
  437. OPT_FORCE_NON_EMPTY_DIRS,
  438. OPT_NO_VERSION_CHECK,
  439. OPT_NO_BACKUP_LOCKS,
  440. OPT_DECOMPRESS,
  441. OPT_INCREMENTAL_HISTORY_NAME,
  442. OPT_INCREMENTAL_HISTORY_UUID,
  443. OPT_REMOVE_ORIGINAL,
  444. OPT_LOCK_WAIT_QUERY_TYPE,
  445. OPT_KILL_LONG_QUERY_TYPE,
  446. OPT_HISTORY,
  447. OPT_KILL_LONG_QUERIES_TIMEOUT,
  448. OPT_LOCK_WAIT_TIMEOUT,
  449. OPT_LOCK_WAIT_THRESHOLD,
  450. OPT_DEBUG_SLEEP_BEFORE_UNLOCK,
  451. OPT_SAFE_SLAVE_BACKUP_TIMEOUT,
  452. OPT_BINLOG_INFO,
  453. OPT_XB_SECURE_AUTH,
  454. OPT_XTRA_TABLES_EXCLUDE,
  455. OPT_XTRA_DATABASES_EXCLUDE,
  456. };
  457. struct my_option xb_client_options[] =
  458. {
  459. {"version", 'v', "print xtrabackup version information",
  460. (G_PTR *) &xtrabackup_version, (G_PTR *) &xtrabackup_version, 0, GET_BOOL,
  461. NO_ARG, 0, 0, 0, 0, 0, 0},
  462. {"target-dir", OPT_XTRA_TARGET_DIR, "destination directory", (G_PTR*) &xtrabackup_target_dir,
  463. (G_PTR*) &xtrabackup_target_dir, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  464. {"backup", OPT_XTRA_BACKUP, "take backup to target-dir",
  465. (G_PTR*) &xtrabackup_backup, (G_PTR*) &xtrabackup_backup,
  466. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  467. {"prepare", OPT_XTRA_PREPARE, "prepare a backup for starting mysql server on the backup.",
  468. (G_PTR*) &xtrabackup_prepare, (G_PTR*) &xtrabackup_prepare,
  469. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  470. {"export", OPT_XTRA_EXPORT, "create files to import to another database when prepare.",
  471. (G_PTR*) &xtrabackup_export, (G_PTR*) &xtrabackup_export,
  472. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  473. {"apply-log-only", OPT_XTRA_APPLY_LOG_ONLY,
  474. "stop recovery process not to progress LSN after applying log when prepare.",
  475. (G_PTR*) &xtrabackup_apply_log_only, (G_PTR*) &xtrabackup_apply_log_only,
  476. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  477. {"print-param", OPT_XTRA_PRINT_PARAM, "print parameter of mysqld needed for copyback.",
  478. (G_PTR*) &xtrabackup_print_param, (G_PTR*) &xtrabackup_print_param,
  479. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  480. {"use-memory", OPT_XTRA_USE_MEMORY, "The value is used instead of buffer_pool_size",
  481. (G_PTR*) &xtrabackup_use_memory, (G_PTR*) &xtrabackup_use_memory,
  482. 0, GET_LL, REQUIRED_ARG, 100*1024*1024L, 1024*1024L, LONGLONG_MAX, 0,
  483. 1024*1024L, 0},
  484. {"throttle", OPT_XTRA_THROTTLE, "limit count of IO operations (pairs of read&write) per second to IOS values (for '--backup')",
  485. (G_PTR*) &xtrabackup_throttle, (G_PTR*) &xtrabackup_throttle,
  486. 0, GET_LONG, REQUIRED_ARG, 0, 0, LONG_MAX, 0, 1, 0},
  487. {"log", OPT_LOG, "Ignored option for MySQL option compatibility",
  488. (G_PTR*) &log_ignored_opt, (G_PTR*) &log_ignored_opt, 0,
  489. GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
  490. {"log-copy-interval", OPT_XTRA_LOG_COPY_INTERVAL, "time interval between checks done by log copying thread in milliseconds (default is 1 second).",
  491. (G_PTR*) &xtrabackup_log_copy_interval, (G_PTR*) &xtrabackup_log_copy_interval,
  492. 0, GET_LONG, REQUIRED_ARG, 1000, 0, LONG_MAX, 0, 1, 0},
  493. {"extra-lsndir", OPT_XTRA_EXTRA_LSNDIR, "(for --backup): save an extra copy of the xtrabackup_checkpoints file in this directory.",
  494. (G_PTR*) &xtrabackup_extra_lsndir, (G_PTR*) &xtrabackup_extra_lsndir,
  495. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  496. {"incremental-lsn", OPT_XTRA_INCREMENTAL, "(for --backup): copy only .ibd pages newer than specified LSN 'high:low'. ##ATTENTION##: If a wrong LSN value is specified, it is impossible to diagnose this, causing the backup to be unusable. Be careful!",
  497. (G_PTR*) &xtrabackup_incremental, (G_PTR*) &xtrabackup_incremental,
  498. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  499. {"incremental-basedir", OPT_XTRA_INCREMENTAL_BASEDIR, "(for --backup): copy only .ibd pages newer than backup at specified directory.",
  500. (G_PTR*) &xtrabackup_incremental_basedir, (G_PTR*) &xtrabackup_incremental_basedir,
  501. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  502. {"incremental-dir", OPT_XTRA_INCREMENTAL_DIR, "(for --prepare): apply .delta files and logfile in the specified directory.",
  503. (G_PTR*) &xtrabackup_incremental_dir, (G_PTR*) &xtrabackup_incremental_dir,
  504. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  505. {"tables", OPT_XTRA_TABLES, "filtering by regexp for table names.",
  506. (G_PTR*) &xtrabackup_tables, (G_PTR*) &xtrabackup_tables,
  507. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  508. {"tables_file", OPT_XTRA_TABLES_FILE, "filtering by list of the exact database.table name in the file.",
  509. (G_PTR*) &xtrabackup_tables_file, (G_PTR*) &xtrabackup_tables_file,
  510. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  511. {"databases", OPT_XTRA_DATABASES, "filtering by list of databases.",
  512. (G_PTR*) &xtrabackup_databases, (G_PTR*) &xtrabackup_databases,
  513. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  514. {"databases_file", OPT_XTRA_DATABASES_FILE,
  515. "filtering by list of databases in the file.",
  516. (G_PTR*) &xtrabackup_databases_file, (G_PTR*) &xtrabackup_databases_file,
  517. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  518. {"tables-exclude", OPT_XTRA_TABLES_EXCLUDE, "filtering by regexp for table names. "
  519. "Operates the same way as --tables, but matched names are excluded from backup. "
  520. "Note that this option has a higher priority than --tables.",
  521. (G_PTR*) &xtrabackup_tables_exclude, (G_PTR*) &xtrabackup_tables_exclude,
  522. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  523. {"databases-exclude", OPT_XTRA_DATABASES_EXCLUDE, "Excluding databases based on name, "
  524. "Operates the same way as --databases, but matched names are excluded from backup. "
  525. "Note that this option has a higher priority than --databases.",
  526. (G_PTR*) &xtrabackup_databases_exclude, (G_PTR*) &xtrabackup_databases_exclude,
  527. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  528. {"create-ib-logfile", OPT_XTRA_CREATE_IB_LOGFILE, "** not work for now** creates ib_logfile* also after '--prepare'. ### If you want create ib_logfile*, only re-execute this command in same options. ###",
  529. (G_PTR*) &xtrabackup_create_ib_logfile, (G_PTR*) &xtrabackup_create_ib_logfile,
  530. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  531. {"stream", OPT_XTRA_STREAM, "Stream all backup files to the standard output "
  532. "in the specified format."
  533. "Supported format is 'xbstream'."
  534. ,
  535. (G_PTR*) &xtrabackup_stream_str, (G_PTR*) &xtrabackup_stream_str, 0, GET_STR,
  536. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  537. {"compress", OPT_XTRA_COMPRESS, "Compress individual backup files using the "
  538. "specified compression algorithm. Currently the only supported algorithm "
  539. "is 'quicklz'. It is also the default algorithm, i.e. the one used when "
  540. "--compress is used without an argument.",
  541. (G_PTR*) &xtrabackup_compress_alg, (G_PTR*) &xtrabackup_compress_alg, 0,
  542. GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
  543. {"compress-threads", OPT_XTRA_COMPRESS_THREADS,
  544. "Number of threads for parallel data compression. The default value is 1.",
  545. (G_PTR*) &xtrabackup_compress_threads, (G_PTR*) &xtrabackup_compress_threads,
  546. 0, GET_UINT, REQUIRED_ARG, 1, 1, UINT_MAX, 0, 0, 0},
  547. {"compress-chunk-size", OPT_XTRA_COMPRESS_CHUNK_SIZE,
  548. "Size of working buffer(s) for compression threads in bytes. The default value is 64K.",
  549. (G_PTR*) &xtrabackup_compress_chunk_size, (G_PTR*) &xtrabackup_compress_chunk_size,
  550. 0, GET_ULL, REQUIRED_ARG, (1 << 16), 1024, ULONGLONG_MAX, 0, 0, 0},
  551. {"incremental-force-scan", OPT_XTRA_INCREMENTAL_FORCE_SCAN,
  552. "Perform a full-scan incremental backup even in the presence of changed "
  553. "page bitmap data",
  554. (G_PTR*)&xtrabackup_incremental_force_scan,
  555. (G_PTR*)&xtrabackup_incremental_force_scan, 0, GET_BOOL, NO_ARG,
  556. 0, 0, 0, 0, 0, 0},
  557. {"close_files", OPT_CLOSE_FILES, "do not keep files opened. Use at your own "
  558. "risk.", (G_PTR*) &xb_close_files, (G_PTR*) &xb_close_files, 0, GET_BOOL,
  559. NO_ARG, 0, 0, 0, 0, 0, 0},
  560. {"core-file", OPT_CORE_FILE, "Write core on fatal signals", 0, 0, 0,
  561. GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0},
  562. {"copy-back", OPT_COPY_BACK, "Copy all the files in a previously made "
  563. "backup from the backup directory to their original locations.",
  564. (uchar *) &xtrabackup_copy_back, (uchar *) &xtrabackup_copy_back, 0,
  565. GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  566. {"move-back", OPT_MOVE_BACK, "Move all the files in a previously made "
  567. "backup from the backup directory to the actual datadir location. "
  568. "Use with caution, as it removes backup files.",
  569. (uchar *) &xtrabackup_move_back, (uchar *) &xtrabackup_move_back, 0,
  570. GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  571. {"galera-info", OPT_GALERA_INFO, "This options creates the "
  572. "xtrabackup_galera_info file which contains the local node state at "
  573. "the time of the backup. Option should be used when performing the "
  574. "backup of Percona-XtraDB-Cluster. Has no effect when backup locks "
  575. "are used to create the backup.",
  576. (uchar *) &opt_galera_info, (uchar *) &opt_galera_info, 0,
  577. GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  578. {"slave-info", OPT_SLAVE_INFO, "This option is useful when backing "
  579. "up a replication slave server. It prints the binary log position "
  580. "and name of the master server. It also writes this information to "
  581. "the \"xtrabackup_slave_info\" file as a \"CHANGE MASTER\" command. "
  582. "A new slave for this master can be set up by starting a slave server "
  583. "on this backup and issuing a \"CHANGE MASTER\" command with the "
  584. "binary log position saved in the \"xtrabackup_slave_info\" file.",
  585. (uchar *) &opt_slave_info, (uchar *) &opt_slave_info, 0,
  586. GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  587. {"no-lock", OPT_NO_LOCK, "Use this option to disable table lock "
  588. "with \"FLUSH TABLES WITH READ LOCK\". Use it only if ALL your "
  589. "tables are InnoDB and you DO NOT CARE about the binary log "
  590. "position of the backup. This option shouldn't be used if there "
  591. "are any DDL statements being executed or if any updates are "
  592. "happening on non-InnoDB tables (this includes the system MyISAM "
  593. "tables in the mysql database), otherwise it could lead to an "
  594. "inconsistent backup. If you are considering to use --no-lock "
  595. "because your backups are failing to acquire the lock, this could "
  596. "be because of incoming replication events preventing the lock "
  597. "from succeeding. Please try using --safe-slave-backup to "
  598. "momentarily stop the replication slave thread, this may help "
  599. "the backup to succeed and you then don't need to resort to "
  600. "using this option.",
  601. (uchar *) &opt_no_lock, (uchar *) &opt_no_lock, 0,
  602. GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  603. {"safe-slave-backup", OPT_SAFE_SLAVE_BACKUP, "Stop slave SQL thread "
  604. "and wait to start backup until Slave_open_temp_tables in "
  605. "\"SHOW STATUS\" is zero. If there are no open temporary tables, "
  606. "the backup will take place, otherwise the SQL thread will be "
  607. "started and stopped until there are no open temporary tables. "
  608. "The backup will fail if Slave_open_temp_tables does not become "
  609. "zero after --safe-slave-backup-timeout seconds. The slave SQL "
  610. "thread will be restarted when the backup finishes.",
  611. (uchar *) &opt_safe_slave_backup,
  612. (uchar *) &opt_safe_slave_backup,
  613. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  614. {"rsync", OPT_RSYNC, "Uses the rsync utility to optimize local file "
  615. "transfers. When this option is specified, innobackupex uses rsync "
  616. "to copy all non-InnoDB files instead of spawning a separate cp for "
  617. "each file, which can be much faster for servers with a large number "
  618. "of databases or tables. This option cannot be used together with "
  619. "--stream.",
  620. (uchar *) &opt_rsync, (uchar *) &opt_rsync,
  621. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  622. {"force-non-empty-directories", OPT_FORCE_NON_EMPTY_DIRS, "This "
  623. "option, when specified, makes --copy-back or --move-back transfer "
  624. "files to non-empty directories. Note that no existing files will be "
  625. "overwritten. If --copy-back or --nove-back has to copy a file from "
  626. "the backup directory which already exists in the destination "
  627. "directory, it will still fail with an error.",
  628. (uchar *) &opt_force_non_empty_dirs,
  629. (uchar *) &opt_force_non_empty_dirs,
  630. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  631. {"no-version-check", OPT_NO_VERSION_CHECK, "This option disables the "
  632. "version check which is enabled by the --version-check option.",
  633. (uchar *) &opt_noversioncheck,
  634. (uchar *) &opt_noversioncheck,
  635. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  636. {"no-backup-locks", OPT_NO_BACKUP_LOCKS, "This option controls if "
  637. "backup locks should be used instead of FLUSH TABLES WITH READ LOCK "
  638. "on the backup stage. The option has no effect when backup locks are "
  639. "not supported by the server. This option is enabled by default, "
  640. "disable with --no-backup-locks.",
  641. (uchar *) &opt_no_backup_locks,
  642. (uchar *) &opt_no_backup_locks,
  643. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  644. {"decompress", OPT_DECOMPRESS, "Decompresses all files with the .qp "
  645. "extension in a backup previously made with the --compress option.",
  646. (uchar *) &opt_decompress,
  647. (uchar *) &opt_decompress,
  648. 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
  649. {"user", 'u', "This option specifies the MySQL username used "
  650. "when connecting to the server, if that's not the current user. "
  651. "The option accepts a string argument. See mysql --help for details.",
  652. (uchar*) &opt_user, (uchar*) &opt_user, 0, GET_STR,
  653. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  654. {"host", 'H', "This option specifies the host to use when "
  655. "connecting to the database server with TCP/IP. The option accepts "
  656. "a string argument. See mysql --help for details.",
  657. (uchar*) &opt_host, (uchar*) &opt_host, 0, GET_STR,
  658. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  659. {"port", 'P', "This option specifies the port to use when "
  660. "connecting to the database server with TCP/IP. The option accepts "
  661. "a string argument. See mysql --help for details.",
  662. &opt_port, &opt_port, 0, GET_UINT, REQUIRED_ARG,
  663. 0, 0, 0, 0, 0, 0},
  664. {"password", 'p', "This option specifies the password to use "
  665. "when connecting to the database. It accepts a string argument. "
  666. "See mysql --help for details.",
  667. 0, 0, 0, GET_STR,
  668. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  669. {"socket", 'S', "This option specifies the socket to use when "
  670. "connecting to the local database server with a UNIX domain socket. "
  671. "The option accepts a string argument. See mysql --help for details.",
  672. (uchar*) &opt_socket, (uchar*) &opt_socket, 0, GET_STR,
  673. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  674. {"incremental-history-name", OPT_INCREMENTAL_HISTORY_NAME,
  675. "This option specifies the name of the backup series stored in the "
  676. "PERCONA_SCHEMA.xtrabackup_history history record to base an "
  677. "incremental backup on. Xtrabackup will search the history table "
  678. "looking for the most recent (highest innodb_to_lsn), successful "
  679. "backup in the series and take the to_lsn value to use as the "
  680. "starting lsn for the incremental backup. This will be mutually "
  681. "exclusive with --incremental-history-uuid, --incremental-basedir "
  682. "and --incremental-lsn. If no valid lsn can be found (no series by "
  683. "that name, no successful backups by that name) xtrabackup will "
  684. "return with an error. It is used with the --incremental option.",
  685. (uchar*) &opt_incremental_history_name,
  686. (uchar*) &opt_incremental_history_name, 0, GET_STR,
  687. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  688. {"incremental-history-uuid", OPT_INCREMENTAL_HISTORY_UUID,
  689. "This option specifies the UUID of the specific history record "
  690. "stored in the PERCONA_SCHEMA.xtrabackup_history to base an "
  691. "incremental backup on. --incremental-history-name, "
  692. "--incremental-basedir and --incremental-lsn. If no valid lsn can be "
  693. "found (no success record with that uuid) xtrabackup will return "
  694. "with an error. It is used with the --incremental option.",
  695. (uchar*) &opt_incremental_history_uuid,
  696. (uchar*) &opt_incremental_history_uuid, 0, GET_STR,
  697. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  698. {"ftwrl-wait-query-type", OPT_LOCK_WAIT_QUERY_TYPE,
  699. "This option specifies which types of queries are allowed to complete "
  700. "before innobackupex will issue the global lock. Default is all.",
  701. (uchar*) &opt_lock_wait_query_type,
  702. (uchar*) &opt_lock_wait_query_type, &query_type_typelib,
  703. GET_ENUM, REQUIRED_ARG, QUERY_TYPE_ALL, 0, 0, 0, 0, 0},
  704. {"kill-long-query-type", OPT_KILL_LONG_QUERY_TYPE,
  705. "This option specifies which types of queries should be killed to "
  706. "unblock the global lock. Default is \"all\".",
  707. (uchar*) &opt_kill_long_query_type,
  708. (uchar*) &opt_kill_long_query_type, &query_type_typelib,
  709. GET_ENUM, REQUIRED_ARG, QUERY_TYPE_SELECT, 0, 0, 0, 0, 0},
  710. {"history", OPT_HISTORY,
  711. "This option enables the tracking of backup history in the "
  712. "PERCONA_SCHEMA.xtrabackup_history table. An optional history "
  713. "series name may be specified that will be placed with the history "
  714. "record for the current backup being taken.",
  715. NULL, NULL, 0, GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
  716. {"kill-long-queries-timeout", OPT_KILL_LONG_QUERIES_TIMEOUT,
  717. "This option specifies the number of seconds innobackupex waits "
  718. "between starting FLUSH TABLES WITH READ LOCK and killing those "
  719. "queries that block it. Default is 0 seconds, which means "
  720. "innobackupex will not attempt to kill any queries.",
  721. (uchar*) &opt_kill_long_queries_timeout,
  722. (uchar*) &opt_kill_long_queries_timeout, 0, GET_UINT,
  723. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  724. {"ftwrl-wait-timeout", OPT_LOCK_WAIT_TIMEOUT,
  725. "This option specifies time in seconds that innobackupex should wait "
  726. "for queries that would block FTWRL before running it. If there are "
  727. "still such queries when the timeout expires, innobackupex terminates "
  728. "with an error. Default is 0, in which case innobackupex does not "
  729. "wait for queries to complete and starts FTWRL immediately.",
  730. (uchar*) &opt_lock_wait_timeout,
  731. (uchar*) &opt_lock_wait_timeout, 0, GET_UINT,
  732. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  733. {"ftwrl-wait-threshold", OPT_LOCK_WAIT_THRESHOLD,
  734. "This option specifies the query run time threshold which is used by "
  735. "innobackupex to detect long-running queries with a non-zero value "
  736. "of --ftwrl-wait-timeout. FTWRL is not started until such "
  737. "long-running queries exist. This option has no effect if "
  738. "--ftwrl-wait-timeout is 0. Default value is 60 seconds.",
  739. (uchar*) &opt_lock_wait_threshold,
  740. (uchar*) &opt_lock_wait_threshold, 0, GET_UINT,
  741. REQUIRED_ARG, 60, 0, 0, 0, 0, 0},
  742. {"debug-sleep-before-unlock", OPT_DEBUG_SLEEP_BEFORE_UNLOCK,
  743. "This is a debug-only option used by the XtraBackup test suite.",
  744. (uchar*) &opt_debug_sleep_before_unlock,
  745. (uchar*) &opt_debug_sleep_before_unlock, 0, GET_UINT,
  746. REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  747. {"safe-slave-backup-timeout", OPT_SAFE_SLAVE_BACKUP_TIMEOUT,
  748. "How many seconds --safe-slave-backup should wait for "
  749. "Slave_open_temp_tables to become zero. (default 300)",
  750. (uchar*) &opt_safe_slave_backup_timeout,
  751. (uchar*) &opt_safe_slave_backup_timeout, 0, GET_UINT,
  752. REQUIRED_ARG, 300, 0, 0, 0, 0, 0},
  753. {"binlog-info", OPT_BINLOG_INFO,
  754. "This option controls how XtraBackup should retrieve server's binary log "
  755. "coordinates corresponding to the backup. Possible values are OFF, ON, "
  756. "LOCKLESS and AUTO. See the XtraBackup manual for more information",
  757. &opt_binlog_info, &opt_binlog_info,
  758. &binlog_info_typelib, GET_ENUM, OPT_ARG, BINLOG_INFO_AUTO, 0, 0, 0, 0, 0},
  759. {"secure-auth", OPT_XB_SECURE_AUTH, "Refuse client connecting to server if it"
  760. " uses old (pre-4.1.1) protocol.", &opt_secure_auth,
  761. &opt_secure_auth, 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0},
  762. #include "sslopt-longopts.h"
  763. { 0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}
  764. };
  765. uint xb_client_options_count = array_elements(xb_client_options);
  766. struct my_option xb_server_options[] =
  767. {
  768. {"datadir", 'h', "Path to the database root.", (G_PTR*) &mysql_data_home,
  769. (G_PTR*) &mysql_data_home, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  770. {"tmpdir", 't',
  771. "Path for temporary files. Several paths may be specified, separated by a "
  772. #if defined(__WIN__) || defined(OS2) || defined(__NETWARE__)
  773. "semicolon (;)"
  774. #else
  775. "colon (:)"
  776. #endif
  777. ", in this case they are used in a round-robin fashion.",
  778. (G_PTR*) &opt_mysql_tmpdir,
  779. (G_PTR*) &opt_mysql_tmpdir, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  780. {"parallel", OPT_XTRA_PARALLEL,
  781. "Number of threads to use for parallel datafiles transfer. "
  782. "The default value is 1.",
  783. (G_PTR*) &xtrabackup_parallel, (G_PTR*) &xtrabackup_parallel, 0, GET_INT,
  784. REQUIRED_ARG, 1, 1, INT_MAX, 0, 0, 0},
  785. {"log", OPT_LOG, "Ignored option for MySQL option compatibility",
  786. (G_PTR*) &log_ignored_opt, (G_PTR*) &log_ignored_opt, 0,
  787. GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
  788. {"log_bin", OPT_LOG, "Base name for the log sequence",
  789. &opt_log_bin, &opt_log_bin, 0, GET_STR_ALLOC, OPT_ARG, 0, 0, 0, 0, 0, 0},
  790. {"innodb", OPT_INNODB, "Ignored option for MySQL option compatibility",
  791. (G_PTR*) &innobase_ignored_opt, (G_PTR*) &innobase_ignored_opt, 0,
  792. GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
  793. {"innodb_adaptive_hash_index", OPT_INNODB_ADAPTIVE_HASH_INDEX,
  794. "Enable InnoDB adaptive hash index (enabled by default). "
  795. "Disable with --skip-innodb-adaptive-hash-index.",
  796. (G_PTR*) &innobase_adaptive_hash_index,
  797. (G_PTR*) &innobase_adaptive_hash_index,
  798. 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0},
  799. {"innodb_additional_mem_pool_size", OPT_INNODB_ADDITIONAL_MEM_POOL_SIZE,
  800. "Size of a memory pool InnoDB uses to store data dictionary information and other internal data structures.",
  801. (G_PTR*) &innobase_additional_mem_pool_size,
  802. (G_PTR*) &innobase_additional_mem_pool_size, 0, GET_LONG, REQUIRED_ARG,
  803. 1*1024*1024L, 512*1024L, LONG_MAX, 0, 1024, 0},
  804. {"innodb_autoextend_increment", OPT_INNODB_AUTOEXTEND_INCREMENT,
  805. "Data file autoextend increment in megabytes",
  806. (G_PTR*) &srv_auto_extend_increment,
  807. (G_PTR*) &srv_auto_extend_increment,
  808. 0, GET_ULONG, REQUIRED_ARG, 8L, 1L, 1000L, 0, 1L, 0},
  809. {"innodb_buffer_pool_size", OPT_INNODB_BUFFER_POOL_SIZE,
  810. "The size of the memory buffer InnoDB uses to cache data and indexes of its tables.",
  811. (G_PTR*) &innobase_buffer_pool_size, (G_PTR*) &innobase_buffer_pool_size, 0,
  812. GET_LL, REQUIRED_ARG, 8*1024*1024L, 1024*1024L, LONGLONG_MAX, 0,
  813. 1024*1024L, 0},
  814. {"innodb_checksums", OPT_INNODB_CHECKSUMS, "Enable InnoDB checksums validation (enabled by default). \
  815. Disable with --skip-innodb-checksums.", (G_PTR*) &innobase_use_checksums,
  816. (G_PTR*) &innobase_use_checksums, 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0},
  817. {"innodb_data_file_path", OPT_INNODB_DATA_FILE_PATH,
  818. "Path to individual files and their sizes.", &innobase_data_file_path,
  819. &innobase_data_file_path, 0, GET_STR_ALLOC, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  820. {"innodb_data_home_dir", OPT_INNODB_DATA_HOME_DIR,
  821. "The common part for InnoDB table spaces.", &innobase_data_home_dir,
  822. &innobase_data_home_dir, 0, GET_STR_ALLOC, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  823. {"innodb_doublewrite", OPT_INNODB_DOUBLEWRITE, "Enable InnoDB doublewrite buffer (enabled by default). \
  824. Disable with --skip-innodb-doublewrite.", (G_PTR*) &innobase_use_doublewrite,
  825. (G_PTR*) &innobase_use_doublewrite, 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0},
  826. {"innodb_io_capacity", OPT_INNODB_IO_CAPACITY,
  827. "Number of IOPs the server can do. Tunes the background IO rate",
  828. (G_PTR*) &srv_io_capacity, (G_PTR*) &srv_io_capacity,
  829. 0, GET_ULONG, OPT_ARG, 200, 100, ~0UL, 0, 0, 0},
  830. {"innodb_file_io_threads", OPT_INNODB_FILE_IO_THREADS,
  831. "Number of file I/O threads in InnoDB.", (G_PTR*) &innobase_file_io_threads,
  832. (G_PTR*) &innobase_file_io_threads, 0, GET_LONG, REQUIRED_ARG, 4, 4, 64, 0,
  833. 1, 0},
  834. {"innodb_read_io_threads", OPT_INNODB_READ_IO_THREADS,
  835. "Number of background read I/O threads in InnoDB.", (G_PTR*) &innobase_read_io_threads,
  836. (G_PTR*) &innobase_read_io_threads, 0, GET_LONG, REQUIRED_ARG, 4, 1, 64, 0,
  837. 1, 0},
  838. {"innodb_write_io_threads", OPT_INNODB_WRITE_IO_THREADS,
  839. "Number of background write I/O threads in InnoDB.", (G_PTR*) &innobase_write_io_threads,
  840. (G_PTR*) &innobase_write_io_threads, 0, GET_LONG, REQUIRED_ARG, 4, 1, 64, 0,
  841. 1, 0},
  842. {"innodb_file_per_table", OPT_INNODB_FILE_PER_TABLE,
  843. "Stores each InnoDB table to an .ibd file in the database dir.",
  844. (G_PTR*) &innobase_file_per_table,
  845. (G_PTR*) &innobase_file_per_table, 0, GET_BOOL, NO_ARG,
  846. FALSE, 0, 0, 0, 0, 0},
  847. {"innodb_flush_method", OPT_INNODB_FLUSH_METHOD,
  848. "With which method to flush data.", (G_PTR*) &innobase_unix_file_flush_method,
  849. (G_PTR*) &innobase_unix_file_flush_method, 0, GET_STR, REQUIRED_ARG, 0, 0, 0,
  850. 0, 0, 0},
  851. /* ####### Should we use this option? ####### */
  852. {"innodb_force_recovery", OPT_INNODB_FORCE_RECOVERY,
  853. "Helps to save your data in case the disk image of the database becomes corrupt.",
  854. (G_PTR*) &innobase_force_recovery, (G_PTR*) &innobase_force_recovery, 0,
  855. GET_LONG, REQUIRED_ARG, 0, 0, 6, 0, 1, 0},
  856. {"innodb_log_buffer_size", OPT_INNODB_LOG_BUFFER_SIZE,
  857. "The size of the buffer which InnoDB uses to write log to the log files on disk.",
  858. (G_PTR*) &innobase_log_buffer_size, (G_PTR*) &innobase_log_buffer_size, 0,
  859. GET_LONG, REQUIRED_ARG, 1024*1024L, 256*1024L, LONG_MAX, 0, 1024, 0},
  860. {"innodb_log_file_size", OPT_INNODB_LOG_FILE_SIZE,
  861. "Size of each log file in a log group.",
  862. (G_PTR*) &innobase_log_file_size, (G_PTR*) &innobase_log_file_size, 0,
  863. GET_LL, REQUIRED_ARG, 48*1024*1024L, 1*1024*1024L, LONGLONG_MAX, 0,
  864. 1024*1024L, 0},
  865. {"innodb_log_files_in_group", OPT_INNODB_LOG_FILES_IN_GROUP,
  866. "Number of log files in the log group. InnoDB writes to the files in a "
  867. "circular fashion. Value 3 is recommended here.",
  868. &innobase_log_files_in_group, &innobase_log_files_in_group,
  869. 0, GET_LONG, REQUIRED_ARG, 2, 2, 100, 0, 1, 0},
  870. {"innodb_log_group_home_dir", OPT_INNODB_LOG_GROUP_HOME_DIR,
  871. "Path to InnoDB log files.", &srv_log_group_home_dir,
  872. &srv_log_group_home_dir, 0, GET_STR_ALLOC, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  873. {"innodb_max_dirty_pages_pct", OPT_INNODB_MAX_DIRTY_PAGES_PCT,
  874. "Percentage of dirty pages allowed in bufferpool.", (G_PTR*) &srv_max_buf_pool_modified_pct,
  875. (G_PTR*) &srv_max_buf_pool_modified_pct, 0, GET_ULONG, REQUIRED_ARG, 90, 0, 100, 0, 0, 0},
  876. {"innodb_open_files", OPT_INNODB_OPEN_FILES,
  877. "How many files at the maximum InnoDB keeps open at the same time.",
  878. (G_PTR*) &innobase_open_files, (G_PTR*) &innobase_open_files, 0,
  879. GET_LONG, REQUIRED_ARG, 300L, 10L, LONG_MAX, 0, 1L, 0},
  880. {"innodb_use_native_aio", OPT_INNODB_USE_NATIVE_AIO,
  881. "Use native AIO if supported on this platform.",
  882. (G_PTR*) &srv_use_native_aio,
  883. (G_PTR*) &srv_use_native_aio, 0, GET_BOOL, NO_ARG,
  884. FALSE, 0, 0, 0, 0, 0},
  885. {"innodb_page_size", OPT_INNODB_PAGE_SIZE,
  886. "The universal page size of the database.",
  887. (G_PTR*) &innobase_page_size, (G_PTR*) &innobase_page_size, 0,
  888. /* Use GET_LL to support numeric suffixes in 5.6 */
  889. GET_LL, REQUIRED_ARG,
  890. (1LL << 14), (1LL << 12), (1LL << UNIV_PAGE_SIZE_SHIFT_MAX), 0, 1L, 0},
  891. {"innodb_log_block_size", OPT_INNODB_LOG_BLOCK_SIZE,
  892. "The log block size of the transaction log file. "
  893. "Changing for created log file is not supported. Use on your own risk!",
  894. (G_PTR*) &innobase_log_block_size, (G_PTR*) &innobase_log_block_size, 0,
  895. GET_ULONG, REQUIRED_ARG, 512, 512, 1 << UNIV_PAGE_SIZE_SHIFT_MAX, 0, 1L, 0},
  896. {"innodb_doublewrite_file", OPT_INNODB_DOUBLEWRITE_FILE,
  897. "Path to special datafile for doublewrite buffer. (default is "": not used)",
  898. (G_PTR*) &innobase_doublewrite_file, (G_PTR*) &innobase_doublewrite_file,
  899. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  900. {"innodb_buffer_pool_filename", OPT_INNODB_BUFFER_POOL_FILENAME,
  901. "Filename to/from which to dump/load the InnoDB buffer pool",
  902. (G_PTR*) &innobase_buffer_pool_filename,
  903. (G_PTR*) &innobase_buffer_pool_filename,
  904. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  905. #ifndef __WIN__
  906. {"debug-sync", OPT_XTRA_DEBUG_SYNC,
  907. "Debug sync point. This is only used by the xtrabackup test suite",
  908. (G_PTR*) &xtrabackup_debug_sync,
  909. (G_PTR*) &xtrabackup_debug_sync,
  910. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  911. #endif
  912. {"innodb_checksum_algorithm", OPT_INNODB_CHECKSUM_ALGORITHM,
  913. "The algorithm InnoDB uses for page checksumming. [CRC32, STRICT_CRC32, "
  914. "INNODB, STRICT_INNODB, NONE, STRICT_NONE]", &srv_checksum_algorithm,
  915. &srv_checksum_algorithm, &innodb_checksum_algorithm_typelib, GET_ENUM,
  916. REQUIRED_ARG, SRV_CHECKSUM_ALGORITHM_INNODB, 0, 0, 0, 0, 0},
  917. {"innodb_log_checksum_algorithm", OPT_INNODB_LOG_CHECKSUM_ALGORITHM,
  918. "The algorithm InnoDB uses for log checksumming. [CRC32, STRICT_CRC32, "
  919. "INNODB, STRICT_INNODB, NONE, STRICT_NONE]", &srv_log_checksum_algorithm,
  920. &srv_log_checksum_algorithm, &innodb_checksum_algorithm_typelib, GET_ENUM,
  921. REQUIRED_ARG, SRV_CHECKSUM_ALGORITHM_INNODB, 0, 0, 0, 0, 0},
  922. {"innodb_undo_directory", OPT_INNODB_UNDO_DIRECTORY,
  923. "Directory where undo tablespace files live, this path can be absolute.",
  924. &srv_undo_dir, &srv_undo_dir, 0, GET_STR_ALLOC, REQUIRED_ARG, 0, 0, 0, 0, 0,
  925. 0},
  926. {"innodb_undo_tablespaces", OPT_INNODB_UNDO_TABLESPACES,
  927. "Number of undo tablespaces to use.",
  928. (G_PTR*)&srv_undo_tablespaces, (G_PTR*)&srv_undo_tablespaces,
  929. 0, GET_ULONG, REQUIRED_ARG, 0, 0, 126, 0, 1, 0},
  930. {"defaults_group", OPT_DEFAULTS_GROUP, "defaults group in config file (default \"mysqld\").",
  931. (G_PTR*) &defaults_group, (G_PTR*) &defaults_group,
  932. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
  933. {"plugin-dir", OPT_PLUGIN_DIR, "Server plugin directory",
  934. &xb_plugin_dir, &xb_plugin_dir,
  935. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0 },
  936. { "plugin-load", OPT_PLUGIN_LOAD, "encrypton plugin to load",
  937. &xb_plugin_load, &xb_plugin_load,
  938. 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0 },
  939. { "innodb-encrypt-log", OPT_INNODB_ENCRYPT_LOG, "encrypton plugin to load",
  940. &srv_encrypt_log, &srv_encrypt_log,
  941. 0, GET_BOOL, REQUIRED_ARG, 0, 0, 0, 0, 0, 0 },
  942. {"open_files_limit", OPT_OPEN_FILES_LIMIT, "the maximum number of file "
  943. "descriptors to reserve with setrlimit().",
  944. (G_PTR*) &xb_open_files_limit, (G_PTR*) &xb_open_files_limit, 0, GET_ULONG,
  945. REQUIRED_ARG, 0, 0, UINT_MAX, 0, 1, 0},
  946. { 0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}
  947. };
  948. uint xb_server_options_count = array_elements(xb_server_options);
  949. #ifndef __WIN__
  950. static int debug_sync_resumed;
  951. static void sigcont_handler(int sig);
  952. static void sigcont_handler(int sig __attribute__((unused)))
  953. {
  954. debug_sync_resumed= 1;
  955. }
  956. #endif
  957. static inline
  958. void
  959. debug_sync_point(const char *name)
  960. {
  961. #ifndef __WIN__
  962. FILE *fp;
  963. pid_t pid;
  964. char pid_path[FN_REFLEN];
  965. if (xtrabackup_debug_sync == NULL) {
  966. return;
  967. }
  968. if (strcmp(xtrabackup_debug_sync, name)) {
  969. return;
  970. }
  971. pid = getpid();
  972. snprintf(pid_path, sizeof(pid_path), "%s/xtrabackup_debug_sync",
  973. xtrabackup_target_dir);
  974. fp = fopen(pid_path, "w");
  975. if (fp == NULL) {
  976. msg("xtrabackup: Error: cannot open %s\n", pid_path);
  977. exit(EXIT_FAILURE);
  978. }
  979. fprintf(fp, "%u\n", (uint) pid);
  980. fclose(fp);
  981. msg("xtrabackup: DEBUG: Suspending at debug sync point '%s'. "
  982. "Resume with 'kill -SIGCONT %u'.\n", name, (uint) pid);
  983. debug_sync_resumed= 0;
  984. kill(pid, SIGSTOP);
  985. while (!debug_sync_resumed) {
  986. sleep(1);
  987. }
  988. /* On resume */
  989. msg("xtrabackup: DEBUG: removing the pid file.\n");
  990. my_delete(pid_path, MYF(MY_WME));
  991. #endif
  992. }
  993. static const char *xb_client_default_groups[]=
  994. { "xtrabackup", "client", 0, 0, 0 };
  995. static const char *xb_server_default_groups[]=
  996. { "xtrabackup", "mysqld", 0, 0, 0 };
  997. static void print_version(void)
  998. {
  999. msg("%s based on MariaDB server %s %s (%s) \n",
  1000. my_progname, MYSQL_SERVER_VERSION, SYSTEM_TYPE, MACHINE_TYPE);
  1001. }
  1002. static void usage(void)
  1003. {
  1004. puts("Open source backup tool for InnoDB and XtraDB\n\
  1005. \n\
  1006. Copyright (C) 2009-2015 Percona LLC and/or its affiliates.\n\
  1007. Portions Copyright (C) 2000, 2011, MySQL AB & Innobase Oy. All Rights Reserved.\n\
  1008. \n\
  1009. This program is free software; you can redistribute it and/or\n\
  1010. modify it under the terms of the GNU General Public License\n\
  1011. as published by the Free Software Foundation version 2\n\
  1012. of the License.\n\
  1013. \n\
  1014. This program is distributed in the hope that it will be useful,\n\
  1015. but WITHOUT ANY WARRANTY; without even the implied warranty of\n\
  1016. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n\
  1017. GNU General Public License for more details.\n\
  1018. \n\
  1019. You can download full text of the license on http://www.gnu.org/licenses/gpl-2.0.txt\n");
  1020. printf("Usage: [%s [--defaults-file=#] --backup | %s [--defaults-file=#] --prepare] [OPTIONS]\n",my_progname,my_progname);
  1021. print_defaults("my", xb_server_default_groups);
  1022. my_print_help(xb_client_options);
  1023. my_print_help(xb_server_options);
  1024. my_print_variables(xb_server_options);
  1025. my_print_variables(xb_client_options);
  1026. }
  1027. #define ADD_PRINT_PARAM_OPT(value) \
  1028. { \
  1029. print_param_str << opt->name << "=" << value << "\n"; \
  1030. param_set.insert(opt->name); \
  1031. }
  1032. /************************************************************************
  1033. Check if parameter is set in defaults file or via command line argument
  1034. @return true if parameter is set. */
  1035. bool
  1036. check_if_param_set(const char *param)
  1037. {
  1038. return param_set.find(param) != param_set.end();
  1039. }
  1040. my_bool
  1041. xb_get_one_option(int optid,
  1042. const struct my_option *opt __attribute__((unused)),
  1043. char *argument)
  1044. {
  1045. switch(optid) {
  1046. case 'h':
  1047. strmake(mysql_real_data_home,argument, FN_REFLEN - 1);
  1048. mysql_data_home= mysql_real_data_home;
  1049. ADD_PRINT_PARAM_OPT(mysql_real_data_home);
  1050. break;
  1051. case 't':
  1052. ADD_PRINT_PARAM_OPT(opt_mysql_tmpdir);
  1053. break;
  1054. case OPT_INNODB_DATA_HOME_DIR:
  1055. ADD_PRINT_PARAM_OPT(innobase_data_home_dir);
  1056. break;
  1057. case OPT_INNODB_DATA_FILE_PATH:
  1058. ADD_PRINT_PARAM_OPT(innobase_data_file_path);
  1059. break;
  1060. case OPT_INNODB_LOG_GROUP_HOME_DIR:
  1061. ADD_PRINT_PARAM_OPT(srv_log_group_home_dir);
  1062. break;
  1063. case OPT_INNODB_LOG_FILES_IN_GROUP:
  1064. ADD_PRINT_PARAM_OPT(innobase_log_files_in_group);
  1065. break;
  1066. case OPT_INNODB_LOG_FILE_SIZE:
  1067. ADD_PRINT_PARAM_OPT(innobase_log_file_size);
  1068. break;
  1069. case OPT_INNODB_FLUSH_METHOD:
  1070. ADD_PRINT_PARAM_OPT(innobase_unix_file_flush_method);
  1071. break;
  1072. case OPT_INNODB_PAGE_SIZE:
  1073. ADD_PRINT_PARAM_OPT(innobase_page_size);
  1074. break;
  1075. case OPT_INNODB_LOG_BLOCK_SIZE:
  1076. ADD_PRINT_PARAM_OPT(innobase_log_block_size);
  1077. break;
  1078. case OPT_INNODB_DOUBLEWRITE_FILE:
  1079. ADD_PRINT_PARAM_OPT(innobase_doublewrite_file);
  1080. break;
  1081. case OPT_INNODB_UNDO_DIRECTORY:
  1082. ADD_PRINT_PARAM_OPT(srv_undo_dir);
  1083. break;
  1084. case OPT_INNODB_UNDO_TABLESPACES:
  1085. ADD_PRINT_PARAM_OPT(srv_undo_tablespaces);
  1086. break;
  1087. case OPT_INNODB_CHECKSUM_ALGORITHM:
  1088. ut_a(srv_checksum_algorithm <= SRV_CHECKSUM_ALGORITHM_STRICT_NONE);
  1089. ADD_PRINT_PARAM_OPT(innodb_checksum_algorithm_names[srv_checksum_algorithm]);
  1090. break;
  1091. case OPT_INNODB_LOG_CHECKSUM_ALGORITHM:
  1092. ut_a(srv_log_checksum_algorithm <= SRV_CHECKSUM_ALGORITHM_STRICT_NONE);
  1093. ADD_PRINT_PARAM_OPT(innodb_checksum_algorithm_names[srv_log_checksum_algorithm]);
  1094. break;
  1095. case OPT_INNODB_BUFFER_POOL_FILENAME:
  1096. ADD_PRINT_PARAM_OPT(innobase_buffer_pool_filename);
  1097. break;
  1098. case OPT_XTRA_TARGET_DIR:
  1099. strmake(xtrabackup_real_target_dir,argument, sizeof(xtrabackup_real_target_dir)-1);
  1100. xtrabackup_target_dir= xtrabackup_real_target_dir;
  1101. break;
  1102. case OPT_XTRA_STREAM:
  1103. if (!strcasecmp(argument, "xbstream"))
  1104. xtrabackup_stream_fmt = XB_STREAM_FMT_XBSTREAM;
  1105. else
  1106. {
  1107. msg("Invalid --stream argument: %s\n", argument);
  1108. return 1;
  1109. }
  1110. xtrabackup_stream = TRUE;
  1111. break;
  1112. case OPT_XTRA_COMPRESS:
  1113. if (argument == NULL)
  1114. xtrabackup_compress_alg = "quicklz";
  1115. else if (strcasecmp(argument, "quicklz"))
  1116. {
  1117. msg("Invalid --compress argument: %s\n", argument);
  1118. return 1;
  1119. }
  1120. xtrabackup_compress = TRUE;
  1121. break;
  1122. case OPT_DECOMPRESS:
  1123. opt_decompress = TRUE;
  1124. xtrabackup_decrypt_decompress = true;
  1125. break;
  1126. case (int) OPT_CORE_FILE:
  1127. test_flags |= TEST_CORE_ON_SIGNAL;
  1128. break;
  1129. case OPT_HISTORY:
  1130. if (argument) {
  1131. opt_history = argument;
  1132. } else {
  1133. opt_history = "";
  1134. }
  1135. break;
  1136. case 'p':
  1137. if (argument)
  1138. {
  1139. char *start= argument;
  1140. my_free(opt_password);
  1141. opt_password= my_strdup(argument, MYF(MY_FAE));
  1142. while (*argument) *argument++= 'x'; // Destroy argument
  1143. if (*start)
  1144. start[1]=0 ;
  1145. }
  1146. break;
  1147. #include "sslopt-case.h"
  1148. case '?':
  1149. usage();
  1150. exit(EXIT_SUCCESS);
  1151. break;
  1152. case 'v':
  1153. print_version();
  1154. exit(EXIT_SUCCESS);
  1155. break;
  1156. default:
  1157. break;
  1158. }
  1159. return 0;
  1160. }
  1161. /***********************************************************************
  1162. Initializes log_block_size */
  1163. static
  1164. ibool
  1165. xb_init_log_block_size(void)
  1166. {
  1167. srv_log_block_size = 0;
  1168. if (innobase_log_block_size != 512) {
  1169. uint n_shift = (uint)get_bit_shift(innobase_log_block_size);;
  1170. if (n_shift > 0) {
  1171. srv_log_block_size = (ulint)(1LL << n_shift);
  1172. msg("InnoDB: The log block size is set to %lu.\n",
  1173. srv_log_block_size);
  1174. }
  1175. } else {
  1176. srv_log_block_size = 512;
  1177. }
  1178. if (!srv_log_block_size) {
  1179. msg("InnoDB: Error: %lu is not valid value for "
  1180. "innodb_log_block_size.\n", innobase_log_block_size);
  1181. return FALSE;
  1182. }
  1183. return TRUE;
  1184. }
  1185. static my_bool
  1186. innodb_init_param(void)
  1187. {
  1188. /* innobase_init */
  1189. static char current_dir[3]; /* Set if using current lib */
  1190. my_bool ret;
  1191. char *default_path;
  1192. srv_is_being_started = TRUE;
  1193. /* === some variables from mysqld === */
  1194. memset((G_PTR) &mysql_tmpdir_list, 0, sizeof(mysql_tmpdir_list));
  1195. if (init_tmpdir(&mysql_tmpdir_list, opt_mysql_tmpdir))
  1196. exit(EXIT_FAILURE);
  1197. xtrabackup_tmpdir = my_tmpdir(&mysql_tmpdir_list);
  1198. /* dummy for initialize all_charsets[] */
  1199. get_charset_name(0);
  1200. srv_page_size = 0;
  1201. srv_page_size_shift = 0;
  1202. if (innobase_page_size != (1LL << 14)) {
  1203. int n_shift = (int)get_bit_shift((ulint) innobase_page_size);
  1204. if (n_shift >= 12 && n_shift <= UNIV_PAGE_SIZE_SHIFT_MAX) {
  1205. srv_page_size_shift = n_shift;
  1206. srv_page_size = 1 << n_shift;
  1207. msg("InnoDB: The universal page size of the "
  1208. "database is set to %lu.\n", srv_page_size);
  1209. } else {
  1210. msg("InnoDB: Error: invalid value of "
  1211. "innobase_page_size: %lld", innobase_page_size);
  1212. exit(EXIT_FAILURE);
  1213. }
  1214. } else {
  1215. srv_page_size_shift = 14;
  1216. srv_page_size = (1 << srv_page_size_shift);
  1217. }
  1218. if (!xb_init_log_block_size()) {
  1219. goto error;
  1220. }
  1221. /* Check that values don't overflow on 32-bit systems. */
  1222. if (sizeof(ulint) == 4) {
  1223. if (xtrabackup_use_memory > UINT_MAX32) {
  1224. msg("xtrabackup: use-memory can't be over 4GB"
  1225. " on 32-bit systems\n");
  1226. }
  1227. if (innobase_buffer_pool_size > UINT_MAX32) {
  1228. msg("xtrabackup: innobase_buffer_pool_size can't be "
  1229. "over 4GB on 32-bit systems\n");
  1230. goto error;
  1231. }
  1232. if (innobase_log_file_size > UINT_MAX32) {
  1233. msg("xtrabackup: innobase_log_file_size can't be "
  1234. "over 4GB on 32-bit systemsi\n");
  1235. goto error;
  1236. }
  1237. }
  1238. os_innodb_umask = (ulint)0664;
  1239. /* First calculate the default path for innodb_data_home_dir etc.,
  1240. in case the user has not given any value.
  1241. Note that when using the embedded server, the datadirectory is not
  1242. necessarily the current directory of this program. */
  1243. /* It's better to use current lib, to keep paths short */
  1244. current_dir[0] = FN_CURLIB;
  1245. current_dir[1] = FN_LIBCHAR;
  1246. current_dir[2] = 0;
  1247. default_path = current_dir;
  1248. ut_a(default_path);
  1249. /* Set InnoDB initialization parameters according to the values
  1250. read from MySQL .cnf file */
  1251. if (xtrabackup_backup) {
  1252. msg("xtrabackup: using the following InnoDB configuration:\n");
  1253. } else {
  1254. msg("xtrabackup: using the following InnoDB configuration "
  1255. "for recovery:\n");
  1256. }
  1257. /*--------------- Data files -------------------------*/
  1258. /* The default dir for data files is the datadir of MySQL */
  1259. srv_data_home = (xtrabackup_backup && innobase_data_home_dir
  1260. ? innobase_data_home_dir : default_path);
  1261. msg("xtrabackup: innodb_data_home_dir = %s\n", srv_data_home);
  1262. /* Set default InnoDB data file size to 10 MB and let it be
  1263. auto-extending. Thus users can use InnoDB in >= 4.0 without having
  1264. to specify any startup options. */
  1265. if (!innobase_data_file_path) {
  1266. innobase_data_file_path = (char*) "ibdata1:10M:autoextend";
  1267. }
  1268. msg("xtrabackup: innodb_data_file_path = %s\n",
  1269. innobase_data_file_path);
  1270. /* Since InnoDB edits the argument in the next call, we make another
  1271. copy of it: */
  1272. internal_innobase_data_file_path = strdup(innobase_data_file_path);
  1273. ret = (my_bool) srv_parse_data_file_paths_and_sizes(
  1274. internal_innobase_data_file_path);
  1275. if (ret == FALSE) {
  1276. msg("xtrabackup: syntax error in innodb_data_file_path\n");
  1277. mem_free_and_error:
  1278. free(internal_innobase_data_file_path);
  1279. internal_innobase_data_file_path = NULL;
  1280. goto error;
  1281. }
  1282. if (xtrabackup_prepare) {
  1283. /* "--prepare" needs filenames only */
  1284. ulint i;
  1285. for (i=0; i < srv_n_data_files; i++) {
  1286. char *p;
  1287. p = srv_data_file_names[i];
  1288. while ((p = strchr(p, SRV_PATH_SEPARATOR)) != NULL)
  1289. {
  1290. p++;
  1291. srv_data_file_names[i] = p;
  1292. }
  1293. }
  1294. }
  1295. /* -------------- Log files ---------------------------*/
  1296. /* The default dir for log files is the datadir of MySQL */
  1297. if (!(xtrabackup_backup && srv_log_group_home_dir)) {
  1298. srv_log_group_home_dir = default_path;
  1299. }
  1300. if (xtrabackup_prepare && xtrabackup_incremental_dir) {
  1301. srv_log_group_home_dir = xtrabackup_incremental_dir;
  1302. }
  1303. msg("xtrabackup: innodb_log_group_home_dir = %s\n",
  1304. srv_log_group_home_dir);
  1305. srv_normalize_path_for_win(srv_log_group_home_dir);
  1306. if (strchr(srv_log_group_home_dir, ';')) {
  1307. msg("syntax error in innodb_log_group_home_dir, ");
  1308. goto mem_free_and_error;
  1309. }
  1310. srv_adaptive_flushing = FALSE;
  1311. srv_use_sys_malloc = TRUE;
  1312. srv_file_format = 1; /* Barracuda */
  1313. srv_max_file_format_at_startup = UNIV_FORMAT_MIN; /* on */
  1314. /* --------------------------------------------------*/
  1315. srv_file_flush_method_str = innobase_unix_file_flush_method;
  1316. srv_n_log_files = (ulint) innobase_log_files_in_group;
  1317. srv_log_file_size = (ulint) innobase_log_file_size;
  1318. msg("xtrabackup: innodb_log_files_in_group = %ld\n",
  1319. srv_n_log_files);
  1320. msg("xtrabackup: innodb_log_file_size = %lld\n",
  1321. (long long int) srv_log_file_size);
  1322. srv_log_buffer_size = (ulint) innobase_log_buffer_size;
  1323. /* We set srv_pool_size here in units of 1 kB. InnoDB internally
  1324. changes the value so that it becomes the number of database pages. */
  1325. //srv_buf_pool_size = (ulint) innobase_buffer_pool_size;
  1326. srv_buf_pool_size = (ulint) xtrabackup_use_memory;
  1327. srv_mem_pool_size = (ulint) innobase_additional_mem_pool_size;
  1328. srv_n_file_io_threads = (ulint) innobase_file_io_threads;
  1329. srv_n_read_io_threads = (ulint) innobase_read_io_threads;
  1330. srv_n_write_io_threads = (ulint) innobase_write_io_threads;
  1331. srv_force_recovery = (ulint) innobase_force_recovery;
  1332. srv_use_doublewrite_buf = (ibool) innobase_use_doublewrite;
  1333. if (!innobase_use_checksums) {
  1334. srv_checksum_algorithm = SRV_CHECKSUM_ALGORITHM_NONE;
  1335. }
  1336. btr_search_enabled = (char) innobase_adaptive_hash_index;
  1337. btr_search_index_num = 1;
  1338. os_use_large_pages = (ibool) innobase_use_large_pages;
  1339. os_large_page_size = (ulint) innobase_large_page_size;
  1340. static char default_dir[3] = "./";
  1341. srv_arch_dir = default_dir;
  1342. row_rollback_on_timeout = (ibool) innobase_rollback_on_timeout;
  1343. srv_file_per_table = (my_bool) innobase_file_per_table;
  1344. srv_locks_unsafe_for_binlog = (ibool) innobase_locks_unsafe_for_binlog;
  1345. srv_max_n_open_files = (ulint) innobase_open_files;
  1346. srv_innodb_status = (ibool) innobase_create_status_file;
  1347. srv_print_verbose_log = 1;
  1348. /* Store the default charset-collation number of this MySQL
  1349. installation */
  1350. /* We cannot treat characterset here for now!! */
  1351. data_mysql_default_charset_coll = (ulint)default_charset_info->number;
  1352. ut_a(DATA_MYSQL_LATIN1_SWEDISH_CHARSET_COLL ==
  1353. my_charset_latin1.number);
  1354. ut_a(DATA_MYSQL_BINARY_CHARSET_COLL == my_charset_bin.number);
  1355. /* Store the latin1_swedish_ci character ordering table to InnoDB. For
  1356. non-latin1_swedish_ci charsets we use the MySQL comparison functions,
  1357. and consequently we do not need to know the ordering internally in
  1358. InnoDB. */
  1359. ut_a(0 == strcmp(my_charset_latin1.name, "latin1_swedish_ci"));
  1360. srv_latin1_ordering = my_charset_latin1.sort_order;
  1361. //innobase_commit_concurrency_init_default();
  1362. /* Since we in this module access directly the fields of a trx
  1363. struct, and due to different headers and flags it might happen that
  1364. mutex_t has a different size in this module and in InnoDB
  1365. modules, we check at run time that the size is the same in
  1366. these compilation modules. */
  1367. /* On 5.5+ srv_use_native_aio is TRUE by default. It is later reset
  1368. if it is not supported by the platform in
  1369. innobase_start_or_create_for_mysql(). As we don't call it in xtrabackup,
  1370. we have to duplicate checks from that function here. */
  1371. #ifdef __WIN__
  1372. switch (os_get_os_version()) {
  1373. case OS_WIN95:
  1374. case OS_WIN31:
  1375. case OS_WINNT:
  1376. /* On Win 95, 98, ME, Win32 subsystem for Windows 3.1,
  1377. and NT use simulated aio. In NT Windows provides async i/o,
  1378. but when run in conjunction with InnoDB Hot Backup, it seemed
  1379. to corrupt the data files. */
  1380. srv_use_native_aio = FALSE;
  1381. break;
  1382. case OS_WIN2000:
  1383. case OS_WINXP:
  1384. /* On 2000 and XP, async IO is available. */
  1385. srv_use_native_aio = TRUE;
  1386. break;
  1387. default:
  1388. /* Vista and later have both async IO and condition variables */
  1389. srv_use_native_aio = TRUE;
  1390. srv_use_native_conditions = TRUE;
  1391. break;
  1392. }
  1393. #elif defined(LINUX_NATIVE_AIO)
  1394. if (srv_use_native_aio) {
  1395. ut_print_timestamp(stderr);
  1396. msg(" InnoDB: Using Linux native AIO\n");
  1397. }
  1398. #else
  1399. /* Currently native AIO is supported only on windows and linux
  1400. and that also when the support is compiled in. In all other
  1401. cases, we ignore the setting of innodb_use_native_aio. */
  1402. srv_use_native_aio = FALSE;
  1403. #endif
  1404. /* Assign the default value to srv_undo_dir if it's not specified, as
  1405. my_getopt does not support default values for string options. We also
  1406. ignore the option and override innodb_undo_directory on --prepare,
  1407. because separate undo tablespaces are copied to the root backup
  1408. directory. */
  1409. if (!srv_undo_dir || !xtrabackup_backup) {
  1410. my_free(srv_undo_dir);
  1411. srv_undo_dir = my_strdup(".", MYF(MY_FAE));
  1412. }
  1413. innodb_log_checksum_func_update(srv_log_checksum_algorithm);
  1414. return(FALSE);
  1415. error:
  1416. msg("xtrabackup: innodb_init_param(): Error occured.\n");
  1417. return(TRUE);
  1418. }
  1419. static my_bool
  1420. innodb_init(void)
  1421. {
  1422. int err;
  1423. srv_is_being_started = TRUE;
  1424. err = innobase_start_or_create_for_mysql();
  1425. if (err != DB_SUCCESS) {
  1426. free(internal_innobase_data_file_path);
  1427. internal_innobase_data_file_path = NULL;
  1428. goto error;
  1429. }
  1430. /* They may not be needed for now */
  1431. // (void) hash_init(&innobase_open_tables,system_charset_info, 32, 0, 0,
  1432. // (hash_get_key) innobase_get_key, 0, 0);
  1433. // pthread_mutex_init(&innobase_share_mutex, MY_MUTEX_INIT_FAST);
  1434. // pthread_mutex_init(&prepare_commit_mutex, MY_MUTEX_INIT_FAST);
  1435. // pthread_mutex_init(&commit_threads_m, MY_MUTEX_INIT_FAST);
  1436. // pthread_mutex_init(&commit_cond_m, MY_MUTEX_INIT_FAST);
  1437. // pthread_cond_init(&commit_cond, NULL);
  1438. innodb_inited= 1;
  1439. return(FALSE);
  1440. error:
  1441. msg("xtrabackup: innodb_init(): Error occured.\n");
  1442. return(TRUE);
  1443. }
  1444. static void
  1445. innodb_end()
  1446. {
  1447. srv_fast_shutdown = (ulint) innobase_fast_shutdown;
  1448. innodb_inited = 0;
  1449. msg("xtrabackup: starting shutdown with innodb_fast_shutdown = %lu\n",
  1450. srv_fast_shutdown);
  1451. innodb_shutdown();
  1452. free(internal_innobase_data_file_path);
  1453. internal_innobase_data_file_path = NULL;
  1454. /* They may not be needed for now */
  1455. // hash_free(&innobase_open_tables);
  1456. // pthread_mutex_destroy(&innobase_share_mutex);
  1457. // pthread_mutex_destroy(&prepare_commit_mutex);
  1458. // pthread_mutex_destroy(&commit_threads_m);
  1459. // pthread_mutex_destroy(&commit_cond_m);
  1460. // pthread_cond_destroy(&commit_cond);
  1461. }
  1462. /* ================= common ================= */
  1463. /***********************************************************************
  1464. Read backup meta info.
  1465. @return TRUE on success, FALSE on failure. */
  1466. static
  1467. my_bool
  1468. xtrabackup_read_metadata(char *filename)
  1469. {
  1470. FILE *fp;
  1471. my_bool r = TRUE;
  1472. int t;
  1473. fp = fopen(filename,"r");
  1474. if(!fp) {
  1475. msg("xtrabackup: Error: cannot open %s\n", filename);
  1476. return(FALSE);
  1477. }
  1478. if (fscanf(fp, "backup_type = %29s\n", metadata_type)
  1479. != 1) {
  1480. r = FALSE;
  1481. goto end;
  1482. }
  1483. /* Use UINT64PF instead of LSN_PF here, as we have to maintain the file
  1484. format. */
  1485. if (fscanf(fp, "from_lsn = " UINT64PF "\n", &metadata_from_lsn)
  1486. != 1) {
  1487. r = FALSE;
  1488. goto end;
  1489. }
  1490. if (fscanf(fp, "to_lsn = " UINT64PF "\n", &metadata_to_lsn)
  1491. != 1) {
  1492. r = FALSE;
  1493. goto end;
  1494. }
  1495. if (fscanf(fp, "last_lsn = " UINT64PF "\n", &metadata_last_lsn)
  1496. != 1) {
  1497. metadata_last_lsn = 0;
  1498. }
  1499. /* Optional fields */
  1500. if (fscanf(fp, "recover_binlog_info = %d\n", &t) == 1) {
  1501. recover_binlog_info = (t == 1);
  1502. }
  1503. end:
  1504. fclose(fp);
  1505. return(r);
  1506. }
  1507. /***********************************************************************
  1508. Print backup meta info to a specified buffer. */
  1509. static
  1510. void
  1511. xtrabackup_print_metadata(char *buf, size_t buf_len)
  1512. {
  1513. /* Use UINT64PF instead of LSN_PF here, as we have to maintain the file
  1514. format. */
  1515. snprintf(buf, buf_len,
  1516. "backup_type = %s\n"
  1517. "from_lsn = " UINT64PF "\n"
  1518. "to_lsn = " UINT64PF "\n"
  1519. "last_lsn = " UINT64PF "\n"
  1520. "recover_binlog_info = %d\n",
  1521. metadata_type,
  1522. metadata_from_lsn,
  1523. metadata_to_lsn,
  1524. metadata_last_lsn,
  1525. MY_TEST(opt_binlog_info == BINLOG_INFO_LOCKLESS));
  1526. }
  1527. /***********************************************************************
  1528. Stream backup meta info to a specified datasink.
  1529. @return TRUE on success, FALSE on failure. */
  1530. static
  1531. my_bool
  1532. xtrabackup_stream_metadata(ds_ctxt_t *ds_ctxt)
  1533. {
  1534. char buf[1024];
  1535. size_t len;
  1536. ds_file_t *stream;
  1537. MY_STAT mystat;
  1538. my_bool rc = TRUE;
  1539. xtrabackup_print_metadata(buf, sizeof(buf));
  1540. len = strlen(buf);
  1541. mystat.st_size = len;
  1542. mystat.st_mtime = my_time(0);
  1543. stream = ds_open(ds_ctxt, XTRABACKUP_METADATA_FILENAME, &mystat);
  1544. if (stream == NULL) {
  1545. msg("xtrabackup: Error: cannot open output stream "
  1546. "for %s\n", XTRABACKUP_METADATA_FILENAME);
  1547. return(FALSE);
  1548. }
  1549. if (ds_write(stream, buf, len)) {
  1550. rc = FALSE;
  1551. }
  1552. if (ds_close(stream)) {
  1553. rc = FALSE;
  1554. }
  1555. return(rc);
  1556. }
  1557. /***********************************************************************
  1558. Write backup meta info to a specified file.
  1559. @return TRUE on success, FALSE on failure. */
  1560. static
  1561. my_bool
  1562. xtrabackup_write_metadata(const char *filepath)
  1563. {
  1564. char buf[1024];
  1565. size_t len;
  1566. FILE *fp;
  1567. xtrabackup_print_metadata(buf, sizeof(buf));
  1568. len = strlen(buf);
  1569. fp = fopen(filepath, "w");
  1570. if(!fp) {
  1571. msg("xtrabackup: Error: cannot open %s\n", filepath);
  1572. return(FALSE);
  1573. }
  1574. if (fwrite(buf, len, 1, fp) < 1) {
  1575. fclose(fp);
  1576. return(FALSE);
  1577. }
  1578. fclose(fp);
  1579. return(TRUE);
  1580. }
  1581. /***********************************************************************
  1582. Read meta info for an incremental delta.
  1583. @return TRUE on success, FALSE on failure. */
  1584. static my_bool
  1585. xb_read_delta_metadata(const char *filepath, xb_delta_info_t *info)
  1586. {
  1587. FILE* fp;
  1588. char key[51];
  1589. char value[51];
  1590. my_bool r = TRUE;
  1591. /* set defaults */
  1592. info->page_size = ULINT_UNDEFINED;
  1593. info->zip_size = ULINT_UNDEFINED;
  1594. info->space_id = ULINT_UNDEFINED;
  1595. fp = fopen(filepath, "r");
  1596. if (!fp) {
  1597. /* Meta files for incremental deltas are optional */
  1598. return(TRUE);
  1599. }
  1600. while (!feof(fp)) {
  1601. if (fscanf(fp, "%50s = %50s\n", key, value) == 2) {
  1602. if (strcmp(key, "page_size") == 0) {
  1603. info->page_size = strtoul(value, NULL, 10);
  1604. } else if (strcmp(key, "zip_size") == 0) {
  1605. info->zip_size = strtoul(value, NULL, 10);
  1606. } else if (strcmp(key, "space_id") == 0) {
  1607. info->space_id = strtoul(value, NULL, 10);
  1608. }
  1609. }
  1610. }
  1611. fclose(fp);
  1612. if (info->page_size == ULINT_UNDEFINED) {
  1613. msg("xtrabackup: page_size is required in %s\n", filepath);
  1614. r = FALSE;
  1615. }
  1616. if (info->space_id == ULINT_UNDEFINED) {
  1617. msg("xtrabackup: Warning: This backup was taken with XtraBackup 2.0.1 "
  1618. "or earlier, some DDL operations between full and incremental "
  1619. "backups may be handled incorrectly\n");
  1620. }
  1621. return(r);
  1622. }
  1623. /***********************************************************************
  1624. Write meta info for an incremental delta.
  1625. @return TRUE on success, FALSE on failure. */
  1626. my_bool
  1627. xb_write_delta_metadata(const char *filename, const xb_delta_info_t *info)
  1628. {
  1629. ds_file_t *f;
  1630. char buf[64];
  1631. my_bool ret;
  1632. size_t len;
  1633. MY_STAT mystat;
  1634. snprintf(buf, sizeof(buf),
  1635. "page_size = %lu\n"
  1636. "zip_size = %lu\n"
  1637. "space_id = %lu\n",
  1638. info->page_size, info->zip_size, info->space_id);
  1639. len = strlen(buf);
  1640. mystat.st_size = len;
  1641. mystat.st_mtime = my_time(0);
  1642. f = ds_open(ds_meta, filename, &mystat);
  1643. if (f == NULL) {
  1644. msg("xtrabackup: Error: cannot open output stream for %s\n",
  1645. filename);
  1646. return(FALSE);
  1647. }
  1648. ret = (ds_write(f, buf, len) == 0);
  1649. if (ds_close(f)) {
  1650. ret = FALSE;
  1651. }
  1652. return(ret);
  1653. }
  1654. /* ================= backup ================= */
  1655. void
  1656. xtrabackup_io_throttling(void)
  1657. {
  1658. if (xtrabackup_backup && xtrabackup_throttle && (io_ticket--) < 0) {
  1659. os_event_reset(wait_throttle);
  1660. os_event_wait(wait_throttle);
  1661. }
  1662. }
  1663. static
  1664. my_bool regex_list_check_match(
  1665. const regex_list_t& list,
  1666. const char* name)
  1667. {
  1668. regmatch_t tables_regmatch[1];
  1669. for (regex_list_t::const_iterator i = list.begin(), end = list.end();
  1670. i != end; ++i) {
  1671. const regex_t& regex = *i;
  1672. int regres = regexec(&regex, name, 1, tables_regmatch, 0);
  1673. if (regres != REG_NOMATCH) {
  1674. return(TRUE);
  1675. }
  1676. }
  1677. return(FALSE);
  1678. }
  1679. static
  1680. my_bool
  1681. find_filter_in_hashtable(
  1682. const char* name,
  1683. hash_table_t* table,
  1684. xb_filter_entry_t** result
  1685. )
  1686. {
  1687. xb_filter_entry_t* found = NULL;
  1688. HASH_SEARCH(name_hash, table, ut_fold_string(name),
  1689. xb_filter_entry_t*,
  1690. found, (void) 0,
  1691. !strcmp(found->name, name));
  1692. if (found && result) {
  1693. *result = found;
  1694. }
  1695. return (found != NULL);
  1696. }
  1697. /************************************************************************
  1698. Checks if a given table name matches any of specifications given in
  1699. regex_list or tables_hash.
  1700. @return TRUE on match or both regex_list and tables_hash are empty.*/
  1701. static my_bool
  1702. check_if_table_matches_filters(const char *name,
  1703. const regex_list_t& regex_list,
  1704. hash_table_t* tables_hash)
  1705. {
  1706. if (regex_list.empty() && !tables_hash) {
  1707. return(FALSE);
  1708. }
  1709. if (regex_list_check_match(regex_list, name)) {
  1710. return(TRUE);
  1711. }
  1712. if (tables_hash && find_filter_in_hashtable(name, tables_hash, NULL)) {
  1713. return(TRUE);
  1714. }
  1715. return FALSE;
  1716. }
  1717. enum skip_database_check_result {
  1718. DATABASE_SKIP,
  1719. DATABASE_SKIP_SOME_TABLES,
  1720. DATABASE_DONT_SKIP,
  1721. DATABASE_DONT_SKIP_UNLESS_EXPLICITLY_EXCLUDED,
  1722. };
  1723. /************************************************************************
  1724. Checks if a database specified by name should be skipped from backup based on
  1725. the --databases, --databases_file or --databases_exclude options.
  1726. @return TRUE if entire database should be skipped,
  1727. FALSE otherwise.
  1728. */
  1729. static
  1730. skip_database_check_result
  1731. check_if_skip_database(
  1732. const char* name /*!< in: path to the database */
  1733. )
  1734. {
  1735. /* There are some filters for databases, check them */
  1736. xb_filter_entry_t* database = NULL;
  1737. if (databases_exclude_hash &&
  1738. find_filter_in_hashtable(name, databases_exclude_hash,
  1739. &database) &&
  1740. !database->has_tables) {
  1741. /* Database is found and there are no tables specified,
  1742. skip entire db. */
  1743. return DATABASE_SKIP;
  1744. }
  1745. if (databases_include_hash) {
  1746. if (!find_filter_in_hashtable(name, databases_include_hash,
  1747. &database)) {
  1748. /* Database isn't found, skip the database */
  1749. return DATABASE_SKIP;
  1750. } else if (database->has_tables) {
  1751. return DATABASE_SKIP_SOME_TABLES;
  1752. } else {
  1753. return DATABASE_DONT_SKIP_UNLESS_EXPLICITLY_EXCLUDED;
  1754. }
  1755. }
  1756. return DATABASE_DONT_SKIP;
  1757. }
  1758. /************************************************************************
  1759. Checks if a database specified by path should be skipped from backup based on
  1760. the --databases, --databases_file or --databases_exclude options.
  1761. @return TRUE if the table should be skipped. */
  1762. my_bool
  1763. check_if_skip_database_by_path(
  1764. const char* path /*!< in: path to the db directory. */
  1765. )
  1766. {
  1767. if (databases_include_hash == NULL &&
  1768. databases_exclude_hash == NULL) {
  1769. return(FALSE);
  1770. }
  1771. const char* db_name = strrchr(path, SRV_PATH_SEPARATOR);
  1772. if (db_name == NULL) {
  1773. db_name = path;
  1774. } else {
  1775. ++db_name;
  1776. }
  1777. return check_if_skip_database(db_name) == DATABASE_SKIP;
  1778. }
  1779. /************************************************************************
  1780. Checks if a table specified as a name in the form "database/name" (InnoDB 5.6)
  1781. or "./database/name.ibd" (InnoDB 5.5-) should be skipped from backup based on
  1782. the --tables or --tables-file options.
  1783. @return TRUE if the table should be skipped. */
  1784. my_bool
  1785. check_if_skip_table(
  1786. /******************/
  1787. const char* name) /*!< in: path to the table */
  1788. {
  1789. char buf[FN_REFLEN];
  1790. const char *dbname, *tbname;
  1791. const char *ptr;
  1792. char *eptr;
  1793. if (regex_exclude_list.empty() &&
  1794. regex_include_list.empty() &&
  1795. tables_include_hash == NULL &&
  1796. tables_exclude_hash == NULL &&
  1797. databases_include_hash == NULL &&
  1798. databases_exclude_hash == NULL) {
  1799. return(FALSE);
  1800. }
  1801. dbname = NULL;
  1802. tbname = name;
  1803. while ((ptr = strchr(tbname, '/')) != NULL) {
  1804. dbname = tbname;
  1805. tbname = ptr + 1;
  1806. }
  1807. if (dbname == NULL) {
  1808. return(FALSE);
  1809. }
  1810. strncpy(buf, dbname, FN_REFLEN);
  1811. buf[tbname - 1 - dbname] = 0;
  1812. const skip_database_check_result skip_database =
  1813. check_if_skip_database(buf);
  1814. if (skip_database == DATABASE_SKIP) {
  1815. return (TRUE);
  1816. }
  1817. buf[FN_REFLEN - 1] = '\0';
  1818. buf[tbname - 1 - dbname] = '.';
  1819. /* Check if there's a suffix in the table name. If so, truncate it. We
  1820. rely on the fact that a dot cannot be a part of a table name (it is
  1821. encoded by the server with the @NNNN syntax). */
  1822. if ((eptr = strchr(&buf[tbname - dbname], '.')) != NULL) {
  1823. *eptr = '\0';
  1824. }
  1825. /* For partitioned tables first try to match against the regexp
  1826. without truncating the #P#... suffix so we can backup individual
  1827. partitions with regexps like '^test[.]t#P#p5' */
  1828. if (check_if_table_matches_filters(buf, regex_exclude_list,
  1829. tables_exclude_hash)) {
  1830. return(TRUE);
  1831. }
  1832. if (check_if_table_matches_filters(buf, regex_include_list,
  1833. tables_include_hash)) {
  1834. return(FALSE);
  1835. }
  1836. if ((eptr = strstr(buf, "#P#")) != NULL) {
  1837. *eptr = 0;
  1838. if (check_if_table_matches_filters(buf, regex_exclude_list,
  1839. tables_exclude_hash)) {
  1840. return (TRUE);
  1841. }
  1842. if (check_if_table_matches_filters(buf, regex_include_list,
  1843. tables_include_hash)) {
  1844. return(FALSE);
  1845. }
  1846. }
  1847. if (skip_database == DATABASE_DONT_SKIP_UNLESS_EXPLICITLY_EXCLUDED) {
  1848. /* Database is in include-list, and qualified name wasn't
  1849. found in any of exclusion filters.*/
  1850. return (FALSE);
  1851. }
  1852. if (skip_database == DATABASE_SKIP_SOME_TABLES ||
  1853. !regex_include_list.empty() ||
  1854. tables_include_hash) {
  1855. /* Include lists are present, but qualified name
  1856. failed to match any.*/
  1857. return(TRUE);
  1858. }
  1859. return(FALSE);
  1860. }
  1861. /***********************************************************************
  1862. Reads the space flags from a given data file and returns the compressed
  1863. page size, or 0 if the space is not compressed. */
  1864. ulint
  1865. xb_get_zip_size(pfs_os_file_t file)
  1866. {
  1867. byte *buf;
  1868. byte *page;
  1869. ulint zip_size = ULINT_UNDEFINED;
  1870. ibool success;
  1871. ulint space;
  1872. buf = static_cast<byte *>(ut_malloc(2 * UNIV_PAGE_SIZE));
  1873. page = static_cast<byte *>(ut_align(buf, UNIV_PAGE_SIZE));
  1874. success = os_file_read(file, page, 0, UNIV_PAGE_SIZE);
  1875. if (!success) {
  1876. goto end;
  1877. }
  1878. space = mach_read_from_4(page + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
  1879. zip_size = (space == 0 ) ? 0 :
  1880. dict_tf_get_zip_size(fsp_header_get_flags(page));
  1881. end:
  1882. ut_free(buf);
  1883. return(zip_size);
  1884. }
  1885. const char*
  1886. xb_get_copy_action(const char *dflt)
  1887. {
  1888. const char *action;
  1889. if (xtrabackup_stream) {
  1890. if (xtrabackup_compress) {
  1891. action = "Compressing and streaming";
  1892. } else {
  1893. action = "Streaming";
  1894. }
  1895. } else {
  1896. if (xtrabackup_compress) {
  1897. action = "Compressing";
  1898. } else {
  1899. action = dflt;
  1900. }
  1901. }
  1902. return(action);
  1903. }
  1904. /* TODO: We may tune the behavior (e.g. by fil_aio)*/
  1905. static
  1906. my_bool
  1907. xtrabackup_copy_datafile(fil_node_t* node, uint thread_n)
  1908. {
  1909. char dst_name[FN_REFLEN];
  1910. ds_file_t *dstfile = NULL;
  1911. xb_fil_cur_t cursor;
  1912. xb_fil_cur_result_t res;
  1913. xb_write_filt_t *write_filter = NULL;
  1914. xb_write_filt_ctxt_t write_filt_ctxt;
  1915. const char *action;
  1916. xb_read_filt_t *read_filter;
  1917. ibool is_system;
  1918. my_bool rc = FALSE;
  1919. /* Get the name and the path for the tablespace. node->name always
  1920. contains the path (which may be absolute for remote tablespaces in
  1921. 5.6+). space->name contains the tablespace name in the form
  1922. "./database/table.ibd" (in 5.5-) or "database/table" (in 5.6+). For a
  1923. multi-node shared tablespace, space->name contains the name of the first
  1924. node, but that's irrelevant, since we only need node_name to match them
  1925. against filters, and the shared tablespace is always copied regardless
  1926. of the filters value. */
  1927. const char* const node_name = node->space->name;
  1928. const char* const node_path = node->name;
  1929. is_system = !fil_is_user_tablespace_id(node->space->id);
  1930. if (!is_system && check_if_skip_table(node_name)) {
  1931. msg("[%02u] Skipping %s.\n", thread_n, node_name);
  1932. return(FALSE);
  1933. }
  1934. if (!changed_page_bitmap) {
  1935. read_filter = &rf_pass_through;
  1936. }
  1937. else {
  1938. read_filter = &rf_bitmap;
  1939. }
  1940. res = xb_fil_cur_open(&cursor, read_filter, node, thread_n);
  1941. if (res == XB_FIL_CUR_SKIP) {
  1942. goto skip;
  1943. } else if (res == XB_FIL_CUR_ERROR) {
  1944. goto error;
  1945. }
  1946. strncpy(dst_name, cursor.rel_path, sizeof(dst_name));
  1947. /* Setup the page write filter */
  1948. if (xtrabackup_incremental) {
  1949. write_filter = &wf_incremental;
  1950. } else {
  1951. write_filter = &wf_write_through;
  1952. }
  1953. memset(&write_filt_ctxt, 0, sizeof(xb_write_filt_ctxt_t));
  1954. ut_a(write_filter->process != NULL);
  1955. if (write_filter->init != NULL &&
  1956. !write_filter->init(&write_filt_ctxt, dst_name, &cursor)) {
  1957. msg("[%02u] xtrabackup: error: "
  1958. "failed to initialize page write filter.\n", thread_n);
  1959. goto error;
  1960. }
  1961. dstfile = ds_open(ds_data, dst_name, &cursor.statinfo);
  1962. if (dstfile == NULL) {
  1963. msg("[%02u] xtrabackup: error: "
  1964. "cannot open the destination stream for %s\n",
  1965. thread_n, dst_name);
  1966. goto error;
  1967. }
  1968. action = xb_get_copy_action();
  1969. if (xtrabackup_stream) {
  1970. msg_ts("[%02u] %s %s\n", thread_n, action, node_path);
  1971. } else {
  1972. msg_ts("[%02u] %s %s to %s\n", thread_n, action,
  1973. node_path, dstfile->path);
  1974. }
  1975. /* The main copy loop */
  1976. while ((res = xb_fil_cur_read(&cursor)) == XB_FIL_CUR_SUCCESS) {
  1977. if (!write_filter->process(&write_filt_ctxt, dstfile)) {
  1978. goto error;
  1979. }
  1980. }
  1981. if (res == XB_FIL_CUR_ERROR) {
  1982. goto error;
  1983. }
  1984. if (write_filter->finalize
  1985. && !write_filter->finalize(&write_filt_ctxt, dstfile)) {
  1986. goto error;
  1987. }
  1988. /* close */
  1989. msg_ts("[%02u] ...done\n", thread_n);
  1990. xb_fil_cur_close(&cursor);
  1991. if (ds_close(dstfile)) {
  1992. rc = TRUE;
  1993. }
  1994. if (write_filter && write_filter->deinit) {
  1995. write_filter->deinit(&write_filt_ctxt);
  1996. }
  1997. return(rc);
  1998. error:
  1999. xb_fil_cur_close(&cursor);
  2000. if (dstfile != NULL) {
  2001. ds_close(dstfile);
  2002. }
  2003. if (write_filter && write_filter->deinit) {
  2004. write_filter->deinit(&write_filt_ctxt);;
  2005. }
  2006. msg("[%02u] xtrabackup: Error: "
  2007. "xtrabackup_copy_datafile() failed.\n", thread_n);
  2008. return(TRUE); /*ERROR*/
  2009. skip:
  2010. if (dstfile != NULL) {
  2011. ds_close(dstfile);
  2012. }
  2013. if (write_filter && write_filter->deinit) {
  2014. write_filter->deinit(&write_filt_ctxt);
  2015. }
  2016. msg("[%02u] xtrabackup: Warning: We assume the "
  2017. "table was dropped during xtrabackup execution "
  2018. "and ignore the file.\n", thread_n);
  2019. msg("[%02u] xtrabackup: Warning: skipping tablespace %s.\n",
  2020. thread_n, node_name);
  2021. return(FALSE);
  2022. }
  2023. static
  2024. void
  2025. xtrabackup_choose_lsn_offset(lsn_t start_lsn)
  2026. {
  2027. #if SUPPORT_PERCONA_5_5
  2028. ulint no, alt_no, expected_no;
  2029. ulint blocks_in_group;
  2030. lsn_t tmp_offset, end_lsn;
  2031. int lsn_chosen = 0;
  2032. log_group_t *group;
  2033. start_lsn = ut_uint64_align_down(start_lsn, OS_FILE_LOG_BLOCK_SIZE);
  2034. end_lsn = start_lsn + RECV_SCAN_SIZE;
  2035. group = UT_LIST_GET_FIRST(log_sys->log_groups);
  2036. if (mysql_server_version < 50500 || mysql_server_version > 50600) {
  2037. /* only make sense for Percona Server 5.5 */
  2038. return;
  2039. }
  2040. if (server_flavor == FLAVOR_PERCONA_SERVER) {
  2041. /* it is Percona Server 5.5 */
  2042. group->alt_offset_chosen = true;
  2043. group->lsn_offset = group->lsn_offset_alt;
  2044. return;
  2045. }
  2046. if (group->lsn_offset_alt == group->lsn_offset ||
  2047. group->lsn_offset_alt == (lsn_t) -1) {
  2048. /* we have only one option */
  2049. return;
  2050. }
  2051. no = alt_no = (ulint) -1;
  2052. lsn_chosen = 0;
  2053. blocks_in_group = log_block_convert_lsn_to_no(
  2054. log_group_get_capacity(group)) - 1;
  2055. /* read log block number from usual offset */
  2056. if (group->lsn_offset < group->file_size * group->n_files &&
  2057. (log_group_calc_lsn_offset(start_lsn, group) %
  2058. UNIV_PAGE_SIZE) % OS_MIN_LOG_BLOCK_SIZE == 0) {
  2059. log_group_read_log_seg(LOG_RECOVER, log_sys->buf,
  2060. group, start_lsn, end_lsn);
  2061. no = log_block_get_hdr_no(log_sys->buf);
  2062. }
  2063. /* read log block number from Percona Server 5.5 offset */
  2064. tmp_offset = group->lsn_offset;
  2065. group->lsn_offset = group->lsn_offset_alt;
  2066. if (group->lsn_offset < group->file_size * group->n_files &&
  2067. (log_group_calc_lsn_offset(start_lsn, group) %
  2068. UNIV_PAGE_SIZE) % OS_MIN_LOG_BLOCK_SIZE == 0) {
  2069. log_group_read_log_seg(LOG_RECOVER, log_sys->buf,
  2070. group, start_lsn, end_lsn);
  2071. alt_no = log_block_get_hdr_no(log_sys->buf);
  2072. }
  2073. expected_no = log_block_convert_lsn_to_no(start_lsn);
  2074. ut_a(!(no == expected_no && alt_no == expected_no));
  2075. group->lsn_offset = tmp_offset;
  2076. if ((no <= expected_no &&
  2077. ((expected_no - no) % blocks_in_group) == 0) ||
  2078. ((expected_no | 0x40000000UL) - no) % blocks_in_group == 0) {
  2079. /* default offset looks ok */
  2080. ++lsn_chosen;
  2081. }
  2082. if ((alt_no <= expected_no &&
  2083. ((expected_no - alt_no) % blocks_in_group) == 0) ||
  2084. ((expected_no | 0x40000000UL) - alt_no) % blocks_in_group == 0) {
  2085. /* PS 5.5 style offset looks ok */
  2086. ++lsn_chosen;
  2087. group->alt_offset_chosen = true;
  2088. group->lsn_offset = group->lsn_offset_alt;
  2089. }
  2090. /* We are in trouble, because we can not make a
  2091. decision to choose one over the other. Die just
  2092. like a Buridan's ass */
  2093. ut_a(lsn_chosen == 1);
  2094. #endif
  2095. }
  2096. extern ibool log_block_checksum_is_ok_or_old_format(const byte* block);
  2097. /*******************************************************//**
  2098. Scans log from a buffer and writes new log data to the outpud datasinc.
  2099. @return true if success */
  2100. static
  2101. bool
  2102. xtrabackup_scan_log_recs(
  2103. /*===============*/
  2104. log_group_t* group, /*!< in: log group */
  2105. bool is_last, /*!< in: whether it is last segment
  2106. to copy */
  2107. lsn_t start_lsn, /*!< in: buffer start lsn */
  2108. lsn_t* contiguous_lsn, /*!< in/out: it is known that all log
  2109. groups contain contiguous log data up
  2110. to this lsn */
  2111. lsn_t* group_scanned_lsn,/*!< out: scanning succeeded up to
  2112. this lsn */
  2113. bool* finished) /*!< out: false if is not able to scan
  2114. any more in this log group */
  2115. {
  2116. lsn_t scanned_lsn;
  2117. ulint data_len;
  2118. ulint write_size;
  2119. const byte* log_block;
  2120. ulint scanned_checkpoint_no = 0;
  2121. *finished = false;
  2122. scanned_lsn = start_lsn;
  2123. log_block = log_sys->buf;
  2124. while (log_block < log_sys->buf + RECV_SCAN_SIZE && !*finished) {
  2125. ulint no = log_block_get_hdr_no(log_block);
  2126. ulint scanned_no = log_block_convert_lsn_to_no(scanned_lsn);
  2127. ibool checksum_is_ok =
  2128. log_block_checksum_is_ok_or_old_format(log_block);
  2129. if (no != scanned_no && checksum_is_ok) {
  2130. ulint blocks_in_group;
  2131. blocks_in_group = log_block_convert_lsn_to_no(
  2132. log_group_get_capacity(group)) - 1;
  2133. if ((no < scanned_no &&
  2134. ((scanned_no - no) % blocks_in_group) == 0) ||
  2135. no == 0 ||
  2136. /* Log block numbers wrap around at 0x3FFFFFFF */
  2137. ((scanned_no | 0x40000000UL) - no) %
  2138. blocks_in_group == 0) {
  2139. /* old log block, do nothing */
  2140. *finished = true;
  2141. break;
  2142. }
  2143. msg("xtrabackup: error:"
  2144. " log block numbers mismatch:\n"
  2145. "xtrabackup: error: expected log block no. %lu,"
  2146. " but got no. %lu from the log file.\n",
  2147. (ulong) scanned_no, (ulong) no);
  2148. if ((no - scanned_no) % blocks_in_group == 0) {
  2149. msg("xtrabackup: error:"
  2150. " it looks like InnoDB log has wrapped"
  2151. " around before xtrabackup could"
  2152. " process all records due to either"
  2153. " log copying being too slow, or "
  2154. " log files being too small.\n");
  2155. }
  2156. return(false);
  2157. } else if (!checksum_is_ok) {
  2158. /* Garbage or an incompletely written log block */
  2159. msg("xtrabackup: warning: Log block checksum mismatch"
  2160. " (block no %lu at lsn " LSN_PF "): \n"
  2161. "expected %lu, calculated checksum %lu\n",
  2162. (ulong) no,
  2163. scanned_lsn,
  2164. (ulong) log_block_get_checksum(log_block),
  2165. (ulong) log_block_calc_checksum(log_block));
  2166. msg("xtrabackup: warning: this is possible when the "
  2167. "log block has not been fully written by the "
  2168. "server, will retry later.\n");
  2169. *finished = true;
  2170. break;
  2171. }
  2172. if (log_block_get_flush_bit(log_block)) {
  2173. /* This block was a start of a log flush operation:
  2174. we know that the previous flush operation must have
  2175. been completed for all log groups before this block
  2176. can have been flushed to any of the groups. Therefore,
  2177. we know that log data is contiguous up to scanned_lsn
  2178. in all non-corrupt log groups. */
  2179. if (scanned_lsn > *contiguous_lsn) {
  2180. *contiguous_lsn = scanned_lsn;
  2181. }
  2182. }
  2183. data_len = log_block_get_data_len(log_block);
  2184. if (
  2185. (scanned_checkpoint_no > 0)
  2186. && (log_block_get_checkpoint_no(log_block)
  2187. < scanned_checkpoint_no)
  2188. && (scanned_checkpoint_no
  2189. - log_block_get_checkpoint_no(log_block)
  2190. > 0x80000000UL)) {
  2191. /* Garbage from a log buffer flush which was made
  2192. before the most recent database recovery */
  2193. *finished = true;
  2194. break;
  2195. }
  2196. scanned_lsn = scanned_lsn + data_len;
  2197. scanned_checkpoint_no = log_block_get_checkpoint_no(log_block);
  2198. if (data_len < OS_FILE_LOG_BLOCK_SIZE) {
  2199. /* Log data for this group ends here */
  2200. *finished = true;
  2201. } else {
  2202. log_block += OS_FILE_LOG_BLOCK_SIZE;
  2203. }
  2204. }
  2205. *group_scanned_lsn = scanned_lsn;
  2206. /* ===== write log to 'xtrabackup_logfile' ====== */
  2207. if (!*finished) {
  2208. write_size = RECV_SCAN_SIZE;
  2209. } else {
  2210. write_size = (ulint)(ut_uint64_align_up(scanned_lsn,
  2211. OS_FILE_LOG_BLOCK_SIZE) - start_lsn);
  2212. if (!is_last && scanned_lsn % OS_FILE_LOG_BLOCK_SIZE) {
  2213. write_size -= OS_FILE_LOG_BLOCK_SIZE;
  2214. }
  2215. }
  2216. if (write_size == 0) {
  2217. return(true);
  2218. }
  2219. if (srv_encrypt_log) {
  2220. log_encrypt_before_write(scanned_checkpoint_no,
  2221. log_sys->buf, write_size);
  2222. }
  2223. if (ds_write(dst_log_file, log_sys->buf, write_size)) {
  2224. msg("xtrabackup: Error: "
  2225. "write to logfile failed\n");
  2226. return(false);
  2227. }
  2228. return(true);
  2229. }
  2230. static my_bool
  2231. xtrabackup_copy_logfile(lsn_t from_lsn, my_bool is_last)
  2232. {
  2233. /* definition from recv_recovery_from_checkpoint_start() */
  2234. log_group_t* group;
  2235. lsn_t group_scanned_lsn;
  2236. lsn_t contiguous_lsn;
  2237. ut_a(dst_log_file != NULL);
  2238. /* read from checkpoint_lsn_start to current */
  2239. contiguous_lsn = ut_uint64_align_down(from_lsn, OS_FILE_LOG_BLOCK_SIZE);
  2240. /* TODO: We must check the contiguous_lsn still exists in log file.. */
  2241. group = UT_LIST_GET_FIRST(log_sys->log_groups);
  2242. while (group) {
  2243. bool finished;
  2244. lsn_t start_lsn;
  2245. lsn_t end_lsn;
  2246. /* reference recv_group_scan_log_recs() */
  2247. finished = false;
  2248. start_lsn = contiguous_lsn;
  2249. while (!finished) {
  2250. end_lsn = start_lsn + RECV_SCAN_SIZE;
  2251. xtrabackup_io_throttling();
  2252. mutex_enter(&log_sys->mutex);
  2253. log_group_read_log_seg(LOG_RECOVER, log_sys->buf,
  2254. group, start_lsn, end_lsn, false);
  2255. if (!xtrabackup_scan_log_recs(group, is_last,
  2256. start_lsn, &contiguous_lsn, &group_scanned_lsn,
  2257. &finished)) {
  2258. goto error;
  2259. }
  2260. mutex_exit(&log_sys->mutex);
  2261. start_lsn = end_lsn;
  2262. }
  2263. group->scanned_lsn = group_scanned_lsn;
  2264. msg_ts(">> log scanned up to (" LSN_PF ")\n",
  2265. group->scanned_lsn);
  2266. group = UT_LIST_GET_NEXT(log_groups, group);
  2267. /* update global variable*/
  2268. log_copy_scanned_lsn = group_scanned_lsn;
  2269. /* innodb_mirrored_log_groups must be 1, no other groups */
  2270. ut_a(group == NULL);
  2271. debug_sync_point("xtrabackup_copy_logfile_pause");
  2272. }
  2273. return(FALSE);
  2274. error:
  2275. mutex_exit(&log_sys->mutex);
  2276. ds_close(dst_log_file);
  2277. msg("xtrabackup: Error: xtrabackup_copy_logfile() failed.\n");
  2278. return(TRUE);
  2279. }
  2280. static
  2281. #ifndef __WIN__
  2282. void*
  2283. #else
  2284. ulint
  2285. #endif
  2286. log_copying_thread(
  2287. void* arg __attribute__((unused)))
  2288. {
  2289. /*
  2290. Initialize mysys thread-specific memory so we can
  2291. use mysys functions in this thread.
  2292. */
  2293. my_thread_init();
  2294. ut_a(dst_log_file != NULL);
  2295. log_copying_running = TRUE;
  2296. while(log_copying) {
  2297. os_event_reset(log_copying_stop);
  2298. os_event_wait_time_low(log_copying_stop,
  2299. xtrabackup_log_copy_interval * 1000ULL,
  2300. 0);
  2301. if (log_copying) {
  2302. if(xtrabackup_copy_logfile(log_copy_scanned_lsn,
  2303. FALSE)) {
  2304. exit(EXIT_FAILURE);
  2305. }
  2306. }
  2307. }
  2308. /* last copying */
  2309. if(xtrabackup_copy_logfile(log_copy_scanned_lsn, TRUE)) {
  2310. exit(EXIT_FAILURE);
  2311. }
  2312. log_copying_running = FALSE;
  2313. my_thread_end();
  2314. os_thread_exit(NULL);
  2315. return(0);
  2316. }
  2317. /* io throttle watching (rough) */
  2318. static
  2319. #ifndef __WIN__
  2320. void*
  2321. #else
  2322. ulint
  2323. #endif
  2324. io_watching_thread(
  2325. void* arg)
  2326. {
  2327. (void)arg;
  2328. /* currently, for --backup only */
  2329. ut_a(xtrabackup_backup);
  2330. io_watching_thread_running = TRUE;
  2331. while (log_copying) {
  2332. os_thread_sleep(1000000); /*1 sec*/
  2333. io_ticket = xtrabackup_throttle;
  2334. os_event_set(wait_throttle);
  2335. }
  2336. /* stop io throttle */
  2337. xtrabackup_throttle = 0;
  2338. os_event_set(wait_throttle);
  2339. io_watching_thread_running = FALSE;
  2340. os_thread_exit(NULL);
  2341. return(0);
  2342. }
  2343. /************************************************************************
  2344. I/o-handler thread function. */
  2345. static
  2346. #ifndef __WIN__
  2347. void*
  2348. #else
  2349. ulint
  2350. #endif
  2351. io_handler_thread(
  2352. /*==============*/
  2353. void* arg)
  2354. {
  2355. ulint segment;
  2356. segment = *((ulint*)arg);
  2357. while (srv_shutdown_state != SRV_SHUTDOWN_EXIT_THREADS) {
  2358. fil_aio_wait(segment);
  2359. }
  2360. /* We count the number of threads in os_thread_exit(). A created
  2361. thread should always use that to exit and not use return() to exit.
  2362. The thread actually never comes here because it is exited in an
  2363. os_event_wait(). */
  2364. os_thread_exit(NULL);
  2365. #ifndef __WIN__
  2366. return(NULL); /* Not reached */
  2367. #else
  2368. return(0);
  2369. #endif
  2370. }
  2371. /**************************************************************************
  2372. Datafiles copying thread.*/
  2373. static
  2374. os_thread_ret_t
  2375. data_copy_thread_func(
  2376. /*==================*/
  2377. void *arg) /* thread context */
  2378. {
  2379. data_thread_ctxt_t *ctxt = (data_thread_ctxt_t *) arg;
  2380. uint num = ctxt->num;
  2381. fil_node_t* node;
  2382. /*
  2383. Initialize mysys thread-specific memory so we can
  2384. use mysys functions in this thread.
  2385. */
  2386. my_thread_init();
  2387. debug_sync_point("data_copy_thread_func");
  2388. while ((node = datafiles_iter_next(ctxt->it)) != NULL) {
  2389. /* copy the datafile */
  2390. if(xtrabackup_copy_datafile(node, num)) {
  2391. msg("[%02u] xtrabackup: Error: "
  2392. "failed to copy datafile.\n", num);
  2393. exit(EXIT_FAILURE);
  2394. }
  2395. }
  2396. os_mutex_enter(ctxt->count_mutex);
  2397. (*ctxt->count)--;
  2398. os_mutex_exit(ctxt->count_mutex);
  2399. my_thread_end();
  2400. os_thread_exit(NULL);
  2401. OS_THREAD_DUMMY_RETURN;
  2402. }
  2403. /************************************************************************
  2404. Initialize the appropriate datasink(s). Both local backups and streaming in the
  2405. 'xbstream' format allow parallel writes so we can write directly.
  2406. Otherwise (i.e. when streaming in the 'tar' format) we need 2 separate datasinks
  2407. for the data stream (and don't allow parallel data copying) and for metainfo
  2408. files (including xtrabackup_logfile). The second datasink writes to temporary
  2409. files first, and then streams them in a serialized way when closed. */
  2410. static void
  2411. xtrabackup_init_datasinks(void)
  2412. {
  2413. /* Start building out the pipelines from the terminus back */
  2414. if (xtrabackup_stream) {
  2415. /* All streaming goes to stdout */
  2416. ds_data = ds_meta = ds_redo = ds_create(xtrabackup_target_dir,
  2417. DS_TYPE_STDOUT);
  2418. } else {
  2419. /* Local filesystem */
  2420. ds_data = ds_meta = ds_redo = ds_create(xtrabackup_target_dir,
  2421. DS_TYPE_LOCAL);
  2422. }
  2423. /* Track it for destruction */
  2424. xtrabackup_add_datasink(ds_data);
  2425. /* Stream formatting */
  2426. if (xtrabackup_stream) {
  2427. ds_ctxt_t *ds;
  2428. ut_a(xtrabackup_stream_fmt == XB_STREAM_FMT_XBSTREAM);
  2429. ds = ds_create(xtrabackup_target_dir, DS_TYPE_XBSTREAM);
  2430. xtrabackup_add_datasink(ds);
  2431. ds_set_pipe(ds, ds_data);
  2432. ds_data = ds;
  2433. ds_redo = ds_meta = ds_data;
  2434. }
  2435. /* Compression for ds_data and ds_redo */
  2436. if (xtrabackup_compress) {
  2437. ds_ctxt_t *ds;
  2438. /* Use a 1 MB buffer for compressed output stream */
  2439. ds = ds_create(xtrabackup_target_dir, DS_TYPE_BUFFER);
  2440. ds_buffer_set_size(ds, 1024 * 1024);
  2441. xtrabackup_add_datasink(ds);
  2442. ds_set_pipe(ds, ds_data);
  2443. if (ds_data != ds_redo) {
  2444. ds_data = ds;
  2445. ds = ds_create(xtrabackup_target_dir, DS_TYPE_BUFFER);
  2446. ds_buffer_set_size(ds, 1024 * 1024);
  2447. xtrabackup_add_datasink(ds);
  2448. ds_set_pipe(ds, ds_redo);
  2449. ds_redo = ds;
  2450. } else {
  2451. ds_redo = ds_data = ds;
  2452. }
  2453. ds = ds_create(xtrabackup_target_dir, DS_TYPE_COMPRESS);
  2454. xtrabackup_add_datasink(ds);
  2455. ds_set_pipe(ds, ds_data);
  2456. if (ds_data != ds_redo) {
  2457. ds_data = ds;
  2458. ds = ds_create(xtrabackup_target_dir, DS_TYPE_COMPRESS);
  2459. xtrabackup_add_datasink(ds);
  2460. ds_set_pipe(ds, ds_redo);
  2461. ds_redo = ds;
  2462. } else {
  2463. ds_redo = ds_data = ds;
  2464. }
  2465. }
  2466. }
  2467. /************************************************************************
  2468. Destroy datasinks.
  2469. Destruction is done in the specific order to not violate their order in the
  2470. pipeline so that each datasink is able to flush data down the pipeline. */
  2471. static void xtrabackup_destroy_datasinks(void)
  2472. {
  2473. for (uint i = actual_datasinks; i > 0; i--) {
  2474. ds_destroy(datasinks[i-1]);
  2475. datasinks[i-1] = NULL;
  2476. }
  2477. ds_data = NULL;
  2478. ds_meta = NULL;
  2479. ds_redo = NULL;
  2480. }
  2481. #define SRV_N_PENDING_IOS_PER_THREAD OS_AIO_N_PENDING_IOS_PER_THREAD
  2482. #define SRV_MAX_N_PENDING_SYNC_IOS 100
  2483. /************************************************************************
  2484. @return TRUE if table should be opened. */
  2485. static
  2486. ibool
  2487. xb_check_if_open_tablespace(
  2488. const char* db,
  2489. const char* table)
  2490. {
  2491. char buf[FN_REFLEN];
  2492. snprintf(buf, sizeof(buf), "%s/%s", db, table);
  2493. return !check_if_skip_table(buf);
  2494. }
  2495. /************************************************************************
  2496. Initializes the I/O and tablespace cache subsystems. */
  2497. static
  2498. void
  2499. xb_fil_io_init(void)
  2500. /*================*/
  2501. {
  2502. srv_n_file_io_threads = srv_n_read_io_threads;
  2503. os_aio_init(8 * SRV_N_PENDING_IOS_PER_THREAD,
  2504. srv_n_read_io_threads,
  2505. srv_n_write_io_threads,
  2506. SRV_MAX_N_PENDING_SYNC_IOS);
  2507. fil_init(srv_file_per_table ? 50000 : 5000, LONG_MAX);
  2508. fsp_init();
  2509. }
  2510. /****************************************************************************
  2511. Populates the tablespace memory cache by scanning for and opening data files.
  2512. @returns DB_SUCCESS or error code.*/
  2513. static
  2514. ulint
  2515. xb_load_tablespaces(void)
  2516. /*=====================*/
  2517. {
  2518. ulint i;
  2519. bool create_new_db;
  2520. ulint err;
  2521. ulint sum_of_new_sizes;
  2522. lsn_t min_arch_logno, max_arch_logno;
  2523. for (i = 0; i < srv_n_file_io_threads; i++) {
  2524. thread_nr[i] = i;
  2525. os_thread_create(io_handler_thread, thread_nr + i,
  2526. thread_ids + i);
  2527. }
  2528. os_thread_sleep(200000); /*0.2 sec*/
  2529. err = open_or_create_data_files(&create_new_db,
  2530. &min_arch_logno, &max_arch_logno,
  2531. &flushed_lsn,
  2532. &sum_of_new_sizes);
  2533. if (err != DB_SUCCESS) {
  2534. msg("xtrabackup: Could not open or create data files.\n"
  2535. "xtrabackup: If you tried to add new data files, and it "
  2536. "failed here,\n"
  2537. "xtrabackup: you should now edit innodb_data_file_path in "
  2538. "my.cnf back\n"
  2539. "xtrabackup: to what it was, and remove the new ibdata "
  2540. "files InnoDB created\n"
  2541. "xtrabackup: in this failed attempt. InnoDB only wrote "
  2542. "those files full of\n"
  2543. "xtrabackup: zeros, but did not yet use them in any way. "
  2544. "But be careful: do not\n"
  2545. "xtrabackup: remove old data files which contain your "
  2546. "precious data!\n");
  2547. return(err);
  2548. }
  2549. /* create_new_db must not be TRUE.. */
  2550. if (create_new_db) {
  2551. msg("xtrabackup: could not find data files at the "
  2552. "specified datadir\n");
  2553. return(DB_ERROR);
  2554. }
  2555. /* Add separate undo tablespaces to fil_system */
  2556. err = srv_undo_tablespaces_init(FALSE,
  2557. TRUE,
  2558. srv_undo_tablespaces,
  2559. &srv_undo_tablespaces_open);
  2560. if (err != DB_SUCCESS) {
  2561. return(err);
  2562. }
  2563. /* It is important to call fil_load_single_table_tablespace() after
  2564. srv_undo_tablespaces_init(), because fil_is_user_tablespace_id() *
  2565. relies on srv_undo_tablespaces_open to be properly initialized */
  2566. msg("xtrabackup: Generating a list of tablespaces\n");
  2567. err = fil_load_single_table_tablespaces(xb_check_if_open_tablespace);
  2568. if (err != DB_SUCCESS) {
  2569. return(err);
  2570. }
  2571. debug_sync_point("xtrabackup_load_tablespaces_pause");
  2572. return(DB_SUCCESS);
  2573. }
  2574. /************************************************************************
  2575. Initialize the tablespace memory cache and populate it by scanning for and
  2576. opening data files.
  2577. @returns DB_SUCCESS or error code.*/
  2578. ulint
  2579. xb_data_files_init(void)
  2580. /*====================*/
  2581. {
  2582. xb_fil_io_init();
  2583. return(xb_load_tablespaces());
  2584. }
  2585. /************************************************************************
  2586. Destroy the tablespace memory cache. */
  2587. void
  2588. xb_data_files_close(void)
  2589. /*====================*/
  2590. {
  2591. ulint i;
  2592. /* Shutdown the aio threads. This has been copied from
  2593. innobase_shutdown_for_mysql(). */
  2594. srv_shutdown_state = SRV_SHUTDOWN_EXIT_THREADS;
  2595. for (i = 0; i < 1000; i++) {
  2596. os_aio_wake_all_threads_at_shutdown();
  2597. if (os_thread_count == 0) {
  2598. break;
  2599. }
  2600. os_thread_sleep(10000);
  2601. }
  2602. if (i == 1000) {
  2603. msg("xtrabackup: Warning: %lu threads created by InnoDB"
  2604. " had not exited at shutdown!\n",
  2605. (ulong) os_thread_count);
  2606. }
  2607. os_aio_free();
  2608. fil_close_all_files();
  2609. /* Free the double write data structures. */
  2610. if (buf_dblwr) {
  2611. buf_dblwr_free();
  2612. }
  2613. /* Reset srv_file_io_threads to its default value to avoid confusing
  2614. warning on --prepare in innobase_start_or_create_for_mysql()*/
  2615. srv_n_file_io_threads = 4;
  2616. srv_shutdown_state = SRV_SHUTDOWN_NONE;
  2617. }
  2618. /***********************************************************************
  2619. Allocate and initialize the entry for databases and tables filtering
  2620. hash tables. If memory allocation is not successful, terminate program.
  2621. @return pointer to the created entry. */
  2622. static
  2623. xb_filter_entry_t *
  2624. xb_new_filter_entry(
  2625. /*================*/
  2626. const char* name) /*!< in: name of table/database */
  2627. {
  2628. xb_filter_entry_t *entry;
  2629. ulint namelen = strlen(name);
  2630. ut_a(namelen <= NAME_LEN * 2 + 1);
  2631. entry = static_cast<xb_filter_entry_t *>
  2632. (ut_malloc(sizeof(xb_filter_entry_t) + namelen + 1));
  2633. memset(entry, '\0', sizeof(xb_filter_entry_t) + namelen + 1);
  2634. entry->name = ((char*)entry) + sizeof(xb_filter_entry_t);
  2635. strcpy(entry->name, name);
  2636. entry->has_tables = FALSE;
  2637. return entry;
  2638. }
  2639. /***********************************************************************
  2640. Add entry to hash table. If hash table is NULL, allocate and initialize
  2641. new hash table */
  2642. static
  2643. xb_filter_entry_t*
  2644. xb_add_filter(
  2645. /*========================*/
  2646. const char* name, /*!< in: name of table/database */
  2647. hash_table_t** hash) /*!< in/out: hash to insert into */
  2648. {
  2649. xb_filter_entry_t* entry;
  2650. entry = xb_new_filter_entry(name);
  2651. if (UNIV_UNLIKELY(*hash == NULL)) {
  2652. *hash = hash_create(1000);
  2653. }
  2654. HASH_INSERT(xb_filter_entry_t,
  2655. name_hash, *hash,
  2656. ut_fold_string(entry->name),
  2657. entry);
  2658. return entry;
  2659. }
  2660. /***********************************************************************
  2661. Validate name of table or database. If name is invalid, program will
  2662. be finished with error code */
  2663. static
  2664. void
  2665. xb_validate_name(
  2666. /*=============*/
  2667. const char* name, /*!< in: name */
  2668. size_t len) /*!< in: length of name */
  2669. {
  2670. const char* p;
  2671. /* perform only basic validation. validate length and
  2672. path symbols */
  2673. if (len > NAME_LEN) {
  2674. msg("xtrabackup: name `%s` is too long.\n", name);
  2675. exit(EXIT_FAILURE);
  2676. }
  2677. p = strpbrk(name, "/\\~");
  2678. if (p && p - name < NAME_LEN) {
  2679. msg("xtrabackup: name `%s` is not valid.\n", name);
  2680. exit(EXIT_FAILURE);
  2681. }
  2682. }
  2683. /***********************************************************************
  2684. Register new filter entry which can be either database
  2685. or table name. */
  2686. static
  2687. void
  2688. xb_register_filter_entry(
  2689. /*=====================*/
  2690. const char* name, /*!< in: name */
  2691. hash_table_t** databases_hash,
  2692. hash_table_t** tables_hash
  2693. )
  2694. {
  2695. const char* p;
  2696. size_t namelen;
  2697. xb_filter_entry_t* db_entry = NULL;
  2698. namelen = strlen(name);
  2699. if ((p = strchr(name, '.')) != NULL) {
  2700. char dbname[NAME_LEN + 1];
  2701. xb_validate_name(name, p - name);
  2702. xb_validate_name(p + 1, namelen - (p - name));
  2703. strncpy(dbname, name, p - name);
  2704. dbname[p - name] = 0;
  2705. if (*databases_hash) {
  2706. HASH_SEARCH(name_hash, (*databases_hash),
  2707. ut_fold_string(dbname),
  2708. xb_filter_entry_t*,
  2709. db_entry, (void) 0,
  2710. !strcmp(db_entry->name, dbname));
  2711. }
  2712. if (!db_entry) {
  2713. db_entry = xb_add_filter(dbname, databases_hash);
  2714. }
  2715. db_entry->has_tables = TRUE;
  2716. xb_add_filter(name, tables_hash);
  2717. } else {
  2718. xb_validate_name(name, namelen);
  2719. xb_add_filter(name, databases_hash);
  2720. }
  2721. }
  2722. static
  2723. void
  2724. xb_register_include_filter_entry(
  2725. const char* name
  2726. )
  2727. {
  2728. xb_register_filter_entry(name, &databases_include_hash,
  2729. &tables_include_hash);
  2730. }
  2731. static
  2732. void
  2733. xb_register_exclude_filter_entry(
  2734. const char* name
  2735. )
  2736. {
  2737. xb_register_filter_entry(name, &databases_exclude_hash,
  2738. &tables_exclude_hash);
  2739. }
  2740. /***********************************************************************
  2741. Register new table for the filter. */
  2742. static
  2743. void
  2744. xb_register_table(
  2745. /*==============*/
  2746. const char* name) /*!< in: name of table */
  2747. {
  2748. if (strchr(name, '.') == NULL) {
  2749. msg("xtrabackup: `%s` is not fully qualified name.\n", name);
  2750. exit(EXIT_FAILURE);
  2751. }
  2752. xb_register_include_filter_entry(name);
  2753. }
  2754. static
  2755. void
  2756. xb_add_regex_to_list(
  2757. const char* regex, /*!< in: regex */
  2758. const char* error_context, /*!< in: context to error message */
  2759. regex_list_t* list) /*! in: list to put new regex to */
  2760. {
  2761. char errbuf[100];
  2762. int ret;
  2763. regex_t compiled_regex;
  2764. ret = regcomp(&compiled_regex, regex, REG_EXTENDED);
  2765. if (ret != 0) {
  2766. regerror(ret, &compiled_regex, errbuf, sizeof(errbuf));
  2767. msg("xtrabackup: error: %s regcomp(%s): %s\n",
  2768. error_context, regex, errbuf);
  2769. exit(EXIT_FAILURE);
  2770. }
  2771. list->push_back(compiled_regex);
  2772. }
  2773. /***********************************************************************
  2774. Register new regex for the include filter. */
  2775. static
  2776. void
  2777. xb_register_include_regex(
  2778. /*==============*/
  2779. const char* regex) /*!< in: regex */
  2780. {
  2781. xb_add_regex_to_list(regex, "tables", &regex_include_list);
  2782. }
  2783. /***********************************************************************
  2784. Register new regex for the exclude filter. */
  2785. static
  2786. void
  2787. xb_register_exclude_regex(
  2788. /*==============*/
  2789. const char* regex) /*!< in: regex */
  2790. {
  2791. xb_add_regex_to_list(regex, "tables-exclude", &regex_exclude_list);
  2792. }
  2793. typedef void (*insert_entry_func_t)(const char*);
  2794. /***********************************************************************
  2795. Scan string and load filter entries from it. */
  2796. static
  2797. void
  2798. xb_load_list_string(
  2799. /*================*/
  2800. char* list, /*!< in: string representing a list */
  2801. const char* delimiters, /*!< in: delimiters of entries */
  2802. insert_entry_func_t ins) /*!< in: callback to add entry */
  2803. {
  2804. char* p;
  2805. char* saveptr;
  2806. p = strtok_r(list, delimiters, &saveptr);
  2807. while (p) {
  2808. ins(p);
  2809. p = strtok_r(NULL, delimiters, &saveptr);
  2810. }
  2811. }
  2812. /***********************************************************************
  2813. Scan file and load filter entries from it. */
  2814. static
  2815. void
  2816. xb_load_list_file(
  2817. /*==============*/
  2818. const char* filename, /*!< in: name of file */
  2819. insert_entry_func_t ins) /*!< in: callback to add entry */
  2820. {
  2821. char name_buf[NAME_LEN*2+2];
  2822. FILE* fp;
  2823. /* read and store the filenames */
  2824. fp = fopen(filename, "r");
  2825. if (!fp) {
  2826. msg("xtrabackup: cannot open %s\n",
  2827. filename);
  2828. exit(EXIT_FAILURE);
  2829. }
  2830. while (fgets(name_buf, sizeof(name_buf), fp) != NULL) {
  2831. char* p = strchr(name_buf, '\n');
  2832. if (p) {
  2833. *p = '\0';
  2834. } else {
  2835. msg("xtrabackup: `%s...` name is too long", name_buf);
  2836. exit(EXIT_FAILURE);
  2837. }
  2838. ins(name_buf);
  2839. }
  2840. fclose(fp);
  2841. }
  2842. static
  2843. void
  2844. xb_filters_init()
  2845. {
  2846. if (xtrabackup_databases) {
  2847. xb_load_list_string(xtrabackup_databases, " \t",
  2848. xb_register_include_filter_entry);
  2849. }
  2850. if (xtrabackup_databases_file) {
  2851. xb_load_list_file(xtrabackup_databases_file,
  2852. xb_register_include_filter_entry);
  2853. }
  2854. if (xtrabackup_databases_exclude) {
  2855. xb_load_list_string(xtrabackup_databases_exclude, " \t",
  2856. xb_register_exclude_filter_entry);
  2857. }
  2858. if (xtrabackup_tables) {
  2859. xb_load_list_string(xtrabackup_tables, ",",
  2860. xb_register_include_regex);
  2861. }
  2862. if (xtrabackup_tables_file) {
  2863. xb_load_list_file(xtrabackup_tables_file, xb_register_table);
  2864. }
  2865. if (xtrabackup_tables_exclude) {
  2866. xb_load_list_string(xtrabackup_tables_exclude, ",",
  2867. xb_register_exclude_regex);
  2868. }
  2869. }
  2870. static
  2871. void
  2872. xb_filter_hash_free(hash_table_t* hash)
  2873. {
  2874. ulint i;
  2875. /* free the hash elements */
  2876. for (i = 0; i < hash_get_n_cells(hash); i++) {
  2877. xb_filter_entry_t* table;
  2878. table = static_cast<xb_filter_entry_t *>
  2879. (HASH_GET_FIRST(hash, i));
  2880. while (table) {
  2881. xb_filter_entry_t* prev_table = table;
  2882. table = static_cast<xb_filter_entry_t *>
  2883. (HASH_GET_NEXT(name_hash, prev_table));
  2884. HASH_DELETE(xb_filter_entry_t, name_hash, hash,
  2885. ut_fold_string(prev_table->name), prev_table);
  2886. ut_free(prev_table);
  2887. }
  2888. }
  2889. /* free hash */
  2890. hash_table_free(hash);
  2891. }
  2892. static void xb_regex_list_free(regex_list_t* list)
  2893. {
  2894. while (list->size() > 0) {
  2895. xb_regfree(&list->front());
  2896. list->pop_front();
  2897. }
  2898. }
  2899. /************************************************************************
  2900. Destroy table filters for partial backup. */
  2901. static
  2902. void
  2903. xb_filters_free()
  2904. {
  2905. xb_regex_list_free(&regex_include_list);
  2906. xb_regex_list_free(&regex_exclude_list);
  2907. if (tables_include_hash) {
  2908. xb_filter_hash_free(tables_include_hash);
  2909. }
  2910. if (tables_exclude_hash) {
  2911. xb_filter_hash_free(tables_exclude_hash);
  2912. }
  2913. if (databases_include_hash) {
  2914. xb_filter_hash_free(databases_include_hash);
  2915. }
  2916. if (databases_exclude_hash) {
  2917. xb_filter_hash_free(databases_exclude_hash);
  2918. }
  2919. }
  2920. /*********************************************************************//**
  2921. Creates or opens the log files and closes them.
  2922. @return DB_SUCCESS or error code */
  2923. static
  2924. ulint
  2925. open_or_create_log_file(
  2926. /*====================*/
  2927. ibool create_new_db, /*!< in: TRUE if we should create a
  2928. new database */
  2929. ibool* log_file_created, /*!< out: TRUE if new log file
  2930. created */
  2931. ibool log_file_has_been_opened,/*!< in: TRUE if a log file has been
  2932. opened before: then it is an error
  2933. to try to create another log file */
  2934. ulint k, /*!< in: log group number */
  2935. ulint i) /*!< in: log file number in group */
  2936. {
  2937. ibool ret;
  2938. os_offset_t size;
  2939. char name[10000];
  2940. ulint dirnamelen;
  2941. UT_NOT_USED(create_new_db);
  2942. UT_NOT_USED(log_file_has_been_opened);
  2943. UT_NOT_USED(k);
  2944. ut_ad(k == 0);
  2945. *log_file_created = FALSE;
  2946. srv_normalize_path_for_win(srv_log_group_home_dir);
  2947. dirnamelen = strlen(srv_log_group_home_dir);
  2948. ut_a(dirnamelen < (sizeof name) - 10 - sizeof "ib_logfile");
  2949. memcpy(name, srv_log_group_home_dir, dirnamelen);
  2950. /* Add a path separator if needed. */
  2951. if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) {
  2952. name[dirnamelen++] = SRV_PATH_SEPARATOR;
  2953. }
  2954. sprintf(name + dirnamelen, "%s%lu", "ib_logfile", (ulong) i);
  2955. files[i] = os_file_create(innodb_file_log_key, name,
  2956. OS_FILE_OPEN, OS_FILE_NORMAL,
  2957. OS_LOG_FILE, &ret,0);
  2958. if (ret == FALSE) {
  2959. fprintf(stderr, "InnoDB: Error in opening %s\n", name);
  2960. return(DB_ERROR);
  2961. }
  2962. size = os_file_get_size(files[i]);
  2963. if (size != srv_log_file_size * UNIV_PAGE_SIZE) {
  2964. fprintf(stderr,
  2965. "InnoDB: Error: log file %s is"
  2966. " of different size " UINT64PF " bytes\n"
  2967. "InnoDB: than specified in the .cnf"
  2968. " file " UINT64PF " bytes!\n",
  2969. name, size, srv_log_file_size * UNIV_PAGE_SIZE);
  2970. return(DB_ERROR);
  2971. }
  2972. ret = os_file_close(files[i]);
  2973. ut_a(ret);
  2974. if (i == 0) {
  2975. /* Create in memory the file space object
  2976. which is for this log group */
  2977. fil_space_create(name,
  2978. 2 * k + SRV_LOG_SPACE_FIRST_ID, 0, FIL_LOG, 0, 0);
  2979. }
  2980. ut_a(fil_validate());
  2981. ut_a(fil_node_create(name, (ulint)srv_log_file_size,
  2982. 2 * k + SRV_LOG_SPACE_FIRST_ID, FALSE));
  2983. if (i == 0) {
  2984. log_group_init(k, srv_n_log_files,
  2985. srv_log_file_size * UNIV_PAGE_SIZE,
  2986. 2 * k + SRV_LOG_SPACE_FIRST_ID,
  2987. SRV_LOG_SPACE_FIRST_ID + 1); /* dummy arch
  2988. space id */
  2989. }
  2990. return(DB_SUCCESS);
  2991. }
  2992. /*********************************************************************//**
  2993. Normalizes init parameter values to use units we use inside InnoDB.
  2994. @return DB_SUCCESS or error code */
  2995. static
  2996. void
  2997. xb_normalize_init_values(void)
  2998. /*==========================*/
  2999. {
  3000. ulint i;
  3001. for (i = 0; i < srv_n_data_files; i++) {
  3002. srv_data_file_sizes[i] = srv_data_file_sizes[i]
  3003. * ((1024 * 1024) / UNIV_PAGE_SIZE);
  3004. }
  3005. srv_last_file_size_max = srv_last_file_size_max
  3006. * ((1024 * 1024) / UNIV_PAGE_SIZE);
  3007. srv_log_file_size = srv_log_file_size / UNIV_PAGE_SIZE;
  3008. srv_log_buffer_size = srv_log_buffer_size / UNIV_PAGE_SIZE;
  3009. srv_lock_table_size = 5 * (srv_buf_pool_size / UNIV_PAGE_SIZE);
  3010. }
  3011. /***********************************************************************
  3012. Set the open files limit. Based on set_max_open_files().
  3013. @return the resulting open files limit. May be less or more than the requested
  3014. value. */
  3015. static uint
  3016. xb_set_max_open_files(
  3017. /*==================*/
  3018. uint max_file_limit) /*!<in: open files limit */
  3019. {
  3020. #if defined(RLIMIT_NOFILE)
  3021. struct rlimit rlimit;
  3022. uint old_cur;
  3023. if (getrlimit(RLIMIT_NOFILE, &rlimit)) {
  3024. goto end;
  3025. }
  3026. old_cur = (uint) rlimit.rlim_cur;
  3027. if (rlimit.rlim_cur == RLIM_INFINITY) {
  3028. rlimit.rlim_cur = max_file_limit;
  3029. }
  3030. if (rlimit.rlim_cur >= max_file_limit) {
  3031. max_file_limit = rlimit.rlim_cur;
  3032. goto end;
  3033. }
  3034. rlimit.rlim_cur = rlimit.rlim_max = max_file_limit;
  3035. if (setrlimit(RLIMIT_NOFILE, &rlimit)) {
  3036. max_file_limit = old_cur; /* Use original value */
  3037. } else {
  3038. rlimit.rlim_cur = 0; /* Safety if next call fails */
  3039. (void) getrlimit(RLIMIT_NOFILE, &rlimit);
  3040. if (rlimit.rlim_cur) {
  3041. /* If call didn't fail */
  3042. max_file_limit = (uint) rlimit.rlim_cur;
  3043. }
  3044. }
  3045. end:
  3046. return(max_file_limit);
  3047. #else
  3048. return(0);
  3049. #endif
  3050. }
  3051. void
  3052. xtrabackup_backup_func(void)
  3053. {
  3054. MY_STAT stat_info;
  3055. lsn_t latest_cp;
  3056. uint i;
  3057. uint count;
  3058. os_ib_mutex_t count_mutex;
  3059. data_thread_ctxt_t *data_threads;
  3060. #ifdef USE_POSIX_FADVISE
  3061. msg("xtrabackup: uses posix_fadvise().\n");
  3062. #endif
  3063. /* cd to datadir */
  3064. if (my_setwd(mysql_real_data_home,MYF(MY_WME)))
  3065. {
  3066. msg("xtrabackup: cannot my_setwd %s\n", mysql_real_data_home);
  3067. exit(EXIT_FAILURE);
  3068. }
  3069. msg("xtrabackup: cd to %s\n", mysql_real_data_home);
  3070. msg("xtrabackup: open files limit requested %u, set to %u\n",
  3071. (uint) xb_open_files_limit,
  3072. xb_set_max_open_files(xb_open_files_limit));
  3073. mysql_data_home= mysql_data_home_buff;
  3074. mysql_data_home[0]=FN_CURLIB; // all paths are relative from here
  3075. mysql_data_home[1]=0;
  3076. srv_n_purge_threads = 1;
  3077. srv_read_only_mode = TRUE;
  3078. srv_backup_mode = TRUE;
  3079. srv_close_files = (bool)xb_close_files;
  3080. if (srv_close_files)
  3081. msg("xtrabackup: warning: close-files specified. Use it "
  3082. "at your own risk. If there are DDL operations like table DROP TABLE "
  3083. "or RENAME TABLE during the backup, inconsistent backup will be "
  3084. "produced.\n");
  3085. /* initialize components */
  3086. if(innodb_init_param())
  3087. exit(EXIT_FAILURE);
  3088. xb_normalize_init_values();
  3089. if (srv_file_flush_method_str == NULL) {
  3090. /* These are the default options */
  3091. srv_unix_file_flush_method = SRV_UNIX_FSYNC;
  3092. } else if (0 == ut_strcmp(srv_file_flush_method_str, "fsync")) {
  3093. srv_unix_file_flush_method = SRV_UNIX_FSYNC;
  3094. } else if (0 == ut_strcmp(srv_file_flush_method_str, "O_DSYNC")) {
  3095. srv_unix_file_flush_method = SRV_UNIX_O_DSYNC;
  3096. } else if (0 == ut_strcmp(srv_file_flush_method_str, "O_DIRECT")) {
  3097. srv_unix_file_flush_method = SRV_UNIX_O_DIRECT;
  3098. msg("xtrabackup: using O_DIRECT\n");
  3099. } else if (0 == ut_strcmp(srv_file_flush_method_str, "littlesync")) {
  3100. srv_unix_file_flush_method = SRV_UNIX_LITTLESYNC;
  3101. } else if (0 == ut_strcmp(srv_file_flush_method_str, "nosync")) {
  3102. srv_unix_file_flush_method = SRV_UNIX_NOSYNC;
  3103. } else if (0 == ut_strcmp(srv_file_flush_method_str, "ALL_O_DIRECT")) {
  3104. srv_unix_file_flush_method = SRV_UNIX_ALL_O_DIRECT;
  3105. msg("xtrabackup: using ALL_O_DIRECT\n");
  3106. } else if (0 == ut_strcmp(srv_file_flush_method_str,
  3107. "O_DIRECT_NO_FSYNC")) {
  3108. srv_unix_file_flush_method = SRV_UNIX_O_DIRECT_NO_FSYNC;
  3109. msg("xtrabackup: using O_DIRECT_NO_FSYNC\n");
  3110. } else {
  3111. msg("xtrabackup: Unrecognized value %s for "
  3112. "innodb_flush_method\n", srv_file_flush_method_str);
  3113. exit(EXIT_FAILURE);
  3114. }
  3115. /* We can only use synchronous unbuffered IO on Windows for now */
  3116. if (srv_file_flush_method_str != NULL) {
  3117. msg("xtrabackupp: Warning: "
  3118. "ignoring innodb_flush_method = %s on Windows.\n", srv_file_flush_method_str);
  3119. }
  3120. #ifdef _WIN32
  3121. srv_win_file_flush_method = SRV_WIN_IO_UNBUFFERED;
  3122. srv_use_native_aio = FALSE;
  3123. #endif
  3124. if (srv_buf_pool_size >= 1000 * 1024 * 1024) {
  3125. /* Here we still have srv_pool_size counted
  3126. in kilobytes (in 4.0 this was in bytes)
  3127. srv_boot() converts the value to
  3128. pages; if buffer pool is less than 1000 MB,
  3129. assume fewer threads. */
  3130. srv_max_n_threads = 50000;
  3131. } else if (srv_buf_pool_size >= 8 * 1024 * 1024) {
  3132. srv_max_n_threads = 10000;
  3133. } else {
  3134. srv_max_n_threads = 1000; /* saves several MB of memory,
  3135. especially in 64-bit
  3136. computers */
  3137. }
  3138. srv_general_init();
  3139. ut_crc32_init();
  3140. crc_init();
  3141. #ifdef WITH_INNODB_DISALLOW_WRITES
  3142. srv_allow_writes_event = os_event_create();
  3143. os_event_set(srv_allow_writes_event);
  3144. #endif
  3145. xb_filters_init();
  3146. {
  3147. ibool log_file_created;
  3148. ibool log_created = FALSE;
  3149. ibool log_opened = FALSE;
  3150. ulint err;
  3151. ulint i;
  3152. xb_fil_io_init();
  3153. log_init();
  3154. lock_sys_create(srv_lock_table_size);
  3155. for (i = 0; i < srv_n_log_files; i++) {
  3156. err = open_or_create_log_file(FALSE, &log_file_created,
  3157. log_opened, 0, i);
  3158. if (err != DB_SUCCESS) {
  3159. //return((int) err);
  3160. exit(EXIT_FAILURE);
  3161. }
  3162. if (log_file_created) {
  3163. log_created = TRUE;
  3164. } else {
  3165. log_opened = TRUE;
  3166. }
  3167. if ((log_opened && log_created)) {
  3168. msg(
  3169. "xtrabackup: Error: all log files must be created at the same time.\n"
  3170. "xtrabackup: All log files must be created also in database creation.\n"
  3171. "xtrabackup: If you want bigger or smaller log files, shut down the\n"
  3172. "xtrabackup: database and make sure there were no errors in shutdown.\n"
  3173. "xtrabackup: Then delete the existing log files. Edit the .cnf file\n"
  3174. "xtrabackup: and start the database again.\n");
  3175. //return(DB_ERROR);
  3176. exit(EXIT_FAILURE);
  3177. }
  3178. }
  3179. /* log_file_created must not be TRUE, if online */
  3180. if (log_file_created) {
  3181. msg("xtrabackup: Something wrong with source files...\n");
  3182. exit(EXIT_FAILURE);
  3183. }
  3184. }
  3185. /* create extra LSN dir if it does not exist. */
  3186. if (xtrabackup_extra_lsndir
  3187. &&!my_stat(xtrabackup_extra_lsndir,&stat_info,MYF(0))
  3188. && (my_mkdir(xtrabackup_extra_lsndir,0777,MYF(0)) < 0)) {
  3189. msg("xtrabackup: Error: cannot mkdir %d: %s\n",
  3190. my_errno, xtrabackup_extra_lsndir);
  3191. exit(EXIT_FAILURE);
  3192. }
  3193. /* create target dir if not exist */
  3194. if (!xtrabackup_stream_str && !my_stat(xtrabackup_target_dir,&stat_info,MYF(0))
  3195. && (my_mkdir(xtrabackup_target_dir,0777,MYF(0)) < 0)){
  3196. msg("xtrabackup: Error: cannot mkdir %d: %s\n",
  3197. my_errno, xtrabackup_target_dir);
  3198. exit(EXIT_FAILURE);
  3199. }
  3200. {
  3201. fil_system_t* f_system = fil_system;
  3202. /* definition from recv_recovery_from_checkpoint_start() */
  3203. log_group_t* max_cp_group;
  3204. ulint max_cp_field;
  3205. byte* buf;
  3206. byte* log_hdr_buf_;
  3207. byte* log_hdr_buf;
  3208. ulint err;
  3209. /* start back ground thread to copy newer log */
  3210. os_thread_id_t log_copying_thread_id;
  3211. datafiles_iter_t *it;
  3212. log_hdr_buf_ = static_cast<byte *>
  3213. (ut_malloc(LOG_FILE_HDR_SIZE + UNIV_PAGE_SIZE_MAX));
  3214. log_hdr_buf = static_cast<byte *>
  3215. (ut_align(log_hdr_buf_, UNIV_PAGE_SIZE_MAX));
  3216. /* get current checkpoint_lsn */
  3217. /* Look for the latest checkpoint from any of the log groups */
  3218. mutex_enter(&log_sys->mutex);
  3219. err = recv_find_max_checkpoint(&max_cp_group, &max_cp_field);
  3220. if (err != DB_SUCCESS) {
  3221. ut_free(log_hdr_buf_);
  3222. exit(EXIT_FAILURE);
  3223. }
  3224. log_group_read_checkpoint_info(max_cp_group, max_cp_field);
  3225. buf = log_sys->checkpoint_buf;
  3226. checkpoint_lsn_start = mach_read_from_8(buf + LOG_CHECKPOINT_LSN);
  3227. checkpoint_no_start = mach_read_from_8(buf + LOG_CHECKPOINT_NO);
  3228. mutex_exit(&log_sys->mutex);
  3229. reread_log_header:
  3230. fil_io(OS_FILE_READ | OS_FILE_LOG, true, max_cp_group->space_id,
  3231. 0,
  3232. 0, 0, LOG_FILE_HDR_SIZE,
  3233. log_hdr_buf, max_cp_group, NULL);
  3234. /* check consistency of log file header to copy */
  3235. mutex_enter(&log_sys->mutex);
  3236. err = recv_find_max_checkpoint(&max_cp_group, &max_cp_field);
  3237. if (err != DB_SUCCESS) {
  3238. ut_free(log_hdr_buf_);
  3239. exit(EXIT_FAILURE);
  3240. }
  3241. log_group_read_checkpoint_info(max_cp_group, max_cp_field);
  3242. buf = log_sys->checkpoint_buf;
  3243. if(checkpoint_no_start != mach_read_from_8(buf + LOG_CHECKPOINT_NO)) {
  3244. checkpoint_lsn_start = mach_read_from_8(buf + LOG_CHECKPOINT_LSN);
  3245. checkpoint_no_start = mach_read_from_8(buf + LOG_CHECKPOINT_NO);
  3246. mutex_exit(&log_sys->mutex);
  3247. goto reread_log_header;
  3248. }
  3249. mutex_exit(&log_sys->mutex);
  3250. xtrabackup_init_datasinks();
  3251. if (!select_history()) {
  3252. exit(EXIT_FAILURE);
  3253. }
  3254. /* open the log file */
  3255. memset(&stat_info, 0, sizeof(MY_STAT));
  3256. dst_log_file = ds_open(ds_redo, XB_LOG_FILENAME, &stat_info);
  3257. if (dst_log_file == NULL) {
  3258. msg("xtrabackup: error: failed to open the target stream for "
  3259. "'%s'.\n", XB_LOG_FILENAME);
  3260. ut_free(log_hdr_buf_);
  3261. exit(EXIT_FAILURE);
  3262. }
  3263. /* label it */
  3264. strcpy((char*) log_hdr_buf + LOG_FILE_WAS_CREATED_BY_HOT_BACKUP,
  3265. "xtrabkup ");
  3266. ut_sprintf_timestamp(
  3267. (char*) log_hdr_buf + (LOG_FILE_WAS_CREATED_BY_HOT_BACKUP
  3268. + (sizeof "xtrabkup ") - 1));
  3269. if (ds_write(dst_log_file, log_hdr_buf, LOG_FILE_HDR_SIZE)) {
  3270. msg("xtrabackup: error: write to logfile failed\n");
  3271. ut_free(log_hdr_buf_);
  3272. exit(EXIT_FAILURE);
  3273. }
  3274. ut_free(log_hdr_buf_);
  3275. /* start flag */
  3276. log_copying = TRUE;
  3277. /* start io throttle */
  3278. if(xtrabackup_throttle) {
  3279. os_thread_id_t io_watching_thread_id;
  3280. io_ticket = xtrabackup_throttle;
  3281. wait_throttle = os_event_create();
  3282. os_thread_create(io_watching_thread, NULL,
  3283. &io_watching_thread_id);
  3284. }
  3285. mutex_enter(&log_sys->mutex);
  3286. xtrabackup_choose_lsn_offset(checkpoint_lsn_start);
  3287. mutex_exit(&log_sys->mutex);
  3288. /* copy log file by current position */
  3289. if(xtrabackup_copy_logfile(checkpoint_lsn_start, FALSE))
  3290. exit(EXIT_FAILURE);
  3291. log_copying_stop = os_event_create();
  3292. os_thread_create(log_copying_thread, NULL, &log_copying_thread_id);
  3293. /* Populate fil_system with tablespaces to copy */
  3294. err = xb_load_tablespaces();
  3295. if (err != DB_SUCCESS) {
  3296. msg("xtrabackup: error: xb_load_tablespaces() failed with"
  3297. "error code %lu\n", err);
  3298. exit(EXIT_FAILURE);
  3299. }
  3300. /* FLUSH CHANGED_PAGE_BITMAPS call */
  3301. if (!flush_changed_page_bitmaps()) {
  3302. exit(EXIT_FAILURE);
  3303. }
  3304. debug_sync_point("xtrabackup_suspend_at_start");
  3305. if (xtrabackup_incremental) {
  3306. if (!xtrabackup_incremental_force_scan) {
  3307. changed_page_bitmap = xb_page_bitmap_init();
  3308. }
  3309. if (!changed_page_bitmap) {
  3310. msg("xtrabackup: using the full scan for incremental "
  3311. "backup\n");
  3312. } else if (incremental_lsn != checkpoint_lsn_start) {
  3313. /* Do not print that bitmaps are used when dummy bitmap
  3314. is build for an empty LSN range. */
  3315. msg("xtrabackup: using the changed page bitmap\n");
  3316. }
  3317. }
  3318. ut_a(xtrabackup_parallel > 0);
  3319. if (xtrabackup_parallel > 1) {
  3320. msg("xtrabackup: Starting %u threads for parallel data "
  3321. "files transfer\n", xtrabackup_parallel);
  3322. }
  3323. it = datafiles_iter_new(f_system);
  3324. if (it == NULL) {
  3325. msg("xtrabackup: Error: datafiles_iter_new() failed.\n");
  3326. exit(EXIT_FAILURE);
  3327. }
  3328. /* Create data copying threads */
  3329. data_threads = (data_thread_ctxt_t *)
  3330. ut_malloc(sizeof(data_thread_ctxt_t) * xtrabackup_parallel);
  3331. count = xtrabackup_parallel;
  3332. count_mutex = os_mutex_create();
  3333. for (i = 0; i < (uint) xtrabackup_parallel; i++) {
  3334. data_threads[i].it = it;
  3335. data_threads[i].num = i+1;
  3336. data_threads[i].count = &count;
  3337. data_threads[i].count_mutex = count_mutex;
  3338. os_thread_create(data_copy_thread_func, data_threads + i,
  3339. &data_threads[i].id);
  3340. }
  3341. /* Wait for threads to exit */
  3342. while (1) {
  3343. os_thread_sleep(1000000);
  3344. os_mutex_enter(count_mutex);
  3345. if (count == 0) {
  3346. os_mutex_exit(count_mutex);
  3347. break;
  3348. }
  3349. os_mutex_exit(count_mutex);
  3350. }
  3351. os_mutex_free(count_mutex);
  3352. ut_free(data_threads);
  3353. datafiles_iter_free(it);
  3354. if (changed_page_bitmap) {
  3355. xb_page_bitmap_deinit(changed_page_bitmap);
  3356. }
  3357. }
  3358. if (!backup_start()) {
  3359. exit(EXIT_FAILURE);
  3360. }
  3361. /* read the latest checkpoint lsn */
  3362. latest_cp = 0;
  3363. {
  3364. log_group_t* max_cp_group;
  3365. ulint max_cp_field;
  3366. ulint err;
  3367. mutex_enter(&log_sys->mutex);
  3368. err = recv_find_max_checkpoint(&max_cp_group, &max_cp_field);
  3369. if (err != DB_SUCCESS) {
  3370. msg("xtrabackup: Error: recv_find_max_checkpoint() failed.\n");
  3371. mutex_exit(&log_sys->mutex);
  3372. goto skip_last_cp;
  3373. }
  3374. log_group_read_checkpoint_info(max_cp_group, max_cp_field);
  3375. xtrabackup_choose_lsn_offset(checkpoint_lsn_start);
  3376. latest_cp = mach_read_from_8(log_sys->checkpoint_buf +
  3377. LOG_CHECKPOINT_LSN);
  3378. mutex_exit(&log_sys->mutex);
  3379. msg("xtrabackup: The latest check point (for incremental): "
  3380. "'" LSN_PF "'\n", latest_cp);
  3381. }
  3382. skip_last_cp:
  3383. /* stop log_copying_thread */
  3384. log_copying = FALSE;
  3385. os_event_set(log_copying_stop);
  3386. msg("xtrabackup: Stopping log copying thread.\n");
  3387. while (log_copying_running) {
  3388. msg(".");
  3389. os_thread_sleep(200000); /*0.2 sec*/
  3390. }
  3391. msg("\n");
  3392. os_event_free(log_copying_stop);
  3393. if (ds_close(dst_log_file)) {
  3394. exit(EXIT_FAILURE);
  3395. }
  3396. if(!xtrabackup_incremental) {
  3397. strcpy(metadata_type, "full-backuped");
  3398. metadata_from_lsn = 0;
  3399. } else {
  3400. strcpy(metadata_type, "incremental");
  3401. metadata_from_lsn = incremental_lsn;
  3402. }
  3403. metadata_to_lsn = latest_cp;
  3404. metadata_last_lsn = log_copy_scanned_lsn;
  3405. if (!xtrabackup_stream_metadata(ds_meta)) {
  3406. msg("xtrabackup: Error: failed to stream metadata.\n");
  3407. exit(EXIT_FAILURE);
  3408. }
  3409. if (xtrabackup_extra_lsndir) {
  3410. char filename[FN_REFLEN];
  3411. sprintf(filename, "%s/%s", xtrabackup_extra_lsndir,
  3412. XTRABACKUP_METADATA_FILENAME);
  3413. if (!xtrabackup_write_metadata(filename)) {
  3414. msg("xtrabackup: Error: failed to write metadata "
  3415. "to '%s'.\n", filename);
  3416. exit(EXIT_FAILURE);
  3417. }
  3418. }
  3419. if (!backup_finish()) {
  3420. exit(EXIT_FAILURE);
  3421. }
  3422. xtrabackup_destroy_datasinks();
  3423. if (wait_throttle) {
  3424. /* wait for io_watching_thread completion */
  3425. while (io_watching_thread_running) {
  3426. os_thread_sleep(1000000);
  3427. }
  3428. os_event_free(wait_throttle);
  3429. wait_throttle = NULL;
  3430. }
  3431. msg("xtrabackup: Transaction log of lsn (" LSN_PF ") to (" LSN_PF
  3432. ") was copied.\n", checkpoint_lsn_start, log_copy_scanned_lsn);
  3433. xb_filters_free();
  3434. xb_data_files_close();
  3435. /* Make sure that the latest checkpoint made it to xtrabackup_logfile */
  3436. if (latest_cp > log_copy_scanned_lsn) {
  3437. msg("xtrabackup: error: last checkpoint LSN (" LSN_PF
  3438. ") is larger than last copied LSN (" LSN_PF ").\n",
  3439. latest_cp, log_copy_scanned_lsn);
  3440. exit(EXIT_FAILURE);
  3441. }
  3442. }
  3443. /* ================= prepare ================= */
  3444. static my_bool
  3445. xtrabackup_init_temp_log(void)
  3446. {
  3447. pfs_os_file_t src_file;
  3448. char src_path[FN_REFLEN];
  3449. char dst_path[FN_REFLEN];
  3450. ibool success;
  3451. ulint field;
  3452. byte* log_buf= (byte *)malloc(UNIV_PAGE_SIZE_MAX * 128); /* 2 MB */
  3453. ib_int64_t file_size;
  3454. lsn_t max_no;
  3455. lsn_t max_lsn;
  3456. lsn_t checkpoint_no;
  3457. ulint fold;
  3458. bool checkpoint_found;
  3459. max_no = 0;
  3460. if (!log_buf) {
  3461. goto error;
  3462. }
  3463. if (!xb_init_log_block_size()) {
  3464. goto error;
  3465. }
  3466. if(!xtrabackup_incremental_dir) {
  3467. sprintf(dst_path, "%s/ib_logfile0", xtrabackup_target_dir);
  3468. sprintf(src_path, "%s/%s", xtrabackup_target_dir,
  3469. XB_LOG_FILENAME);
  3470. } else {
  3471. sprintf(dst_path, "%s/ib_logfile0", xtrabackup_incremental_dir);
  3472. sprintf(src_path, "%s/%s", xtrabackup_incremental_dir,
  3473. XB_LOG_FILENAME);
  3474. }
  3475. srv_normalize_path_for_win(dst_path);
  3476. srv_normalize_path_for_win(src_path);
  3477. retry:
  3478. src_file = os_file_create_simple_no_error_handling(0, src_path,
  3479. OS_FILE_OPEN,
  3480. OS_FILE_READ_WRITE,
  3481. &success,0);
  3482. if (!success) {
  3483. /* The following call prints an error message */
  3484. os_file_get_last_error(TRUE);
  3485. msg("xtrabackup: Warning: cannot open %s. will try to find.\n",
  3486. src_path);
  3487. /* check if ib_logfile0 may be xtrabackup_logfile */
  3488. src_file = os_file_create_simple_no_error_handling(0, dst_path,
  3489. OS_FILE_OPEN,
  3490. OS_FILE_READ_WRITE,
  3491. &success,0);
  3492. if (!success) {
  3493. os_file_get_last_error(TRUE);
  3494. msg(" xtrabackup: Fatal error: cannot find %s.\n",
  3495. src_path);
  3496. goto error;
  3497. }
  3498. success = os_file_read(src_file, log_buf, 0,
  3499. LOG_FILE_HDR_SIZE);
  3500. if (!success) {
  3501. goto error;
  3502. }
  3503. if ( ut_memcmp(log_buf + LOG_FILE_WAS_CREATED_BY_HOT_BACKUP,
  3504. (byte*)"xtrabkup", (sizeof "xtrabkup") - 1) == 0) {
  3505. msg(" xtrabackup: 'ib_logfile0' seems to be "
  3506. "'xtrabackup_logfile'. will retry.\n");
  3507. os_file_close(src_file);
  3508. src_file = XB_FILE_UNDEFINED;
  3509. /* rename and try again */
  3510. success = os_file_rename(0, dst_path, src_path);
  3511. if (!success) {
  3512. goto error;
  3513. }
  3514. goto retry;
  3515. }
  3516. msg(" xtrabackup: Fatal error: cannot find %s.\n",
  3517. src_path);
  3518. os_file_close(src_file);
  3519. src_file = XB_FILE_UNDEFINED;
  3520. goto error;
  3521. }
  3522. file_size = os_file_get_size(src_file);
  3523. /* TODO: We should skip the following modifies, if it is not the first time. */
  3524. /* read log file header */
  3525. success = os_file_read(src_file, log_buf, 0, LOG_FILE_HDR_SIZE);
  3526. if (!success) {
  3527. goto error;
  3528. }
  3529. if ( ut_memcmp(log_buf + LOG_FILE_WAS_CREATED_BY_HOT_BACKUP,
  3530. (byte*)"xtrabkup", (sizeof "xtrabkup") - 1) != 0 ) {
  3531. msg("xtrabackup: notice: xtrabackup_logfile was already used "
  3532. "to '--prepare'.\n");
  3533. goto skip_modify;
  3534. } else {
  3535. /* clear it later */
  3536. //memset(log_buf + LOG_FILE_WAS_CREATED_BY_HOT_BACKUP,
  3537. // ' ', 4);
  3538. }
  3539. checkpoint_found = false;
  3540. /* read last checkpoint lsn */
  3541. for (field = LOG_CHECKPOINT_1; field <= LOG_CHECKPOINT_2;
  3542. field += LOG_CHECKPOINT_2 - LOG_CHECKPOINT_1) {
  3543. if (!recv_check_cp_is_consistent(const_cast<const byte *>
  3544. (log_buf + field)))
  3545. goto not_consistent;
  3546. checkpoint_no = mach_read_from_8(log_buf + field +
  3547. LOG_CHECKPOINT_NO);
  3548. if (checkpoint_no >= max_no) {
  3549. max_no = checkpoint_no;
  3550. max_lsn = mach_read_from_8(log_buf + field +
  3551. LOG_CHECKPOINT_LSN);
  3552. checkpoint_found = true;
  3553. }
  3554. not_consistent:
  3555. ;
  3556. }
  3557. if (!checkpoint_found) {
  3558. msg("xtrabackup: No valid checkpoint found.\n");
  3559. goto error;
  3560. }
  3561. /* It seems to be needed to overwrite the both checkpoint area. */
  3562. mach_write_to_8(log_buf + LOG_CHECKPOINT_1 + LOG_CHECKPOINT_LSN,
  3563. max_lsn);
  3564. mach_write_to_4(log_buf + LOG_CHECKPOINT_1
  3565. + LOG_CHECKPOINT_OFFSET_LOW32,
  3566. LOG_FILE_HDR_SIZE +
  3567. (ulint)(max_lsn -
  3568. ut_uint64_align_down(max_lsn,
  3569. OS_FILE_LOG_BLOCK_SIZE)));
  3570. mach_write_to_4(log_buf + LOG_CHECKPOINT_1
  3571. + LOG_CHECKPOINT_OFFSET_HIGH32, 0);
  3572. fold = ut_fold_binary(log_buf + LOG_CHECKPOINT_1, LOG_CHECKPOINT_CHECKSUM_1);
  3573. mach_write_to_4(log_buf + LOG_CHECKPOINT_1 + LOG_CHECKPOINT_CHECKSUM_1, fold);
  3574. fold = ut_fold_binary(log_buf + LOG_CHECKPOINT_1 + LOG_CHECKPOINT_LSN,
  3575. LOG_CHECKPOINT_CHECKSUM_2 - LOG_CHECKPOINT_LSN);
  3576. mach_write_to_4(log_buf + LOG_CHECKPOINT_1 + LOG_CHECKPOINT_CHECKSUM_2, fold);
  3577. mach_write_to_8(log_buf + LOG_CHECKPOINT_2 + LOG_CHECKPOINT_LSN,
  3578. max_lsn);
  3579. mach_write_to_4(log_buf + LOG_CHECKPOINT_2
  3580. + LOG_CHECKPOINT_OFFSET_LOW32,
  3581. LOG_FILE_HDR_SIZE +
  3582. (ulint)(max_lsn -
  3583. ut_uint64_align_down(max_lsn,
  3584. OS_FILE_LOG_BLOCK_SIZE)));
  3585. mach_write_to_4(log_buf + LOG_CHECKPOINT_2
  3586. + LOG_CHECKPOINT_OFFSET_HIGH32, 0);
  3587. fold = ut_fold_binary(log_buf + LOG_CHECKPOINT_2, LOG_CHECKPOINT_CHECKSUM_1);
  3588. mach_write_to_4(log_buf + LOG_CHECKPOINT_2 + LOG_CHECKPOINT_CHECKSUM_1, fold);
  3589. fold = ut_fold_binary(log_buf + LOG_CHECKPOINT_2 + LOG_CHECKPOINT_LSN,
  3590. LOG_CHECKPOINT_CHECKSUM_2 - LOG_CHECKPOINT_LSN);
  3591. mach_write_to_4(log_buf + LOG_CHECKPOINT_2 + LOG_CHECKPOINT_CHECKSUM_2, fold);
  3592. success = os_file_write(src_path, src_file, log_buf, 0,
  3593. LOG_FILE_HDR_SIZE);
  3594. if (!success) {
  3595. goto error;
  3596. }
  3597. /* expand file size (9/8) and align to UNIV_PAGE_SIZE_MAX */
  3598. if (file_size % UNIV_PAGE_SIZE_MAX) {
  3599. memset(log_buf, 0, UNIV_PAGE_SIZE_MAX);
  3600. success = os_file_write(src_path, src_file, log_buf,
  3601. file_size,
  3602. UNIV_PAGE_SIZE_MAX
  3603. - (ulint) (file_size
  3604. % UNIV_PAGE_SIZE_MAX));
  3605. if (!success) {
  3606. goto error;
  3607. }
  3608. file_size = os_file_get_size(src_file);
  3609. }
  3610. /* TODO: We should judge whether the file is already expanded or not... */
  3611. {
  3612. ulint expand;
  3613. memset(log_buf, 0, UNIV_PAGE_SIZE_MAX * 128);
  3614. expand = (ulint) (file_size / UNIV_PAGE_SIZE_MAX / 8);
  3615. for (; expand > 128; expand -= 128) {
  3616. success = os_file_write(src_path, src_file, log_buf,
  3617. file_size,
  3618. UNIV_PAGE_SIZE_MAX * 128);
  3619. if (!success) {
  3620. goto error;
  3621. }
  3622. file_size += UNIV_PAGE_SIZE_MAX * 128;
  3623. }
  3624. if (expand) {
  3625. success = os_file_write(src_path, src_file, log_buf,
  3626. file_size,
  3627. expand * UNIV_PAGE_SIZE_MAX);
  3628. if (!success) {
  3629. goto error;
  3630. }
  3631. file_size += UNIV_PAGE_SIZE_MAX * expand;
  3632. }
  3633. }
  3634. /* make larger than 2MB */
  3635. if (file_size < 2*1024*1024L) {
  3636. memset(log_buf, 0, UNIV_PAGE_SIZE_MAX);
  3637. while (file_size < 2*1024*1024L) {
  3638. success = os_file_write(src_path, src_file, log_buf,
  3639. file_size,
  3640. UNIV_PAGE_SIZE_MAX);
  3641. if (!success) {
  3642. goto error;
  3643. }
  3644. file_size += UNIV_PAGE_SIZE_MAX;
  3645. }
  3646. file_size = os_file_get_size(src_file);
  3647. }
  3648. msg("xtrabackup: xtrabackup_logfile detected: size=" INT64PF ", "
  3649. "start_lsn=(" LSN_PF ")\n", file_size, max_lsn);
  3650. os_file_close(src_file);
  3651. src_file = XB_FILE_UNDEFINED;
  3652. /* fake InnoDB */
  3653. innobase_log_files_in_group_save = innobase_log_files_in_group;
  3654. srv_log_group_home_dir_save = srv_log_group_home_dir;
  3655. innobase_log_file_size_save = innobase_log_file_size;
  3656. srv_log_group_home_dir = NULL;
  3657. innobase_log_file_size = file_size;
  3658. innobase_log_files_in_group = 1;
  3659. srv_thread_concurrency = 0;
  3660. /* rename 'xtrabackup_logfile' to 'ib_logfile0' */
  3661. success = os_file_rename(0, src_path, dst_path);
  3662. if (!success) {
  3663. goto error;
  3664. }
  3665. xtrabackup_logfile_is_renamed = TRUE;
  3666. free(log_buf);
  3667. return(FALSE);
  3668. skip_modify:
  3669. free(log_buf);
  3670. os_file_close(src_file);
  3671. src_file = XB_FILE_UNDEFINED;
  3672. return(FALSE);
  3673. error:
  3674. free(log_buf);
  3675. if (src_file != XB_FILE_UNDEFINED)
  3676. os_file_close(src_file);
  3677. msg("xtrabackup: Error: xtrabackup_init_temp_log() failed.\n");
  3678. return(TRUE); /*ERROR*/
  3679. }
  3680. /***********************************************************************
  3681. Generates path to the meta file path from a given path to an incremental .delta
  3682. by replacing trailing ".delta" with ".meta", or returns error if 'delta_path'
  3683. does not end with the ".delta" character sequence.
  3684. @return TRUE on success, FALSE on error. */
  3685. static
  3686. ibool
  3687. get_meta_path(
  3688. const char *delta_path, /* in: path to a .delta file */
  3689. char *meta_path) /* out: path to the corresponding .meta
  3690. file */
  3691. {
  3692. size_t len = strlen(delta_path);
  3693. if (len <= 6 || strcmp(delta_path + len - 6, ".delta")) {
  3694. return FALSE;
  3695. }
  3696. memcpy(meta_path, delta_path, len - 6);
  3697. strcpy(meta_path + len - 6, XB_DELTA_INFO_SUFFIX);
  3698. return TRUE;
  3699. }
  3700. /****************************************************************//**
  3701. Create a new tablespace on disk and return the handle to its opened
  3702. file. Code adopted from fil_create_new_single_table_tablespace with
  3703. the main difference that only disk file is created without updating
  3704. the InnoDB in-memory dictionary data structures.
  3705. @return TRUE on success, FALSE on error. */
  3706. static
  3707. ibool
  3708. xb_space_create_file(
  3709. /*==================*/
  3710. const char* path, /*!<in: path to tablespace */
  3711. ulint space_id, /*!<in: space id */
  3712. ulint flags __attribute__((unused)),/*!<in: tablespace
  3713. flags */
  3714. pfs_os_file_t* file) /*!<out: file handle */
  3715. {
  3716. ibool ret;
  3717. byte* buf;
  3718. byte* page;
  3719. *file = os_file_create_simple_no_error_handling(0, path, OS_FILE_CREATE,
  3720. OS_FILE_READ_WRITE,
  3721. &ret,0);
  3722. if (!ret) {
  3723. msg("xtrabackup: cannot create file %s\n", path);
  3724. return ret;
  3725. }
  3726. ret = os_file_set_size(path, *file,
  3727. FIL_IBD_FILE_INITIAL_SIZE * UNIV_PAGE_SIZE);
  3728. if (!ret) {
  3729. msg("xtrabackup: cannot set size for file %s\n", path);
  3730. os_file_close(*file);
  3731. os_file_delete(0, path);
  3732. return ret;
  3733. }
  3734. buf = static_cast<byte *>(ut_malloc(3 * UNIV_PAGE_SIZE));
  3735. /* Align the memory for file i/o if we might have O_DIRECT set */
  3736. page = static_cast<byte *>(ut_align(buf, UNIV_PAGE_SIZE));
  3737. memset(page, '\0', UNIV_PAGE_SIZE);
  3738. fsp_header_init_fields(page, space_id, flags);
  3739. mach_write_to_4(page + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID, space_id);
  3740. if (!fsp_flags_is_compressed(flags)) {
  3741. buf_flush_init_for_writing(page, NULL, 0);
  3742. ret = os_file_write(path, *file, page, 0, UNIV_PAGE_SIZE);
  3743. }
  3744. else {
  3745. page_zip_des_t page_zip;
  3746. ulint zip_size;
  3747. zip_size = fsp_flags_get_zip_size(flags);
  3748. page_zip_set_size(&page_zip, zip_size);
  3749. page_zip.data = page + UNIV_PAGE_SIZE;
  3750. fprintf(stderr, "zip_size = %lu\n", zip_size);
  3751. #ifdef UNIV_DEBUG
  3752. page_zip.m_start =
  3753. #endif /* UNIV_DEBUG */
  3754. page_zip.m_end = page_zip.m_nonempty =
  3755. page_zip.n_blobs = 0;
  3756. buf_flush_init_for_writing(page, &page_zip, 0);
  3757. ret = os_file_write(path, *file, page_zip.data, 0,
  3758. zip_size);
  3759. }
  3760. ut_free(buf);
  3761. if (!ret) {
  3762. msg("xtrabackup: could not write the first page to %s\n",
  3763. path);
  3764. os_file_close(*file);
  3765. os_file_delete(0, path);
  3766. return ret;
  3767. }
  3768. return TRUE;
  3769. }
  3770. /***********************************************************************
  3771. Searches for matching tablespace file for given .delta file and space_id
  3772. in given directory. When matching tablespace found, renames it to match the
  3773. name of .delta file. If there was a tablespace with matching name and
  3774. mismatching ID, renames it to xtrabackup_tmp_#ID.ibd. If there was no
  3775. matching file, creates a new tablespace.
  3776. @return file handle of matched or created file */
  3777. static
  3778. pfs_os_file_t
  3779. xb_delta_open_matching_space(
  3780. const char* dbname, /* in: path to destination database dir */
  3781. const char* name, /* in: name of delta file (without .delta) */
  3782. ulint space_id, /* in: space id of delta file */
  3783. ulint zip_size, /* in: zip_size of tablespace */
  3784. char* real_name, /* out: full path of destination file */
  3785. size_t real_name_len, /* out: buffer size for real_name */
  3786. ibool* success) /* out: indicates error. TRUE = success */
  3787. {
  3788. char dest_dir[FN_REFLEN];
  3789. char dest_space_name[FN_REFLEN];
  3790. ibool ok;
  3791. fil_space_t* fil_space;
  3792. pfs_os_file_t file;
  3793. ulint tablespace_flags;
  3794. xb_filter_entry_t* table;
  3795. ut_a(dbname != NULL ||
  3796. !fil_is_user_tablespace_id(space_id) ||
  3797. space_id == ULINT_UNDEFINED);
  3798. *success = FALSE;
  3799. if (dbname) {
  3800. snprintf(dest_dir, FN_REFLEN, "%s/%s",
  3801. xtrabackup_target_dir, dbname);
  3802. srv_normalize_path_for_win(dest_dir);
  3803. snprintf(dest_space_name, FN_REFLEN, "%s/%s", dbname, name);
  3804. } else {
  3805. snprintf(dest_dir, FN_REFLEN, "%s", xtrabackup_target_dir);
  3806. srv_normalize_path_for_win(dest_dir);
  3807. snprintf(dest_space_name, FN_REFLEN, "%s", name);
  3808. }
  3809. snprintf(real_name, real_name_len,
  3810. "%s/%s",
  3811. xtrabackup_target_dir, dest_space_name);
  3812. srv_normalize_path_for_win(real_name);
  3813. /* Truncate ".ibd" */
  3814. dest_space_name[strlen(dest_space_name) - 4] = '\0';
  3815. /* Create the database directory if it doesn't exist yet */
  3816. if (!os_file_create_directory(dest_dir, FALSE)) {
  3817. msg("xtrabackup: error: cannot create dir %s\n", dest_dir);
  3818. return file;
  3819. }
  3820. if (!fil_is_user_tablespace_id(space_id)) {
  3821. goto found;
  3822. }
  3823. /* remember space name for further reference */
  3824. table = static_cast<xb_filter_entry_t *>
  3825. (ut_malloc(sizeof(xb_filter_entry_t) +
  3826. strlen(dest_space_name) + 1));
  3827. table->name = ((char*)table) + sizeof(xb_filter_entry_t);
  3828. strcpy(table->name, dest_space_name);
  3829. HASH_INSERT(xb_filter_entry_t, name_hash, inc_dir_tables_hash,
  3830. ut_fold_string(table->name), table);
  3831. mutex_enter(&fil_system->mutex);
  3832. fil_space = fil_space_get_by_name(dest_space_name);
  3833. mutex_exit(&fil_system->mutex);
  3834. if (fil_space != NULL) {
  3835. if (fil_space->id == space_id || space_id == ULINT_UNDEFINED) {
  3836. /* we found matching space */
  3837. goto found;
  3838. } else {
  3839. char tmpname[FN_REFLEN];
  3840. snprintf(tmpname, FN_REFLEN, "%s/xtrabackup_tmp_#%lu",
  3841. dbname, fil_space->id);
  3842. msg("xtrabackup: Renaming %s to %s.ibd\n",
  3843. fil_space->name, tmpname);
  3844. if (!fil_rename_tablespace(NULL, fil_space->id,
  3845. tmpname, NULL))
  3846. {
  3847. msg("xtrabackup: Cannot rename %s to %s\n",
  3848. fil_space->name, tmpname);
  3849. goto exit;
  3850. }
  3851. }
  3852. }
  3853. if (space_id == ULINT_UNDEFINED)
  3854. {
  3855. msg("xtrabackup: Error: Cannot handle DDL operation on tablespace "
  3856. "%s\n", dest_space_name);
  3857. exit(EXIT_FAILURE);
  3858. }
  3859. mutex_enter(&fil_system->mutex);
  3860. fil_space = fil_space_get_by_id(space_id);
  3861. mutex_exit(&fil_system->mutex);
  3862. if (fil_space != NULL) {
  3863. char tmpname[FN_REFLEN];
  3864. strncpy(tmpname, dest_space_name, FN_REFLEN);
  3865. msg("xtrabackup: Renaming %s to %s\n",
  3866. fil_space->name, dest_space_name);
  3867. if (!fil_rename_tablespace(NULL, fil_space->id, tmpname,
  3868. NULL))
  3869. {
  3870. msg("xtrabackup: Cannot rename %s to %s\n",
  3871. fil_space->name, dest_space_name);
  3872. goto exit;
  3873. }
  3874. goto found;
  3875. }
  3876. /* No matching space found. create the new one. */
  3877. if (!fil_space_create(dest_space_name, space_id, 0,
  3878. FIL_TABLESPACE, 0, false)) {
  3879. msg("xtrabackup: Cannot create tablespace %s\n",
  3880. dest_space_name);
  3881. goto exit;
  3882. }
  3883. /* Calculate correct tablespace flags for compressed tablespaces. */
  3884. if (!zip_size || zip_size == ULINT_UNDEFINED) {
  3885. tablespace_flags = 0;
  3886. }
  3887. else {
  3888. tablespace_flags
  3889. = (get_bit_shift(zip_size >> PAGE_ZIP_MIN_SIZE_SHIFT
  3890. << 1)
  3891. << DICT_TF_ZSSIZE_SHIFT)
  3892. | DICT_TF_COMPACT
  3893. | (DICT_TF_FORMAT_ZIP << DICT_TF_FORMAT_SHIFT);
  3894. ut_a(dict_tf_get_zip_size(tablespace_flags)
  3895. == zip_size);
  3896. }
  3897. *success = xb_space_create_file(real_name, space_id, tablespace_flags,
  3898. &file);
  3899. goto exit;
  3900. found:
  3901. /* open the file and return it's handle */
  3902. file = os_file_create_simple_no_error_handling(0, real_name,
  3903. OS_FILE_OPEN,
  3904. OS_FILE_READ_WRITE,
  3905. &ok,0);
  3906. if (ok) {
  3907. *success = TRUE;
  3908. } else {
  3909. msg("xtrabackup: Cannot open file %s\n", real_name);
  3910. }
  3911. exit:
  3912. return file;
  3913. }
  3914. /************************************************************************
  3915. Applies a given .delta file to the corresponding data file.
  3916. @return TRUE on success */
  3917. static
  3918. ibool
  3919. xtrabackup_apply_delta(
  3920. const char* dirname, /* in: dir name of incremental */
  3921. const char* dbname, /* in: database name (ibdata: NULL) */
  3922. const char* filename, /* in: file name (not a path),
  3923. including the .delta extension */
  3924. void* /*data*/)
  3925. {
  3926. pfs_os_file_t src_file;
  3927. pfs_os_file_t dst_file;
  3928. char src_path[FN_REFLEN];
  3929. char dst_path[FN_REFLEN];
  3930. char meta_path[FN_REFLEN];
  3931. char space_name[FN_REFLEN];
  3932. ibool success;
  3933. ibool last_buffer = FALSE;
  3934. ulint page_in_buffer;
  3935. ulint incremental_buffers = 0;
  3936. xb_delta_info_t info;
  3937. ulint page_size;
  3938. ulint page_size_shift;
  3939. byte* incremental_buffer_base = NULL;
  3940. byte* incremental_buffer;
  3941. size_t offset;
  3942. ut_a(xtrabackup_incremental);
  3943. if (dbname) {
  3944. snprintf(src_path, sizeof(src_path), "%s/%s/%s",
  3945. dirname, dbname, filename);
  3946. snprintf(dst_path, sizeof(dst_path), "%s/%s/%s",
  3947. xtrabackup_real_target_dir, dbname, filename);
  3948. } else {
  3949. snprintf(src_path, sizeof(src_path), "%s/%s",
  3950. dirname, filename);
  3951. snprintf(dst_path, sizeof(dst_path), "%s/%s",
  3952. xtrabackup_real_target_dir, filename);
  3953. }
  3954. dst_path[strlen(dst_path) - 6] = '\0';
  3955. strncpy(space_name, filename, FN_REFLEN);
  3956. space_name[strlen(space_name) - 6] = 0;
  3957. if (!get_meta_path(src_path, meta_path)) {
  3958. goto error;
  3959. }
  3960. srv_normalize_path_for_win(dst_path);
  3961. srv_normalize_path_for_win(src_path);
  3962. srv_normalize_path_for_win(meta_path);
  3963. if (!xb_read_delta_metadata(meta_path, &info)) {
  3964. goto error;
  3965. }
  3966. page_size = info.page_size;
  3967. page_size_shift = get_bit_shift(page_size);
  3968. msg("xtrabackup: page size for %s is %lu bytes\n",
  3969. src_path, page_size);
  3970. if (page_size_shift < 10 ||
  3971. page_size_shift > UNIV_PAGE_SIZE_SHIFT_MAX) {
  3972. msg("xtrabackup: error: invalid value of page_size "
  3973. "(%lu bytes) read from %s\n", page_size, meta_path);
  3974. goto error;
  3975. }
  3976. src_file = os_file_create_simple_no_error_handling(0, src_path,
  3977. OS_FILE_OPEN,
  3978. OS_FILE_READ_WRITE,
  3979. &success,0);
  3980. if (!success) {
  3981. os_file_get_last_error(TRUE);
  3982. msg("xtrabackup: error: cannot open %s\n", src_path);
  3983. goto error;
  3984. }
  3985. posix_fadvise(src_file, 0, 0, POSIX_FADV_SEQUENTIAL);
  3986. os_file_set_nocache(src_file, src_path, "OPEN");
  3987. dst_file = xb_delta_open_matching_space(
  3988. dbname, space_name, info.space_id, info.zip_size,
  3989. dst_path, sizeof(dst_path), &success);
  3990. if (!success) {
  3991. msg("xtrabackup: error: cannot open %s\n", dst_path);
  3992. goto error;
  3993. }
  3994. posix_fadvise(dst_file, 0, 0, POSIX_FADV_DONTNEED);
  3995. os_file_set_nocache(dst_file, dst_path, "OPEN");
  3996. /* allocate buffer for incremental backup (4096 pages) */
  3997. incremental_buffer_base = static_cast<byte *>
  3998. (ut_malloc((page_size / 4 + 1) *
  3999. page_size));
  4000. incremental_buffer = static_cast<byte *>
  4001. (ut_align(incremental_buffer_base,
  4002. page_size));
  4003. msg("Applying %s to %s...\n", src_path, dst_path);
  4004. while (!last_buffer) {
  4005. ulint cluster_header;
  4006. /* read to buffer */
  4007. /* first block of block cluster */
  4008. offset = ((incremental_buffers * (page_size / 4))
  4009. << page_size_shift);
  4010. success = os_file_read(src_file, incremental_buffer,
  4011. offset, page_size);
  4012. if (!success) {
  4013. goto error;
  4014. }
  4015. cluster_header = mach_read_from_4(incremental_buffer);
  4016. switch(cluster_header) {
  4017. case 0x78747261UL: /*"xtra"*/
  4018. break;
  4019. case 0x58545241UL: /*"XTRA"*/
  4020. last_buffer = TRUE;
  4021. break;
  4022. default:
  4023. msg("xtrabackup: error: %s seems not "
  4024. ".delta file.\n", src_path);
  4025. goto error;
  4026. }
  4027. for (page_in_buffer = 1; page_in_buffer < page_size / 4;
  4028. page_in_buffer++) {
  4029. if (mach_read_from_4(incremental_buffer + page_in_buffer * 4)
  4030. == 0xFFFFFFFFUL)
  4031. break;
  4032. }
  4033. ut_a(last_buffer || page_in_buffer == page_size / 4);
  4034. /* read whole of the cluster */
  4035. success = os_file_read(src_file, incremental_buffer,
  4036. offset, page_in_buffer * page_size);
  4037. if (!success) {
  4038. goto error;
  4039. }
  4040. posix_fadvise(src_file, offset, page_in_buffer * page_size,
  4041. POSIX_FADV_DONTNEED);
  4042. for (page_in_buffer = 1; page_in_buffer < page_size / 4;
  4043. page_in_buffer++) {
  4044. ulint offset_on_page;
  4045. offset_on_page = mach_read_from_4(incremental_buffer + page_in_buffer * 4);
  4046. if (offset_on_page == 0xFFFFFFFFUL)
  4047. break;
  4048. success = os_file_write(dst_path, dst_file,
  4049. incremental_buffer +
  4050. page_in_buffer * page_size,
  4051. (offset_on_page <<
  4052. page_size_shift),
  4053. page_size);
  4054. if (!success) {
  4055. goto error;
  4056. }
  4057. }
  4058. incremental_buffers++;
  4059. }
  4060. if (incremental_buffer_base)
  4061. ut_free(incremental_buffer_base);
  4062. if (src_file != XB_FILE_UNDEFINED)
  4063. os_file_close(src_file);
  4064. if (dst_file != XB_FILE_UNDEFINED)
  4065. os_file_close(dst_file);
  4066. return TRUE;
  4067. error:
  4068. if (incremental_buffer_base)
  4069. ut_free(incremental_buffer_base);
  4070. if (src_file != XB_FILE_UNDEFINED)
  4071. os_file_close(src_file);
  4072. if (dst_file != XB_FILE_UNDEFINED)
  4073. os_file_close(dst_file);
  4074. msg("xtrabackup: Error: xtrabackup_apply_delta(): "
  4075. "failed to apply %s to %s.\n", src_path, dst_path);
  4076. return FALSE;
  4077. }
  4078. /************************************************************************
  4079. Callback to handle datadir entry. Function of this type will be called
  4080. for each entry which matches the mask by xb_process_datadir.
  4081. @return should return TRUE on success */
  4082. typedef ibool (*handle_datadir_entry_func_t)(
  4083. /*=========================================*/
  4084. const char* data_home_dir, /*!<in: path to datadir */
  4085. const char* db_name, /*!<in: database name */
  4086. const char* file_name, /*!<in: file name with suffix */
  4087. void* arg); /*!<in: caller-provided data */
  4088. /************************************************************************
  4089. Callback to handle datadir entry. Deletes entry if it has no matching
  4090. fil_space in fil_system directory.
  4091. @return FALSE if delete attempt was unsuccessful */
  4092. static
  4093. ibool
  4094. rm_if_not_found(
  4095. const char* data_home_dir, /*!<in: path to datadir */
  4096. const char* db_name, /*!<in: database name */
  4097. const char* file_name, /*!<in: file name with suffix */
  4098. void* arg __attribute__((unused)))
  4099. {
  4100. char name[FN_REFLEN];
  4101. xb_filter_entry_t* table;
  4102. snprintf(name, FN_REFLEN, "%s/%s", db_name, file_name);
  4103. /* Truncate ".ibd" */
  4104. name[strlen(name) - 4] = '\0';
  4105. HASH_SEARCH(name_hash, inc_dir_tables_hash, ut_fold_string(name),
  4106. xb_filter_entry_t*,
  4107. table, (void) 0,
  4108. !strcmp(table->name, name));
  4109. if (!table) {
  4110. snprintf(name, FN_REFLEN, "%s/%s/%s", data_home_dir,
  4111. db_name, file_name);
  4112. return os_file_delete(0, name);
  4113. }
  4114. return(TRUE);
  4115. }
  4116. /************************************************************************
  4117. Function enumerates files in datadir (provided by path) which are matched
  4118. by provided suffix. For each entry callback is called.
  4119. @return FALSE if callback for some entry returned FALSE */
  4120. static
  4121. ibool
  4122. xb_process_datadir(
  4123. const char* path, /*!<in: datadir path */
  4124. const char* suffix, /*!<in: suffix to match
  4125. against */
  4126. handle_datadir_entry_func_t func, /*!<in: callback */
  4127. void* data) /*!<in: additional argument for
  4128. callback */
  4129. {
  4130. ulint ret;
  4131. char dbpath[FN_REFLEN];
  4132. os_file_dir_t dir;
  4133. os_file_dir_t dbdir;
  4134. os_file_stat_t dbinfo;
  4135. os_file_stat_t fileinfo;
  4136. ulint suffix_len;
  4137. dberr_t err = DB_SUCCESS;
  4138. static char current_dir[2];
  4139. current_dir[0] = FN_CURLIB;
  4140. current_dir[1] = 0;
  4141. srv_data_home = current_dir;
  4142. suffix_len = strlen(suffix);
  4143. /* datafile */
  4144. dbdir = os_file_opendir(path, FALSE);
  4145. if (dbdir != NULL) {
  4146. ret = fil_file_readdir_next_file(&err, path, dbdir,
  4147. &fileinfo);
  4148. while (ret == 0) {
  4149. if (fileinfo.type == OS_FILE_TYPE_DIR) {
  4150. goto next_file_item_1;
  4151. }
  4152. if (strlen(fileinfo.name) > suffix_len
  4153. && 0 == strcmp(fileinfo.name +
  4154. strlen(fileinfo.name) - suffix_len,
  4155. suffix)) {
  4156. if (!func(
  4157. path, NULL,
  4158. fileinfo.name, data))
  4159. {
  4160. return(FALSE);
  4161. }
  4162. }
  4163. next_file_item_1:
  4164. ret = fil_file_readdir_next_file(&err,
  4165. path, dbdir,
  4166. &fileinfo);
  4167. }
  4168. os_file_closedir(dbdir);
  4169. } else {
  4170. msg("xtrabackup: Cannot open dir %s\n",
  4171. path);
  4172. }
  4173. /* single table tablespaces */
  4174. dir = os_file_opendir(path, FALSE);
  4175. if (dir == NULL) {
  4176. msg("xtrabackup: Cannot open dir %s\n",
  4177. path);
  4178. }
  4179. ret = fil_file_readdir_next_file(&err, path, dir,
  4180. &dbinfo);
  4181. while (ret == 0) {
  4182. if (dbinfo.type == OS_FILE_TYPE_FILE
  4183. || dbinfo.type == OS_FILE_TYPE_UNKNOWN) {
  4184. goto next_datadir_item;
  4185. }
  4186. sprintf(dbpath, "%s/%s", path,
  4187. dbinfo.name);
  4188. srv_normalize_path_for_win(dbpath);
  4189. dbdir = os_file_opendir(dbpath, FALSE);
  4190. if (dbdir != NULL) {
  4191. ret = fil_file_readdir_next_file(&err, dbpath, dbdir,
  4192. &fileinfo);
  4193. while (ret == 0) {
  4194. if (fileinfo.type == OS_FILE_TYPE_DIR) {
  4195. goto next_file_item_2;
  4196. }
  4197. if (strlen(fileinfo.name) > suffix_len
  4198. && 0 == strcmp(fileinfo.name +
  4199. strlen(fileinfo.name) -
  4200. suffix_len,
  4201. suffix)) {
  4202. /* The name ends in suffix; process
  4203. the file */
  4204. if (!func(
  4205. path,
  4206. dbinfo.name,
  4207. fileinfo.name, data))
  4208. {
  4209. return(FALSE);
  4210. }
  4211. }
  4212. next_file_item_2:
  4213. ret = fil_file_readdir_next_file(&err,
  4214. dbpath, dbdir,
  4215. &fileinfo);
  4216. }
  4217. os_file_closedir(dbdir);
  4218. }
  4219. next_datadir_item:
  4220. ret = fil_file_readdir_next_file(&err,
  4221. path,
  4222. dir, &dbinfo);
  4223. }
  4224. os_file_closedir(dir);
  4225. return(TRUE);
  4226. }
  4227. /************************************************************************
  4228. Applies all .delta files from incremental_dir to the full backup.
  4229. @return TRUE on success. */
  4230. static
  4231. ibool
  4232. xtrabackup_apply_deltas()
  4233. {
  4234. return xb_process_datadir(xtrabackup_incremental_dir, ".delta",
  4235. xtrabackup_apply_delta, NULL);
  4236. }
  4237. static my_bool
  4238. xtrabackup_close_temp_log(my_bool clear_flag)
  4239. {
  4240. pfs_os_file_t src_file;
  4241. char src_path[FN_REFLEN];
  4242. char dst_path[FN_REFLEN];
  4243. ibool success;
  4244. byte log_buf[UNIV_PAGE_SIZE_MAX];
  4245. if (!xtrabackup_logfile_is_renamed)
  4246. return(FALSE);
  4247. /* rename 'ib_logfile0' to 'xtrabackup_logfile' */
  4248. if(!xtrabackup_incremental_dir) {
  4249. sprintf(dst_path, "%s/ib_logfile0", xtrabackup_target_dir);
  4250. sprintf(src_path, "%s/%s", xtrabackup_target_dir,
  4251. XB_LOG_FILENAME);
  4252. } else {
  4253. sprintf(dst_path, "%s/ib_logfile0", xtrabackup_incremental_dir);
  4254. sprintf(src_path, "%s/%s", xtrabackup_incremental_dir,
  4255. XB_LOG_FILENAME);
  4256. }
  4257. srv_normalize_path_for_win(dst_path);
  4258. srv_normalize_path_for_win(src_path);
  4259. success = os_file_rename(0, dst_path, src_path);
  4260. if (!success) {
  4261. goto error;
  4262. }
  4263. xtrabackup_logfile_is_renamed = FALSE;
  4264. if (!clear_flag)
  4265. return(FALSE);
  4266. /* clear LOG_FILE_WAS_CREATED_BY_HOT_BACKUP field */
  4267. src_file = os_file_create_simple_no_error_handling(0, src_path,
  4268. OS_FILE_OPEN,
  4269. OS_FILE_READ_WRITE,
  4270. &success,0);
  4271. if (!success) {
  4272. goto error;
  4273. }
  4274. success = os_file_read(src_file, log_buf, 0, LOG_FILE_HDR_SIZE);
  4275. if (!success) {
  4276. goto error;
  4277. }
  4278. memset(log_buf + LOG_FILE_WAS_CREATED_BY_HOT_BACKUP, ' ', 4);
  4279. success = os_file_write(src_path, src_file, log_buf, 0,
  4280. LOG_FILE_HDR_SIZE);
  4281. if (!success) {
  4282. goto error;
  4283. }
  4284. os_file_close(src_file);
  4285. src_file = XB_FILE_UNDEFINED;
  4286. innobase_log_files_in_group = innobase_log_files_in_group_save;
  4287. srv_log_group_home_dir = srv_log_group_home_dir_save;
  4288. innobase_log_file_size = innobase_log_file_size_save;
  4289. return(FALSE);
  4290. error:
  4291. if (src_file != XB_FILE_UNDEFINED)
  4292. os_file_close(src_file);
  4293. msg("xtrabackup: Error: xtrabackup_close_temp_log() failed.\n");
  4294. return(TRUE); /*ERROR*/
  4295. }
  4296. /*********************************************************************//**
  4297. Write the meta data (index user fields) config file.
  4298. @return true in case of success otherwise false. */
  4299. static
  4300. bool
  4301. xb_export_cfg_write_index_fields(
  4302. /*===========================*/
  4303. const dict_index_t* index, /*!< in: write the meta data for
  4304. this index */
  4305. FILE* file) /*!< in: file to write to */
  4306. {
  4307. byte row[sizeof(ib_uint32_t) * 2];
  4308. for (ulint i = 0; i < index->n_fields; ++i) {
  4309. byte* ptr = row;
  4310. const dict_field_t* field = &index->fields[i];
  4311. mach_write_to_4(ptr, field->prefix_len);
  4312. ptr += sizeof(ib_uint32_t);
  4313. mach_write_to_4(ptr, field->fixed_len);
  4314. if (fwrite(row, 1, sizeof(row), file) != sizeof(row)) {
  4315. msg("xtrabackup: Error: writing index fields.");
  4316. return(false);
  4317. }
  4318. /* Include the NUL byte in the length. */
  4319. ib_uint32_t len = (ib_uint32_t)strlen(field->name) + 1;
  4320. ut_a(len > 1);
  4321. mach_write_to_4(row, len);
  4322. if (fwrite(row, 1, sizeof(len), file) != sizeof(len)
  4323. || fwrite(field->name, 1, len, file) != len) {
  4324. msg("xtrabackup: Error: writing index column.");
  4325. return(false);
  4326. }
  4327. }
  4328. return(true);
  4329. }
  4330. /*********************************************************************//**
  4331. Write the meta data config file index information.
  4332. @return true in case of success otherwise false. */
  4333. static __attribute__((nonnull, warn_unused_result))
  4334. bool
  4335. xb_export_cfg_write_indexes(
  4336. /*======================*/
  4337. const dict_table_t* table, /*!< in: write the meta data for
  4338. this table */
  4339. FILE* file) /*!< in: file to write to */
  4340. {
  4341. {
  4342. byte row[sizeof(ib_uint32_t)];
  4343. /* Write the number of indexes in the table. */
  4344. mach_write_to_4(row, UT_LIST_GET_LEN(table->indexes));
  4345. if (fwrite(row, 1, sizeof(row), file) != sizeof(row)) {
  4346. msg("xtrabackup: Error: writing index count.");
  4347. return(false);
  4348. }
  4349. }
  4350. bool ret = true;
  4351. /* Write the index meta data. */
  4352. for (const dict_index_t* index = UT_LIST_GET_FIRST(table->indexes);
  4353. index != 0 && ret;
  4354. index = UT_LIST_GET_NEXT(indexes, index)) {
  4355. byte* ptr;
  4356. byte row[sizeof(ib_uint64_t)
  4357. + sizeof(ib_uint32_t) * 8];
  4358. ptr = row;
  4359. ut_ad(sizeof(ib_uint64_t) == 8);
  4360. mach_write_to_8(ptr, index->id);
  4361. ptr += sizeof(ib_uint64_t);
  4362. mach_write_to_4(ptr, index->space);
  4363. ptr += sizeof(ib_uint32_t);
  4364. mach_write_to_4(ptr, index->page);
  4365. ptr += sizeof(ib_uint32_t);
  4366. mach_write_to_4(ptr, index->type);
  4367. ptr += sizeof(ib_uint32_t);
  4368. mach_write_to_4(ptr, index->trx_id_offset);
  4369. ptr += sizeof(ib_uint32_t);
  4370. mach_write_to_4(ptr, index->n_user_defined_cols);
  4371. ptr += sizeof(ib_uint32_t);
  4372. mach_write_to_4(ptr, index->n_uniq);
  4373. ptr += sizeof(ib_uint32_t);
  4374. mach_write_to_4(ptr, index->n_nullable);
  4375. ptr += sizeof(ib_uint32_t);
  4376. mach_write_to_4(ptr, index->n_fields);
  4377. if (fwrite(row, 1, sizeof(row), file) != sizeof(row)) {
  4378. msg("xtrabackup: Error: writing index meta-data.");
  4379. return(false);
  4380. }
  4381. /* Write the length of the index name.
  4382. NUL byte is included in the length. */
  4383. ib_uint32_t len = (ib_uint32_t)strlen(index->name) + 1;
  4384. ut_a(len > 1);
  4385. mach_write_to_4(row, len);
  4386. if (fwrite(row, 1, sizeof(len), file) != sizeof(len)
  4387. || fwrite(index->name, 1, len, file) != len) {
  4388. msg("xtrabackup: Error: writing index name.");
  4389. return(false);
  4390. }
  4391. ret = xb_export_cfg_write_index_fields(index, file);
  4392. }
  4393. return(ret);
  4394. }
  4395. /*********************************************************************//**
  4396. Write the meta data (table columns) config file. Serialise the contents of
  4397. dict_col_t structure, along with the column name. All fields are serialized
  4398. as ib_uint32_t.
  4399. @return true in case of success otherwise false. */
  4400. static __attribute__((nonnull, warn_unused_result))
  4401. bool
  4402. xb_export_cfg_write_table(
  4403. /*====================*/
  4404. const dict_table_t* table, /*!< in: write the meta data for
  4405. this table */
  4406. FILE* file) /*!< in: file to write to */
  4407. {
  4408. dict_col_t* col;
  4409. byte row[sizeof(ib_uint32_t) * 7];
  4410. col = table->cols;
  4411. for (ulint i = 0; i < table->n_cols; ++i, ++col) {
  4412. byte* ptr = row;
  4413. mach_write_to_4(ptr, col->prtype);
  4414. ptr += sizeof(ib_uint32_t);
  4415. mach_write_to_4(ptr, col->mtype);
  4416. ptr += sizeof(ib_uint32_t);
  4417. mach_write_to_4(ptr, col->len);
  4418. ptr += sizeof(ib_uint32_t);
  4419. mach_write_to_4(ptr, col->mbminmaxlen);
  4420. ptr += sizeof(ib_uint32_t);
  4421. mach_write_to_4(ptr, col->ind);
  4422. ptr += sizeof(ib_uint32_t);
  4423. mach_write_to_4(ptr, col->ord_part);
  4424. ptr += sizeof(ib_uint32_t);
  4425. mach_write_to_4(ptr, col->max_prefix);
  4426. if (fwrite(row, 1, sizeof(row), file) != sizeof(row)) {
  4427. msg("xtrabackup: Error: writing table column data.");
  4428. return(false);
  4429. }
  4430. /* Write out the column name as [len, byte array]. The len
  4431. includes the NUL byte. */
  4432. ib_uint32_t len;
  4433. const char* col_name;
  4434. col_name = dict_table_get_col_name(table, dict_col_get_no(col));
  4435. /* Include the NUL byte in the length. */
  4436. len = (ib_uint32_t)strlen(col_name) + 1;
  4437. ut_a(len > 1);
  4438. mach_write_to_4(row, len);
  4439. if (fwrite(row, 1, sizeof(len), file) != sizeof(len)
  4440. || fwrite(col_name, 1, len, file) != len) {
  4441. msg("xtrabackup: Error: writing column name.");
  4442. return(false);
  4443. }
  4444. }
  4445. return(true);
  4446. }
  4447. /*********************************************************************//**
  4448. Write the meta data config file header.
  4449. @return true in case of success otherwise false. */
  4450. static __attribute__((nonnull, warn_unused_result))
  4451. bool
  4452. xb_export_cfg_write_header(
  4453. /*=====================*/
  4454. const dict_table_t* table, /*!< in: write the meta data for
  4455. this table */
  4456. FILE* file) /*!< in: file to write to */
  4457. {
  4458. byte value[sizeof(ib_uint32_t)];
  4459. /* Write the meta-data version number. */
  4460. mach_write_to_4(value, IB_EXPORT_CFG_VERSION_V1);
  4461. if (fwrite(&value, 1, sizeof(value), file) != sizeof(value)) {
  4462. msg("xtrabackup: Error: writing meta-data version number.");
  4463. return(false);
  4464. }
  4465. /* Write the server hostname. */
  4466. ib_uint32_t len;
  4467. const char* hostname = "Hostname unknown";
  4468. /* The server hostname includes the NUL byte. */
  4469. len = (ib_uint32_t)strlen(hostname) + 1;
  4470. mach_write_to_4(value, len);
  4471. if (fwrite(&value, 1, sizeof(value), file) != sizeof(value)
  4472. || fwrite(hostname, 1, len, file) != len) {
  4473. msg("xtrabackup: Error: writing hostname.");
  4474. return(false);
  4475. }
  4476. /* The table name includes the NUL byte. */
  4477. ut_a(table->name != 0);
  4478. len = (ib_uint32_t)strlen(table->name) + 1;
  4479. /* Write the table name. */
  4480. mach_write_to_4(value, len);
  4481. if (fwrite(&value, 1, sizeof(value), file) != sizeof(value)
  4482. || fwrite(table->name, 1, len, file) != len) {
  4483. msg("xtrabackup: Error: writing table name.");
  4484. return(false);
  4485. }
  4486. byte row[sizeof(ib_uint32_t) * 3];
  4487. /* Write the next autoinc value. */
  4488. mach_write_to_8(row, table->autoinc);
  4489. if (fwrite(row, 1, sizeof(ib_uint64_t), file) != sizeof(ib_uint64_t)) {
  4490. msg("xtrabackup: Error: writing table autoinc value.");
  4491. return(false);
  4492. }
  4493. byte* ptr = row;
  4494. /* Write the system page size. */
  4495. mach_write_to_4(ptr, UNIV_PAGE_SIZE);
  4496. ptr += sizeof(ib_uint32_t);
  4497. /* Write the table->flags. */
  4498. mach_write_to_4(ptr, table->flags);
  4499. ptr += sizeof(ib_uint32_t);
  4500. /* Write the number of columns in the table. */
  4501. mach_write_to_4(ptr, table->n_cols);
  4502. if (fwrite(row, 1, sizeof(row), file) != sizeof(row)) {
  4503. msg("xtrabackup: Error: writing table meta-data.");
  4504. return(false);
  4505. }
  4506. return(true);
  4507. }
  4508. /*********************************************************************//**
  4509. Write MySQL 5.6-style meta data config file.
  4510. @return true in case of success otherwise false. */
  4511. static
  4512. bool
  4513. xb_export_cfg_write(
  4514. const fil_node_t* node,
  4515. const dict_table_t* table) /*!< in: write the meta data for
  4516. this table */
  4517. {
  4518. char file_path[FN_REFLEN];
  4519. FILE* file;
  4520. bool success;
  4521. strcpy(file_path, node->name);
  4522. strcpy(file_path + strlen(file_path) - 4, ".cfg");
  4523. file = fopen(file_path, "w+b");
  4524. if (file == NULL) {
  4525. msg("xtrabackup: Error: cannot open %s\n", node->name);
  4526. success = false;
  4527. } else {
  4528. success = xb_export_cfg_write_header(table, file);
  4529. if (success) {
  4530. success = xb_export_cfg_write_table(table, file);
  4531. }
  4532. if (success) {
  4533. success = xb_export_cfg_write_indexes(table, file);
  4534. }
  4535. if (fclose(file) != 0) {
  4536. msg("xtrabackup: Error: cannot close %s\n", node->name);
  4537. success = false;
  4538. }
  4539. }
  4540. return(success);
  4541. }
  4542. static
  4543. void
  4544. innodb_free_param()
  4545. {
  4546. srv_free_paths_and_sizes();
  4547. free(internal_innobase_data_file_path);
  4548. internal_innobase_data_file_path = NULL;
  4549. free_tmpdir(&mysql_tmpdir_list);
  4550. }
  4551. /**************************************************************************
  4552. Store the current binary log coordinates in a specified file.
  4553. @return 'false' on error. */
  4554. static bool
  4555. store_binlog_info(
  4556. /*==============*/
  4557. const char *filename) /*!< in: output file name */
  4558. {
  4559. FILE *fp;
  4560. if (trx_sys_mysql_bin_log_name[0] == '\0') {
  4561. return(true);
  4562. }
  4563. fp = fopen(filename, "w");
  4564. if (!fp) {
  4565. msg("xtrabackup: failed to open '%s'\n", filename);
  4566. return(false);
  4567. }
  4568. fprintf(fp, "%s\t" UINT64PF "\n",
  4569. trx_sys_mysql_bin_log_name, trx_sys_mysql_bin_log_pos);
  4570. fclose(fp);
  4571. return(true);
  4572. }
  4573. static void
  4574. xtrabackup_prepare_func(int argc, char ** argv)
  4575. {
  4576. ulint err;
  4577. datafiles_iter_t *it;
  4578. fil_node_t *node;
  4579. fil_space_t *space;
  4580. char metadata_path[FN_REFLEN];
  4581. /* cd to target-dir */
  4582. if (my_setwd(xtrabackup_real_target_dir,MYF(MY_WME)))
  4583. {
  4584. msg("xtrabackup: cannot my_setwd %s\n",
  4585. xtrabackup_real_target_dir);
  4586. exit(EXIT_FAILURE);
  4587. }
  4588. msg("xtrabackup: cd to %s\n", xtrabackup_real_target_dir);
  4589. encryption_plugin_prepare_init(argc, argv);
  4590. xtrabackup_target_dir= mysql_data_home_buff;
  4591. xtrabackup_target_dir[0]=FN_CURLIB; // all paths are relative from here
  4592. xtrabackup_target_dir[1]=0;
  4593. /*
  4594. read metadata of target
  4595. */
  4596. sprintf(metadata_path, "%s/%s", xtrabackup_target_dir,
  4597. XTRABACKUP_METADATA_FILENAME);
  4598. if (!xtrabackup_read_metadata(metadata_path)) {
  4599. msg("xtrabackup: Error: failed to read metadata from '%s'\n",
  4600. metadata_path);
  4601. exit(EXIT_FAILURE);
  4602. }
  4603. if (!strcmp(metadata_type, "full-backuped")) {
  4604. msg("xtrabackup: This target seems to be not prepared yet.\n");
  4605. } else if (!strcmp(metadata_type, "log-applied")) {
  4606. msg("xtrabackup: This target seems to be already "
  4607. "prepared with --apply-log-only.\n");
  4608. goto skip_check;
  4609. } else if (!strcmp(metadata_type, "full-prepared")) {
  4610. msg("xtrabackup: This target seems to be already prepared.\n");
  4611. } else {
  4612. msg("xtrabackup: This target seems not to have correct "
  4613. "metadata...\n");
  4614. exit(EXIT_FAILURE);
  4615. }
  4616. if (xtrabackup_incremental) {
  4617. msg("xtrabackup: error: applying incremental backup "
  4618. "needs target prepared with --apply-log-only.\n");
  4619. exit(EXIT_FAILURE);
  4620. }
  4621. skip_check:
  4622. if (xtrabackup_incremental
  4623. && metadata_to_lsn != incremental_lsn) {
  4624. msg("xtrabackup: error: This incremental backup seems "
  4625. "not to be proper for the target.\n"
  4626. "xtrabackup: Check 'to_lsn' of the target and "
  4627. "'from_lsn' of the incremental.\n");
  4628. exit(EXIT_FAILURE);
  4629. }
  4630. /* Create logfiles for recovery from 'xtrabackup_logfile', before start InnoDB */
  4631. srv_max_n_threads = 1000;
  4632. srv_n_purge_threads = 1;
  4633. ut_mem_init();
  4634. /* temporally dummy value to avoid crash */
  4635. srv_page_size_shift = 14;
  4636. srv_page_size = (1 << srv_page_size_shift);
  4637. os_sync_init();
  4638. sync_init();
  4639. os_io_init_simple();
  4640. mem_init(srv_mem_pool_size);
  4641. ut_crc32_init();
  4642. #ifdef WITH_INNODB_DISALLOW_WRITES
  4643. srv_allow_writes_event = os_event_create();
  4644. os_event_set(srv_allow_writes_event);
  4645. #endif
  4646. xb_filters_init();
  4647. if (xtrabackup_init_temp_log())
  4648. goto error_cleanup;
  4649. if(innodb_init_param()) {
  4650. goto error_cleanup;
  4651. }
  4652. xb_normalize_init_values();
  4653. if (xtrabackup_incremental) {
  4654. err = xb_data_files_init();
  4655. if (err != DB_SUCCESS) {
  4656. msg("xtrabackup: error: xb_data_files_init() failed "
  4657. "with error code %lu\n", err);
  4658. goto error_cleanup;
  4659. }
  4660. }
  4661. if (xtrabackup_incremental) {
  4662. inc_dir_tables_hash = hash_create(1000);
  4663. if(!xtrabackup_apply_deltas()) {
  4664. xb_data_files_close();
  4665. xb_filter_hash_free(inc_dir_tables_hash);
  4666. goto error_cleanup;
  4667. }
  4668. }
  4669. if (xtrabackup_incremental) {
  4670. xb_data_files_close();
  4671. }
  4672. if (xtrabackup_incremental) {
  4673. /* Cleanup datadir from tablespaces deleted between full and
  4674. incremental backups */
  4675. xb_process_datadir("./", ".ibd", rm_if_not_found, NULL);
  4676. xb_filter_hash_free(inc_dir_tables_hash);
  4677. }
  4678. if (fil_system) {
  4679. fil_close();
  4680. }
  4681. mem_close();
  4682. ut_free_all_mem();
  4683. innodb_free_param();
  4684. sync_close();
  4685. sync_initialized = FALSE;
  4686. /* Reset the configuration as it might have been changed by
  4687. xb_data_files_init(). */
  4688. if(innodb_init_param()) {
  4689. goto error_cleanup;
  4690. }
  4691. srv_apply_log_only = (bool) xtrabackup_apply_log_only;
  4692. /* increase IO threads */
  4693. if(srv_n_file_io_threads < 10) {
  4694. srv_n_read_io_threads = 4;
  4695. srv_n_write_io_threads = 4;
  4696. }
  4697. msg("xtrabackup: Starting InnoDB instance for recovery.\n"
  4698. "xtrabackup: Using %lld bytes for buffer pool "
  4699. "(set by --use-memory parameter)\n", xtrabackup_use_memory);
  4700. srv_max_buf_pool_modified_pct = (double)max_buf_pool_modified_pct;
  4701. if (srv_max_dirty_pages_pct_lwm > srv_max_buf_pool_modified_pct) {
  4702. srv_max_dirty_pages_pct_lwm = srv_max_buf_pool_modified_pct;
  4703. }
  4704. if(innodb_init())
  4705. goto error_cleanup;
  4706. if (xtrabackup_incremental) {
  4707. it = datafiles_iter_new(fil_system);
  4708. if (it == NULL) {
  4709. msg("xtrabackup: Error: datafiles_iter_new() failed.\n");
  4710. exit(EXIT_FAILURE);
  4711. }
  4712. while ((node = datafiles_iter_next(it)) != NULL) {
  4713. byte *header;
  4714. ulint size;
  4715. ulint actual_size;
  4716. mtr_t mtr;
  4717. buf_block_t *block;
  4718. ulint flags;
  4719. space = node->space;
  4720. /* Align space sizes along with fsp header. We want to process
  4721. each space once, so skip all nodes except the first one in a
  4722. multi-node space. */
  4723. if (UT_LIST_GET_PREV(chain, node) != NULL) {
  4724. continue;
  4725. }
  4726. mtr_start(&mtr);
  4727. mtr_s_lock(fil_space_get_latch(space->id, &flags), &mtr);
  4728. block = buf_page_get(space->id,
  4729. dict_tf_get_zip_size(flags),
  4730. 0, RW_S_LATCH, &mtr);
  4731. header = FSP_HEADER_OFFSET + buf_block_get_frame(block);
  4732. size = mtr_read_ulint(header + FSP_SIZE, MLOG_4BYTES,
  4733. &mtr);
  4734. mtr_commit(&mtr);
  4735. fil_extend_space_to_desired_size(&actual_size, space->id, size);
  4736. }
  4737. datafiles_iter_free(it);
  4738. } /* if (xtrabackup_incremental) */
  4739. if (xtrabackup_export) {
  4740. msg("xtrabackup: export option is specified.\n");
  4741. pfs_os_file_t info_file;
  4742. char info_file_path[FN_REFLEN];
  4743. ibool success;
  4744. char table_name[FN_REFLEN];
  4745. byte* page;
  4746. byte* buf = NULL;
  4747. buf = static_cast<byte *>(ut_malloc(UNIV_PAGE_SIZE * 2));
  4748. page = static_cast<byte *>(ut_align(buf, UNIV_PAGE_SIZE));
  4749. /* flush insert buffer at shutdwon */
  4750. innobase_fast_shutdown = 0;
  4751. it = datafiles_iter_new(fil_system);
  4752. if (it == NULL) {
  4753. msg("xtrabackup: Error: datafiles_iter_new() "
  4754. "failed.\n");
  4755. exit(EXIT_FAILURE);
  4756. }
  4757. while ((node = datafiles_iter_next(it)) != NULL) {
  4758. int len;
  4759. char *next, *prev, *p;
  4760. dict_table_t* table;
  4761. dict_index_t* index;
  4762. ulint n_index;
  4763. space = node->space;
  4764. /* treat file_per_table only */
  4765. if (!fil_is_user_tablespace_id(space->id)) {
  4766. continue;
  4767. }
  4768. /* node exist == file exist, here */
  4769. strcpy(info_file_path, node->name);
  4770. #ifdef _WIN32
  4771. for (int i = 0; info_file_path[i]; i++)
  4772. if (info_file_path[i] == '\\')
  4773. info_file_path[i]= '/';
  4774. #endif
  4775. strcpy(info_file_path +
  4776. strlen(info_file_path) -
  4777. 4, ".exp");
  4778. len =(ib_uint32_t)strlen(info_file_path);
  4779. p = info_file_path;
  4780. prev = NULL;
  4781. while ((next = strchr(p, '/')) != NULL)
  4782. {
  4783. prev = p;
  4784. p = next + 1;
  4785. }
  4786. info_file_path[len - 4] = 0;
  4787. strncpy(table_name, prev, FN_REFLEN);
  4788. info_file_path[len - 4] = '.';
  4789. mutex_enter(&(dict_sys->mutex));
  4790. table = dict_table_get_low(table_name);
  4791. if (!table) {
  4792. msg("xtrabackup: error: "
  4793. "cannot find dictionary "
  4794. "record of table %s\n",
  4795. table_name);
  4796. goto next_node;
  4797. }
  4798. /* Write MySQL 5.6 .cfg file */
  4799. if (!xb_export_cfg_write(node, table)) {
  4800. goto next_node;
  4801. }
  4802. index = dict_table_get_first_index(table);
  4803. n_index = UT_LIST_GET_LEN(table->indexes);
  4804. if (n_index > 31) {
  4805. msg("xtrabackup: warning: table '%s' has more "
  4806. "than 31 indexes, .exp file was not "
  4807. "generated. Table will fail to import "
  4808. "on server version prior to 5.6.\n",
  4809. table->name);
  4810. goto next_node;
  4811. }
  4812. /* init exp file */
  4813. memset(page, 0, UNIV_PAGE_SIZE);
  4814. mach_write_to_4(page , 0x78706f72UL);
  4815. mach_write_to_4(page + 4, 0x74696e66UL);/*"xportinf"*/
  4816. mach_write_to_4(page + 8, n_index);
  4817. strncpy((char *) page + 12,
  4818. table_name, 500);
  4819. msg("xtrabackup: export metadata of "
  4820. "table '%s' to file `%s` "
  4821. "(%lu indexes)\n",
  4822. table_name, info_file_path,
  4823. n_index);
  4824. n_index = 1;
  4825. while (index) {
  4826. mach_write_to_8(page + n_index * 512, index->id);
  4827. mach_write_to_4(page + n_index * 512 + 8,
  4828. index->page);
  4829. strncpy((char *) page + n_index * 512 +
  4830. 12, index->name, 500);
  4831. msg("xtrabackup: name=%s, "
  4832. "id.low=%lu, page=%lu\n",
  4833. index->name,
  4834. (ulint)(index->id &
  4835. 0xFFFFFFFFUL),
  4836. (ulint) index->page);
  4837. index = dict_table_get_next_index(index);
  4838. n_index++;
  4839. }
  4840. srv_normalize_path_for_win(info_file_path);
  4841. info_file = os_file_create(
  4842. 0,
  4843. info_file_path,
  4844. OS_FILE_OVERWRITE,
  4845. OS_FILE_NORMAL, OS_DATA_FILE,
  4846. &success,0);
  4847. if (!success) {
  4848. os_file_get_last_error(TRUE);
  4849. goto next_node;
  4850. }
  4851. success = os_file_write(info_file_path,
  4852. info_file, page,
  4853. 0, UNIV_PAGE_SIZE);
  4854. if (!success) {
  4855. os_file_get_last_error(TRUE);
  4856. goto next_node;
  4857. }
  4858. success = os_file_flush(info_file);
  4859. if (!success) {
  4860. os_file_get_last_error(TRUE);
  4861. goto next_node;
  4862. }
  4863. next_node:
  4864. if (info_file != XB_FILE_UNDEFINED) {
  4865. os_file_close(info_file);
  4866. info_file = XB_FILE_UNDEFINED;
  4867. }
  4868. mutex_exit(&(dict_sys->mutex));
  4869. }
  4870. ut_free(buf);
  4871. }
  4872. /* print the binary log position */
  4873. trx_sys_print_mysql_binlog_offset();
  4874. msg("\n");
  4875. /* output to xtrabackup_binlog_pos_innodb and (if
  4876. backup_safe_binlog_info was available on the server) to
  4877. xtrabackup_binlog_info. In the latter case xtrabackup_binlog_pos_innodb
  4878. becomes redundant and is created only for compatibility. */
  4879. if (!store_binlog_info("xtrabackup_binlog_pos_innodb") ||
  4880. (recover_binlog_info &&
  4881. !store_binlog_info(XTRABACKUP_BINLOG_INFO))) {
  4882. exit(EXIT_FAILURE);
  4883. }
  4884. /* Check whether the log is applied enough or not. */
  4885. if ((xtrabackup_incremental
  4886. && srv_start_lsn < incremental_to_lsn)
  4887. ||(!xtrabackup_incremental
  4888. && srv_start_lsn < metadata_to_lsn)) {
  4889. msg("xtrabackup: error: "
  4890. "The transaction log file is corrupted.\n"
  4891. "xtrabackup: error: "
  4892. "The log was not applied to the intended LSN!\n");
  4893. msg("xtrabackup: Log applied to lsn " LSN_PF "\n",
  4894. srv_start_lsn);
  4895. if (xtrabackup_incremental) {
  4896. msg("xtrabackup: The intended lsn is " LSN_PF "\n",
  4897. incremental_to_lsn);
  4898. } else {
  4899. msg("xtrabackup: The intended lsn is " LSN_PF "\n",
  4900. metadata_to_lsn);
  4901. }
  4902. exit(EXIT_FAILURE);
  4903. }
  4904. #ifdef WITH_WSREP
  4905. xb_write_galera_info(xtrabackup_incremental);
  4906. #endif
  4907. innodb_end();
  4908. innodb_free_param();
  4909. sync_initialized = FALSE;
  4910. /* re-init necessary components */
  4911. ut_mem_init();
  4912. os_sync_init();
  4913. sync_init();
  4914. os_io_init_simple();
  4915. if(xtrabackup_close_temp_log(TRUE))
  4916. exit(EXIT_FAILURE);
  4917. /* output to metadata file */
  4918. {
  4919. char filename[FN_REFLEN];
  4920. strcpy(metadata_type, srv_apply_log_only ?
  4921. "log-applied" : "full-prepared");
  4922. if(xtrabackup_incremental
  4923. && metadata_to_lsn < incremental_to_lsn)
  4924. {
  4925. metadata_to_lsn = incremental_to_lsn;
  4926. metadata_last_lsn = incremental_last_lsn;
  4927. }
  4928. sprintf(filename, "%s/%s", xtrabackup_target_dir, XTRABACKUP_METADATA_FILENAME);
  4929. if (!xtrabackup_write_metadata(filename)) {
  4930. msg("xtrabackup: Error: failed to write metadata "
  4931. "to '%s'\n", filename);
  4932. exit(EXIT_FAILURE);
  4933. }
  4934. if(xtrabackup_extra_lsndir) {
  4935. sprintf(filename, "%s/%s", xtrabackup_extra_lsndir, XTRABACKUP_METADATA_FILENAME);
  4936. if (!xtrabackup_write_metadata(filename)) {
  4937. msg("xtrabackup: Error: failed to write "
  4938. "metadata to '%s'\n", filename);
  4939. exit(EXIT_FAILURE);
  4940. }
  4941. }
  4942. }
  4943. if (!apply_log_finish()) {
  4944. exit(EXIT_FAILURE);
  4945. }
  4946. sync_close();
  4947. sync_initialized = FALSE;
  4948. if (fil_system) {
  4949. fil_close();
  4950. }
  4951. ut_free_all_mem();
  4952. /* start InnoDB once again to create log files */
  4953. if (!xtrabackup_apply_log_only) {
  4954. /* xtrabackup_incremental_dir is used to indicate that
  4955. we are going to apply incremental backup. Here we already
  4956. applied incremental backup and are about to do final prepare
  4957. of the full backup */
  4958. xtrabackup_incremental_dir = NULL;
  4959. if(innodb_init_param()) {
  4960. goto error;
  4961. }
  4962. srv_apply_log_only = false;
  4963. /* increase IO threads */
  4964. if(srv_n_file_io_threads < 10) {
  4965. srv_n_read_io_threads = 4;
  4966. srv_n_write_io_threads = 4;
  4967. }
  4968. srv_shutdown_state = SRV_SHUTDOWN_NONE;
  4969. if(innodb_init())
  4970. goto error;
  4971. innodb_end();
  4972. innodb_free_param();
  4973. }
  4974. xb_filters_free();
  4975. return;
  4976. error_cleanup:
  4977. xtrabackup_close_temp_log(FALSE);
  4978. xb_filters_free();
  4979. error:
  4980. exit(EXIT_FAILURE);
  4981. }
  4982. /**************************************************************************
  4983. Append group name to xb_load_default_groups list. */
  4984. static
  4985. void
  4986. append_defaults_group(const char *group, const char *default_groups[],
  4987. size_t default_groups_size)
  4988. {
  4989. uint i;
  4990. bool appended = false;
  4991. for (i = 0; i < default_groups_size - 1; i++) {
  4992. if (default_groups[i] == NULL) {
  4993. default_groups[i] = group;
  4994. appended = true;
  4995. break;
  4996. }
  4997. }
  4998. ut_a(appended);
  4999. }
  5000. bool
  5001. xb_init()
  5002. {
  5003. const char *mixed_options[4] = {NULL, NULL, NULL, NULL};
  5004. int n_mixed_options;
  5005. /* sanity checks */
  5006. if (opt_slave_info
  5007. && opt_no_lock
  5008. && !opt_safe_slave_backup) {
  5009. msg("Error: --slave-info is used with --no-lock but "
  5010. "without --safe-slave-backup. The binlog position "
  5011. "cannot be consistent with the backup data.\n");
  5012. return(false);
  5013. }
  5014. if (opt_rsync && xtrabackup_stream_fmt) {
  5015. msg("Error: --rsync doesn't work with --stream\n");
  5016. return(false);
  5017. }
  5018. n_mixed_options = 0;
  5019. if (opt_decompress) {
  5020. mixed_options[n_mixed_options++] = "--decompress";
  5021. }
  5022. if (xtrabackup_copy_back) {
  5023. mixed_options[n_mixed_options++] = "--copy-back";
  5024. }
  5025. if (xtrabackup_move_back) {
  5026. mixed_options[n_mixed_options++] = "--move-back";
  5027. }
  5028. if (xtrabackup_prepare) {
  5029. mixed_options[n_mixed_options++] = "--apply-log";
  5030. }
  5031. if (n_mixed_options > 1) {
  5032. msg("Error: %s and %s are mutually exclusive\n",
  5033. mixed_options[0], mixed_options[1]);
  5034. return(false);
  5035. }
  5036. if (xtrabackup_backup) {
  5037. if ((mysql_connection = xb_mysql_connect()) == NULL) {
  5038. return(false);
  5039. }
  5040. if (!get_mysql_vars(mysql_connection)) {
  5041. return(false);
  5042. }
  5043. encryption_plugin_backup_init(mysql_connection);
  5044. history_start_time = time(NULL);
  5045. }
  5046. return(true);
  5047. }
  5048. extern void init_signals(void);
  5049. #include <sql_locale.h>
  5050. /* Messages . Avoid loading errmsg.sys file */
  5051. void setup_error_messages()
  5052. {
  5053. static const char *all_msgs[ER_ERROR_LAST - ER_ERROR_FIRST +1];
  5054. my_default_lc_messages = &my_locale_en_US;
  5055. my_default_lc_messages->errmsgs->errmsgs = all_msgs;
  5056. /* Populate the necessary error messages */
  5057. struct {
  5058. int id;
  5059. const char *fmt;
  5060. }
  5061. xb_msgs[] =
  5062. {
  5063. { ER_DATABASE_NAME,"Database" },
  5064. { ER_TABLE_NAME,"Table"},
  5065. { ER_PARTITION_NAME, "Partition" },
  5066. { ER_SUBPARTITION_NAME, "Subpartition" },
  5067. { ER_TEMPORARY_NAME, "Temporary"},
  5068. { ER_RENAMED_NAME, "Renamed"},
  5069. { ER_CANT_FIND_DL_ENTRY, "Can't find symbol '%-.128s' in library"},
  5070. { ER_CANT_OPEN_LIBRARY, "Can't open shared library '%-.192s' (errno: %d, %-.128s)" },
  5071. { ER_OUTOFMEMORY, "Out of memory; restart server and try again (needed %d bytes)" },
  5072. { ER_CANT_OPEN_LIBRARY, "Can't open shared library '%-.192s' (errno: %d, %-.128s)" },
  5073. { ER_UDF_NO_PATHS, "No paths allowed for shared library" },
  5074. { ER_CANT_INITIALIZE_UDF,"Can't initialize function '%-.192s'; %-.80s"},
  5075. { ER_PLUGIN_IS_NOT_LOADED,"Plugin '%-.192s' is not loaded" }
  5076. };
  5077. for (int i = 0; i < (int)array_elements(all_msgs); i++)
  5078. all_msgs[i] = "Unknown error";
  5079. for (int i = 0; i < (int)array_elements(xb_msgs); i++)
  5080. all_msgs[xb_msgs[i].id - ER_ERROR_FIRST] = xb_msgs[i].fmt;
  5081. }
  5082. extern my_bool(*dict_check_if_skip_table)(const char* name) ;
  5083. void
  5084. handle_options(int argc, char **argv, char ***argv_client, char ***argv_server)
  5085. {
  5086. /* Setup some variables for Innodb.*/
  5087. srv_xtrabackup = true;
  5088. files_charset_info = &my_charset_utf8_general_ci;
  5089. dict_check_if_skip_table = check_if_skip_table;
  5090. setup_error_messages();
  5091. sys_var_init();
  5092. plugin_mutex_init();
  5093. mysql_rwlock_init(key_rwlock_LOCK_system_variables_hash, &LOCK_system_variables_hash);
  5094. opt_stack_trace = 1;
  5095. test_flags |= TEST_SIGINT;
  5096. init_signals();
  5097. #ifndef _WIN32
  5098. /* Exit process on SIGINT. */
  5099. my_sigset(SIGINT, SIG_DFL);
  5100. #endif
  5101. sf_leaking_memory = 0; /* don't report memory leaks on early exist */
  5102. int i;
  5103. int ho_error;
  5104. char* target_dir = NULL;
  5105. bool prepare = false;
  5106. char conf_file[FN_REFLEN];
  5107. int argc_client = argc;
  5108. int argc_server = argc;
  5109. /* scan options for group and config file to load defaults from */
  5110. for (i = 1; i < argc; i++) {
  5111. char *optend = strcend(argv[i], '=');
  5112. if (strncmp(argv[i], "--defaults-group",
  5113. optend - argv[i]) == 0) {
  5114. defaults_group = optend + 1;
  5115. append_defaults_group(defaults_group,
  5116. xb_server_default_groups,
  5117. array_elements(xb_server_default_groups));
  5118. }
  5119. if (strncmp(argv[i], "--login-path",
  5120. optend - argv[i]) == 0) {
  5121. append_defaults_group(optend + 1,
  5122. xb_client_default_groups,
  5123. array_elements(xb_client_default_groups));
  5124. }
  5125. if (!strncmp(argv[i], "--prepare",
  5126. optend - argv[i])) {
  5127. prepare = true;
  5128. }
  5129. if (!strncmp(argv[i], "--apply-log",
  5130. optend - argv[i])) {
  5131. prepare = true;
  5132. }
  5133. if (!strncmp(argv[i], "--target-dir",
  5134. optend - argv[i]) && *optend) {
  5135. target_dir = optend + 1;
  5136. }
  5137. if (!*optend && argv[i][0] != '-') {
  5138. target_dir = argv[i];
  5139. }
  5140. }
  5141. snprintf(conf_file, sizeof(conf_file), "my");
  5142. if (prepare && target_dir) {
  5143. snprintf(conf_file, sizeof(conf_file),
  5144. "%s/backup-my.cnf", target_dir);
  5145. if (!strncmp(argv[1], "--defaults-file=", 16)) {
  5146. /* Remove defaults-file*/
  5147. for (int i = 2; ; i++) {
  5148. if ((argv[i-1]= argv[i]) == 0)
  5149. break;
  5150. }
  5151. argc--;
  5152. }
  5153. }
  5154. *argv_client = argv;
  5155. *argv_server = argv;
  5156. if (load_defaults(conf_file, xb_server_default_groups,
  5157. &argc_server, argv_server)) {
  5158. exit(EXIT_FAILURE);
  5159. }
  5160. int n;
  5161. for (n = 0; (*argv_server)[n]; n++) {};
  5162. argc_server = n;
  5163. print_param_str <<
  5164. "# This MySQL options file was generated by XtraBackup.\n"
  5165. "[" << defaults_group << "]\n";
  5166. /* We want xtrabackup to ignore unknown options, because it only
  5167. recognizes a small subset of server variables */
  5168. my_getopt_skip_unknown = TRUE;
  5169. /* Reset u_max_value for all options, as we don't want the
  5170. --maximum-... modifier to set the actual option values */
  5171. for (my_option *optp= xb_server_options; optp->name; optp++) {
  5172. optp->u_max_value = (G_PTR *) &global_max_value;
  5173. }
  5174. /* Throw a descriptive error if --defaults-file or --defaults-extra-file
  5175. is not the first command line argument */
  5176. for (int i = 2 ; i < argc ; i++) {
  5177. char *optend = strcend((argv)[i], '=');
  5178. if (optend - argv[i] == 15 &&
  5179. !strncmp(argv[i], "--defaults-file", optend - argv[i])) {
  5180. msg("xtrabackup: Error: --defaults-file "
  5181. "must be specified first on the command "
  5182. "line\n");
  5183. exit(EXIT_FAILURE);
  5184. }
  5185. if (optend - argv[i] == 21 &&
  5186. !strncmp(argv[i], "--defaults-extra-file",
  5187. optend - argv[i])) {
  5188. msg("xtrabackup: Error: --defaults-extra-file "
  5189. "must be specified first on the command "
  5190. "line\n");
  5191. exit(EXIT_FAILURE);
  5192. }
  5193. }
  5194. if (argc_server > 0
  5195. && (ho_error=handle_options(&argc_server, argv_server,
  5196. xb_server_options, xb_get_one_option)))
  5197. exit(ho_error);
  5198. if (load_defaults(conf_file, xb_client_default_groups,
  5199. &argc_client, argv_client)) {
  5200. exit(EXIT_FAILURE);
  5201. }
  5202. for (n = 0; (*argv_client)[n]; n++) {};
  5203. argc_client = n;
  5204. if (innobackupex_mode && argc_client > 0) {
  5205. /* emulate innobackupex script */
  5206. innobackupex_mode = true;
  5207. if (!ibx_handle_options(&argc_client, argv_client)) {
  5208. exit(EXIT_FAILURE);
  5209. }
  5210. }
  5211. if (argc_client > 0
  5212. && (ho_error=handle_options(&argc_client, argv_client,
  5213. xb_client_options, xb_get_one_option)))
  5214. exit(ho_error);
  5215. /* Reject command line arguments that don't look like options, i.e. are
  5216. not of the form '-X' (single-character options) or '--option' (long
  5217. options) */
  5218. for (int i = 0 ; i < argc_client ; i++) {
  5219. const char * const opt = (*argv_client)[i];
  5220. if (strncmp(opt, "--", 2) &&
  5221. !(strlen(opt) == 2 && opt[0] == '-')) {
  5222. bool server_option = true;
  5223. for (int j = 0; j < argc_server; j++) {
  5224. if (opt == (*argv_server)[j]) {
  5225. server_option = false;
  5226. break;
  5227. }
  5228. }
  5229. if (!server_option) {
  5230. msg("xtrabackup: Error:"
  5231. " unknown argument: '%s'\n", opt);
  5232. exit(EXIT_FAILURE);
  5233. }
  5234. }
  5235. }
  5236. }
  5237. /* ================= main =================== */
  5238. extern my_bool(*fil_check_if_skip_database_by_path)(const char* name);
  5239. int main(int argc, char **argv)
  5240. {
  5241. char **client_defaults, **server_defaults;
  5242. char cwd[FN_REFLEN];
  5243. if (argc > 1 && (strcmp(argv[1], "--innobackupex") == 0))
  5244. {
  5245. argv++;
  5246. argc--;
  5247. innobackupex_mode = true;
  5248. }
  5249. /* Setup skip fil_load_single_tablespaces callback.*/
  5250. fil_check_if_skip_database_by_path = check_if_skip_database_by_path;
  5251. init_signals();
  5252. MY_INIT(argv[0]);
  5253. pthread_key_create(&THR_THD, NULL);
  5254. my_pthread_setspecific_ptr(THR_THD, NULL);
  5255. xb_regex_init();
  5256. capture_tool_command(argc, argv);
  5257. if (mysql_server_init(-1, NULL, NULL))
  5258. {
  5259. exit(EXIT_FAILURE);
  5260. }
  5261. system_charset_info = &my_charset_utf8_general_ci;
  5262. key_map_full.set_all();
  5263. handle_options(argc, argv, &client_defaults, &server_defaults);
  5264. int argc_server;
  5265. for (argc_server = 0; server_defaults[argc_server]; argc_server++) {}
  5266. int argc_client;
  5267. for (argc_client = 0; client_defaults[argc_client]; argc_client++) {}
  5268. if (innobackupex_mode) {
  5269. if (!ibx_init()) {
  5270. exit(EXIT_FAILURE);
  5271. }
  5272. }
  5273. if ((!xtrabackup_print_param) && (!xtrabackup_prepare) && (strcmp(mysql_data_home, "./") == 0)) {
  5274. if (!xtrabackup_print_param)
  5275. usage();
  5276. msg("\nxtrabackup: Error: Please set parameter 'datadir'\n");
  5277. exit(EXIT_FAILURE);
  5278. }
  5279. /* Expand target-dir, incremental-basedir, etc. */
  5280. my_getwd(cwd, sizeof(cwd), MYF(0));
  5281. my_load_path(xtrabackup_real_target_dir,
  5282. xtrabackup_target_dir, cwd);
  5283. unpack_dirname(xtrabackup_real_target_dir,
  5284. xtrabackup_real_target_dir);
  5285. xtrabackup_target_dir= xtrabackup_real_target_dir;
  5286. if (xtrabackup_incremental_basedir) {
  5287. my_load_path(xtrabackup_real_incremental_basedir,
  5288. xtrabackup_incremental_basedir, cwd);
  5289. unpack_dirname(xtrabackup_real_incremental_basedir,
  5290. xtrabackup_real_incremental_basedir);
  5291. xtrabackup_incremental_basedir =
  5292. xtrabackup_real_incremental_basedir;
  5293. }
  5294. if (xtrabackup_incremental_dir) {
  5295. my_load_path(xtrabackup_real_incremental_dir,
  5296. xtrabackup_incremental_dir, cwd);
  5297. unpack_dirname(xtrabackup_real_incremental_dir,
  5298. xtrabackup_real_incremental_dir);
  5299. xtrabackup_incremental_dir = xtrabackup_real_incremental_dir;
  5300. }
  5301. if (xtrabackup_extra_lsndir) {
  5302. my_load_path(xtrabackup_real_extra_lsndir,
  5303. xtrabackup_extra_lsndir, cwd);
  5304. unpack_dirname(xtrabackup_real_extra_lsndir,
  5305. xtrabackup_real_extra_lsndir);
  5306. xtrabackup_extra_lsndir = xtrabackup_real_extra_lsndir;
  5307. }
  5308. /* get default temporary directory */
  5309. if (!opt_mysql_tmpdir || !opt_mysql_tmpdir[0]) {
  5310. opt_mysql_tmpdir = getenv("TMPDIR");
  5311. #if defined(__WIN__)
  5312. if (!opt_mysql_tmpdir) {
  5313. opt_mysql_tmpdir = getenv("TEMP");
  5314. }
  5315. if (!opt_mysql_tmpdir) {
  5316. opt_mysql_tmpdir = getenv("TMP");
  5317. }
  5318. #endif
  5319. if (!opt_mysql_tmpdir || !opt_mysql_tmpdir[0]) {
  5320. opt_mysql_tmpdir = const_cast<char*>(DEFAULT_TMPDIR);
  5321. }
  5322. }
  5323. /* temporary setting of enough size */
  5324. srv_page_size_shift = UNIV_PAGE_SIZE_SHIFT_MAX;
  5325. srv_page_size = UNIV_PAGE_SIZE_MAX;
  5326. if (xtrabackup_backup && xtrabackup_incremental) {
  5327. /* direct specification is only for --backup */
  5328. /* and the lsn is prior to the other option */
  5329. char* endchar;
  5330. int error = 0;
  5331. incremental_lsn = strtoll(xtrabackup_incremental, &endchar, 10);
  5332. if (*endchar != '\0')
  5333. error = 1;
  5334. if (error) {
  5335. msg("xtrabackup: value '%s' may be wrong format for "
  5336. "incremental option.\n", xtrabackup_incremental);
  5337. exit(EXIT_FAILURE);
  5338. }
  5339. } else if (xtrabackup_backup && xtrabackup_incremental_basedir) {
  5340. char filename[FN_REFLEN];
  5341. sprintf(filename, "%s/%s", xtrabackup_incremental_basedir, XTRABACKUP_METADATA_FILENAME);
  5342. if (!xtrabackup_read_metadata(filename)) {
  5343. msg("xtrabackup: error: failed to read metadata from "
  5344. "%s\n", filename);
  5345. exit(EXIT_FAILURE);
  5346. }
  5347. incremental_lsn = metadata_to_lsn;
  5348. xtrabackup_incremental = xtrabackup_incremental_basedir; //dummy
  5349. } else if (xtrabackup_prepare && xtrabackup_incremental_dir) {
  5350. char filename[FN_REFLEN];
  5351. sprintf(filename, "%s/%s", xtrabackup_incremental_dir, XTRABACKUP_METADATA_FILENAME);
  5352. if (!xtrabackup_read_metadata(filename)) {
  5353. msg("xtrabackup: error: failed to read metadata from "
  5354. "%s\n", filename);
  5355. exit(EXIT_FAILURE);
  5356. }
  5357. incremental_lsn = metadata_from_lsn;
  5358. incremental_to_lsn = metadata_to_lsn;
  5359. incremental_last_lsn = metadata_last_lsn;
  5360. xtrabackup_incremental = xtrabackup_incremental_dir; //dummy
  5361. } else if (opt_incremental_history_name) {
  5362. xtrabackup_incremental = opt_incremental_history_name;
  5363. } else if (opt_incremental_history_uuid) {
  5364. xtrabackup_incremental = opt_incremental_history_uuid;
  5365. } else {
  5366. xtrabackup_incremental = NULL;
  5367. }
  5368. if (!xb_init()) {
  5369. exit(EXIT_FAILURE);
  5370. }
  5371. /* --print-param */
  5372. if (xtrabackup_print_param) {
  5373. printf("%s", print_param_str.str().c_str());
  5374. exit(EXIT_SUCCESS);
  5375. }
  5376. print_version();
  5377. if (xtrabackup_incremental) {
  5378. msg("incremental backup from " LSN_PF " is enabled.\n",
  5379. incremental_lsn);
  5380. }
  5381. if (xtrabackup_export && innobase_file_per_table == FALSE) {
  5382. msg("xtrabackup: auto-enabling --innodb-file-per-table due to "
  5383. "the --export option\n");
  5384. innobase_file_per_table = TRUE;
  5385. }
  5386. /* cannot execute both for now */
  5387. {
  5388. int num = 0;
  5389. if (xtrabackup_backup) num++;
  5390. if (xtrabackup_prepare) num++;
  5391. if (xtrabackup_copy_back) num++;
  5392. if (xtrabackup_move_back) num++;
  5393. if (xtrabackup_decrypt_decompress) num++;
  5394. if (num != 1) { /* !XOR (for now) */
  5395. usage();
  5396. exit(EXIT_FAILURE);
  5397. }
  5398. }
  5399. #ifndef __WIN__
  5400. if (xtrabackup_debug_sync) {
  5401. signal(SIGCONT, sigcont_handler);
  5402. }
  5403. #endif
  5404. /* --backup */
  5405. if (xtrabackup_backup)
  5406. xtrabackup_backup_func();
  5407. /* --prepare */
  5408. if (xtrabackup_prepare) {
  5409. xtrabackup_prepare_func(argc_server, server_defaults);
  5410. }
  5411. if (xtrabackup_copy_back || xtrabackup_move_back) {
  5412. if (!check_if_param_set("datadir")) {
  5413. msg("Error: datadir must be specified.\n");
  5414. exit(EXIT_FAILURE);
  5415. }
  5416. if (!copy_back())
  5417. exit(EXIT_FAILURE);
  5418. }
  5419. if (xtrabackup_decrypt_decompress && !decrypt_decompress()) {
  5420. exit(EXIT_FAILURE);
  5421. }
  5422. backup_cleanup();
  5423. if (innobackupex_mode) {
  5424. ibx_cleanup();
  5425. }
  5426. free_defaults(client_defaults);
  5427. free_defaults(server_defaults);
  5428. if (THR_THD)
  5429. (void) pthread_key_delete(THR_THD);
  5430. msg_ts("completed OK!\n");
  5431. exit(EXIT_SUCCESS);
  5432. }