You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

110 lines
6.1 KiB

MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
MDEV-12610: MariaDB start is slow Problem appears to be that the function fsp_flags_try_adjust() is being unconditionally invoked on every .ibd file on startup. Based on performance investigation also the top function fsp_header_get_crypt_offset() needs to addressed. Ported implementation of fsp_header_get_encryption_offset() function from 10.2 to fsp_header_get_crypt_offset(). Introduced a new function fil_crypt_read_crypt_data() to read page 0 if it is not yet read. fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd file is not read on startup we need to check has page 0 read from space that we investigate for key rotation, if it is not read we read it. fil_space_crypt_get_status(): Now that page 0 for every .ibd file is not read on startup here also we need to read page 0 if it is not yet read it. This is needed as tests use IS query to wait until background encryption or decryption has finished and this function is used to produce results. fil_crypt_thread(): Add is_stopping condition for tablespace so that we do not rotate pages if usage of tablespace should be stopped. This was needed for failure seen on regression testing. fil_space_create: Remove page_0_crypt_read and extra unnecessary info output. fil_open_single_table_tablespace(): We call fsp_flags_try_adjust only when when no errors has happened and server was not started on read only mode and tablespace validation was requested or flags contain other table options except low order bits to FSP_FLAGS_POS_PAGE_SSIZE position. fil_space_t::page_0_crypt_read removed. Added test case innodb-first-page-read to test startup when encryption is on and when encryption is off to check that not for all tables page 0 is read on startup.
9 years ago
MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.
8 years ago
  1. -- source include/have_innodb.inc
  2. -- source include/have_file_key_management_plugin.inc
  3. -- source include/not_embedded.inc
  4. FLUSH STATUS;
  5. create database innodb_test;
  6. use innodb_test;
  7. create table innodb_normal(c1 bigint not null, b char(200)) engine=innodb;
  8. create table innodb_compact(c1 bigint not null, b char(200)) engine=innodb row_format=compact;
  9. create table innodb_dynamic(c1 bigint not null, b char(200)) engine=innodb row_format=dynamic;
  10. create table innodb_compressed(c1 bigint not null, b char(200)) engine=innodb row_format=compressed;
  11. create table innodb_compressed1(c1 bigint not null, b char(200)) engine=innodb row_format=compressed key_block_size=1;
  12. create table innodb_compressed2(c1 bigint not null, b char(200)) engine=innodb row_format=compressed key_block_size=2;
  13. create table innodb_compressed4(c1 bigint not null, b char(200)) engine=innodb row_format=compressed key_block_size=4;
  14. create table innodb_compressed8(c1 bigint not null, b char(200)) engine=innodb row_format=compressed key_block_size=8;
  15. create table innodb_compressed16(c1 bigint not null, b char(200)) engine=innodb row_format=compressed key_block_size=16;
  16. create table innodb_redundant(c1 bigint not null, b char(200)) engine=innodb row_format=redundant;
  17. create table innodb_pagecomp(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes;
  18. create table innodb_pagecomp1(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=1;
  19. create table innodb_pagecomp2(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=2;
  20. create table innodb_pagecomp3(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=3;
  21. create table innodb_pagecomp4(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=4;
  22. create table innodb_pagecomp5(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=5;
  23. create table innodb_pagecomp6(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=6;
  24. create table innodb_pagecomp7(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=7;
  25. create table innodb_pagecomp8(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=8;
  26. create table innodb_pagecomp9(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes page_compression_level=9;
  27. --replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR
  28. eval create table innodb_datadir1(c1 bigint not null, b char(200)) engine=innodb DATA DIRECTORY='$MYSQL_TMP_DIR';
  29. --replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR
  30. eval create table innodb_datadir2(c1 bigint not null, b char(200)) engine=innodb row_format=compressed DATA DIRECTORY='$MYSQL_TMP_DIR';
  31. --replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR
  32. eval create table innodb_datadir3(c1 bigint not null, b char(200)) engine=innodb page_compressed=yes DATA DIRECTORY='$MYSQL_TMP_DIR';
  33. begin;
  34. insert into innodb_normal values (1,'secret');
  35. insert into innodb_compact select * from innodb_normal;
  36. insert into innodb_dynamic select * from innodb_normal;
  37. insert into innodb_compressed select * from innodb_normal;
  38. insert into innodb_compressed1 select * from innodb_normal;
  39. insert into innodb_compressed2 select * from innodb_normal;
  40. insert into innodb_compressed4 select * from innodb_normal;
  41. insert into innodb_compressed8 select * from innodb_normal;
  42. insert into innodb_compressed16 select * from innodb_normal;
  43. insert into innodb_redundant select * from innodb_normal;
  44. insert into innodb_pagecomp select * from innodb_normal;
  45. insert into innodb_pagecomp1 select * from innodb_normal;
  46. insert into innodb_pagecomp2 select * from innodb_normal;
  47. insert into innodb_pagecomp3 select * from innodb_normal;
  48. insert into innodb_pagecomp4 select * from innodb_normal;
  49. insert into innodb_pagecomp5 select * from innodb_normal;
  50. insert into innodb_pagecomp6 select * from innodb_normal;
  51. insert into innodb_pagecomp7 select * from innodb_normal;
  52. insert into innodb_pagecomp8 select * from innodb_normal;
  53. insert into innodb_pagecomp9 select * from innodb_normal;
  54. insert into innodb_datadir1 select * from innodb_normal;
  55. insert into innodb_datadir2 select * from innodb_normal;
  56. insert into innodb_datadir3 select * from innodb_normal;
  57. commit;
  58. FLUSH STATUS;
  59. --echo # Restart server and see how many page 0's are read
  60. --source include/restart_mysqld.inc
  61. --echo # result should actual number of tables except remote tables could be read twice
  62. --echo # i.e. < 23 + 3*2 = 29
  63. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  64. use innodb_test;
  65. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  66. use test;
  67. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  68. set global innodb_encrypt_tables=OFF;
  69. --echo # wait until tables are decrypted
  70. SET GLOBAL innodb_max_dirty_pages_pct=0.0;
  71. SET GLOBAL innodb_max_dirty_pages_pct_lwm=0.0;
  72. let $wait_condition =
  73. SELECT variable_value = 0
  74. FROM information_schema.global_status
  75. WHERE variable_name = 'INNODB_BUFFER_POOL_PAGES_DIRTY';
  76. --source include/wait_condition.inc
  77. --echo # result should be actual number of tables except remote tables could be read twice
  78. --echo # i.e. < 23 + 3*2 = 29
  79. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  80. use innodb_test;
  81. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  82. use test;
  83. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  84. FLUSH STATUS;
  85. --echo # restart and see number read page 0
  86. -- source include/restart_mysqld.inc
  87. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  88. use innodb_test;
  89. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  90. use test;
  91. SELECT VARIABLE_VALUE <= 29 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_pages0_read';
  92. drop database innodb_test;
  93. FLUSH STATUS;