You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

255 lines
8.1 KiB

MDEV-17958 Make bug-endian innodb_checksum_algorithm=crc32 optional In MySQL 5.7, it was noticed that files are not portable between big-endian and little-endian processor architectures (such as SPARC and x86), because the original implementation of innodb_checksum_algorithm=crc32 was not byte order agnostic. A byte order agnostic implementation of innodb_checksum_algorithm=crc32 was only added to MySQL 5.7, not backported to 5.6. Consequently, MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C implementation that works incorrectly on big-endian architectures, and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C implementation from MySQL 5.7. MySQL 5.7 introduced a "legacy crc32" variant that is functionally equivalent to the big-endian version of the original crc32 implementation. Thanks to this variant, old data files can be transferred from big-endian systems to newer versions. Introducing new variants of checksum algorithms (without introducing new names for them, or something on the pages themselves to identify the algorithm) generally is a bad idea, because each checksum algorithm is like a lottery ticket. The more algorithms you try, the more likely it will be for the checksum to match on a corrupted page. So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32, and MariaDB 10.2.2 inherited this weakening. We introduce a build option that together with MDEV-17957 makes innodb_checksum_algorithm=strict_crc32 strict again by only allowing one variant of the checksum to match. WITH_INNODB_BUG_ENDIAN_CRC32: A new cmake option for enabling the bug-compatible "legacy crc32" checksum. This is only enabled on big-endian systems by default, to facilitate an upgrade from MariaDB 10.0 or 10.1. Checked by #ifdef INNODB_BUG_ENDIAN_CRC32. ut_crc32_byte_by_byte: Remove (unused function). legacy_big_endian_checksum: Remove. This variable seems to have unnecessarily complicated the logic. When the weakening is enabled, we must always fall back to the buggy checksum. buf_page_check_crc32(): A helper function to compute one or two CRC-32C variants.
7 years ago
MDEV-17958 Make bug-endian innodb_checksum_algorithm=crc32 optional In MySQL 5.7, it was noticed that files are not portable between big-endian and little-endian processor architectures (such as SPARC and x86), because the original implementation of innodb_checksum_algorithm=crc32 was not byte order agnostic. A byte order agnostic implementation of innodb_checksum_algorithm=crc32 was only added to MySQL 5.7, not backported to 5.6. Consequently, MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C implementation that works incorrectly on big-endian architectures, and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C implementation from MySQL 5.7. MySQL 5.7 introduced a "legacy crc32" variant that is functionally equivalent to the big-endian version of the original crc32 implementation. Thanks to this variant, old data files can be transferred from big-endian systems to newer versions. Introducing new variants of checksum algorithms (without introducing new names for them, or something on the pages themselves to identify the algorithm) generally is a bad idea, because each checksum algorithm is like a lottery ticket. The more algorithms you try, the more likely it will be for the checksum to match on a corrupted page. So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32, and MariaDB 10.2.2 inherited this weakening. We introduce a build option that together with MDEV-17957 makes innodb_checksum_algorithm=strict_crc32 strict again by only allowing one variant of the checksum to match. WITH_INNODB_BUG_ENDIAN_CRC32: A new cmake option for enabling the bug-compatible "legacy crc32" checksum. This is only enabled on big-endian systems by default, to facilitate an upgrade from MariaDB 10.0 or 10.1. Checked by #ifdef INNODB_BUG_ENDIAN_CRC32. ut_crc32_byte_by_byte: Remove (unused function). legacy_big_endian_checksum: Remove. This variable seems to have unnecessarily complicated the logic. When the weakening is enabled, we must always fall back to the buggy checksum. buf_page_check_crc32(): A helper function to compute one or two CRC-32C variants.
7 years ago
MDEV-17958 Make bug-endian innodb_checksum_algorithm=crc32 optional In MySQL 5.7, it was noticed that files are not portable between big-endian and little-endian processor architectures (such as SPARC and x86), because the original implementation of innodb_checksum_algorithm=crc32 was not byte order agnostic. A byte order agnostic implementation of innodb_checksum_algorithm=crc32 was only added to MySQL 5.7, not backported to 5.6. Consequently, MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C implementation that works incorrectly on big-endian architectures, and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C implementation from MySQL 5.7. MySQL 5.7 introduced a "legacy crc32" variant that is functionally equivalent to the big-endian version of the original crc32 implementation. Thanks to this variant, old data files can be transferred from big-endian systems to newer versions. Introducing new variants of checksum algorithms (without introducing new names for them, or something on the pages themselves to identify the algorithm) generally is a bad idea, because each checksum algorithm is like a lottery ticket. The more algorithms you try, the more likely it will be for the checksum to match on a corrupted page. So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32, and MariaDB 10.2.2 inherited this weakening. We introduce a build option that together with MDEV-17957 makes innodb_checksum_algorithm=strict_crc32 strict again by only allowing one variant of the checksum to match. WITH_INNODB_BUG_ENDIAN_CRC32: A new cmake option for enabling the bug-compatible "legacy crc32" checksum. This is only enabled on big-endian systems by default, to facilitate an upgrade from MariaDB 10.0 or 10.1. Checked by #ifdef INNODB_BUG_ENDIAN_CRC32. ut_crc32_byte_by_byte: Remove (unused function). legacy_big_endian_checksum: Remove. This variable seems to have unnecessarily complicated the logic. When the weakening is enabled, we must always fall back to the buggy checksum. buf_page_check_crc32(): A helper function to compute one or two CRC-32C variants.
7 years ago
MDEV-17958 Make bug-endian innodb_checksum_algorithm=crc32 optional In MySQL 5.7, it was noticed that files are not portable between big-endian and little-endian processor architectures (such as SPARC and x86), because the original implementation of innodb_checksum_algorithm=crc32 was not byte order agnostic. A byte order agnostic implementation of innodb_checksum_algorithm=crc32 was only added to MySQL 5.7, not backported to 5.6. Consequently, MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C implementation that works incorrectly on big-endian architectures, and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C implementation from MySQL 5.7. MySQL 5.7 introduced a "legacy crc32" variant that is functionally equivalent to the big-endian version of the original crc32 implementation. Thanks to this variant, old data files can be transferred from big-endian systems to newer versions. Introducing new variants of checksum algorithms (without introducing new names for them, or something on the pages themselves to identify the algorithm) generally is a bad idea, because each checksum algorithm is like a lottery ticket. The more algorithms you try, the more likely it will be for the checksum to match on a corrupted page. So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32, and MariaDB 10.2.2 inherited this weakening. We introduce a build option that together with MDEV-17957 makes innodb_checksum_algorithm=strict_crc32 strict again by only allowing one variant of the checksum to match. WITH_INNODB_BUG_ENDIAN_CRC32: A new cmake option for enabling the bug-compatible "legacy crc32" checksum. This is only enabled on big-endian systems by default, to facilitate an upgrade from MariaDB 10.0 or 10.1. Checked by #ifdef INNODB_BUG_ENDIAN_CRC32. ut_crc32_byte_by_byte: Remove (unused function). legacy_big_endian_checksum: Remove. This variable seems to have unnecessarily complicated the logic. When the weakening is enabled, we must always fall back to the buggy checksum. buf_page_check_crc32(): A helper function to compute one or two CRC-32C variants.
7 years ago
  1. # Copyright (c) 2006, 2016, Oracle and/or its affiliates. All rights reserved.
  2. # Copyright (c) 2017, 2022, MariaDB Corporation.
  3. #
  4. # This program is free software; you can redistribute it and/or modify
  5. # it under the terms of the GNU General Public License as published by
  6. # the Free Software Foundation; version 2 of the License.
  7. #
  8. # This program is distributed in the hope that it will be useful,
  9. # but WITHOUT ANY WARRANTY; without even the implied warranty of
  10. # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  11. # GNU General Public License for more details.
  12. #
  13. # You should have received a copy of the GNU General Public License
  14. # along with this program; if not, write to the Free Software
  15. # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
  16. # This is the CMakeLists for InnoDB
  17. INCLUDE(CheckFunctionExists)
  18. INCLUDE(CheckCSourceCompiles)
  19. INCLUDE(CheckCSourceRuns)
  20. INCLUDE(lz4.cmake)
  21. INCLUDE(lzo.cmake)
  22. INCLUDE(lzma.cmake)
  23. INCLUDE(bzip2.cmake)
  24. INCLUDE(snappy.cmake)
  25. INCLUDE(numa)
  26. INCLUDE(TestBigEndian)
  27. MYSQL_CHECK_LZ4()
  28. MYSQL_CHECK_LZO()
  29. MYSQL_CHECK_LZMA()
  30. MYSQL_CHECK_BZIP2()
  31. MYSQL_CHECK_SNAPPY()
  32. MYSQL_CHECK_NUMA()
  33. TEST_BIG_ENDIAN(IS_BIG_ENDIAN)
  34. INCLUDE(${MYSQL_CMAKE_SCRIPT_DIR}/compile_flags.cmake)
  35. IF(CMAKE_CROSSCOMPILING)
  36. # Use CHECK_C_SOURCE_COMPILES instead of CHECK_C_SOURCE_RUNS when
  37. # cross-compiling. Not as precise, but usually good enough.
  38. # This only make sense for atomic tests in this file, this trick doesn't
  39. # work in a general case.
  40. MACRO(CHECK_C_SOURCE SOURCE VAR)
  41. CHECK_C_SOURCE_COMPILES("${SOURCE}" "${VAR}")
  42. ENDMACRO()
  43. ELSE()
  44. MACRO(CHECK_C_SOURCE SOURCE VAR)
  45. CHECK_C_SOURCE_RUNS("${SOURCE}" "${VAR}")
  46. ENDMACRO()
  47. ENDIF()
  48. # OS tests
  49. IF(UNIX)
  50. IF(CMAKE_SYSTEM_NAME STREQUAL "Linux")
  51. ADD_DEFINITIONS("-DUNIV_LINUX -D_GNU_SOURCE=1")
  52. CHECK_INCLUDE_FILES (libaio.h HAVE_LIBAIO_H)
  53. CHECK_LIBRARY_EXISTS(aio io_queue_init "" HAVE_LIBAIO)
  54. IF(HAVE_LIBAIO_H AND HAVE_LIBAIO)
  55. ADD_DEFINITIONS(-DLINUX_NATIVE_AIO=1)
  56. LINK_LIBRARIES(aio)
  57. ENDIF()
  58. IF(HAVE_LIBNUMA)
  59. LINK_LIBRARIES(numa)
  60. ENDIF()
  61. ELSEIF(CMAKE_SYSTEM_NAME MATCHES "HP*")
  62. ADD_DEFINITIONS("-DUNIV_HPUX")
  63. ELSEIF(CMAKE_SYSTEM_NAME STREQUAL "AIX")
  64. ADD_DEFINITIONS("-DUNIV_AIX")
  65. ELSEIF(CMAKE_SYSTEM_NAME STREQUAL "SunOS")
  66. ADD_DEFINITIONS("-DUNIV_SOLARIS")
  67. ENDIF()
  68. ENDIF()
  69. OPTION(INNODB_COMPILER_HINTS "Compile InnoDB with compiler hints" ON)
  70. MARK_AS_ADVANCED(INNODB_COMPILER_HINTS)
  71. IF(INNODB_COMPILER_HINTS)
  72. ADD_DEFINITIONS("-DCOMPILER_HINTS")
  73. ENDIF()
  74. ADD_FEATURE_INFO(INNODB_COMPILER_HINTS INNODB_COMPILER_HINTS "InnoDB compiled with compiler hints")
  75. SET(MUTEXTYPE "event" CACHE STRING "Mutex type: event, sys or futex")
  76. IF(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
  77. # After: WL#5825 Using C++ Standard Library with MySQL code
  78. # we no longer use -fno-exceptions
  79. # SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions")
  80. # Add -Wconversion if compiling with GCC
  81. ## As of Mar 15 2011 this flag causes 3573+ warnings. If you are reading this
  82. ## please fix them and enable the following code:
  83. #SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wconversion")
  84. IF (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64" OR
  85. CMAKE_SYSTEM_PROCESSOR MATCHES "i386" AND
  86. CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.6)
  87. INCLUDE(CheckCXXCompilerFlag)
  88. CHECK_CXX_COMPILER_FLAG("-fno-builtin-memcmp" HAVE_NO_BUILTIN_MEMCMP)
  89. IF (HAVE_NO_BUILTIN_MEMCMP)
  90. # Work around http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43052
  91. SET_SOURCE_FILES_PROPERTIES(${CMAKE_CURRENT_SOURCE_DIR}/rem/rem0cmp.cc
  92. PROPERTIES COMPILE_FLAGS -fno-builtin-memcmp)
  93. ENDIF()
  94. ENDIF()
  95. ENDIF()
  96. # Enable InnoDB's UNIV_DEBUG in debug builds
  97. SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DUNIV_DEBUG")
  98. OPTION(WITH_INNODB_AHI "Include innodb_adaptive_hash_index" ON)
  99. OPTION(WITH_INNODB_ROOT_GUESS "Cache index root block descriptors" ON)
  100. IF(WITH_INNODB_AHI)
  101. ADD_DEFINITIONS(-DBTR_CUR_HASH_ADAPT -DBTR_CUR_ADAPT)
  102. IF(NOT WITH_INNODB_ROOT_GUESS)
  103. MESSAGE(WARNING "WITH_INNODB_AHI implies WITH_INNODB_ROOT_GUESS")
  104. SET(WITH_INNODB_ROOT_GUESS ON)
  105. ENDIF()
  106. ELSEIF(WITH_INNODB_ROOT_GUESS)
  107. ADD_DEFINITIONS(-DBTR_CUR_ADAPT)
  108. ENDIF()
  109. ADD_FEATURE_INFO(INNODB_AHI WITH_INNODB_AHI "InnoDB Adaptive Hash Index")
  110. ADD_FEATURE_INFO(INNODB_ROOT_GUESS WITH_INNODB_ROOT_GUESS
  111. "Cache index root block descriptors in InnoDB")
  112. OPTION(WITH_INNODB_BUG_ENDIAN_CRC32 "Weaken innodb_checksum_algorithm=crc32 by supporting upgrade from big-endian systems running 5.6/10.0/10.1" ${IS_BIG_ENDIAN})
  113. IF(WITH_INNODB_BUG_ENDIAN_CRC32)
  114. ADD_DEFINITIONS(-DINNODB_BUG_ENDIAN_CRC32)
  115. ENDIF()
  116. ADD_FEATURE_INFO(INNODB_BUG_ENDIAN_CRC32 WITH_INNODB_BUG_ENDIAN_CRC32 "Big endian weaker CRC32 to support upgrade from big-endian systems running 5.6/10.0/10.1")
  117. OPTION(WITH_INNODB_EXTRA_DEBUG "Enable extra InnoDB debug checks" OFF)
  118. IF(WITH_INNODB_EXTRA_DEBUG)
  119. ADD_DEFINITIONS(-DUNIV_ZIP_DEBUG)
  120. ENDIF()
  121. ADD_FEATURE_INFO(INNODB_EXTRA_DEBUG WITH_INNODB_EXTRA_DEBUG "Extra InnoDB debug checks")
  122. CHECK_FUNCTION_EXISTS(sched_getcpu HAVE_SCHED_GETCPU)
  123. IF(HAVE_SCHED_GETCPU)
  124. ADD_DEFINITIONS(-DHAVE_SCHED_GETCPU=1)
  125. ENDIF()
  126. CHECK_FUNCTION_EXISTS(nanosleep HAVE_NANOSLEEP)
  127. IF(HAVE_NANOSLEEP)
  128. ADD_DEFINITIONS(-DHAVE_NANOSLEEP=1)
  129. ENDIF()
  130. IF(HAVE_FALLOC_PUNCH_HOLE_AND_KEEP_SIZE)
  131. ADD_DEFINITIONS(-DHAVE_FALLOC_PUNCH_HOLE_AND_KEEP_SIZE=1)
  132. ENDIF()
  133. IF(NOT MSVC)
  134. # Work around MDEV-18417, MDEV-18656, MDEV-18417
  135. IF(WITH_ASAN AND CMAKE_COMPILER_IS_GNUCC AND
  136. CMAKE_C_COMPILER_VERSION VERSION_LESS "6.0.0")
  137. SET_SOURCE_FILES_PROPERTIES(trx/trx0rec.cc PROPERTIES COMPILE_FLAGS -O1)
  138. ENDIF()
  139. # Only use futexes on Linux if GCC atomics are available
  140. IF(NOT MSVC AND NOT CMAKE_CROSSCOMPILING)
  141. CHECK_C_SOURCE_RUNS(
  142. "
  143. #include <stdio.h>
  144. #include <unistd.h>
  145. #include <errno.h>
  146. #include <assert.h>
  147. #include <linux/futex.h>
  148. #include <unistd.h>
  149. #include <sys/syscall.h>
  150. int futex_wait(int* futex, int v) {
  151. return(syscall(SYS_futex, futex, FUTEX_WAIT_PRIVATE, v, NULL, NULL, 0));
  152. }
  153. int futex_signal(int* futex) {
  154. return(syscall(SYS_futex, futex, FUTEX_WAKE, 1, NULL, NULL, 0));
  155. }
  156. int main() {
  157. int ret;
  158. int m = 1;
  159. /* It is setup to fail and return EWOULDBLOCK. */
  160. ret = futex_wait(&m, 0);
  161. assert(ret == -1 && errno == EWOULDBLOCK);
  162. /* Shouldn't wake up any threads. */
  163. assert(futex_signal(&m) == 0);
  164. return(0);
  165. }"
  166. HAVE_IB_LINUX_FUTEX)
  167. ENDIF()
  168. IF(HAVE_IB_LINUX_FUTEX)
  169. ADD_DEFINITIONS(-DHAVE_IB_LINUX_FUTEX=1)
  170. ENDIF()
  171. ENDIF(NOT MSVC)
  172. CHECK_FUNCTION_EXISTS(vasprintf HAVE_VASPRINTF)
  173. SET(MUTEXTYPE "event" CACHE STRING "Mutex type: event, sys or futex")
  174. IF(MUTEXTYPE MATCHES "event")
  175. ADD_DEFINITIONS(-DMUTEX_EVENT)
  176. ELSEIF(MUTEXTYPE MATCHES "futex" AND DEFINED HAVE_IB_LINUX_FUTEX)
  177. ADD_DEFINITIONS(-DMUTEX_FUTEX)
  178. ELSE()
  179. ADD_DEFINITIONS(-DMUTEX_SYS)
  180. ENDIF()
  181. # Include directories under innobase
  182. INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/storage/innobase/include
  183. ${CMAKE_SOURCE_DIR}/storage/innobase/handler)
  184. # Sun Studio bug with -xO2
  185. IF(CMAKE_CXX_COMPILER_ID MATCHES "SunPro"
  186. AND CMAKE_CXX_FLAGS_RELEASE MATCHES "O2"
  187. AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
  188. # Sun Studio 12 crashes with -xO2 flag, but not with higher optimization
  189. # -xO3
  190. SET_SOURCE_FILES_PROPERTIES(${CMAKE_CURRENT_SOURCE_DIR}/rem/rem0rec.cc
  191. PROPERTIES COMPILE_FLAGS -xO3)
  192. ENDIF()
  193. # Avoid generating Hardware Capabilities due to crc32 instructions
  194. IF(CMAKE_SYSTEM_NAME MATCHES "SunOS" AND CMAKE_SYSTEM_PROCESSOR MATCHES "i386")
  195. MY_CHECK_CXX_COMPILER_FLAG("-Wa,-nH")
  196. IF(have_CXX__Wa__nH)
  197. ADD_COMPILE_FLAGS(
  198. ut/ut0crc32.cc
  199. COMPILE_FLAGS "-Wa,-nH"
  200. )
  201. ENDIF()
  202. ENDIF()
  203. IF(MSVC)
  204. # Avoid "unreferenced label" warning in generated file
  205. GET_FILENAME_COMPONENT(_SRC_DIR ${CMAKE_CURRENT_LIST_FILE} PATH)
  206. SET_SOURCE_FILES_PROPERTIES(${_SRC_DIR}/pars/pars0grm.c
  207. PROPERTIES COMPILE_FLAGS "/wd4102")
  208. SET_SOURCE_FILES_PROPERTIES(${_SRC_DIR}/pars/lexyy.c
  209. PROPERTIES COMPILE_FLAGS "/wd4003")
  210. ENDIF()
  211. # Include directories under innobase
  212. INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/storage/innobase/include
  213. ${CMAKE_SOURCE_DIR}/storage/innobase/handler
  214. ${CMAKE_SOURCE_DIR}/libbinlogevents/include )