mariadb

Tree: 3daef523af

Author	SHA1	Message	Date
Eugene Kosov	3daef523af	MDEV-17084 Optimize append only files for NVDIMM Optionally use libpmem for InnoDB redo log writing. When server is built -DWITH_PMEM=ON InnoDB tries to detect that redo log is located on persistent memory storage and uses faster file access method. When server is built with -DWITH_PMEM=OFF preprocessor is used to ensure that no slowdown will present due to allocations and virtual function calls. So, we don't slow down server in a common case. mapped_file_t: an map file, unmap file and returns mapped memory buffer file_io: abstraction around memory mapped files and file descriptors. Allows writing, reading and flushing to files. file_io::writes_are_durable(): notable method of a class. When it returns true writes are flushed immediately. file_os_io: file descriptor based file access. Depends on a global state like srv_read_only_mode file_pmem_io: file access via libpmem This is a collaboration work with Sergey Vojtovich	6 years ago
Marko Mäkelä	f8a9f90667	MDEV-12353: Remove support for crash-upgrade We tighten some assertions regarding dict_index_t::is_dummy and crash recovery, now that redo log processing will no longer create dummy objects.	6 years ago
Vladislav Vaintroub	98a67eccc5	MDEV-21327 : MDL wait notification for innodb background threadpool	6 years ago
Eugene Kosov	3d3d8f1012	MDEV-21337 fix aligned_malloc() do not fallback to malloc(), always return properly aligned buffer	6 years ago
Vladislav Vaintroub	5e62b6a5e0	MDEV-16264 Use threadpool for Innodb background work. Almost all threads have gone - the "ticking" threads, that sleep a while then do some work) (srv_monitor_thread, srv_error_monitor_thread, srv_master_thread) were replaced with timers. Some timers are periodic, e.g the "master" timer. - The btr_defragment_thread is also replaced by a timer , which reschedules it self when current defragment "item" needs throttling - the buf_resize_thread and buf_dump_threads are substitutes with tasks Ditto with page cleaner workers. - purge workers threads are not tasks as well, and purge cleaner coordinator is a combination of a task and timer. - All AIO is outsourced to tpool, Innodb just calls thread_pool::submit_io() and provides the callback. - The srv_slot_t was removed, and innodb_debug_sync used in purge is currently not working, and needs reimplementation.	6 years ago
Marko Mäkelä	f6ea0389a4	Replace ut_timer() with my_interval_timer() The function pointer ut_timer() was only used by the InnoDB defragmenting thread. Let InnoDB use a single monotonic high-precision timer, my_interval_timer() [in nanoseconds], occasionally wrapped by microsecond_interval_timer(). srv_defragment_interval: Change from "timer" units to nanoseconds. This concludes the InnoDB time function cleanup that was motivated by MDEV-14154. Only ut_time_ms() will remain for now, wrapping my_interval_timer().	6 years ago
Marko Mäkelä	e32212c63c	MDEV-19582 Out-of-bounds memory accesses by WolfSSL Fix errors caught by clang-7 in the encrypted variants of the tests innodb.innodb-table-online innodb.innodb-index-online	6 years ago
Vicențiu Ciorbaru	5543b75550	Update FSF Address * Update wrong zip-code	7 years ago
Marko Mäkelä	828cc2ba7c	MDEV-18417/MDEV-18656/MDEV-18417: Work around compiler ASAN bug In a Ubuntu Xenial build environment, the compiler identified as g++-5.real (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 seems to be emitting incorrect code for the compilation unit trx0rec.cc, triggering a bogus-looking AddressSanitizer report of an invalid read of something in the function trx_undo_rec_get_pars(). This is potentially affecting any larger tests where the InnoDB purge subsystem is being exercised. When the optimization level of trx0rec.cc is limited to -O1, no bogus failure is being reported. With -O2 or -O3, a lot of things seemed to be inlined in the function, and the disassembly of the generated code did not make sense to me.	7 years ago
Sergei Golubchik	3d2d060b62	fix gcc 8 compiler warnings There were two newly enabled warnings: 1. cast for a function pointers. Affected sql_analyse.h, mi_write.c and ma_write.cc, mf_iocache-t.cc, mysqlbinlog.cc, encryption.cc, etc 2. memcpy/memset of nontrivial structures. Fixed as: * the warning disabled for InnoDB * TABLE, TABLE_SHARE, and TABLE_LIST got a new method reset() which does the bzero(), which is safe for these classes, but any other bzero() will still cause a warning * Table_scope_and_contents_source_st uses `TABLE_LIST ` (trivial) instead of `SQL_I_List<TABLE_LIST>` (not trivial) so it's safe to bzero now. added casts in debug_sync.cc and sql_select.cc (for JOIN) * move assignment method for MDL_request instead of memcpy() * PARTIAL_INDEX_INTERSECT_INFO::init() instead of bzero() * remove constructor from READ_RECORD() to make it trivial * replace some memcpy() with c++ copy assignments	7 years ago
Vladislav Vaintroub	ac31ff6275	MDEV-15760 - don't build mariabackup with -DPLUGIN_INNOBASE=DYNAMIC	7 years ago
Marko Mäkelä	09af00cbde	MDEV-13564: Remove old crash-upgrade logic in 10.4 Stop supporting the additional trunc.log files that were introduced via MySQL 5.7 to MariaDB Server 10.2 and 10.3. DB_TABLESPACE_TRUNCATED: Remove. purge_sys.truncate: A new structure to track undo tablespace file truncation. srv_start(): Remove the call to buf_pool_invalidate(). It is no longer necessary, given that we no longer access things in ways that violate the ARIES protocol. This call was originally added for innodb_file_format, and it may later have been necessary for the proper function of the MySQL 5.7 TRUNCATE recovery, which we are now removing. trx_purge_cleanse_purge_queue(): Take the undo tablespace as a parameter. trx_purge_truncate_history(): Rewrite everything mostly in a single function, replacing references to undo::Truncate. recv_apply_hashed_log_recs(): If any redo log is to be applied, and if the log_sys.log.subformat indicates that separately logged truncate may have been used, refuse to proceed except if innodb_force_recovery is set. We will still refuse crash-upgrade if TRUNCATE TABLE was logged. Undo tablespace truncation would only be logged in undotrunc.log files, which we are no longer checking for.	7 years ago
Yuqi	ee5124d714	Make MariaDB CRC32-lib platform independence (#780 ) Make mariadb crc32 lib platform independent It looks strange that someone can make use of 2 crc libraries (Power64 or AArch64) at the same time. The patch sets macros 'CRC32_LIBRARY' to make platform independence as an optional crc32 library. Change-Id: I68bbf73cafb6a12f7fb105ad57d117b114a8c4af Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>	7 years ago
Ashutosh Parida	dda808e662	fix gcc row0trunc compilation error aarch64	8 years ago
Sergey Vojtovich	5ccf3f96ac	Fix misuse of MY_CHECK_CXX_COMPILER_FLAG - compile_flags already include from top CMakeLists.txt - MY_CHECK_CXX_COMPILER_FLAG() accepts only one parameter - output variable of MY_CHECK_CXX_COMPILER_FLAG() is have_CXX__Wa__nH - same check for mariabackup Based on contribution by satanson (PR#466).	8 years ago
Sergei Golubchik	88d1c1c551	MDEV-15288 Configure errors when building without INNOBASE	8 years ago
Sergey Vojtovich	83b471348d	MDEV-14318 - cmake updates to build on arm64 Added more files affected by GCC ICE: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67143	8 years ago
Marko Mäkelä	03400d974d	Dead code removal: sess_t The session object is not really needed for anything. We can directly create and free the dummy purge_sys->query->trx.	8 years ago
Marko Mäkelä	6f314edac7	MDEV-14648 Restore fix for MySQL BUG#39053 - UNINSTALL PLUGIN does not allow the storage engine to cleanup open connections Also, allow the MariaDB 10.2 server to link InnoDB dynamically against ha_innodb.so (which is what mysql-test-run.pl expects to exist, instead of the default name ha_innobase.so). wsrep_load_data_split(): Instead of referring to innodb_hton_ptr, check the handlerton::db_type. This was recently broken by me in MDEV-11415. innodb_lock_schedule_algorithm: Define as a weak global symbol, so that WITH_WSREP will not depend on InnoDB being linked statically. I tested this manually. Notably, running a test that only does SET GLOBAL wsrep_on=1; with a static or dynamic InnoDB and ./mtr --mysqld=--loose-innodb-lock-schedule-algorithm=fcfs will crash with SIGSEGV at shutdown. With the default VATS combination the wsrep_on is properly refused for both the static and dynamic InnoDB. ha_close_connection(): Do invoke the method also for plugins for which UNINSTALL PLUGIN was deferred due to open connections. Thanks to @svoj for pointing this out. thd_to_trx(): Return a pointer, not a reference to a pointer. check_trx_exists(): Invoke thd_set_ha_data() for assigning a transaction. log_write_checkpoint_info(): Remove an unused DEBUG_SYNC point that would cause an assertion failure on shutdown after deferred UNINSTALL PLUGIN. This was tested as follows: cmake -DWITH_WSREP=1 -DPLUGIN_INNOBASE:STRING=DYNAMIC \ -DWITH_MARIABACKUP:BOOL=OFF ... make cd mysql-test ./mtr innodb.innodb_uninstall	8 years ago
Marko Mäkelä	27ea2963fc	Dead code removal: sess_t The session object is not really needed for anything. We can directly create and free the dummy purge_sys->query->trx.	8 years ago
Marko Mäkelä	5fe9b4a7ae	MDEV-14648 Restore fix for MySQL BUG#39053 - UNINSTALL PLUGIN does not allow the storage engine to cleanup open connections Also, allow the MariaDB 10.2 server to link InnoDB dynamically against ha_innodb.so (which is what mysql-test-run.pl expects to exist, instead of the default name ha_innobase.so). wsrep_load_data_split(): Instead of referring to innodb_hton_ptr, check the handlerton::db_type. This was recently broken by me in MDEV-11415. innodb_lock_schedule_algorithm: Define as a weak global symbol, so that WITH_WSREP will not depend on InnoDB being linked statically. I tested this manually. Notably, running a test that only does SET GLOBAL wsrep_on=1; with a static or dynamic InnoDB and ./mtr --mysqld=--loose-innodb-lock-schedule-algorithm=fcfs will crash with SIGSEGV at shutdown. With the default VATS combination the wsrep_on is properly refused for both the static and dynamic InnoDB. ha_close_connection(): Do invoke the method also for plugins for which UNINSTALL PLUGIN was deferred due to open connections. Thanks to @svoj for pointing this out. thd_to_trx(): Return a pointer, not a reference to a pointer. check_trx_exists(): Invoke thd_set_ha_data() for assigning a transaction. log_write_checkpoint_info(): Remove an unused DEBUG_SYNC point that would cause an assertion failure on shutdown after deferred UNINSTALL PLUGIN. This was tested as follows: cmake -DWITH_WSREP=1 -DPLUGIN_INNOBASE:STRING=DYNAMIC \ -DWITH_MARIABACKUP:BOOL=OFF ... make cd mysql-test ./mtr innodb.innodb_uninstall	8 years ago
Vladislav Vaintroub	6c279ad6a7	MDEV-15091 : Windows, 64bit: reenable and fix warning C4267 (conversion from 'size_t' to 'type', possible loss of data) Handle string length as size_t, consistently (almost always:)) Change function prototypes to accept size_t, where in the past ulong or uint were used. change local/member variables to size_t when appropriate. This fix excludes rocksdb, spider,spider, sphinx and connect for now.	8 years ago
Vicențiu Ciorbaru	3dfe148074	5.6.39	8 years ago
wlad	b6d72ed44d	MDEV-14283 : Fix Solaris 10 build. - introduce system check for posix_memalign (not available on Solaris 10) - Disable dtrace probes, to fix weird link errors in mariabackup	8 years ago
Aleksey Midenkov	33085349e9	IB, SQL: removed VTQ, added TRT on SQL layer [closes #305 ]	8 years ago
Vladislav Vaintroub	7354dc6773	MDEV-13384 - misc Windows warnings fixed	8 years ago
Jan Lindström	016c35a7f2	MDEV-13690: Remove unnecessary innodb_use_mtflush, innodb_mtflush_threads parameters and related code Users can use innodb-page-cleaners instead.	8 years ago
Vladislav Vaintroub	49f3fb8feb	small refactoring o innobase/CMakeLists.txt allow to build with -DPLUGIN_INNOBASE=NO	8 years ago
Sergei Golubchik	58b56f14a0	cleanup: remove ha_innopart.cc we'll copy it from 5.7 when we'll need it	8 years ago
Sergei Golubchik	7581fb23e2	compilation fix for SLES 11 SP4 also fix innodb	8 years ago
Marko Mäkelä	8c71c6aa8b	MDEV-12548 Initial implementation of Mariabackup for MariaDB 10.2 InnoDB I/O and buffer pool interfaces and the redo log format have been changed between MariaDB 10.1 and 10.2, and the backup code has to be adjusted accordingly. The code has been simplified, and many memory leaks have been fixed. Instead of the file name xtrabackup_logfile, the file name ib_logfile0 is being used for the copy of the redo log. Unnecessary InnoDB startup and shutdown and some unnecessary threads have been removed. Some help was provided by Vladislav Vaintroub. Parameters have been cleaned up and aligned with those of MariaDB 10.2. The --dbug option has been added, so that in debug builds, --dbug=d,ib_log can be specified to enable diagnostic messages for processing redo log entries. By default, innodb_doublewrite=OFF, so that --prepare works faster. If more crash-safety for --prepare is needed, double buffering can be enabled. The parameter innodb_log_checksums=OFF can be used to ignore redo log checksums in --backup. Some messages have been cleaned up. Unless --export is specified, Mariabackup will not deal with undo log. The InnoDB mini-transaction redo log is not only about user-level transactions; it is actually about mini-transactions. To avoid confusion, call it the redo log, not transaction log. We disable any undo log processing in --prepare. Because MariaDB 10.2 supports indexed virtual columns, the undo log processing would need to be able to evaluate virtual column expressions. To reduce the amount of code dependencies, we will not process any undo log in prepare. This means that the --export option must be disabled for now. This also means that the following options are redundant and have been removed: xtrabackup --apply-log-only innobackupex --redo-only In addition to disabling any undo log processing, we will disable any further changes to data pages during --prepare, including the change buffer merge. This means that restoring incremental backups should reliably work even when change buffering is being used on the server. Because of this, preparing a backup will not generate any further redo log, and the redo log file can be safely deleted. (If the --export option is enabled in the future, it must generate redo log when processing undo logs and buffered changes.) In --prepare, we cannot easily know if a partial backup was used, especially when restoring a series of incremental backups. So, we simply warn about any missing files, and ignore the redo log for them. FIXME: Enable the --export option. FIXME: Improve the handling of the MLOG_INDEX_LOAD record, and write a test that initiates a backup while an ALGORITHM=INPLACE operation is creating indexes or rebuilding a table. An error should be detected when preparing the backup. FIXME: In --incremental --prepare, xtrabackup_apply_delta() should ensure that if FSP_SIZE is modified, the file size will be adjusted accordingly.	8 years ago
Aleksey Midenkov	afe64a450e	IB: moved VTQ funcs to separate file	9 years ago
Aleksey Midenkov	fc7da4dd4f	IB, SQL: InnoDB partitioning [closes #118 ] * native InnoDB partitioning for BY SYSTEM_TIME partitions.	9 years ago
Sergei Golubchik	6935d66053	MDEV-12591 MariaDB embedded server refuses start with innodb_plugin innodb calls sd_notify(), it must be linked with libsystemd	9 years ago
Sergei Golubchik	3a3b3d8ba8	cleanup: innodb files in cmake/ moved to storage/innobase/ also removed duplicate checks from storage/innobase/CMakeLists.txt (they're all in storage/innobase/innodb.cmake)	9 years ago
Marko Mäkelä	4e1116b2c6	MDEV-12271 Port MySQL 8.0 Bug#23150562 REMOVE UNIV_MUST_NOT_INLINE AND UNIV_NONINL Also, remove empty .ic files that were not removed by my MySQL commit. Problem: InnoDB used to support a compilation mode that allowed to choose whether the function definitions in .ic files are to be inlined or not. This stopped making sense when InnoDB moved to C++ in MySQL 5.6 (and ha_innodb.cc started to #include .ic files), and more so in MySQL 5.7 when inline methods and functions were introduced in .h files. Solution: Remove all references to UNIV_NONINL and UNIV_MUST_NOT_INLINE from all files, assuming that the symbols are never defined. Remove the files fut0fut.cc and ut0byte.cc which only mattered when UNIV_NONINL was defined.	10 years ago
Marko Mäkelä	a45866c6db	MDEV-12050 Remove unused InnoDB Memcached hooks Oracle introduced a Memcached plugin interface to the InnoDB storage engine in MySQL 5.6. That interface is essentially a fork of Memcached development snapshot 1.6.0-beta1 of an old development branch 'engine-pu'. To my knowledge, there have not been any updates to the Memcached code between MySQL 5.6 and 5.7; only bug fixes and extensions related to the Oracle modifications. The Memcached plugin is not part of the MariaDB Server. Therefore it does not make sense to include the InnoDB interfaces for the Memcached plugin, or to have any related configuration parameters: innodb_api_bk_commit_interval innodb_api_disable_rowlock innodb_api_enable_binlog innodb_api_enable_mdl innodb_api_trx_level Removing this code in one commit makes it possible to easily restore it, in case it turns out to be needed later.	9 years ago
Daniel Black	58ac8dd255	MDEV-11195: Simplify enablement of NUMA in innodb/xtradb Move common parts of {innodb,xtradb}/CmakeLists.txt to the macro.	9 years ago
Daniel Black	ab0e5031e4	MDEV-11195: Correct enablement of NUMA in innodb/xtradb No -DHAVE_LIBNUMA=1 was passed to the source compile (and the global include/my_config.h wasn't used). This also is Linux only so corrected the cmake macro. Fixed indenting in cmake macro. Removed NUMA defination from include/my_config.h as its only in the storage engine. Thanks Elena Stepanova and Vladislav Vaintroub for the detailed list of bugs/questions. Signed-off-by: Daniel Black <daniel.black@au.ibm.com>	9 years ago
Daniel Black	8103f6f223	MDEV-10829: Enable Innodb NUMA interleave Code was already existing within the innobase/xtradb storage engines however without this cmake code it was never enabled. num.cmake heavily based off work by Annamalai Gurusami <annamalai.gurusami@oracle.com> Signed-off-by: Daniel Black <daniel.black@au.ibm.com>	9 years ago
Jan Lindström	fec844aca8	Merge InnoDB 5.7 from mysql-5.7.14. Contains also: MDEV-10549 mysqld: sql/handler.cc:2692: int handler::ha_index_first(uchar*): Assertion `table_share->tmp_table != NO_TMP_TABLE \|\| m_lock_type != 2' failed. (branch bb-10.2-jan) Unlike MySQL, InnoDB still uses THR_LOCK in MariaDB MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) enable tests that were fixed in MDEV-10549 MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) fix main.innodb_mysql_sync - re-enable online alter for partitioned innodb tables	9 years ago
Jan Lindström	2e814d4702	Merge InnoDB 5.7 from mysql-5.7.9. Contains also MDEV-10547: Test multi_update_innodb fails with InnoDB 5.7 The failure happened because 5.7 has changed the signature of the bool handler::primary_key_is_clustered() const virtual function ("const" was added). InnoDB was using the old signature which caused the function not to be used. MDEV-10550: Parallel replication lock waits/deadlock handling does not work with InnoDB 5.7 Fixed mutexing problem on lock_trx_handle_wait. Note that rpl_parallel and rpl_optimistic_parallel tests still fail. MDEV-10156 : Group commit tests fail on 10.2 InnoDB (branch bb-10.2-jan) Reason: incorrect merge MDEV-10550: Parallel replication can't sync with master in InnoDB 5.7 (branch bb-10.2-jan) Reason: incorrect merge	9 years ago
Sergey Vojtovich	9c2215e022	MDEV-9872 - Add common optimized CRC32 function interface Move crc32-vpmsum to extra. Compile static crc32-vpmsum instead of adding sources directly. Make use of crc32-vpmsum via my_checksum(). Based on contribution by Daniel Black.	9 years ago
sjaakola	1ce821b509	Refs codership/mysql-wsrep#221 - disabling certain IB atomic builtins, which caused complete hangs	10 years ago
Daniel Black	e4e2d9dd3c	Do not build optimised power crc32 on bigendian	10 years ago
Daniel Axtens	2538c7cf89	Use POWER8 accelerated crc32 - Make accelerated checksum available to InnoDB and XtraDB. - Fall back to slice-by-eight if not available. The mode used is printed on startup. - Will only build on POWER systems at the moment until CMakeLists are modified to only add the crc32_power8/ files when building on POWER. running MySQL-5.7 unittest/gunit/innodb/ut0crc32-t Before: 1..2 Using software crc32 implementation, CPU is little-endian ok 1 Using software crc32 implementation, CPU is little-endian normal CRC32: real 0.148006 sec normal CRC32: user 0.148000 sec normal CRC32: sys 0.000000 sec big endian CRC32: real 0.144293 sec big endian CRC32: user 0.144000 sec big endian CRC32: sys 0.000000 sec ok 2 After: 1..2 Using POWER8 crc32 implementation, CPU is little-endian ok 1 Using POWER8 crc32 implementation, CPU is little-endian normal CRC32: real 0.008097 sec normal CRC32: user 0.008000 sec normal CRC32: sys 0.000000 sec big endian CRC32: real 0.147043 sec big endian CRC32: user 0.144000 sec big endian CRC32: sys 0.000000 sec ok 2 Author CRC32 ASM code: Anton Blanchard <anton@au.ibm.com> ref: https://github.com/antonblanchard/crc32-vpmsum Signed-off-by: Daniel Black <daniel.black@au.ibm.com>	10 years ago
Sergei Golubchik	65d69c84cf	cmake: remove unused checks, options, and symbols	10 years ago
Sergei Golubchik	84da1547e6	MDEV-8883 more cross-compiling fixes use CHECK_C_SOURCE_COMPILES for atomic tests if cross-compiling (continue to use CHECK_C_SOURCE_RUNS otherwise)	10 years ago
Sergei Golubchik	86ff4da14d	5.6.27	10 years ago
Shaohua Wang	f59d68eeae	BUG#21102971 data corruption on arm64 The root cause is that x86 has a stronger memory model than the ARM processors. And the GCC builtins didn't issue the correct fences when setting/unsetting the lock word. In particular during the mutex release. The solution is rewriting atomic TAS operations: replace '__sync_' by '__atomic_' if possible. Reviewed-by: Sunny Bains <sunny.bains@oracle.com> Reviewed-by: Bin Su <bin.x.su@oracle.com> Reviewed-by: Debarun Banerjee <debarun.banerjee@oracle.com> Reviewed-by: Krunal Bauskar <krunal.bauskar@oracle.com> RB: 9782 RB: 9665 RB: 9783	10 years ago

1 2 3 4 5

239 Commits (3daef523af25e4f1e4e75d2c26a9b25475f0c679)