mariadb

Tree: 21430e4378

Author	SHA1	Message	Date
Sergei Golubchik	21430e4378	encryption keys service	11 years ago
Sergei Golubchik	cf8bf0b68e	encryption key management plugin api	11 years ago
Jan Lindström	9cdf494197	Fixed XtraDB implementation of encryption and page encryption.	11 years ago
Sergei Golubchik	d7d589dc01	Further fixes (copied from `d7d589dc`)	11 years ago
Michael Widenius	1e3e19a4dd	Fixes for encryption from eperi	11 years ago
Monty	fb7a504ee1	Pushed files missing in last commit	11 years ago
Jan Lindström	f12dbc1611	Fix for tablespace import/export failure.	11 years ago
Jan Lindström	da41a9683c	Merge Google's Scrubing feature: commit `c01fc2134d` Author: Jonas Oreland <jonaso@google.com> Date: Thu Apr 10 15:04:25 2014 +0000 innodb datafile scrubbing this patch implements scrubbing of innodb datafiles either by scrubbing as part of purge (non compressed) or scrubbing by scanning whole tablespaces (added into key rotation threads). the background scrubbing is further divided into btr_optimistic_scrub (which calls btr_page_reorganize) or if that fails btr_pessimistic_scrub (which calls page split routines). Change-Id: I55915e9274ab3b0c5ac90b42d7304386124b0b0f commit `aad1e79709` Author: Minli Zhu <minliz@google.com> Date: Mon Feb 10 15:42:26 2014 -0800 Innodb redo log scrubbing. Change-Id: I0d4a55b0021dde7129f6ff701c7b75c4186f9e9a	11 years ago
Jan Lindström	c5ae630582	Merge Google encryption commit `195158e988` Author: Minli Zhu <minliz@google.com> Date: Mon Nov 25 11:05:55 2013 -0800 Innodb redo log encryption/decryption. Use start lsn of a log block as part of AES CTR counter. Record key version with each checkpoint. Internally key version 0 means no encryption. Tests done (see test_innodb_log_encryption.sh for detail): - Verify flag innodb_encrypt_log on or off, combined with various key versions passed through CLI, and dynamically set after startup, will not corrupt database. This includes tests from being unencrypted to encrypted, and encrypted to unencrypted. - Verify start-up with no redo logs succeeds. - Verify fresh start-up succeeds. Change-Id: I4ce4c2afdf3076be2fce90ebbc2a7ce01184b612 commit `c1b9727365` Author: Jonas Oreland <jonaso@google.com> Date: Tue Dec 3 18:47:27 2013 +0100 encryption of aria data&index files this patch implements encryption of aria data & index files. this is implemented as 1) add read/write hooks (renamed from callbacks) that does encrypt/decrypt (also add pre_read and post_write hooks) 2) modify page headers for data/index to contain key version (making the data-page header size different for with/without encryption) 3) modify index page 0 to contain IV (and crypt header) 4) AES CRT crypt functions 5) counter block is implemented using combination of page no, lsn and table specific id NOTE: 1) log files are not encrypted, this is not needed for if aria is only used for internal temporary tables and they are not transactional (i.e not logged) 2) all encrypted tables are using PAGE_CHECKSUM (crc) normal internal temporary tables are (currently) not CHECKSUM:ed 3) This patch adds insert-order semantics to aria block_format. The default behaviour of aria block-format is best-fit, meaning that rows gets allocated to page trying to fill the pages as much as possible. However, certain sql constructs materialize temporary result in tmp-tables, and expect that a table scan will later return the rows in the same order they were inserted. This implementation of insert-order is only enabled when explicitly requested by sql-layer. CHANGES: 1) found bug in ma_write that made code try to abort a record that was never written unsure why this is not exposed Change-Id: Ia82bbaa92e2c0629c08693c5add2f56b815c0509 commit `89dc1ab651` Author: Jonas Oreland <jonaso@google.com> Date: Mon Feb 17 08:04:50 2014 -0800 Implement encryption of innodb datafiles. Pages are encrypted before written to disk and decrypted when read from disk. Each page except first page (page 0) in tablespace is encrypted. Page 0 is unencrypted and contains IV for the tablespace. FIL_PAGE_FILE_FLUSH_LSN on each page (except page 0) is used to store a 32-bit key-version, so that multiple keys can be active in a tablespace simultaneous. The other 32-bit of the FIL_PAGE_FILE_FLUSH_LSN field contains a checksum that is computed after encryption. This checksum is used by innochecksum and when restoring from double-write-buffer. The encryption is performed using AES CRT. Monitoring of encryption is enabled using new IS-table INNODB_TABLESPACES_ENCRYPTION. In addition to that new status variables innodb_encryption_rotation_{ pages_read_from_cache, pages_read_from_disk, pages_modified,pages_flushed } has been added. The following tunables are introduces - innodb_encrypt_tables - innodb_encryption_threads - innodb_encryption_rotate_key_age - innodb_encryption_rotation_iops Change-Id: I8f651795a30b52e71b16d6bc9cb7559be349d0b2 commit `a17eef2f69` Author: Andrew Ford <andrewford@google.com> Date: Thu Jan 2 15:43:09 2014 -0800 Key management skeleton with debug hooks. Change-Id: Ifd6aa3743d7ea291c70083f433a059c439aed866 commit `68a399838a` Author: Andrew Ford <andrewford@google.com> Date: Mon Oct 28 16:27:44 2013 -0700 Add AES-128 CTR and GCM encryption classes. Change-Id: I116305eced2a233db15306bc2ef5b9d398d1a3a2	11 years ago
Jan Lindström	0d9a43d4c4	Suspect fix for buildbot failure on page compressed and page encrypted tables.	11 years ago
Jan Lindström	df09bd359c	MDEV-6992: Code review of changes in page encryption	11 years ago
Sergei Golubchik	f2be9c0fea	after merge. innodb/xtradb to work on Windows	11 years ago
Jan Lindström	0fe39b6856	MDEV-7262: innodb.innodb-mdev7046 and innodb-page_compression* fail on BuildBot If persistent trim is not used some OS require that we write full page.	11 years ago
Jan Lindström	24a6b41348	Move page initialization to better place.	11 years ago
Jan Lindström	01590005ba	Fix buildbot valgrind errors on test innodb.innodb-page_compression_tables Problem was that temporal buffers allocated for page compression are not initialized and rest of the page that is actually writen was als not initialized after previous usage.	11 years ago
Jan Lindström	1a05bb4010	MDEV-7166: innodb.innodb-page_compression_zip fails in buildbot Analysis: If innodb_use_trim is not enabled or system does not support fallocate to make persistent trim, we should always write full page not only partial pages.	11 years ago
Jan Lindström	251fa7ffc5	Fix error on trim operation alligment. Furthermore, make sure that we do not return simulated out of file space on read operation, that would cause crash.	11 years ago
Jan Lindström	2d2d11f02b	MDEV-6968: CREATE TABLE crashes with InnoDB plugin Analysis: fil_extend_space_to_desired_size() does not provide file node to os_aio(). This failed on Windows only because on Windows we do not use posix_fallocate() to extend file space. Fix: Add file node to os_aio() function call and make sure that we do not use NULL pointer at os_aio_array_reserve_slot(). Additionally, make sure that we do not use 0 as file_block_size (512 is the minimum).	11 years ago
Jan Lindström	b96697d286	MDEV-6648: InnoDB: Add support for 4K sector size if supported New generation hard drives, SSDs and NVM devices support 4K sector size. Supported sector size can be found using fstatvfs() or GetDiskFreeSpace() functions.	11 years ago
Jan Lindström	cff0012d28	MDEV-6927: Share more structures Merge Facebook commit `f981a51a47` authored by Steaphan Greene from https://github.com/facebook/mysql-5.6. This just moves some structure definitions from inside a single .cc file to a shared .h file, with a few tweaks to allow these structures to be shared. On its own, it should have no actual effect. This is needed later.	11 years ago
Jan Lindström	c1f5f61c9b	Make sure that information schema knows also page compressed page type and that we decompress fist page if it is page compressed before really accessing it.	11 years ago
Monty	4af97bc0e3	Removed files that had been accidentally committed Removed compiler warnings	11 years ago
Jan Lindström	d3ceb934f1	MDEV-6701: InnoDB tests fail in 10.1 Fixed test failures seen on defragment tests, innodb.innodb-wl5522-debug-zip and innodb.innodb_bug12902967.	11 years ago
Jan Lindström	6907da2341	Fix small error on LZMA compression error printout.	11 years ago
Jan Lindström	df4dd593f2	MDEV-6247: Merge 10.0-galera to 10.1. Merged lp:maria/maria-10.0-galera up to revision 3879. Added a new functions to handler API to forcefully abort_transaction, producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These were added for future possiblity to add more storage engines that could use galera replication.	11 years ago
Jan Lindström	d87ffeb491	MDEV-6548: Incorrect compression on LZMA. Analysis: Provided incorrect parameter to output buffer size and incorrectly determined actual payload size after compression.	11 years ago
Jan Lindström	5ebb396638	MDEV-6247: Merge 10.0-galera to 10.1. Merged lp:maria/maria-10.0-galera up to revision 3867.	11 years ago
Jan Lindström	e974b56438	MDEV-6512: InnoDB: Assertion failure in thread 4537024512 in file buf0buf.cc line 2642. Analysis: innodb_compression_algorithm is a global variable and can change while we are building page compressed page. This could lead page corruption. Fix: Cache innodb_compression_algorithm on local variable before doing any compression or page formating to avoid concurrent change. Improved page verification on debug builds.	11 years ago
Jan Lindström	56c4b016ad	Fiix random test failures on fil_decompress_page_2 function. Analysis: InnoDB writes also files that do not contain FIL-header. This could lead incorrect analysis on os_fil_read_func function when it tries to see is page page compressed based on FIL_PAGE_TYPE field on FIL-header. With bad luck uncompressed page that does not contain FIL-headed, the byte on FIL_PAGE_TYPE position could indicate that page is page comrpessed. Fix: Upper layer must indicate is file space page compressed or not. If this is not yet known, we need to read the FIL-header and find it out. Files that we know that are not page compressed we can always just provide FALSE.	11 years ago
Sergei Petrunia	fa0628a6ac	MDEV-6456: Add progress indication for "Reading tablespace information from the .ibd files" # of processed files is printed every 15 sec.	11 years ago
Jan Lindström	dbc79ce055	MDEV-6354: Implement a way to read MySQL 5.7.4-labs-tplc page compression format (Fusion-IO). Addeed LZMA and BZIP2 compression methods.	11 years ago
Jan Lindström	6bd2f900b2	MDEV-6288: Innodb causes server crash after disk full, then can't ALTER TABLE any more.	11 years ago
Jan Lindström	3bca01930a	MDEV-6361: innodb_compression_algorithm configuration variable can be set to unsupported value. MDEV-6350: Excessive unnecessary memory allocation at InnoDB/XtraDB startup if LZO is installed.	11 years ago
Sergei Golubchik	1e702f3d20	don't include <linux/falloc.h> when it's not really needed	12 years ago
Sergei Golubchik	4e68fafad4	compilation failure on Win64 cannot use &ulint_var (where ulint == unsigned __int64) where (ulong *) is expected (in uncompress() and in "%lu")	12 years ago
Sergei Golubchik	7e7e1bf8aa	don't include the file that 1) not present everywhere 2) not used anyway	12 years ago
Jan Lindström	6d460760eb	Fix compiler warnings.	12 years ago
Jan Lindström	105060e307	Fix compiler warnings.	12 years ago
Jan Lindström	9d399c9f35	MDEV-6075: Allow > 16K pages on InnoDB This patch allows up to 64K pages for tables with DYNAMIC, COMPACT and REDUNDANT row types. Tables with COMPRESSED row type allows still only <= 16K page size. Note that single row size must be still <= 16K and max key length is not affected.	12 years ago
Jan Lindström	d6afa8004e	Fixed small error on compression failure error text.	12 years ago
Jan Lindström	2d340f9a67	Fixed bug on free buffer space calculation when LZO is used. Fixed bug on function call when InnoDB plugin is used.	12 years ago
Jan Lindström	2f46e5b9fc	MDEV-6070: FusionIO: Failure to create a table with ATOMIC_WRITES option leaves the database in inconsistent state, Analysis: Problem was that atomic writes variable had incorrect type on same places leading to fact that e.g. OFF option was not regognized. Furthermore, some error check code was missing from both InnoDB and XtraDB engines. Finally, when table is created we have already created the .ibd file and if we can't set atomic writes it stays there. Fix: Fix atomic writes variable type to ulint as it should be. Fix: Add proper error code checking on os errors on both InnoDB and XtraDB Fix: Remove the .idb file when atomic writes can't be enabled to a new table.	12 years ago
Jan Lindström	13c73c31c3	Added support for LZO compression method. Removed: innodb_use_lz4 configuration parameter Added: innodb_compression_algorithm configuration parameter 0 = no compression, 1 = ZLIB, 2 = LZ4, 3 = LZO Fixed issue with incorrect trim calculations	12 years ago
Jan Lindström	0b92fe9c18	Fixed windows compiler errors.	12 years ago
Jan Lindström	6a756b3a44	Code cleanup: Removed some unnecessary outputs from standard builds (available on special builds UNIV_PAGECOMPRESS_DEBUG and UNIV_MTFLUSH_DEBUG). Added a new status variable compress_pages_page_compression_error to count possible compression errors.	12 years ago
Jan Lindström	f1ca1f37c9	MDEV-5878: Failing assertion: mutex_own(mutex) with innodb_use_fallocate=ON. Analysis: This was merge error on file fil0fil.cc. fil_system mutex was taken twice because of this. Fix: Remove unnecessary mutex_enter and fixed the issue with slow posix_fallocate usage.	12 years ago
Jan Lindström	3ea72a2ba9	Removed options innodb_compress_index_pages and innodb_trim_pct. Both are unnecessary. There is a lot more index pages than there is normal pages. Earlier all pages were compressed and this provided best performance and compression ratio. Added status variable to show how many non index pages are written.	12 years ago
Jan Lindström	7322270a05	Set actual compressed page size also on read code path to buffer pool so that we can later use it to avoid unnecessary trim operations.	12 years ago
Jan Lindström	772aa0c575	MDEV-6160: InnoDB: Failing assertion: page_is_comp(next_page) == page_is_comp(page) This is not yet a fix. This is change to print additional information at the point when this assertion is going to happen. Print as much information about the pages and index to find out why next page is not a compact format.	12 years ago
Jan Lindström	752ba087c8	MDEV-5746: Slow file extend when innodb_use_fallocate=1 and SSD file storage. Analysis: posix_fallocate was called using 0 as offset and len as desired size. This is not optimal for SSDs. Fix: Call posix_fallocate with correct offset i.e. current file size and extend the file from there len bytes.	12 years ago

1 2 3

116 Commits (21430e43786edaf5b689c8b4aed49341bb7021c8)