mariadb

Tree: 1c4aed7c68

Author	SHA1	Message	Date
Otto Kekalainen	d1ba623677	All-green GitLab CI in main branch in January 2025 Ensure that the GitLab CI is green again. Ideally all failures would be fixed, but at the moment it is not feasible. As the purpose of a CI is to protect the code base from having new testable regressions slip in, update the CI pipeline to be all green by disabling multiple failing test or reduce their scope. - Avoid timeouts and make buid faster with '-DCPACK_RPM_DEBUGINFO_PACKAGE=OFF' - Remove 'GIT_CLONE_PATH' which isn't universally supported on all GitLab instances - Bump 'MARIADB_MAJOR_VERSION' to current 'main' contents (11.8). Also update the upgrade test matrix to modern MariaDB versions. - Remove UBsan builds as they are failing and can't quickly be fixed, along with the MTR test that depended on it. - Switch the CentOS 7 build to use Amazon Linux 2 instead, which has equivalent Fedora package versions, but is still maintained and package archive mirrors active. - Extend skiplist to include currently failing MTR tests. These are not quick to fix, so adding them to the skiplist will at least make the CI green and allow us to catch any new regressions. - Drop the default QPS limit in the mini-benchmark.sh to match what is currently passing. The drop is large because it wasn't adjusted after `4016c905` got merged. - Remove the currently failing faketime tests for 2028 compatibility. - Remove unnecessary `dependencies:` definitions that have no effect in the presence of `needs:` - Reduce the scope of `cppcheck` to avoid timeouts from job. - Update the ignorelists for both `cppcheck` and `flawfinder` as the jobs were failing. This way the jobs will be green again, and reveal is any new commits introduce issues. - Drop the duplicate `cppcheck` and `flawfinder` job definitions that got erroneously merged when both `9c287c0` and `f4ce1e4` got merged on the `main` branch, when the original intent was to have the latter commit only on old MariaDB versions. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	10 months ago
Robin Newhouse	008bddaa6c	GitLab CI Upgrade CentOS 8 to CentOS 9 build > After May 31, 2024, CentOS Stream 8 will be archived and no further updates will be provided. [1] CentOS Stream 8 is now EOL and should be updated to using CentOS Stream 9 for compatibility testing in GitLab CI. [1] https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/ https://www.centos.org/centos-linux-eol/ All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services.	1 year ago
Anson Chung	df35072c8d	Refactor GitLab cppcheck and update SAST ignorelists Line numbers had to be removed from the ignorelists in order to be diffed against since locations of the same findings can differ across runs. Therefore preprocessing has to be done on the CI findings so that it can be compared to the ignorelist and new findings can be outputted. However, since line numbers have to be removed, a situation occurs where it is difficult to reference the location of findings in code given the output of the CI job. To lessen this pain, change the cppcheck template to include code snippets which make it easier to reference where in the code the finding is referring to, even in the absence of line numbers. Ignorelisting works as before since locations of the finding may change but not the code it is referring to. Furthermore, due to the innate difficulty in maintaining ignorelists across branches and triaging new findings, allow failure as to not have constantly failing pipelines as a result of a new findings that have not been addressed yet. Lastly, update SAST ignorelists to match the newly refactored cppcheck job and the current state of the codebase. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	1 year ago
Anson Chung	eb0c719947	Add GitLab MTR runs to ensure support past 2038 A new GitLab MTR run is added which uses libfaketime to simulate starting the server and running the test suite in the future. The job simulates running in two times in the year 2038, once before the signed 32 bit UNIX limit and once after. This change helps to ensure that all future changes preserve the ability for the engine start and function past the year 2038. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	1 year ago
Robin Newhouse	6efa75a8cb	Enable mini-benchmark to run with perf The mini-benchmark.sh script failed to run in the latest Fedora distributions in GitLab CI. Executing the benchmark inside a Docker container had failed because the check for `perf` was done in a way that caused the benchmark to exit because of the `set -e` option. Test and skip `perf` to allowing the remaining benchmark activities to proceed. This check was added in `acb6684` but inadvertantly reverted in `42a1f94`. Logic was corrected to only run perf when the flag is enabled, and to prevent perf stat and perf record from being simultaneously enabled. Set -ex is also added to enable easier identification of mini-benchmark issues in the future. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	2 years ago
Robin Newhouse	a63c558b20	Implement mysql_upgrade upgrade testing in CI Performs an upgrade of mariadb from an earlier version to the rpms built in CI. Then checks whether log contains evidence of upgrade in the form of "Needs upgrade" or "Table rebuild required". Designed to check minor version upgrades which should not trigger rebuilds. The test is written in bash script so it can be executed from other CI systems. $ test_upgrade.sh source_version target_version $ test_upgrade.sh source_version # defaults to rpm/ $ test_upgrade.sh source_version --rpm-dir <directory> Binaries must be created with performance schema enabled, or mysql_upgrade complains about missing tables. Upgrade testing is parallelized with a "matrix" of source versions. Others can be introduced later. This was partially designed to catch issues like that seen in https://jira.mariadb.org/browse/MDEV-28727 where a minor version upgrade (e.g. 10.4.8 -> 10.4.26) triggered a system table rebuild. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Robin Newhouse	03d4fd3214	Backport GitLab CI to 10.5 Add .gitlab-ci.yml file to earliest supported branch to enable automated building and testing for all MariaDB major branches. Note to mergers: GitLab CI is available for branches >= 10.6. This commit includes a GitLab CI file identical to that in branches >= 10.6, except for the MARIADB_MAJOR_VERSION variable which should reflect the branch version. A modified CI will be included in branches 10.4 with PR !2418. Also changed is the `allow_failure: true` for the MSAN build, which should be merged up to later branches. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Robin Newhouse	f4ce1e487e	All-green GitLab CI in 10.4 branch Note to mergers: Do not merge this commit to 10.5+. An additional PR will be created for the 10.5 branch which is compatible with later branches. Include cppcheck and FlawFinder for SAST scanning. From 10.6, cherry-picked `12bf5c46` (Remove unused French translations in Connect engine) and `c6072ed9` (Ensure that source files contain only valid UTF8 encodings). Necessary for FlawFinder to execute and useful anyway. Removing MSAN build and test as it was not introduced until 10.5 and does not successfully build. Remove failing upgrade test since Fedora installs MariaDB 10.5 and the 10.5->10.4 upgrade rightfully complains Add to skiplist failing test: main.func_math (MDEV-20966) All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Daniel Black	320a4b52c9	MDEV-30205 Move /usr/share/mysql to /usr/share/mariadb These are mainly internal files so is a low impact change. The few scripts/mysqlsql where renames to mariadb_ prefix on the name. mysql-test renamed to mariadb-test in the final packages	3 years ago
Robin Newhouse	9c287c0a90	All-green GitLab CI in 11.0 branch Include cppcheck and FlawFinder for SAST scanning. Ignorelists are present for both, so only new problems will trigger a CI failure. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Robin Newhouse	20d2c9038a	Fix mini-benchmark The mini-benchmark.sh script failed to run in the latest Fedora distributions in GitLab CI. It requires `lscpu` resolved by installing util-linux. Additionally, executing the benchmark inside a Docker container had failed because of increased Docker security in recent updates. In particular the `renice` and `taskset` operations are not permitted. Neither are the required `perf` operations. https://docs.docker.com/engine/security/seccomp/ Allow these operations to fail gracefully, and test then skip `perf`, allowing the remaining benchmark activities to proceed. Other minor changes to the CI are included such as allowing sanitizer jobs to fail and using "needs" in the mini-benchmark pipeline. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Robin Newhouse	60f96b58e4	Backport GitLab CI to earlier branches Add .gitlab-ci.yml file to earliest supported branch to enable automated building and testing for all MariaDB major branches. For 10.4 we use the bundled SSL to build MariaDB when the platform does not have OpenSSL 1.1 available. This requires the installation of gnutls-devel as a dependency of MariaDB Connector/C. OpenSSL 3.0 support was backported to MariaDB 10.5 (see https://github.com/MariaDB/server/pull/2036, `f0fa40ef`, `8a9c1e9c`) All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Robin Newhouse	76c2402812	Fix build failure in sanitizer on GitLab-CI Sanitizer tests were introduced in `617f45b` for GitLab CI, but started failing on latest Fedora version with error: $ yum install -y /usr/lib64/libasan.so.6.0.0 /usr/lib64/libtsan.so.0.0.0 /usr/lib64/libubsan.so.1.0.0 Last metadata expiration check: 0:00:51 ago on Fri Dec 9 20:05:01 2022. No match for argument: /usr/lib64/libasan.so.6.0.0 No match for argument: /usr/lib64/libtsan.so.0.0.0 Error: Unable to find a match: /usr/lib64/libasan.so.6.0.0 /usr/lib64/libtsan.so.0.0.0 The reason for using specific library versions is unknown. Switch to simply using latest package versions, as is works and is likely to work best in the long run. Also, enclose "../rpmlist-$CI_JOB_NAME-$CI_COMMIT_REF_SLUG.log" in quotes to avoid `ambiguous redirect` error when $CI_JOB_NAME has spaces. Additionally use "needs" statements to allow tests to run immediately after dependent jobs passed instead of waiting for the full stage to complete. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services	3 years ago
Otto Kekäläinen	95d71272ef	Gitlab-CI: Upgrade Fedora build always use latest (now 37) version The version was fixed to be Fedora 36 due to previous issues on Gitlab-CI, but those seem to be solved now. Use 'mariadb' name in scripts and server binary as Fedora switched name in `df76620f9e` Switch to using the `default:` section supported by newer Gitlab-CI, see https://docs.gitlab.com/ee/ci/yaml/#default. Also define an explicit timeout of 3 hours to ensure builds don't time out if the default timeout is too short. NOTE TO MERGERS: These changes are version independent and should be merged up on all MariaDB branches 10.6 -> 10.11.	3 years ago
anson1014	c1de3776a6	Extend GitLab CI with build and test jobs for sanitizers (#2174 ) Add a build and test job for each of ASAN, MSAN, TSAN, and UBSAN to the GitLab pipeline such that current vulnerabilities will be more easily visible and on each new commit, we can ensure that there are no additional errors introduced. Furthermore, sanitizer test runs are run separate from the existing mysql-test-run to isolate sanitizer error from functional errors. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Anson Chung	3a9cb4c1d7	Fix Ninja builds on Gitlab-CI by limiting parallelism In previous versions it was stated that MDEV-25968 was causing other jobs in the pipeline to fail if not run with "-j 2" but this bug was not affecting fedora-ninja. This is still true for the public gitlab runners. However, running the fedora-ninja job on custom runners with more processors without the "-j 2" flag will cause the compiler to crash. When running the build with 2,4,8,16,32 threads, build times were consistent indicating that the typical bottleneck is I/O and not CPU cores. Therefore, "-j 2" is not a big drawback and it was chosen in order to remain consistent with the other builds affected by MDEV-25968. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.	3 years ago
Otto Kekäläinen	c61249eef1	Fix failing Gitlab-CI by fixing CentOS 8 repo urls	4 years ago
Otto Kekäläinen	ffb7f8854a	Fix failing Gitlab-CI by adding pcre2-devel as a build dependency The commits `a73acf6c06` and `4d74bac8bc` updated the PCRE library to a new version, which in turn requires CMake 3.0. That does not exist in CentOS 7 nor 8, so builds started failing. Actually the build should not be downloading anything at all. The root cause was that pcre2-devel was missing from the dependencies. This was originally not detected, as the download fallback had masked the issue.	4 years ago
Otto Kekäläinen	c5c61b51b6	Extend the Gitlab-CI pipeline to run mini benchmark Implement new mini-benchmark script for simple CPU bound benchmark for the duration of 5 minutes. The script can be run stand-alone or as part of a CI pipeline. Extend Gitlab-CI to run mini-benchmark on every commit to catch if there are severe performance regressions. Also bump MARIADB_MAJOR_VERSION to 10.8 which is needed on the 10.8 branch.	4 years ago
Otto Kekäläinen	260649de04	Misc improvements to the Gitlab-CI pipeline for MariaDB - Add new Ninja and Clang build jobs. This helps to ensure those toolchains also work in addition to default CMake/gcc. - Generate dependencies.dot/png to illustrate the CMake/Make/Ninja build dependencies. Viewing this image and identifying bottle necks in parallelism can help make the build run faster. - Enable CUnit tests now as they are fixed on 10.6 (MDEV-25820). - Limit parallel builds to 2 CPUs (full parallelism needs MDEV-25968) on CMake/Make. Now only the Ninja builds run full parallel builds as only Ninja is smart enough to prevent builds failing on resource over-consumption. - Enable Gitlab-CI cache for job 'centos8' for ccache so that it builds faster. Don't use Gitlab-CI cache for other jobs, as it would too easily use up all free tier storage on Gitlab.com and force users to get a paid account just for MariaDB builds. - On other jobs clean away ccache, as it only had a 5% hit rate on single builds with no downloaded cache. - Dump full database contents during the test install so that one can use diff to compare the database contents at different stages and thus track/debug potential bugs in mariadb-install-db and mariadb-upgrade code. Bugfixes: - Zero out ccache stats before each run so that 'ccache -s' would actually show the stats for the latest run.	4 years ago
Otto Kekäläinen	04369f9cee	Implement simple Gitlab-CI pipeline for MariaDB with RPM builds As Travis-CI has stopped offering free testing for open source projects, and they don't seem to have any plans to revert their new restrictions, MariaDB no longer has a good CI system outside contributors could run independently for basic validation before submitting Pull Requests. Implement a simple Gitlab-CI pipeline that runs basic RPM builds on one old, one less old and one very new distro release and then do some basic tests on the RPM packages to validate they installed and the server actually runs.	4 years ago

49 Commits (1c4aed7c680c0402d6e97e097f03815c0e9bf4c5)