MariaDB

MariaDB Upgrades & Change Buffer Errors

I upgraded MariaDB from 10.11 to 11.4 a few months ago and everything appeared to go smoothly until trouble brewed a few days later. After restarting MariaDB following a minor update, MariaDB bombed out with an error. After unsuccessfully troubleshooting for an hour, I bit the bullet and rolled back to a previous backup. What I didn't know at the time, was this was the beginning of my journey down the rabbit hole of what should've been a painless upgrade.

Knowing what I know now, the amount of people that will actually hit this error should likely be small. If you happen to be one of the unlucky ones like me, read on.

*NOTE: At the time of writing this article, MariaDB already released version 11.8 which is also long-term-support (LTS) release, but in this guide I've stuck with 11.4 which is the closest LTS release to 10.11

The Error

This is the error I received when restarting MariaDB after upgrading to 11.4. If you see this error your only options are to rollback to a previous backup or if you were a reckless admin, you'll have to bite the bullet and try downgrading back to 10.11.

[ERROR] InnoDB: The change buffer is corrupted

Why Me?

After a lot of research, I found that the issue actually started 2 upgrades ago when I migrated to MariaDB 10.6. This is when devs decided to disable the InnoDB change buffer. Prior to experiencing this issue, I hadn't even heard of the InnoDB change buffer, but apparently it has been the cause of a lot of database corruptions and MariaDB had set down the path to completely remove it from their future releases.

One major flaw in MariaDB's plan was that disabling the change buffer didn't actually clear out the buffer. It just merely stopped it from caching new changes. Little did I know, my change buffer still had data in it. When I then upgraded to MariaDB 11.4, I had unknowingly sealed my fate as the change buffer had been completely removed and there was no way for me to reconcile the changes left behind.

The Workaround

Assuming you're back to a working MariaDB 10.11 installation, this guide will show you how to upgrade to 11.4 without experiencing the dreaded change buffer errors.

Preparing for the Upgrade

Start by doing a full backup of all of your databases.

mariadb-dump -u username -p -v --routines --events --all-databases --single-transaction > alldbbackups.sql

Now shutdown MariaDB.

sudo systemctl stop mariadb

Go ahead and uninstall the current version of MariaDB.

sudo apt remove mariadb-server

Our last step before upgrading will be moving the /var/lib/mysql directory which contains all of your MariaDB files including your DB

sudo mv /var/lib/mysql /var/lib/mysql-old

We're now ready to upgrade to MariaDB 11.4.

Upgrading MariaDB

We'll start by adding the 11.4 repository by going to MariaDB Repo Download site.

You'll need to select an Ubuntu distribution, the MariaDB Server version (11.4) and a Mirror

MariaDB Repo Downloadd Site

After making a selection for all 3 options, you'll want to run the first 3 commands below that which will import the repository key

sudo apt-get install apt-transport-https curl
sudo mkdir -p /etc/apt/keyrings
sudo curl -o /etc/apt/keyrings/mariadb-keyring.pgp 'https://mariadb.org/mariadb_release_signing_key.pgp'

Next copy the repository info below those commands and save it to a new apt source file.

sudo nano /etc/apt/sources/mariadb.sources
# MariaDB 11.4 repository list
# https://mariadb.org/download/
X-Repolib-Name: MariaDB
Types: deb
# deb.mariadb.org is a dynamic mirror if your preferred mirror goes offline. See https://mariadb.org/mirrorbits/ for details.
# URIs: https://deb.mariadb.org/11.4/ubuntu
URIs: https://ftp.osuosl.org/pub/mariadb/repo/11.4/ubuntu
Suites: noble
Components: main main/debug
Signed-By: /etc/apt/keyrings/mariadb-keyring.pgp

Now run apt update and install MariaDB 11.4

sudo apt update
sudo apt install mariadb-server

Import your previous backup

sudo mariadb < alldbbackups.sql

*NOTE: If you run into errors importing your databases back into MariaDB, you may need to disable InnoDB strict mode. In your /etc/mysql/mariadb.cnf, add the following line innodb_strict_mode = OFF and then restart MariaDB (sudo systemctl restart mariadb)

Checking MariaDB

With all of your databases re-imported into MariaDB, we want to make sure we no longer see the error that previously plagued us by restarting MariaDB

sudo systemctl restart MariaDB

If no error shows, do one last check by showing the status of MariaDB

sudo systemctl status MariaDB

If everything went as expected, you should see MariaDB is now on version 11.4 and is active & running.

MariaDB Status

The Cleanup

When you're sure your new MariaDB installation is working and all databases are verified, you can delete the old MariaDB directory & the SQL backup.

sudo rm -rf /var/lib/mysql-old
sudo rm alldbbackups.sql
Please Share Me, I'm Lonely