InnoDB: Assertion failure in file .../padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.13/storage/innobase/btr/ line 7940

th flag

Every hour one MariaDB version 10.4 database stops responding and I see this output in the log file:

Feb  9 10:00:02 maria55 mysqld: 2023-02-09 10:00:02 0x7f72547a5700  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.13/storage/innobase/btr/ line 7940
Feb  9 10:00:02 maria55 mysqld: InnoDB: Failing assertion: space_id == page_get_space_id(page)
Feb  9 10:00:02 maria55 mysqld: InnoDB: We intentionally generate a memory trap.
Feb  9 10:00:02 maria55 mysqld: InnoDB: Submit a detailed bug report to
Feb  9 10:00:02 maria55 mysqld: InnoDB: If you get repeated assertion failures or crashes, even
Feb  9 10:00:02 maria55 mysqld: InnoDB: immediately after the mysqld startup, there may be
Feb  9 10:00:02 maria55 mysqld: InnoDB: corruption in the InnoDB tablespace. Please refer to
Feb  9 10:00:02 maria55 mysqld: InnoDB:
Feb  9 10:00:02 maria55 mysqld: InnoDB: about forcing recovery.

The server is running Kopano and although I can't find any prominent CRONs that cause this problem exactly on the hour, I fully suspect Kopano has some maintenance routines every hour triggering the crash.

The advice about corruption in InnoDB sounds legitimate, but I don't know how to do that the links are not clear.

For now we did a mysqlcheck kopano and database checks works 100%

Googling further doesn't really show anything.

I'm at a loss what to try next.

My one idea is to upgrade the database to see if any 'bugs' might be squashed, but when we did a dump to the same disk, this happened:

mysqldump --single-transaction --routines kopano -r kopanodb.sql

mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table 'lob' at row: 166961

Now I have a secondary problem in that I can't make a backup. The total size of the database is 80GB!

My idea was to upgrade MariaDB but now I am scared things go catastrophically wrong and I won't have a good backup.

My other idea was to maybe stop the MariaDB service and then copy /var/lib/mysql to a temporary location, all 80 GB, and then at least maybe this will be an okay backup.

Any other ideas or tips?

th flag

The solution was to do a MariaDB upgrade. The clues were version 10.4 of MariaDB is very old, and this line:

Submit a detailed bug report...

To get around the problem of database backup, I took the database offline by stopping the service, and then just did a directory/file copy. It's not an ideal backup but I think I would have been able to restore had something gone wrong.


Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.