Score:0

Why /var/lib/mysql takes too much space whereas actual db is small enough

fr flag

In my server Db takes too much space:

Filesystem       Size  Used Avail Use% Mounted on
/dev/root         58G   58G  461M 100% /
tmpfs            966M     0  966M   0% /dev/shm
tmpfs            387M  856K  386M   1% /run
tmpfs            5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p15  105M  6.1M   99M   6% /boot/efi
tmpfs            194M  4.0K  194M   1% /run/user/1000

Ans whilst I was investigating it I saw that /vae/lib/mysql is the one who takes up the space:

ncdu 1.15.1 ~ Use the arrow keys to navigate, press ? for help                                                                                                                                                                                
--- /var/lib 
   50.6 GiB [##########] /mysql                                                                                                                                                                                                               
  801.2 MiB [          ] /snapd
  138.4 MiB [          ] /apt
   90.1 MiB [          ] /mecab
   39.4 MiB [          ] /dpkg
    3.8 MiB [          ] /ubuntu-advantage

And I tried to investigate what is the thing that takes too much space. All of the databases do serve wordpress site therefore I cannot modify the schema too much.

But I cheched the db size though:

SELECT table_schema AS "Database", SUM(data_length + index_length) / 1024 / 1024 AS "Size (MB)" FROM information_schema.TABLES GROUP BY table_schema;
+--------------------+--------------+
| Database           | Size (MB)    |
+--------------------+--------------+
| blog1              | 166.00000000 |
| information_schema |   0.00000000 |
| mysql              |   2.76562500 |
| performance_schema |   0.00000000 |
| blog2              |  31.79687500 |
| sys                |   0.01562500 |
+--------------------+--------------+
6 rows in set (0.35 sec)

Both of them Are relatively small Databases. Therefore how on Jesus' Christ name the /var/lib/mysql takes too much space?

The setup is just a LEMP stack with mysql without replication.

Edit 1

I have looked upon /var/lib/mysql and I see many records for binlog:

sudo du -sk /var/lib/mysql
53151572    /var/lib/mysql

$ sudo ls -l /var/lib/mysql | wc -l
623
# I susspect 653 lines would be too much if pasted here

$ sudo ls -l /var/lib/mysql | grep binlog | head
-rw-r----- 1 mysql mysql  43170892 Jun  4 00:00 binlog.000233
-rw-r----- 1 mysql mysql  35975939 Jun  5 00:00 binlog.000234
-rw-r----- 1 mysql mysql  36153241 Jun  6 00:00 binlog.000235
-rw-r----- 1 mysql mysql  37148526 Jun  7 00:00 binlog.000236
-rw-r----- 1 mysql mysql  34947871 Jun  8 00:00 binlog.000237
-rw-r----- 1 mysql mysql  34058129 Jun  9 00:00 binlog.000238
-rw-r----- 1 mysql mysql  36786212 Jun 10 00:00 binlog.000239
-rw-r----- 1 mysql mysql  34790230 Jun 11 00:00 binlog.000240
-rw-r----- 1 mysql mysql  37634381 Jun 12 00:00 binlog.000241
-rw-r----- 1 mysql mysql  35801131 Jun 13 00:00 binlog.000242

$ sudo ls -l /var/lib/mysql | grep binlog | wc -l
545

# 545 lines are binlog files therefore I have binlogs

And I have looked upon a way to mitigate it:

$ cat /etc/mysql/mysql.conf.d/rj.cnf
# Added by Rick James
[mysqld]
log_bin =                          # turn off
binlog_expire_logs_seconds = 86400  # 1 day
max_binlog_size = 1         # 100M

And I did:

mysql> SHOW BINARY LOGS;
PURGE BINARY LOGS BEFORE '2023-07-30 00:00:00';
Query OK, 0 rows affected, 1 warning (0.01 sec)

But still files keep filling up upon /var/lib/mysql Would be safe if I rm -rf them or if somehow could tell that binlog storage it /dev/null ?

Nikita Kipriyanov avatar
za flag
So what's `ls -lah /var/lib/mysql`? I suspect a single default InnoDB tablespace had grown and there is no practical way to shrink it other than recreate. For this reason, it is recommended to set `innodb_per_file_table` to 1, so default InnoDB tablespace isn't being used for storing user table data. [Look here.](https://dba.stackexchange.com/questions/8982/what-is-the-best-way-to-reduce-the-size-of-ibdata-in-mysql)
Romeo Ninov avatar
in flag
Have you check for binary logs? WHat is the result of `du -sk /var/lib/mysql`?
Dimitrios Desyllas avatar
fr flag
I have done that you mention both, and I edited my question upon commands and mitigations.
Nikita Kipriyanov avatar
za flag
See [here](https://dba.stackexchange.com/questions/72770/disable-mysql-binary-logging-with-log-bin-variable) and [here](https://dba.stackexchange.com/questions/41050/is-it-safe-to-delete-mysql-bin-files/41054#41054). **Don't** just `rm` them. Do `RESET MASTER` instead.
ua flag
https://dev.mysql.com/doc/refman/8.0/en/purge-binary-logs.html -- `PURGE BINLOGS BEFORE '2023-06-25'; (or something like that.)
ua flag
Also, did you restart the server after adding the cnf settings?
Dimitrios Desyllas avatar
fr flag
Yeap @RickJames
Score:1
fr flag

In order to fix the issue do the following:

Remove the /etc/mysql/mysql.conf.d/rj.cnf

rm -rf /etc/mysql/mysql.conf.d/rj.cnf

Create a new config /etc/mysql/mysql.conf.d/no-binlog.cnf and add the follosing config:

[mysqld]
disable_log_bin

Then do:

sudo service mysql restart

Afterwards connect using the mysql command and run:

RESET MASTER

This will fix your issue. It is assumed that no replication is used for example having a single server that has a typical LAMP or LEMP stack. (eg personal server)


The reason why

The reason was that you had some logfiles that mysql uses upon replication. These logfiles are generated from binlog and can be accumulated over time.

In our approach we disable this type of logging due to lack of replication.

Score:0
ws flag

While, if you DO NOT have any replication in place and your backup/restore plan does not depend on bin logs, then you can simply limit the history retained or turn off binary logging altogether. It appears that bin logs are the source of the issue here. But there's more to the story.

Deleted records

Depending on how your MySQL data is configured, it will rarely free up disk space previously used for records which are now deleted. It will re-use this space for new records.

The easiest way to recover space is to do a dump/restore.

Wordpress

Wordpress on its own is OK. But it does have a large ecosystem of plugins which vary greatly in quality. I've seen a lot of installations were there are more writes (by orders of magnitude) to the database than reads. Worse than these plugins storing lots of data is that they typically use the Wordpress tables to store the data in - so you can uninstall the plugin but you can never delete the data.

Dimitrios Desyllas avatar
fr flag
My system is not too critical it is just secondary sites. For not the space had been freed up. If a problem occurs again I will re-tackle it. The site is for 5-4 years anmd the db is just 166Mb. Therefore I do not expect in my case the plugins to cause too much trouble.
Dimitrios Desyllas avatar
fr flag
If the servar was a Shared hosting I would use a different approach.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.