patternsqlMinor
MySQL 5.7.42, Gradual memory growth over time, memory leak?
Viewed 0 times
gradualtimemysqlleakgrowthmemoryover
Problem
I'm currently running a MySQL server (version 5.7.42) in production, and I've noticed a gradual increase in memory usage throughout the day. I suspect there might be a memory leak issue causing this behavior.
Here are some details about my server configuration:
I'm using the Prometheus exporter to monitor the server, and the memory usage keeps growing gradually over time.
Before upgrading to version 5.7.42, I was using version 5.7.22, and I didn't encounter this issue with memory usage. I'm wondering why this problem has arisen after the upgrade.
To investigate the issue further, I tried reproducing the problem in a similar environment using Ubuntu 18.04, as the MySQL 5.7.42 repositories are not officially compatible with Ubuntu 22 bu I obtained the same results.
It significantly worsens when the automysqlbackup process is triggered during the early morning hours. Also I'm using the --quick and --single-transaction options in process to avoid locking issues and improve the performance.
This is the actual configuration that I'm using:
```
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
default-character-set = utf8mb4
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
max_connections = 200
max_allowed_packet = 128M
interactive_timeout = 600
wait_timeout = 600
table_definition_cache = 4096
key_buffer_size = 128M
max_heap_table_size = 256M
tmp_table_size = 256M
join_buffer_size = 4M
thread_cache_size = 16
log_error = /var/log/mysql/mysql.err
log_error_verbosity = 2
long_query_time = 5
slow-query-log = on
slow_query_log_fil
Here are some details about my server configuration:
MySQL version: 5.7.42
Server specs: 4 vCPUs, 8 GB of RAM
MySQL data directory: SSD (no IOPS limit applied)
SO: Ubuntu 22.04I'm using the Prometheus exporter to monitor the server, and the memory usage keeps growing gradually over time.
Before upgrading to version 5.7.42, I was using version 5.7.22, and I didn't encounter this issue with memory usage. I'm wondering why this problem has arisen after the upgrade.
To investigate the issue further, I tried reproducing the problem in a similar environment using Ubuntu 18.04, as the MySQL 5.7.42 repositories are not officially compatible with Ubuntu 22 bu I obtained the same results.
It significantly worsens when the automysqlbackup process is triggered during the early morning hours. Also I'm using the --quick and --single-transaction options in process to avoid locking issues and improve the performance.
This is the actual configuration that I'm using:
```
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
default-character-set = utf8mb4
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
max_connections = 200
max_allowed_packet = 128M
interactive_timeout = 600
wait_timeout = 600
table_definition_cache = 4096
key_buffer_size = 128M
max_heap_table_size = 256M
tmp_table_size = 256M
join_buffer_size = 4M
thread_cache_size = 16
log_error = /var/log/mysql/mysql.err
log_error_verbosity = 2
long_query_time = 5
slow-query-log = on
slow_query_log_fil
Solution
See if any of these apply:
----- 2022-04-26 5.7.38 General Availability -- -- -----
A memory leak occurred if mysqldump was used on more than one table with the --order-by-primary option. The memory allocated for sorting each table’s rows is now freed after every table, rather than only once. (Bug #30042589, Bug #96178)
----- 2020-08-24 Perc 5.7.31-34 -- Bugs Fixed -- -----
PS-7203: Fixed audit plugin memory leak on replicas when opening tables
----- 2020-05-20 Perc 5.7.30-33 -- Bugs Fixed -- -----
PS-5844: Fix a memory leak after ‘innodb.alter_crash’ in ‘prepare_inplace_alter_table_dict()’ (Upstream #96472)
----- 2020-04-27 5.7.30 General Availability -- -- -----
The Event Scheduler had a memory leak. (Bug #30628268)
----- 2020-02-05 Perc 5.7.29-32 -- Bugs Fixed -- -----
PS-5843: A memory leak could occur after “group_replication.gr_majority_loss_restart”. (Upstream #96471)
----- 2020-01-13 5.7.29 General Availability -- -- Replication -----
A memory leak could occur when a failed replication group member tried to rejoin a minority group and was disallowed from doing so. (Bug #30162547, Bug #96471)
----- 2019-07-22 5.7.27 General Availability -- -- -----
The parser could leak memory for certain multiple-statement queries. (Bug #29419820)
----- 2019-07-22 5.7.27 General Availability -- -- -----
audit_log filtering operations could leak memory. (Bug #29201747)
----- 2019-04-25 5.7.26 General Availability -- -- InnoDB -----
Memory leaks discovered in the innochecksum utility were removed. (Bug #28917614, Bug #93164)
----- 2019-01-21 5.7.25 General Availability -- -- InnoDB -----
A dangling pointer caused a memory leak. (Bug #28693568)
----- 2019-01-21 5.7.25 General Availability -- -- -----
A memory leak was caused by GET_LOCK() calls with a zero timeout that failed due to concurrent connections holding the same user-level lock. (Bug #28714367)
----- 2019-01-21 5.7.25 General Availability -- -- -----
mysqlpump did not free all allocated resources when it encountered an error, resulting in memory leaks. (Bug #28538971, Bug #92131)
----- 2018-11-21 Perc 5.7.23-25 -- Bugs Fixed -- -----
#5049: A severe memory leak regression in the RocksDB Block Cache
----- 2018-07-27 5.7.23 General Availability -- -- -----
A memory leak in the pfs-t unit test was fixed. Thanks to Yura Sorokin for the patch. (Bug #27440735, Bug #89384)
(I scraped those from the Changelogs and/or Release notes.)
----- 2022-04-26 5.7.38 General Availability -- -- -----
A memory leak occurred if mysqldump was used on more than one table with the --order-by-primary option. The memory allocated for sorting each table’s rows is now freed after every table, rather than only once. (Bug #30042589, Bug #96178)
----- 2020-08-24 Perc 5.7.31-34 -- Bugs Fixed -- -----
PS-7203: Fixed audit plugin memory leak on replicas when opening tables
----- 2020-05-20 Perc 5.7.30-33 -- Bugs Fixed -- -----
PS-5844: Fix a memory leak after ‘innodb.alter_crash’ in ‘prepare_inplace_alter_table_dict()’ (Upstream #96472)
----- 2020-04-27 5.7.30 General Availability -- -- -----
The Event Scheduler had a memory leak. (Bug #30628268)
----- 2020-02-05 Perc 5.7.29-32 -- Bugs Fixed -- -----
PS-5843: A memory leak could occur after “group_replication.gr_majority_loss_restart”. (Upstream #96471)
----- 2020-01-13 5.7.29 General Availability -- -- Replication -----
A memory leak could occur when a failed replication group member tried to rejoin a minority group and was disallowed from doing so. (Bug #30162547, Bug #96471)
----- 2019-07-22 5.7.27 General Availability -- -- -----
The parser could leak memory for certain multiple-statement queries. (Bug #29419820)
----- 2019-07-22 5.7.27 General Availability -- -- -----
audit_log filtering operations could leak memory. (Bug #29201747)
----- 2019-04-25 5.7.26 General Availability -- -- InnoDB -----
Memory leaks discovered in the innochecksum utility were removed. (Bug #28917614, Bug #93164)
----- 2019-01-21 5.7.25 General Availability -- -- InnoDB -----
A dangling pointer caused a memory leak. (Bug #28693568)
----- 2019-01-21 5.7.25 General Availability -- -- -----
A memory leak was caused by GET_LOCK() calls with a zero timeout that failed due to concurrent connections holding the same user-level lock. (Bug #28714367)
----- 2019-01-21 5.7.25 General Availability -- -- -----
mysqlpump did not free all allocated resources when it encountered an error, resulting in memory leaks. (Bug #28538971, Bug #92131)
----- 2018-11-21 Perc 5.7.23-25 -- Bugs Fixed -- -----
#5049: A severe memory leak regression in the RocksDB Block Cache
----- 2018-07-27 5.7.23 General Availability -- -- -----
A memory leak in the pfs-t unit test was fixed. Thanks to Yura Sorokin for the patch. (Bug #27440735, Bug #89384)
(I scraped those from the Changelogs and/or Release notes.)
Context
StackExchange Database Administrators Q#327606, answer score: 2
Revisions (0)
No revisions yet.