patternsqlMinor
Mysql slowly uses memory until it starts to use swap
Viewed 0 times
startsuntilswapusesmysqlmemoryuseslowly
Problem
I am running a 1gb of RAM rackspace database server. For some reason in about 2 days the memory usage goes from using very little swap, to using 100mb. If I don't restart sql it will keep using more swap. (My my.cnf file is shown below and memory usage shown below)
Some background: I have about 50 active databases that have the same schema that use INNODB for there tables. I have a couple databases with little traffic that use MyISAM.
On the INNODB tables I do NOT use persistent connections. I also have a reporting function that creates a temporary table. (This can be resource intensive, but does NOT happen often)
I am using CENTOS 6.3 and mysql 5.5.28-log
Even though I am using swap, the performance is still pretty good. I am just afraid that if I don't restart every few days I will have a problem.
Here is my log of free -m for about 2 days: (The first record is right after a mysql restart)
```
12/26 2:08 PM EST
total used free shared buffers cached
Mem: 992 697 295 0 74 362
-/+ buffers/cache: 260 732
Swap: 976 15 961
12/26 4:10 PM EST
[root@php-pos-db ~]# free -m
total used free shared buffers cached
Mem: 992 791 201 0 97 405
-/+ buffers/cache: 287 705
Swap: 976 14 961
12/27 2:52 PM EST
[root@php-pos-db ~]# free -m
total used free shared buffers cached
Mem: 992 947 45 0 55 169
-/+ buffers/cache: 722 270
Swap: 976 34 942
12/28 1:41 PM EST
total used free shared buffers cached
Mem: 992 963 29 0 45 119
-/+ buffers/cache: 797 195
Swap: 976 48 927
12/28 7:24 PM EST
[root@php-po
Some background: I have about 50 active databases that have the same schema that use INNODB for there tables. I have a couple databases with little traffic that use MyISAM.
On the INNODB tables I do NOT use persistent connections. I also have a reporting function that creates a temporary table. (This can be resource intensive, but does NOT happen often)
I am using CENTOS 6.3 and mysql 5.5.28-log
Even though I am using swap, the performance is still pretty good. I am just afraid that if I don't restart every few days I will have a problem.
Here is my log of free -m for about 2 days: (The first record is right after a mysql restart)
```
12/26 2:08 PM EST
total used free shared buffers cached
Mem: 992 697 295 0 74 362
-/+ buffers/cache: 260 732
Swap: 976 15 961
12/26 4:10 PM EST
[root@php-pos-db ~]# free -m
total used free shared buffers cached
Mem: 992 791 201 0 97 405
-/+ buffers/cache: 287 705
Swap: 976 14 961
12/27 2:52 PM EST
[root@php-pos-db ~]# free -m
total used free shared buffers cached
Mem: 992 947 45 0 55 169
-/+ buffers/cache: 722 270
Swap: 976 34 942
12/28 1:41 PM EST
total used free shared buffers cached
Mem: 992 963 29 0 45 119
-/+ buffers/cache: 797 195
Swap: 976 48 927
12/28 7:24 PM EST
[root@php-po
Solution
MySQL does have the nasty habit being swap-happy. Jeremy Cole has best addressed this in his blog : http://blog.jcole.us/2012/04/16/a-brief-update-on-numa-and-mysql/. From that blog, you learn that there is something you can do: Add
SUGGESTIONS
If the Server is dedicated to doing only MySQL, please change the following in /etc/my.cnf:
If the server is at least dual-core, add these
Next, login to mysql run
Next, run the following in the OS
Give it a Try !!!
UPDATE 2012-12-31 08:30 EDT
From your last comment
It stopped climbing around 1 gb. I removed unused databases and it just seems mysql 5.5 stores a lot of data in memory as this didn't happen in 5.0. Did mysql change a lot?
Yes, MySQL has changed a lot. In fact, there are many cases where upgrading from MySQL 5.0 to MySQL 5.5 and resulted in performance degradation. InnoDB 5.5 is now equipped to do hyperthreading and multicore engagement.
Percona actually tested this out awhile ago.
Please read me past posts on this subject
I also wrote about this in ServerFault and StackOverflow
numactl --interleave=all inside /etc/init.d/mysql.SUGGESTIONS
If the Server is dedicated to doing only MySQL, please change the following in /etc/my.cnf:
[mysqld]
innodb_open_files=1000
innodb_flush_method=O_DIRECT
innodb_buffer_pool_size=768M
innodb_log_file_size=192MIf the server is at least dual-core, add these
innodb_buffer_pool_instances=2
innodb_read_io_threads=16
innodb_write_io_threads=16
innodb_io_capacity=2000Next, login to mysql run
SET GLOBAL innodb_fast_shutdown = 0;Next, run the following in the OS
cd /var/lib/mysql
service mysql stop
mv ib_logfile0 ib_logfile0.bak
mv ib_logfile1 ib_logfile1.bak
service mysql startGive it a Try !!!
UPDATE 2012-12-31 08:30 EDT
From your last comment
It stopped climbing around 1 gb. I removed unused databases and it just seems mysql 5.5 stores a lot of data in memory as this didn't happen in 5.0. Did mysql change a lot?
Yes, MySQL has changed a lot. In fact, there are many cases where upgrading from MySQL 5.0 to MySQL 5.5 and resulted in performance degradation. InnoDB 5.5 is now equipped to do hyperthreading and multicore engagement.
Percona actually tested this out awhile ago.
Please read me past posts on this subject
Jul 16, 2012: decreased performance of stored procedure when migrated from mysql server 5.0 to 5.5
Nov 24, 2011: Why mysql 5.5 slower than 5.1 (linux,using mysqlslap)
Oct 05, 2011: Query runs a long time in some newer MySQL versions
Sep 20, 2011: Multi cores and MySQL Performance
Jun 19, 2011: How do I properly perform a MySQL bake-off?
I also wrote about this in ServerFault and StackOverflow
Feb 22, 2012: https://serverfault.com/questions/362388/mysql-5-1-vs-mysql-5-5-5-1-twice-as-fast/362706#362706
Feb 08, 2012: https://stackoverflow.com/questions/9202825/mysql-5-5-perfomance/9202860#9202860
Code Snippets
[mysqld]
innodb_open_files=1000
innodb_flush_method=O_DIRECT
innodb_buffer_pool_size=768M
innodb_log_file_size=192Minnodb_buffer_pool_instances=2
innodb_read_io_threads=16
innodb_write_io_threads=16
innodb_io_capacity=2000cd /var/lib/mysql
service mysql stop
mv ib_logfile0 ib_logfile0.bak
mv ib_logfile1 ib_logfile1.bak
service mysql startContext
StackExchange Database Administrators Q#31182, answer score: 5
Revisions (0)
No revisions yet.