InnoDB Performance Tuning

I'm new to database administration so please excuse any omissions of what would seem like basic info to provide for this problem.

I'm a RoR developer and our team is currently short a DBA, so we're all filling the role.

We have a couple large databases with tables ranging anywhere from 50k rows to 84 million rows.

I'm trying to tune our settings for the server we're running on, and would appreciate any help or advice or areas to research.

I'm mostly looking for low-hanging fruit here that we could improve until we can hire a full time DBA to handle this kind of stuff.

```
Server Statistics:
Ubuntu 10.04
8 CPUs
32 GB RAM

Show Variables Output
auto_increment_increment 1
auto_increment_offset 1
autocommit ON
automatic_sp_privileges ON
back_log 50
basedir /usr/
big_tables OFF
binlog_cache_size 32768
binlog_direct_non_transactional_updates OFF
binlog_format STATEMENT
bulk_insert_buffer_size 8388608
character_set_client latin1
character_set_connection latin1
character_set_database latin1
character_set_filesystem binary
character_set_results latin1
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
collation_connection latin1_swedish_ci
collation_database latin1_swedish_ci
collation_server latin1_swedish_ci
completion_type 0
concurrent_insert 2
connect_timeout 20
datadir /var/lib/mysql/
date_format %Y-%m-%d
datetime_format %Y-%m-%d %H:%i:%s
default_week_format 0
delay_key_write ON
delayed_insert_limit 100
delayed_insert_timeout 300
delayed_queue_size 1000
div_precision_increment 4
engine_condition_pushdown ON
error_count 0
event_scheduler OFF
expire_logs_days 10
flush OFF
flush_time 0
foreign_key_checks ON
ft_boolean_syntax + -><()~*:""&|
ft_max_word_len 84
ft_min_word_len 4
ft_query_expansion_limit 20
ft_stopword_file (built-in)
general_log OFF
general_log_file /var/run/mysqld/mysqld.log
group_concat_max_len 1024
have_community_features YES

Firstly, I must point out that variable tuning is very application specific and highly depends on what problems you are running into. That being said, you need to know your dataset and which engines you are using (and so does anyone answering). Here is a query to get you that, but be aware it could potentially run a while and might lock up your application (anecdotal: I can run it on 80GB in less than 5 seconds):

SELECT IFNULL(B.engine,'Total') "Storage Engine", CONCAT(LPAD(FORMAT(
B.DSize/POWER(1024,pw),3),17,' '),' ',SUBSTR(' KMGTP',pw+1,1),'B') "Data Size",
CONCAT(LPAD(FORMAT(B.ISize/POWER(1024,pw),3),17,' '),' ',
SUBSTR(' KMGTP',pw+1,1),'B') "Index Size",CONCAT(LPAD(FORMAT(B.TSize/
POWER(1024,pw),3),17,' '),' ',SUBSTR(' KMGTP',pw+1,1),'B') "Table Size"
FROM (SELECT engine,SUM(data_length) DSize,
SUM(index_length) ISize,SUM(data_length+index_length) TSize FROM information_schema.tables
WHERE table_schema NOT IN ('mysql','information_schema','performance_schema') AND
engine IS NOT NULL GROUP BY engine WITH ROLLUP) B,(SELECT 2 pw) A ORDER BY TSize;

I stole this query from Rolando. It is very nice!

This will output the storage engines you are using and the amount of data, index and total space is being used by each.

This is important because you are allocation 5GB for MyISAM indexes:

key_buffer_size 5242880000

Do you have that much MyISAM index usage?

For InnoDB, here are some quick suggestions.

-
innodb_buffer_pool_size 16106127360 is 15GB. Normally if you run all InnoDB (no MyISAM) and MySQL is the only thing running on the server, this can be most of your memory (saving room for per-thread buffers). But 15GB is decent so I wouldn't start your tuning there.

-
InnoDB Log files are low:

innodb_log_buffer_size  1048576 #1MB
innodb_log_file_size    5242880 #5MB

Since you mention database sizes up to 84GB, I would start by tuning this up to innodb_log_file_size = 100MB and innodb_log_buffer_size = 8MB. Here is how to change this directive safely.

-
innodb_flush_log_at_trx_commit is at 1, which helps make your InnoDB as ACID compliant as MySQL can get. It means that after every update commit (insert, deleted, update), InnoDB with flush the transaction to the logfile. Depending on your disk speeds, this can be a bottleneck. By setting it to 2, InnoDB will flush it to the OS cache, and every second will flush it to the logfile. So if your OS crashes, you lose 1-2 seconds, but if MySQL crashes, you won't lose anything.

This is a good article to read to get started, and in general anything posted on that blog.

StackExchange Database Administrators Q#28738, answer score: 4