HiveBrain v1.2.0
Get Started
← Back to all entries
gotchasqlMinor

Why does MySQL slow down in loading data?

Submitted by: @import:stackexchange-dba··
0
Viewed 0 times
whyslowmysqldownloadingdoesdata

Problem

We have a really large file (26GB) that we need to upload to MySQL. In order to do so we split it into nine 3GB files.

What's confusing is that the first file took only 2 minutes to load, whereas the second file took 16 minutes and the third file took 40 minutes.

We're loading these files using load data local infile syntax, without any indexes or keys. The table engine is MyISAM.

Does anyone know why MySQL slows down on each successive load?

Solution

You might need to LOCK the tables before you insert the data into the database and then you unlock after the LOAD statement.

There are some tips on MySQL manual as well, check it out:

http://dev.mysql.com/doc/refman/5.6/en/optimizing-myisam-bulk-data-loading.html

Context

StackExchange Database Administrators Q#62949, answer score: 2

Revisions (0)

No revisions yet.