patternsqlMinor
Does mysql use deleted rows space in a table?
Viewed 0 times
rowsspacemysqldeleteddoesusetable
Problem
I am deleting a large data around 60GB from one of my innodb table which is around 70GB.
This is a rapidly growing table so i keep on inserting and deleting the records over a period of time. I cannot do DB optimization every time I delete.
So, does mysql use this deleted rows space (which is not given back to OS) for newly inserted rows?
update: innodb_file_per_table is ON already.
This is a rapidly growing table so i keep on inserting and deleting the records over a period of time. I cannot do DB optimization every time I delete.
So, does mysql use this deleted rows space (which is not given back to OS) for newly inserted rows?
update: innodb_file_per_table is ON already.
Solution
Here is something to consider: you need to determine how much fragmentation exists before deciding to defragment the table or not.
The most efficient way is to measure the space usage from the OS and from INFORMATION_SCHEMA
For this example, assume the following
OS
INFORMATION_SCHEMA
Simply subtract @table_size_inf from TABLE_SIZE_OS to get the number of bytes unused
Since you deleted 85.7142 % of the table (60GB out of 70GB), you could then run
innodb_file_per_table is disabled
You could do this against an entire database
I have suggested stuff like this before
UPDATE 2017-01-31 07:40 EST
You just asked the following
so i am assuming , if i don't do optimization and keep on inserting data , newly inserted records will use the space which is left over by previously deleted records and table size won't grow. However it depends on the newly inserted data size and contiguous space left over by previously deleted data . Am i correct?
My answer to that is Yes. Deleted rows are simply marked and are available for reuse.
However, each 16K block would have its own internal fragmentation. Even my answer does not compute the granularity of fragmentation within 16K blocks. The more sparse a 16K block is due to deleted space, the more disk I/O InnoDB will experience in traversing 16K blocks in search of free space wide enough for row insertion.
If you are satisfied with the rate of your bulk insertions, then there is no need to
The most efficient way is to measure the space usage from the OS and from INFORMATION_SCHEMA
For this example, assume the following
- datadir is
/var/lib/mysql
- database is
mydb
- table is
mytable
OS
TABLE_SIZE_OS=`ls -l /var/lib/mysql/mydb/mytable.ibd | awk '{print $5}'`
echo ${TABLE_SIZE_OS}INFORMATION_SCHEMA
SELECT (data_length+index_length) INTO @table_size_inf
FROM information_schema.tables
WHERE table_schema'mydb'
AND table_name='mytable';
SELECT @table_size_inf;Simply subtract @table_size_inf from TABLE_SIZE_OS to get the number of bytes unused
Since you deleted 85.7142 % of the table (60GB out of 70GB), you could then run
OPTIMIZE TABLE mydb.mtable;innodb_file_per_table is disabled
You could do this against an entire database
cd /var/lib/mysql
IBDATA_SIZE=0
for X in `ls -l ibdata* | awk '{print $5}'` ; do (( IBDATA_SIZE += X )) ; done
SQLSTMT="SELECT SUM(data_length+index_length)"
SQLSTMT="${SQLSTMT} FROM information_schema.tables"
SQLSTMT="${SQLSTMT} WHERE engine='InnoDB'"
INNODB_SIZE=`mysql -uroot -ppassword -ANe"${SQLSTMT}"`
(( IBDATA_FRAG = IBDATA_SIZE - INNODB_SIZE ))I have suggested stuff like this before
Apr 11, 2012: How do you remove fragmentation from InnoDB tables?
Jul 03, 2013: Information about Disk Storage MySQL
UPDATE 2017-01-31 07:40 EST
You just asked the following
so i am assuming , if i don't do optimization and keep on inserting data , newly inserted records will use the space which is left over by previously deleted records and table size won't grow. However it depends on the newly inserted data size and contiguous space left over by previously deleted data . Am i correct?
My answer to that is Yes. Deleted rows are simply marked and are available for reuse.
However, each 16K block would have its own internal fragmentation. Even my answer does not compute the granularity of fragmentation within 16K blocks. The more sparse a 16K block is due to deleted space, the more disk I/O InnoDB will experience in traversing 16K blocks in search of free space wide enough for row insertion.
If you are satisfied with the rate of your bulk insertions, then there is no need to
OPTIMIZE TABLE that one table. If your index statistics against that table make SELECT queries bad, just run ANALYZE TABLE instead of OPTIMIZE TABLE.Code Snippets
TABLE_SIZE_OS=`ls -l /var/lib/mysql/mydb/mytable.ibd | awk '{print $5}'`
echo ${TABLE_SIZE_OS}SELECT (data_length+index_length) INTO @table_size_inf
FROM information_schema.tables
WHERE table_schema'mydb'
AND table_name='mytable';
SELECT @table_size_inf;OPTIMIZE TABLE mydb.mtable;cd /var/lib/mysql
IBDATA_SIZE=0
for X in `ls -l ibdata* | awk '{print $5}'` ; do (( IBDATA_SIZE += X )) ; done
SQLSTMT="SELECT SUM(data_length+index_length)"
SQLSTMT="${SQLSTMT} FROM information_schema.tables"
SQLSTMT="${SQLSTMT} WHERE engine='InnoDB'"
INNODB_SIZE=`mysql -uroot -ppassword -ANe"${SQLSTMT}"`
(( IBDATA_FRAG = IBDATA_SIZE - INNODB_SIZE ))Context
StackExchange Database Administrators Q#162632, answer score: 2
Revisions (0)
No revisions yet.