patternMinor
What to do after deleting a million records
Viewed 0 times
aftermilliondeletingwhatrecords
Problem
We are implementing a new feature in our system that will cause about a million records (each record is tiny, basically a GUID, a date, and four
This will run every night at 1am, and will purge about a million records every time it runs.
Is there anything I should be noting or doing for a table like this? Any properties I should put on the table, or any routines I should run regularly to "clean up"? I've never dealt with a table like this before.
The table has a single clustered index (GUID + one of the
smallint fields) to be purged from a table every night. Basically it's a caching table, and once the data is 7 days old we do:DELETE FROM scheduleCache WHERE schDateCreated < '2013-08-26This will run every night at 1am, and will purge about a million records every time it runs.
Is there anything I should be noting or doing for a table like this? Any properties I should put on the table, or any routines I should run regularly to "clean up"? I've never dealt with a table like this before.
The table has a single clustered index (GUID + one of the
smallint fields), and we have a weekly index rebuild that runs Sunday mornings.Solution
The problem of deleting large portions of a table is far from a trivial problem. The best approach, by far, is partitioning. A daily partition scheme with a sliding window is really a magic bullet for this problem, see How to Implement an Automatic Sliding Window in a Partitioned Table.
If you cannot afford partitioning (eg. non-enterprise license on site) then I would recommend clustering by
If you cannot afford partitioning (eg. non-enterprise license on site) then I would recommend clustering by
schDateCreated. If you need primary key on the GUID+smallint then move it to non-clustered. Delete in batches (eg. TOP 10000), in a loop, to reduce pressure on the log. Consider updating stats after the operation.Context
StackExchange Database Administrators Q#48708, answer score: 7
Revisions (0)
No revisions yet.