patternsqlMinor
Estimate transaction log size for single transaction
Viewed 0 times
estimatelogsizesinglefortransaction
Problem
Today I have tried to execute an update operation on 50M+ rows, something like:
after 30min of waiting I have seen that the log has started to filling itself, after another 30min the entire transaction rolled-back due to the log being full.
DB was in the simple recovery mode but this is meaningless as this was done as a single transaction. SQL Server on it's own couldn't predict that it will use the entire log size (lets say that it was locked size for this matter), but my question to you my dear experienced DBA's - could you predict that size? Is there any way to calculate such an estimation?
UPDATE:
Yes, I understand the process, I know I can batch, I know how to make it performing well also. My real question is how big the log size is going to be?
How would you calculate update on a simple table like:
On my test box, the update part of the operation was 6 pages large = 48KB.
When I repeated the same for 10k rows I received 144 pages of transaction log.
For 100k rows it was 1314 pages, for 1M it was 13970 pages. This shows that we can concider this as linear function (on the we have extra pages which has to be there anyway no matter did we update anything or not --> 2-3 pages).
Going back to the begeining I know that I could run a single batch on 5% of my total operation, check the page chenge content (as blogged by Paul Randal):
```
DECLARE @Extent_ID INT;
DECLARE @Size_Total BIGINT = 0;
DECLARE @File_ID INT;
DECLARE @File_Size_Pages INT;
DECLARE @Log_Page_ID INT;
DECLARE @Log_Total BIGINT = 0;
DECLARE @Log_Total_Changed BIGINT;
DECLARE @DBCC_PAGE_String_Log VARCHAR (200);
DECLARE [files] CURSOR FOR
SELECT [file_id], [size]
FR
UPDATE [table] SET [Column] += 1;after 30min of waiting I have seen that the log has started to filling itself, after another 30min the entire transaction rolled-back due to the log being full.
DB was in the simple recovery mode but this is meaningless as this was done as a single transaction. SQL Server on it's own couldn't predict that it will use the entire log size (lets say that it was locked size for this matter), but my question to you my dear experienced DBA's - could you predict that size? Is there any way to calculate such an estimation?
UPDATE:
Yes, I understand the process, I know I can batch, I know how to make it performing well also. My real question is how big the log size is going to be?
How would you calculate update on a simple table like:
CREATE TABLE [TABL] ([ID] int identity(1,1) PRIMARY KEY CLUSTERED, [X] int not null);
INSERT INTO [TABL] VALUES(1)
GO 1000
UPDATE [TABL]
SET [X] = 2;On my test box, the update part of the operation was 6 pages large = 48KB.
When I repeated the same for 10k rows I received 144 pages of transaction log.
For 100k rows it was 1314 pages, for 1M it was 13970 pages. This shows that we can concider this as linear function (on the we have extra pages which has to be there anyway no matter did we update anything or not --> 2-3 pages).
Going back to the begeining I know that I could run a single batch on 5% of my total operation, check the page chenge content (as blogged by Paul Randal):
```
DECLARE @Extent_ID INT;
DECLARE @Size_Total BIGINT = 0;
DECLARE @File_ID INT;
DECLARE @File_Size_Pages INT;
DECLARE @Log_Page_ID INT;
DECLARE @Log_Total BIGINT = 0;
DECLARE @Log_Total_Changed BIGINT;
DECLARE @DBCC_PAGE_String_Log VARCHAR (200);
DECLARE [files] CURSOR FOR
SELECT [file_id], [size]
FR
Solution
Is there any way to calculate such an estimation?
No. You cannot predict that. You should always do your updates or deletes in batches.
No. You cannot predict that. You should always do your updates or deletes in batches.
Context
StackExchange Database Administrators Q#214785, answer score: 5
Revisions (0)
No revisions yet.