patternsqlMinor

SQL Server Index and Statistics Maintenance performance question

Submitted by: @import:stackexchange-dba·Mar 10, 2026·

Viewed 0 times

serversqlmaintenancestatisticsperformanceandindexquestion

Problem

I tried to use IndexOptimize stored procedure on a database with a very high number of indexes (> 250.000). The initial step where stored procedure collects data what needs to be processed takes hours, even if I set @Indexes parameter to narrow down the work.

SQL Server Maintenance Solution Version: 2019-02-10 10:40:47 SQL
Server 2017 Standard Edition with latest CU14 installed.

At a customer I saw a database with > 500.000 indexes. After 12 hours data collection step was still running.

I would expect if I set @Indexes to a single index execution should be started immediately.

Here is an example of my stored procedure call.

EXECUTE dbo.IndexOptimize
@Databases = 'db',
@Indexes =  'db.dbo.'
@FragmentationLow = NULL,
@FragmentationMedium = 'INDEX_REORGANIZE,INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationHigh = 'INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationLevel1 = 30,
@FragmentationLevel2 = 50,
@UpdateStatistics = 'ALL',
@OnlyModifiedStatistics = 'Y',
@LogToTable = 'Y',
@LockTimeout = 60

Can somebody share with me his experience in using IndexOptimize for a database with a very high number of indexes ?

Solution

Can somebody share with me his experience in using IndexOptimize for a
database with a very high number of indexes ?

You are right, Ola's script first gets data from all the index related DMV's and inserts them inside of @tmpIndexesStatistics, even if one table or one index is specified.
For a lot of indexes, it gets stuck on this first command, which is a fairly huge query. See below for the query and even further below for a possible fix for this.

TL;DR

In one of the queries, the dynamic management views are called a couple of times each with multiple filters.

All this results in a fairly big execution plan.

Creating temp tables and storing the data from the DMV's prior to running the query is a workaround. You could optimize further with indexes on these temp tables / Rewrites.

Test data: >100K empty tables + >200k empty indexes

USE Test2
GO
SET NOCOUNT ON;
DECLARE @SQL NVARCHAR(MAX)
DECLARE @i int = 0

DECLARE @counter int = 1000
WHILE @i <+ @counter
BEGIN
SET @SQL = N'CREATE TABLE dbo.'+QUOTENAME(cast(@i as nvarchar(20)))+'( id int, val varchar(255));'
EXEC(@sql)
SET @SQL = N'CREATE INDEX IX_'+cast(@i as nvarchar(20))+ ' ON '+QUOTENAME(cast(@i as nvarchar(20)))+'(id) '
EXEC(@sql)
SET @SQL = N'CREATE INDEX IX_'+cast(@i as nvarchar(20))+ '_2 ON '+QUOTENAME(cast(@i as nvarchar(20)))+'(val) '
EXEC(@sql)
set @I +=1

IF @i = @counter
BEGIN

  IF @counter < 100000
  BEGIN
        SET @counter += 1000;
  END
END
END

select count(*) from sys.tables;
--100731

select count(*) from sys.indexes  where index_id != 0;
--201614

When changing the parameters to work on all the indexes, and printing instead of executing the statement ( @Execute='N')

EXECUTE MNGDB.dbo.IndexOptimize
@Databases = 'test2',
@Indexes =  'test2.dbo.',
@FragmentationLow = NULL,
@FragmentationMedium = 'INDEX_REORGANIZE,INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationHigh = 'INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationLevel1 = 30,
@FragmentationLevel2 = 50,
@UpdateStatistics = 'ALL',
@OnlyModifiedStatistics = 'Y',
@LogToTable = 'Y',
@LockTimeout = 60,
@Execute='N';

This huge query comes into play

```
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
USE [test2];
SELECT SchemaID, SchemaName, ObjectID, ObjectName, ObjectType, IsMemoryOptimized, IndexID, IndexName, IndexType, AllowPageLocks, IsImageText, IsNewLOB, IsFileStream, IsColumnStore, IsComputed, IsTimestamp, OnReadOnlyFileGroup, ResumableIndexOperation, StatisticsID, StatisticsName, NoRecompute, IsIncremental, PartitionID, PartitionNumber, PartitionCount, [Order], Selected, Completed
FROM (SELECT schemas.[schema_id] AS SchemaID, schemas.[name] AS SchemaName, objects.[object_id] AS ObjectID, objects.[name] AS ObjectName, RTRIM(objects.[type]) AS ObjectType, tables.is_memory_optimized AS IsMemoryOptimized, indexes.index_id AS IndexID, indexes.[name] AS IndexName, indexes.[type] AS IndexType, indexes.allow_page_locks AS AllowPageLocks,
CASE WHEN indexes.[type] = 1 AND EXISTS(SELECT * FROM sys.columns columns INNER JOIN sys.types types ON columns.system_type_id = types.user_type_id WHERE columns.[object_id] = objects.object_id AND types.name IN('image','text','ntext')) THEN 1 ELSE 0 END AS IsImageText,
CASE WHEN indexes.[type] = 1 AND EXISTS(SELECT * FROM sys.columns columns INNER JOIN sys.types types ON columns.system_type_id = types.user_type_id OR (columns.user_type_id = types.user_type_id AND types.is_assembly_type = 1) WHERE columns.[object_id] = objects.object_id AND (types.name IN('xml') OR (types.name IN('varchar','nvarchar','varbinary') AND columns.max_length = -1) OR (types.is_assembly_type = 1 AND columns.max_length = -1))) THEN 1 WHEN indexes.[type] = 2
AND EXISTS(SELECT * FROM sys.index_columns index_columns INNER JOIN sys.columns columns ON index_columns.[object_id] = columns.[object_id]
AND index_columns.column_id = columns.column_id
INNER JOIN sys.types types ON columns.system_type_id = types.user_type_id OR (columns.user_type_id = types.user_type_id AND types.is_assembly_type = 1) WHERE index_columns.[object_id] = objects.object_id AND index_columns.index_id = indexes.index_id AND (types.[name] IN('xml') OR (types.[name] IN('varchar','nvarchar','varbinary') AND columns.max_length = -1) OR (types.is_assembly_type = 1 AND columns.max_length = -1)))
THEN 1 ELSE 0 END AS IsNewLOB,
CASE WHEN indexes.[type] = 1 AND EXISTS(SELECT * FROM sys.columns columns WHERE columns.[object_id] = objects.object_id AND columns.is_filestream = 1) THEN 1 ELSE 0 END AS IsFileStream,
CASE WHEN EXISTS(SELECT * FROM sys.indexes indexes WHERE indexes.[object_id] = objects.object_id AND [type] IN(5,6)) THEN 1 ELSE 0 END AS IsColumnStore,
CASE WHEN EXISTS(SELECT * FROM sys.index_columns index_columns INNER JOIN sys.columns columns ON index_columns.object_id = columns.object_id AND index_columns.column_id = columns.column_id WHERE (index_columns.key_ordinal > 0 OR index_columns.partitio

Code Snippets

USE Test2
GO
SET NOCOUNT ON;
DECLARE @SQL NVARCHAR(MAX)
DECLARE @i int = 0

DECLARE @counter int = 1000
WHILE @i <+ @counter
BEGIN
SET @SQL = N'CREATE TABLE dbo.'+QUOTENAME(cast(@i as nvarchar(20)))+'( id int, val varchar(255));'
EXEC(@sql)
SET @SQL = N'CREATE INDEX IX_'+cast(@i as nvarchar(20))+ ' ON '+QUOTENAME(cast(@i as nvarchar(20)))+'(id) '
EXEC(@sql)
SET @SQL = N'CREATE INDEX IX_'+cast(@i as nvarchar(20))+ '_2 ON '+QUOTENAME(cast(@i as nvarchar(20)))+'(val) '
EXEC(@sql)
set @I +=1

IF @i = @counter
BEGIN

  IF @counter < 100000
  BEGIN
        SET @counter += 1000;
  END
END
END

select count(*) from sys.tables;
--100731

select count(*) from sys.indexes  where index_id != 0;
--201614

EXECUTE MNGDB.dbo.IndexOptimize
@Databases = 'test2',
@Indexes =  'test2.dbo.',
@FragmentationLow = NULL,
@FragmentationMedium = 'INDEX_REORGANIZE,INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationHigh = 'INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationLevel1 = 30,
@FragmentationLevel2 = 50,
@UpdateStatistics = 'ALL',
@OnlyModifiedStatistics = 'Y',
@LogToTable = 'Y',
@LockTimeout = 60,
@Execute='N';

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
USE [test2]; 
SELECT SchemaID, SchemaName, ObjectID, ObjectName, ObjectType, IsMemoryOptimized, IndexID, IndexName, IndexType, AllowPageLocks, IsImageText, IsNewLOB, IsFileStream, IsColumnStore, IsComputed, IsTimestamp, OnReadOnlyFileGroup, ResumableIndexOperation, StatisticsID, StatisticsName, NoRecompute, IsIncremental, PartitionID, PartitionNumber, PartitionCount, [Order], Selected, Completed 
FROM (SELECT schemas.[schema_id] AS SchemaID, schemas.[name] AS SchemaName, objects.[object_id] AS ObjectID, objects.[name] AS ObjectName, RTRIM(objects.[type]) AS ObjectType, tables.is_memory_optimized AS IsMemoryOptimized, indexes.index_id AS IndexID, indexes.[name] AS IndexName, indexes.[type] AS IndexType, indexes.allow_page_locks AS AllowPageLocks, 
CASE WHEN indexes.[type] = 1 AND EXISTS(SELECT * FROM sys.columns columns INNER JOIN sys.types types ON columns.system_type_id = types.user_type_id WHERE columns.[object_id] = objects.object_id AND types.name IN('image','text','ntext')) THEN 1 ELSE 0 END AS IsImageText, 
CASE WHEN indexes.[type] = 1 AND EXISTS(SELECT * FROM sys.columns columns INNER JOIN sys.types types ON columns.system_type_id = types.user_type_id OR (columns.user_type_id = types.user_type_id AND types.is_assembly_type = 1) WHERE columns.[object_id] = objects.object_id AND (types.name IN('xml') OR (types.name IN('varchar','nvarchar','varbinary') AND columns.max_length = -1) OR (types.is_assembly_type = 1 AND columns.max_length = -1))) THEN 1 WHEN indexes.[type] = 2 
            AND EXISTS(SELECT * FROM sys.index_columns index_columns INNER JOIN sys.columns columns ON index_columns.[object_id] = columns.[object_id] 
            AND index_columns.column_id = columns.column_id 
            INNER JOIN sys.types types ON columns.system_type_id = types.user_type_id OR (columns.user_type_id = types.user_type_id AND types.is_assembly_type = 1) WHERE index_columns.[object_id] = objects.object_id AND index_columns.index_id = indexes.index_id AND (types.[name] IN('xml') OR (types.[name] IN('varchar','nvarchar','varbinary') AND columns.max_length = -1) OR (types.is_assembly_type = 1 AND columns.max_length = -1))) 
            THEN 1 ELSE 0 END AS IsNewLOB, 
CASE WHEN indexes.[type] = 1 AND EXISTS(SELECT * FROM sys.columns columns WHERE columns.[object_id] = objects.object_id  AND columns.is_filestream = 1) THEN 1 ELSE 0 END AS IsFileStream, 
CASE WHEN EXISTS(SELECT * FROM sys.indexes indexes WHERE indexes.[object_id] = objects.object_id AND [type] IN(5,6)) THEN 1 ELSE 0 END AS IsColumnStore, 
CASE WHEN EXISTS(SELECT * FROM sys.index_columns index_columns INNER JOIN sys.columns columns ON index_columns.object_id = columns.object_id AND index_columns.column_id = columns.column_id WHERE (index_columns.key_ordinal > 0 OR index_columns.partition_ordinal > 0) AND columns.is_computed = 1 AND index_columns.object_id = indexes.object_id AND index_columns.index_id = indexes.index_id) THEN 1 ELSE 0 END AS IsC

IF EXISTS(SELECT * FROM @ActionsPreferred) OR @UpdateStatistics IN('ALL','INDEX')

EXECUTE MNGDB.dbo.IndexOptimize
@Databases = 'test2',
@Indexes =  'test2.dbo.[83631]',
@FragmentationLow = NULL,
@FragmentationMedium = 'INDEX_REORGANIZE,INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationHigh = 'INDEX_REBUILD_ONLINE,INDEX_REBUILD_OFFLINE',
@FragmentationLevel1 = 30,
@FragmentationLevel2 = 50,
@UpdateStatistics = 'ALL',
@OnlyModifiedStatistics = 'Y',
@LogToTable = 'Y',
@LockTimeout = 60,
@EXECUTE= 'N'

Context

StackExchange Database Administrators Q#234520, answer score: 7

Revisions (0)

No revisions yet.