Question

I have the following query:

SELECT TOP (100000) 
    [Filter1].[ID] AS [ID], 
    [Filter1].[FIELD1] AS [FIELD1], 
    [Filter1].[FIELD2] AS [FIELD2], 
    [Filter1].[FIELD3] AS [FIELD3], 
    [Filter1].[FIELD4] AS [FIELD4], 
    ...
    [Filter1].[FIELD30] AS [FIELD30], 
    FROM ( SELECT [Extent1].[ID] AS [ID],
         [Extent1].[FIELD1] AS [FIELD1],
         [Extent1].[FIELD2] AS [FIELD2],
         [Extent1].[FIELD3] AS [FIELD3]
         ...
         [Filter1].[FIELD30] AS [FIELD30], 
         row_number() OVER (ORDER BY [Extent1].[ID] ASC) AS [row_number]
         FROM [dbo].[TABLE] AS [Extent1]
         WHERE (N'VALUE1' <> [Extent1].[**FIELD2**]
         AND (N'VALUE2' <> ([Extent1].[**FIELD3**])
         AND ([Extent1].[**FIELD4**] IN (VALUE1, VALUE2, VALUE3, .... VALUE9)))
         AS [Filter1]
    WHERE [Filter1].[row_number] > 0
    ORDER BY [Filter1].[ID] ASC

Due to the amount of rows that need to be selected (a few million) I am doing it in batches, hence the row_number filtering. Currently the query analyzer says that a Clustered index scan is conducted on FIELD1. Still I would like better performance which is why I've tried indexing on the fields in the WHERE and ORDER BY clauses.

What I've tried so far:

Non-clustered indexes on

FIELD2 ASC,
FIELD3 ASC,
FIELD4 ASC,
ID ASC

And every possible permutation. The query execution time doubles and triples.

Why is this happening and what sort of index can I create to speed this up?

By the way, I'm running SQL Server 2005, so can't use filtered indexes. Compatibility level is 7.0.

Was it helpful?

Solution 2

Solution here was to ignore indexes on where and order by clauses and instead change the order by clause to "ORDER BY [Filter1].[FIELD2] ASC" where a clustered index already existed. In this way 100k rows were returned in 3 seconds. Ordering within the file was changed, however server-side paging was not affected.

OTHER TIPS

Hey I did some testing with dummy data on my local. I am making this recommendations considering you do not have any index on that table right now. or if you have please drop before you do any test with this suggestions.

1) [FIELD1] is not being used in any where filters. Make that Primary KEY as NON-CLUSTERED.

2) Now make Clustered index on columns (FIELD2 ASC,FIELD3 ASC,FIELD4 ASC)

3) You have ROW_NUMBER function ordered by on [ID] column. So use that [Row_Number] in outer order by clause instead [ID]

4) Change the Where Filters Order in the Query. Keep the [FIELD4] filter very first and use [FIELD2] and [FIELD3] filters after that.

5) If The Data Type of Columns [FIELD2] AND [FIELD3] is INT/NUMERIC/DATE then you can replace the "<>" operator with combination of "(> OR <=)".

6) If the Datatype of Columns [FIELD2] AND [FIELD3] is STRING then leave those WHERE filters with "<>" operator.

check these suggestions on SQL FIDDLE. This one does not have any data but explains above index and query suggestions

With above suggestions you will get the "Index Seek" and this should give you the good performance improvement. in my testing with dummy data of 2M row table it is returning 50k rows in 2 seconds in SSMS.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top