When SharePoint list with 5000+ items queried with ID returns expected result but with any other column gives threshold error

Question

I am trying to query a SharePoint list which has more than 5000 items. When I run filter query with ID, it returns the expected results. But when I run the same query with any other column it throws an error as shown below.

I know that there is a threshold limit of 5000. But I want to understand why am I getting the values when queried with ID column and not with any other column.

Answer 1

Add indexes to columns

Lets say it is sorting by category and filtering by status, then both those fields need to be indexed. It’s very easy to add something a column to the index.

• Go the list settings
• Click on the column "Indexed Columns"
• Click on Create a new Index
• Under primary column select the column you want.
• Click Create

Answer 2

The ID column is a built-in column (in table AllUserData) that is shared by all list items from all lists. In the SQL table it is naturally indexed in the SQL sense.

User-defined columns are not created as new SQL columns; instead, SharePoint uses one of the many generic columns available in the AllUserData table. There are a lot of these columns (~10 for text fields, ~10s for number fields, etc.) and they are not indexed in the SQL sense.

You have to explicitly tell SharePoint to index one of this column for a given list, so SharePoint copies values from them to a special table where it is indexed. See:

• https://www.dynatrace.com/blog/sharepoint-list-performance-how-list-column-indices-really-work-under-the-hood/ if you want more details on the indexing process.

• https://support.office.com/en-us/article/Add-an-index-to-a-SharePoint-column-f3f00554-b7dc-44d1-a2ed-d477eac463b0?ui=en-US&rs=en-US&ad=US for the detailed procedure to turn a SharePoint user column to an indexed column.

Answer 3

The reason behind working on ID column and throwing error on other column type is very simple and clear.

We aware about the fact that if we want to fetch data or filter data from list containing more than 5000 items, we will need to use indexed column.

And index of ID column is by default maintained by SharePoint. So if you need to filter data using other column except ID, you need to do indexing on it.

How to do indexing is already shown by Tarun. In addition to this I would like to share some more interesting fact about Indexing.

Not all column type supports indexing. The supported column type for indexing are:

• Single line of text
• Choice field (but not multichoice)
• Number
• Currency
• Date/ Time
• Lookup (but not multivalue)
• Person or Group (but not multivalue)
• Title (but not in a document library)

And the column which does not support indexing are:

• Multiple lines of text
• Hyperlink/Picture
• Custom Field Types
• Calculated Field
• Boolean (yes/no)
• UI version
• Checked out to
• Content type ID

So, please keep this things in mind while indexing any column. I found the reference here