Sharepoint List Threshold even when returning less then 5000 records

Question

I work for a company that creates software that interfaces with Sharepoint using Web Services. We have run into an issue at a few clients where we are hitting the threshold limit on document libraries that contain records greater then 5000, even though our query is only returning a few records. It was my understanding that you would only hit the limit if your query or view returned 5000 or more records.

Here's an example of a request using the lists.asmx web service.

var libID = "{97769110-C075-4BB8-93F3-8FD97A1C5AEB}";
var viewName = null;
var xnodeQuery = "<Query><Where><Eq><FieldRef Name=\"BATCH_NO\" /><Value Type=\"Text\">123456ABC12345</Value></Eq></Where><OrderBy><FieldRef Name=\"Created\" Ascending=\"False\" /></OrderBy></Query>";
var xnodeViewFields = "<ViewFields Properties=\"True\" />";
var xnodeQueryOptions = "<QueryOptions><IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns><DateInUtc>TRUE</DateInUtc><ViewAttributes Scope=\"Recursive\" /></QueryOptions>"
var xListItems = listService.GetListItems(libID, viewName, xnodeQuery, xnodeViewFields, "", xnodeQueryOptions, "");

Answer 1

Apart from improving performance, indexing may help in averting the list throttling issues if applied properly.

Suppose, you create a view and apply a filter on non indexed column and the view returns 4000 items and total number of items in a list is 7000. Normal users with non admin access will get throttling exception. This is because the view will try to scan all items (7000) in the list and then apply the filter. However, if the field is indexed, it will only scan 4000 items and users won't see throttling exception.

See this for more information: I have list contains 7000 item, and I need to show latest 10 items on the home page