Improve the query

Question

I have two List. One with new objects, lets call it NewCol, and one that I have read out of my database(lets call it CurrentCol). The CurrentCol is about 1.2 million rows(and will grow daily) and the NewCol can be any thing from 1 to thousands of rows at a time. Can any one please help me improve the following code as it runs way too slow:

foreach (PortedNumberCollection element in NewCol)
{
    if (CurrentCol.AsParallel().Select(x => x.MSISDN).Contains(element.MSISDN))
    {
        CurrentCol.AsParallel().Where(y => y.MSISDN == element.MSISDN)
            .Select
                (x =>
                    {      
                       //if the MSISDN exists in both the old and new collection
                       // then update the old collection                                              
                       x.RouteAction = element.RouteAction;
                       x.RoutingLabel = element.RoutingLabel;
                       return x;
                    }
                 );
    }
    else
    {
        //The item does not exist in the old collection so add it
        CurrentCol.Add(element);
    }
}

Answer 1

It's a pretty bad idea to read the whole database into memory and search it there. Searching is what databases are optimized for. The best thing you can do is to move that whole code into the database somehow, for example by executing a MERGE statement for each item in NewCol.

---

However, if you can't do that, the next best thing would be to make CurrentCol a dictionary with MSISDN as the key:

// Somewhere... Only once, not every time you want to add new items
// In fact, there shouldn't be any CurrentCol, only the dictionary.
var currentItems = CurrentCol.ToDictionary(x => x.MSISDN, x => x);

// ...

foreach(var newItem in NewCol)
{
    PortedNumberCollection existingItem;
    if(currentItems.TryGetValue(newItem.MSISDN, out existingItem)
    {
        existingItem.RouteAction = newItem.RouteAction;
        existingItem.RoutingLabel = newItem.RoutingLabel;
    }
    else
    {
        currentItems.Add(newItem.MSISDN, newItem);
    }
}