Hitting computer limitations?

Question

I wrote a simple application that builds a Concurrent Dictionary of 999 items, and then fire off 50 threads using the ThreadPool.SetMinThreads() method. I then loop through the 999 dictionary entries and update a record in the database to flag that an entry has been processed.

When running the application, I can see the threads have started and then I can run a SQL query to see the records being updated. So far, all of this is working rather well. When some of the initial threads have finished, the next batch of threads start (which is exactly what I want it to do). I can still see my records in the database are still being updated telling me that the application is working as expected. I can still see new threads being created and then I get a deadlock. When I look at the deadlock, it is from one of the initial 50 threads that started. This is where my question comes in.

I'm running the application on a 3Ghz dual core processor with 6Gb RAM. My SQL Server instance is also running on the same machine, but I wouldn't have thought this would have been a problem. The app is a proof of concept, but not being able to run 50 threads in a dev environment doesn't look promising. I know in the production environment the SQL instance will be on a seperate machine, as will the application. Any ideas?

Answer 1

A typical Windows install easily has 900 threads going. Check out Taskmgr.exe, Performance tab. The vast majority of them will be blocked waiting for something to happen. Adding another 50 won't put much of a dent on that. They won't execute much code, they'll be constantly waiting for the dbase server doing its job. As such, starting that many threads isn't very useful, perf of your code is entirely throttled by how fast the dbase engine can execute your queries. You can easily tell from Taskmgr.exe again, if the cpu load isn't 100% then adding more threads won't help.

No, deadlock is caused by code, not lack of resources. And keeping 50 balls in the air without dropping one is a very hard programming problem. Use the Debug + Windows + Threads debugging window to have an idea why a thread isn't making progress.

Answer 2

Database deadlock is possible if you update the same table from multiple threads. It seems that you need to rerun failed SQL statement and/or change locking level of your statements. It can be done using locking hints in the statement itself.