How to investigate why sql script that runs every day taking 2 min is taking 2 hours?

Question

My colleague asked me a question today

"I have a SQL script containing 4 select queries. I have been using it daily for more than a month but yesterday same query took 2 hours and I had to aborting execution."

His questions were

• Q1. What happened to this script on that day?
• Q2. How can I check of those 4 queries which of them got executed and which one culprit for abort?

My answer to Q2 was to use SQL profiler and check trace for Sql statement event.

For Q1: I asked few questions to him

1. What was the volume of data on that day? His answer: No change
2. Was there any change in indexing i.e. someone might have dropped indexing? His answer: No Change
3. Did it trapped in a deadlock by checking data management views to track it? His answer: Not in a deadlock

What else do you think I should have considered to ask? Can there be any other reason for this?

Since I didn't see the query so I can't paste it here.

Answer 1

Things to look at (SQL Server):

1. Statistics out of date? Has somebody run a large bulk insert operation? Run update statistics.

2. Change in indexing? If so, if it's a stored procedure, check the execution plan and/or recompile it...then check the execution plan again and correct any problems.

3. SQL Server caches execution plans. If you query is parameterized or uses if-then-else logic, the first time it runs, if the parameters are an edge case, the execution plan cached can work poorly for ordinary executions. You can read more about this...ah...feature at:

   • http://www.solidq.com/sqj/Pages/2011-April-Issue/Parameter-Sniffing-Problem-with-SQL-Server-Stored-Procedures.aspx

   • http://social.msdn.microsoft.com/Forums/en-US/transactsql/thread/88ff51a4-bfea-404c-a828-d50d25fa0f59

   • SQL poor stored procedure execution plan performance - parameter sniffing

Answer 2

In this case my approach would be:

1. Here is the case, he had to abort the execution because the query was taking more than expected time and finally it didn't complete. As per my understanding, there might be any blocking session/uncommitted transaction for the table you are querying(executed by any different user on the day). Since you were executing 'select' statement and as I know, 'select' statements used to wait for any other transactions to get completed(if the transaction executed before 'select'). Your query might be waiting for any other transaction to get completed(the transaction might have update/insert or delete). Check for the blocking session if any.

2. For a single session sql server switches between threads. You need to check either the thread containing your query is in 'suspended'/'running' or 'runnable' mode. In your case your query might be in suspended mode. Investigate in which mode the query is and why.

3. Next thing is fragmentation. Best practice is to have a index rebuild/reorganize job configured in your environment which helps to remove unnecessary fragmentation. So that your query will need to scan less amount of pages while returning data. Otherwise , your query will be taking more and more time for returning data. Configure the job and execute the job at least once in a week. It will keep refreshing your indexes and pages.

Answer 3

• Use EXPLAIN to analyze the four queries. That will tell you how the optimizer will be using indexes (or not using indexes).

• Add queries to the script to SELECT NOW() in between the statements, so you can measure how long each query took. You can also have MySQL do arithmetic for you, by storing NOW() into a session variable and then use TIMEDIFF() to calculate the difference between start and finish of the statement.

  SELECT NOW() INTO @start;
  SELECT SLEEP(5); -- or whatever query you need to measure
  SELECT TIMEDIFF(@start, NOW());
  

• @Scott suggests in his comment, use the slow query log to measure the time for long-running queries.

• Once you have identified the long-running query, use the query PROFILER while executing the query to see exactly where it's spending its time.