what is the right way to spawn thread for database IO in asmx web service?

Question

I have a short lock guarded section in a method (that serves the request entirely) that makes all initializations (etc. log-related). So only 1 thread can be there at time. In this section I also load system data from database if not loaded. This is naturally executed only on 1st request and it does not matter it takes time and no threads can propagate since it's done only once (by dummy request).

    static public void LoadAllSystemData()
    {
        SystemData newData = new SystemData(); //own type (etc. Hashtables in Hashtables).
        LoadTables(ref newData);
        LoadClasses(ref newData);
        LoadAllSysDescrs(ref newData);
        LoadFatFields(ref newData);
        LoadAllFields(ref newData);
        _allData = newData;
    }

After the lock-guarded section the system data is accessed from concurrent threads only by reading and no locks are needed:

    static public Hashtable GetTables()
    {
        return _allData.Tables;
    }

Now the lock guarded section must have method that checks if system data is older than 24h and refresh it. If it done just by calling method (from lock guarded section) below that thread takes a long time and no other thread can enter the lock guarded section.

   static public void CheckStatus()
   {
       DateTime timeStamp = DateTime.Now;
       TimeSpan span = timeStamp.Subtract(_cacheTimeStamp);
       if (span.Hours >= 24)
       {
           LoadAllSystemData();
           _cacheTimeStamp = DateTime.Now;
       }
    }

My questions are:

1. How to spawn a non-threadpool thread best way to handle IO so the threadpool worker thread can propagate and all the threads spend minimum time in lock guarded section?

2. Is the _allData = newData; in LoadAllSystemData atomic? If it is, it feels the best way to implement that so GetXxx-methods like GetTables do not need any locking!

3. Is there any way to get LoadAllSystemData to be called before requests? For example on iisreset?

Thanks in advance for your answers!

Answer 1

Matti, you're asking multiple questions that point to the best structure for your application. I would summarize your questions as:

• How do I pre-load data needed by my service prior to handling any legitimate requests?
• How do I ensure the pre-loaded data is loaded "once"?
• How do I refresh the pre-loaded data on a schedule, i.e. every 24 hours?

Understanding the Asp.Net pipeline and event structure will help you understand the answers to these questions.

First, the Asp.Net pipeline provides a one-time execution region in Application_Start. This is fired once per application cycle. This would also fire in 'iisreset', which cycles every application for a given server. However, application cycles themselves will recycle on their own, based on their configuration. All of which are controlled through IIS settings.

Second, Asp.Net is a request system; it won't fire refresh events for you, nor can you use it by itself as a scheduler. You need an outside agent to act on that for you.

Here's what you could do:

1. Add your pre-loaded data routine to Application_Start.
2. Configure your web site's Application cycle settings for every 24 hours.

This would ensure your data is loaded once, via Application_Start. It would ensure your data is loaded prior to your site serving any requests. Last, it would ensure your data is refreshed every 24 hours.

Here's what I would do:

1. Add pre-loaded data routine to Application_Start.
2. Modify pre-loaded data routine to use Asp.Net Cache, instead of static usage.
3. Modify lookup data to retrieve data from cache, and re-load if data is not found.
4. Provide capability for diagnostic/monitoring system, i.e. nagios, to refresh data asynchronously via web method call, i.e. "preload_refresh()".

This would provide essentially the same forward effect as the above solution, but with better reliability for your service.

As always, your mileage may vary. Hope this helps.

Answer 2

To answer part of your question anyway, you can use the "Application_Start" method in the Global.asax to execute the statement on the start of the application if you want to pre-fill the data as soon as the application comes online.