So you need to do some work that is not a logical part of a web request and you are looking for options. Well, we got options and a lot of questions.

• Do you need it to process all the time. Will recycling the app pool cause you pain?
• Is it truly batch processing or processing a lot of items?
• Are you on shared hosting?
• Can you install a service?
• Will your processing demands spike periodically?
• Have more than one node?
• Can you put an executable on the box?
• Can you use the windows Task scheduler?
• Thought of moving to Azure or App engine?

Ok, all kidding aside, some of these questions have real impact on how good your batch processing job will perform and how often you will be babysitting it. For a simple site with a small batch processing need, I would use the old timer trick in a http module right in the web site.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

public class BatchModule : IHttpModule
{
  protected static Timer _timer = null;
  
  public void Init(HttpApplication application)
  {
      if (_timer == null)
      {
          var timerCallback = new TimerCallback(DoWork);
          _timer = new Timer(timerCallback, null, 10000, 60000);
      }
  }
  
  public void DoWork(object state)
  {
      //do something
  }
}

This technique has many issues and should only be used for the simplest things. Some issues include the app pool recycling and shutting down your batch processor. Others include having to make your processing code reentrant.

If I was asked today to do batch processing for a client, I would use a Windows Service with WCF backed by MSMQ or some other distributed asynchronous message system that would scale when nodes are added to the cluster. Even if this app was run on a single server, I would still use a local MSMQ queue to start. The atomic way that MSMQ distributes messages and the benefits of throttling, discreet work packets, dead lettering and scalability outweigh the initial complexity of the solution.

There are many platforms today that make batch jobs easier like Azure worker roles, AppEngine and Heroku, not to mention, cron on Unix platforms. I know about the Windows Scheduler and the pain of using it in production. I was hoping that Microsoft would add the worker role functionality to IIS and Asp.Net but it does not look like the cool app fabric features will be coming to IIS.