Handling long Web Requests with Asynchronous Request Processing

Web Applications tend to be stateless, and running long requests can be problematic for Web backends.

Long-running requests can tie up valuable Web server connections and resources. In this article, Rick describes one approach that can be used to handle lengthy requests. A polling mechanism and an Event manager class can be used to pass messages between a Web application and a processing server running the actual long task.

When designing a Web application, the issue of how to handle long-running requests invariably comes up. Almost any Web application will have at least one or two backend or administrative tasks that take a fair amount of time to run. At first blush, you may think, “What's the big deal here? After all, Web Servers are multi-threaded and can run multiple requests simultaneously.” Well, in many cases this arrangement is still problematic, because of the resource use involved on the Web server.

The problems with long requests run directly off a Web application are many:

• Long requests can time out the Web server Web servers are set up with certain timeouts to kill connections after the timeout period is up. Typically, this value should be left small, and running a couple of long requests should not be sufficient reason to change this setting.
• Users don't see progress Browsers accessing your long request see no progress display. The request is handled by your Web backend, but there's no way to communicate the progress.
• Long requests tie up Web Server resources While users are waiting for the request to complete, they are using up a valuable HTTP connection, which is fairly resource intensive for a Web server to maintain.
• Local processing is often required to handle the request Because long requests processed by a Web app typically need to be handled on the Web server itself, this may overload the Web server's CPU resources. For example, running that long report with 10 users simultaneously on the Web server box may slow it to a crawl. A better way is to have one or more separate application servers handle the report processing.

The solution: Message-based application servers

There are a number of ways that these issues can be addressed, but most of them have a simple concept in common: The client application submits a request to the Web Server and the Web server passes off the request for actual processing to another application or application server. The Web app then checks back occasionally to see if the process has completed and, if it has, retrieves the result to send back to the client.

There are many ways that this can be implemented. One solid approach is to use Microsoft Message Queue (MSMQ) to submit messages into a queue, and have another application pick up the incoming request to process. The result is then returned in a response message that the Web application can poll for.

I wrote an implementation of this type of Async manager a while back, but it ended up being too complex for many people to implement and required Windows 2000 with Message Queue installed and configured properly.

So, I set out to create a simpler interface using a simple table-based event mechanism that accomplishes the same functionality in a much simpler interface wrapped in a class. The result is the wwAsyncWebRequest class, which I'll talk about here. Before I dig into the details of how the actual class implements this functionality, let's take a closer look at what's required to build a sophisticated async event processing manager.

Figure 1 - Asynchronous events require coordination of the Web browser, Web Server application and a backend application that actually runs the processing task. Note that the backend application can run either on the Web server or on a separate machine, providing a means of scalability through this process.

How it works

There are several components involved in this scenario. The client application running the browser that provides the user interface and basic process information; the Web Server application that provides the main Web processing; and finally, a backend application service/server that handles processing the actual long task in a separate process or even on a separate machine.

The Browser

The end user will be accessing some functionality over the Web. Typically, the user will initiate some operation such as running a long report. Once the user clicks the link or form button to start the operation, he'll get a result page back that says the request is still processing. This page is refreshed now and then to indicate progress by displaying some sort of updated status information. This progress can either be real progress as provided by the Application Server (more on that later), or something that the Web application simulates, such as an increasing number of dots or an animated gif that changes to give the user the impression that something is happening.

The browser can automatically refresh the page using the <META REFRESH> browser tag, which causes the page to reload.

The Web Server Application

The Web Server application is responsible for actually submitting the Async request to the event queue when the user clicks the submit button to start processing. It then also handles each of the requests from the client to see whether the process is complete. If not complete, another update page containing a META tag is sent back to the browser, saying that the process is still running. If it turns out that the process is complete, the Web application handler picks up the result value(s) and uses those values to build the final output that the user will see in his browser.

The format of data passed back and forth here is crucial. When a request is submitted and retrieved, the format needs to be agreed upon and it must be something that can be stored in a database table. In many cases, this will mean XML data inputs and outputs. The wwAsyncWebRequest class provides input and output data members as well as a simple property storage mechanism that makes it easy to pass this data between the Web application and the Application server.

The Backend Application

This is the actual application that handles the long request. In most scenarios, this will be a listener type application that looks for incoming requests in the event queue and picks up any events that are to be processed. The application then either processes this request itself or offloads processing to yet another application or server. The application server could be very, very simple and simply be directly fired from the Web server application via a CreateProcess or Run command. However, in most real world scenarios you probably have a listener application that polls the event queue for incoming requests and acts accordingly. For example, a generic handler might run any COM object via a SOAP request stored in the event queue.

Putting it all together

As you can see, there is a bit of interaction involved to make this happen. Running an asynchronous request is quite a bit more complex than running a normal request, as you have to coordinate the client side, the Web server and the backend application.

To encapsulate the most common functionality and make it easy to perform the tasks related to the managing the communication process, I built a Visual FoxPro class that I'll introduce here. The class handles event management via FoxPro or SQL Server tables that store information about each asynchronous event you want to fire. The basic concept is that of a queue where messages are passed in and returned out, either by specific message ID or in sequential first in/first out order.

At this point you might ask, why build a class like this when there are tools like MSMQ that handle queueing. There are a couple of reasons. Message queues tend to be somewhat limited in the amount of data they can provide about the messages that are sent. In particular one of the goals of the class I created is to provide inter-application communication so that the client process can ‘see’ what the backend application is doing, if so desired. Special fields in the event table make this very easy, where this same type offunctionality with message queues would require additional messages to be sent and retrieved. For this purpose, a table-based approach is actually much more flexible. For what it's worth, it's possible to subclass this class to use MSMQ behind the scenes, although this has not been provided yet. For now, classes for VFP and SQL Server are provided.

Introducing the wwAsyncRequest class

Let's take a look at the wwAsyncRequest class. The class provides a host of useful features that make it real easy to create message-based. The class provides:

• A table-based event manager that runs on VFP or SQL Server tables
• An easy object-based interface to post, retrieve and check for pending events
• Input and output properties for large data content
• A flexible XML-based property manager to pass data between client and server

To see how the class works let's start by looking at a simple example applet that demonstrates its use. The first step of the operation is the user clicking on a hyperlink to submit the request - in this case, a simulated query that generates an XML document to be returned to the browser. After the initial click, the user sees a page like the one shown in Figure 2.

Figure 2 - Once the request has been submitted, the browser ‘pings’ the server every few seconds for progress information. In this case, the server application even provides information for how much longer the request will run.

The first request that comes back will not show any status information, because the request has just been submitted. The URL for the first request is:

AsyncWebRequest.wwd

which simply sets up the request and submits it into the event queue. For this simplistic example, the Web application posts a SQL statement into the Property manager, posts the event, then starts up a separate EXE file to process the event by passing the event id to it. The external program will pick up the event and process it, while the Web server app simply returns a status page that has no update info from the Application server yet. This first page and all subsequent status pages include a refresh header at the top:

<html>
<head>
    <title>Running Report</title>
    <META HTTP-EQUIV="Refresh" CONTENT="4;URL=AsyncWebRequest.wwd?Action=Check&RequestId=0CU0QTCAG1836">
</head>
<body>
...

The <META> refresh tag forces the page to automatically reload, in this case after 4 seconds, and go the following URL:

The request Id identifies this particular event that we're tracking, and the action asks that we want to check for completion of the request. If the request is still pending, the same kind of page is displayed again, with this same <META> header to continue refreshing after each page.

Each time the check occurs, the Web application has the opportunity to show progress in some form. The wwAsyncWebRequest class provides several mechanisms to do this:

• A chkCounter property that keeps track of how many times the client checked for completion
• A Status property that can be updated by the application server

This example uses both of them. The line of dots you see in Figure 2 is lengthed each time the page is refreshed, representing the number of times we have checked for completion. The “Done in xx seconds” text is retrieved from the Status property that was set by the application server. In this case, the server knows how long the request will take, but the more common scenario will be to provide information about the stages of processing, like Running Accounting Report, Summarizing Totals and so on. Providing status strings with changing data is important to let the user know that his Web browser has not locked up and that he should not click refresh.

When the application server completes its task, it writes the result - in this case an XML document - into the ReturnData property of the object. This time, when the Web application checks for completion, it'll find the request completed and can pick up the value stored in the ReturnData field and send the XML document for display in the browser.

The Web server request

The following code demonstrates the Web application code, using West Wind Web Connection to perform the server-side task for this operation (note you can easily adapt this code to work with any implementation, including COM objects or plain ASP pages using the wwAsyncWebRequest object as a COM object):

************************************************************
* wwDemo :: AsyncWebRequest
****************************************
FUNCTION AsyncWebRequest

SET PROCEDURE TO wwAsyncWebRequest ADDITIVE
SET PROCEDURE TO wwXMLState ADDITIVE

*** Refresh page every 8 seconds
lnPageRefresh = 4
lnPageTimeout = 15   && try 15 times to get result

*** Choose SQL or VFP tables
#IF WWC_USE_SQL_SYSTEMFILES
    loAsync = CREATEOBJECT("wwSQLAsyncWebRequest")
    loAsync.Connect(SERVER.oSQL)
#ELSE
    loAsync = CREATEOBJECT("wwAsyncWebRequest")
#ENDIF

*** Retrieve ID and Action
lcId = Request.QueryString("RequestId")
lcAction = lower(Request.QueryString("Action"))
IF empty(lcAction)
    lcAction = "submit"
ENDIF

DO CASE
    *** Place the event
    CASE lcAction = "submit"
        *** Create new event, but don't save yet (.T. parm)
        lcId = loAsync.SubmitEvent(,"wwDemo TestEvent",.T.)
        loAsync.SetProperty("Report","CustList")
        loAsync.SetProperty("SQL","select * from tt_Cust")
        loAsync.SaveEvent()

        *** Run the demo Handler Server
        lcExe = FULLPATH("wwasyncwebrequesthandler.exe") + " " + lcID + IIF(WWC_USE_SQL_SYSTEMFILES," SQL","")
        RUN /n4 &lcEXE

    *** Check for completion
    CASE lcAction = "check"
        lnResult = loAsync.CheckForCompletion(lcID)
        DO CASE
            CASE lnResult = 1
                *** Display result - XML doc return here
                Response.ContentTypeHeader("text/xml")
                Response.Write(loAsync.oEvent.ReturnData)
                RETURN
            CASE lnResult = -2  && No Event found
                THIS.ErrorMsg("Invalid Event ID", "Couldn't find a matching event.")
                RETURN
            CASE lnResult = -1  && Cancelled
                THIS.ErrorMsg("Event Cancelled", "The event has been cancelled.")
                RETURN
        ENDCASE
        
    *** Cancel the Event by user
    CASE lcAction = "cancel"
        loAsync.CancelEvent(lcID)
        THIS.StandardPage("Async Request Cancelled")
        RETURN
ENDCASE

*** Check for timeout on the Event
IF loAsync.oEvent.chkCounter > lnPageTimeOut
    loAsync.CancelEvent(lcId)
    THIS.StandardPage("Sorry, this request timed out", "Timed out after " + TRANSFORM(lnPageTimeOut) + " requests...")
    RETURN
ENDIF

*** Create the waiting output page
lcBody = "<hr><b>Waiting for report to complete" + ;
    REPLICATE(". ",loAsync.oEvent.ChkCounter + 1) + "</b>" + ;
    IIF(!EMPTY(loAsync.oEvent.Status)," (" + loAsync.oEvent.Status + ")","") + ;
    "<hr><p>" + "This report,... <more text omitted here>"

*** Create the 'Waiting...' page. META refresh is generated
*** via the 4th and 5th parameters to refresh the page
THIS.StandardPage("Running Report",lcBody,,lnPageRefresh, "AsyncWebRequest.wwd?Action=Check&RequestId=" + lcId)
RETURN

There are two blocks of code that are important: The CASE statement with the handling of the Submit and Check actions, and the call to StandardPage(), which is responsible for generating the HTML for the refresh page. Web Connection's wwProcess::StandardPage() method includes support for <META> refresh via its 4th and 5th parameters by supplying the timeout value and URL, respectively.

The CASE statement's SUBMIT section demonstrates several features of the wwAsyncWebRequest class. SubmitEvent() is used to create a new event with which you can pass in a block of input data (XML inputs are often a great choice for this) and a title for the event. The final parameter of .T., in this case, says to not submit this event to the queue just yet, because we'll want to set a few additional properties. At this point, the event does not yet exist in the Event table.

The oAsync object has an oEvent member that maps to all the fields in the underlying Event table. So, you have oAsync.oEvent.InputData and oAsync.oEvent.ReturnData, for example. Other properties include status, chkCounter, userid, submitted, started, completed, expire, cancelled and a free form Properties field. The Properties property can be set with the Get/SetProperty methods of the oAsync object, as is shown in the example. These methods deal with XML-based keys that can be easily set and retrieved. You can assign data to any of these properties to control operation your event handler.

Once you've set up the object completely and you're ready to submit, you call oAsync.Save() to actually write/update the event record. In this case, the event record is written for the first time.

On the CHECK action in the CASE statement, the key method is CheckForCompletion(), which checks whether a specific event has finished processing, has timed or has been cancelled. This method returns a numeric value that identifies the current event status:

• 1 - Completed
• 0 - Still processing
• -1 - Cancelled
• -2 - Invalid Event Id

In the above example, the first check is made for completion and if the value is indeed 1 (completed), we simply retrieve the ReturnData property from the oAsync.oEvent member that is set by the CheckForCompletion() call. Here, we simply echo back the XML by writing it out to the browser. In a more real world scenario, you'd probably do something with the XML like write back to a cursor and perform further processing.

If the request is still processing (status = 0), then we simply fall through the CASE statement and let the StandardPage() call at the end of the request handle the display of the status page.

Check out how the Cancel operation is handled, as well. The call to CancelEvent() sets the Cancel flag on the object, which can then be picked up by the application server to potentially stop processing and abort. In this example, it works because the server-side code runs in a loop that can check for the cancel flag and simply get out. This is very powerful to allow users to abort operations.

The Application server code

As I mentioned above, the application server here is very basic and is primarily used to demonstrate the operations that the server would use to handle requests. In this example, the application server is simply launched from the Web Server application with a RUN command that passes the Event ID to it. The server then picks up the id, retrieves the inputs and goes off processing.

This is a specific handler, totally non-generic for this example, which happens to compile a single operation into a standalone EXE file. Here's the code for the simple procedural function that makes up the EXE file:

*** wwAsyncWebRequestHandler
LPARAMETERS lcID, lcSQL
#INCLUDE WCONNECT.H
LOCAL lcID

IF EMPTY(lcID)
    RETURN
ENDIF

SET EXCLUSIVE OFF
SET DELETED OFF
SET SAFETY OFF

SET PROCEDURE TO wwUtils ADDITIVE
SET PROCEDURE TO wwXMLState Additive
SET PROCEDURE TO wwAsyncWebRequest Additive
SET CLASSLIB TO wwXML ADDITIVE
SET CLASSLIB TO wwSQL Additive

*** Make sure we can see the event file
DO PATH WITH "wwdemo\"
DO PATH WITH ".."

IF !EMPTY(lcSQL)
    loAsync = CREATEOBJECT("wwSQLAsyncWebRequest")
    loAsync.Connect("driver={SQL Server};server=(local);database=WestWind;uid=sa;pwd=")
ELSE
    loAsync = CREATEOBJECT("wwAsyncWebRequest")
ENDIF

IF !loAsync.LoadEvent(lcID)
    RETURN
ENDIF

*** Update the Started Time Stamp
loAsync.oEvent.Started = DATETIME()
loAsync.SaveEvent()

FOR x=1 to 40
    lnSecsLeft = 40 - x
    WAIT WINDOW "Simulating long request taking " + TRANS(lnSecsLeft) + " seconds..."  TIMEOUT 1
    IF !loAsync.LoadEvent(lcID)  && Get latest data!
        WAIT WINDOW "Failed reading " + lcId NOWAIT
        LOOP
    ENDIF
    
    *** Check for cancellation
    IF loAsyn.oEvent.Cancelled
        RETURN && Just exit and get out
    ENDIF
    
    loAsync.oEvent.Status = "Done in " + TRANS (lnSecsLeft) + " secs"
    loAsync.SaveEvent()
ENDFOR

lcSQL = loAsync.GetProperty("SQL")

*** Run the SQL Statement
? lcSQL
&lcSQL INTO Cursor TTCustList

loXML = CREATEOBJECT("wwXML")
lcXML = loXML.CursorToXML()
lcXML = loXML.EncodeXML(lcXML)

*** Close out the request and pass the return data
*** into the ResultData property
loAsync.CompleteEvent(lcID,lcXML)

*** EXIT app
RETURN

The first thing that happens is that the event is loaded with LoadEvent(), which is the base method used to access an event by ID. To let the client know that the application has started, set the Started property and call SaveEvent() to write the updated data to the event table.

The actual long request here is totally simulated with a FOR loop and a timed WAIT window. Note the check for the Cancelled property inside the loop, in case we need to exit the operation. In this case, cancelling is quite easy because we are running in a loop that executes a certain number of times. Most other applications will have specific commands or database operations that take a long time to run, so checking for Cancelled will have to be sprinkled throughout the code, and the cancel operation may not be so immediate.

The FOR loop also handles updating the Status property with the number of remaining seconds for this request. Note the use of LoadEvent(), updating of properties, then calling SaveEvent() to update the table. You will want to call LoadEvent() to make sure you have a recent copy of the data, since the client can change some things like the Cancel flag and the chKCounter property. And, the client can also pass you additional information by using any of the properties provided on the oAsync.oEvent object.

The ‘actual’ task performed by this handler is to run a SQL statement that was stored into a property with SetProperty("SQL",lcSQLStatement) on the WebServer. Here we pull out that property with GetProperty(“SQL”) and run the SQL statement, convert the result to XML and set the ReturnData property with this result string. Use SaveEvent() to write the data, and this application server code is done.

As I mentioned above, that's a really simple handler that is very specific to this request. To write a more generic handler for more than one type of request, you can use wwAsyncWebRequest's GetNextEvent() method, which pulls the next waiting event out of the event queue:

DECLARE SLEEP IN WIN32API INTEGER
DO WHILE .T.
    IF !oAsync.GetNextEvent()
        Sleep(500)  && Wait half second
        LOOP
    ENDIF
    
    lcAction = oASync.GetProperty("Action")
    DO CASE
        CASE lcAction = "SQLQuery"
            ...
        CASE lcAction = "COMObject"
            ...
        CASE lcAction = "PRREPORT"
            ...
        CASE lcAction = "EXIT"
            EXIT
    ENDCASE
ENDDO

Or, this could run off a timer in a form. The handler can be totally generic. For example, it could be set up to pass SOAP messages to the server and, based on the SOAP message, the server could run the request and return the result, using the InputData and ReturnData properties to pass the SOAP packets around.

Taking a closer look at wwASyncWebRequest

The wwAsyncWebRequest class is built to be easy to use. It supports event data either in VFP or SQL Server tables. The SQL Server version of wwAsyncWebRequest uses a different subclass, actually:

*** In server startup code
Server.oSQL = CREATE("wwSQL")
Server.oSQL.Connect("DSN=westwind;uid=sa;pwd=")
...

*** In Process Code
oAsync = CREATE("wwSQLAsyncWebRequest")
oAsync.Connect(oSQL)
* oAysnc.Connect("DSN=westwind;uid=sa;pwd=")

Use a separate oSQL object if the connection to the database is to be persisted across Web requests.

The key methods of the object are LoadEvent and SaveEvent, which are low level and reused throughout the class's higher level methods like CheckForCompletion, GetNextEvent and CompleteEvent, which your code typically will call.

*** wwAsyncWebRequest :: LoadEvent ****************************************
***  Function: Loads a Event from the event table by ID
***      Pass: lcID
***    Return: .T. or .F.   oEvent set afterwards
************************************************************
FUNCTION LoadEvent(lcID)

    IF EMPTY(lcId)
        THIS.ErrorMsg("No ID passed")
        RETURN .F.
    ENDIF
    
    THIS.Open()
    
    IF lcID = "BLANK"
        SCATTER NAME THIS.oEvent MEMO BLANK
        THIS.oEvent.Expire = THIS.nDefaultExpire
        RETURN .T.
    ENDIF
    
    *** Force a refresh always
    REPLACE ID WITH ID
    
    lcID = PADR(lcID,FSIZE("ID"))
    LOCATE FOR ID = lcID
    IF FOUND()
        SCATTER NAME THIS.oEvent MEMO
        RETURN .T.
    ENDIF
    
    SCATTER NAME THIS.oEvent MEMO BLANK
    
    THIS.SetError("Event not found")
    RETURN .F.
ENDFUNC
*  wwAsyncWebRequest :: LoadEvent

*** wwAsyncWebRequest :: SaveEvent ****************************************
***  Function: Saves the currently open Event object
***      Pass: nothing
***    Return: .T. or .F.
************************************************************
FUNCTION SaveEvent
    LOCAL lcID
    
    lcID = THIS.oEvent.id
    
    THIS.Open()
    LOCATE FOR ID == lcID
    IF !FOUND()
        APPEND BLANK
    ENDIF
    
    GATHER NAME THIS.oEvent MEMO
    
    RETURN .T.
ENDFUNC
*  wwAsyncWebRequest :: SaveEvent

NOTE: Be sure to check out the sidebar, “VFP Data Update Problems” for an important “gotcha.”

You can see in these two methods that the oEvent member is key to the operation of this class. The member is created with a SCATTER NAME command, which creates an object with all of the fieldnames of the underlying table. The SQL version uses this same context, but retrieves the data from SQL Server via SQLExec statements. Special Update and Insert statement builder code creates auto-update code to write the object content back to the SQL database in the overridden class.

Most other methods make use of the LoadEvent and SaveEvent methods. For example, CheckForCompletion:

*** wwAsyncWebRequest :: CheckForCompletion *****************************************
***  Function: Checks to see if an event has been completed
***            and simply returns a result value of the status.
***    Assume: You can check oEvent for details
***      Pass: lcID
***    Return: 1 - Completed and oEvent is set
***            0 - Still running and oEvent is set
***           -1 - Cancelled and oEvent is set
***           -2 - Event ID is invalid
************************************************************
FUNCTION CheckForCompletion( lcID )

    *** Invalid Event ID
    IF !THIS.LoadEvent(lcID)
        THIS.SetError("Invalid Event ID")
        RETURN -2
    ENDIF
    
    *** Cancelled Event
    IF THIS.oEvent.Cancelled
        RETURN -1
    ENDIF
    
    *** Event is done
    IF THIS.oEvent.Completed > {01/01/1990}
        RETURN 1
    ENDIF
    
    *** Increase the number of checks
    THIS.oEvent.chkCounter = THIS.oEvent.chkCounter + 1
    THIS.SaveEvent()
    
    *** Still waiting
    RETURN 0
ENDFUNC
*  wwAsyncWebRequest :: CheckForCompletion

CheckForCompletion() loads an event, checks for various object settings, then updates the counter if still waiting for the server to complete the request.

The GetNextEvent() method is a little tricky in that it has to make sure that only one client retrieves an event at a single time. Using record locks, this is easy to accomplish:

*** wwAsyncWebRequest :: GetNextEvent ****************************************
***  Function: Sets the current event with the next event in the queue.
***    Assume: sets the oEvent member
***      Pass: Optinal - the type of event to look for
***    Return: .T. if oEvent was set. .F. if no events pending.
************************************************************
FUNCTION GetNextEvent
    LPARAMETERS lcType
    
    THIS.Open()
    
    IF EMPTY(lcType)
        lcType = "  "
    ENDIF

    DO WHILE .T.
        LOCATE FOR STARTED = { : } AND COMPLETED = { : } and TYPE = PADR(lcType,FSIZE("type"))
        IF FOUND()
            IF RLOCK()  && Make sure we can lock it
                SCATTER NAME THIS.oEvent MEMO
                THIS.oEvent.Started = DATETIME()
                REPLACE Started WITH THIS.oEvent.Started
                UNLOCK
                RETURN .T.  && Got an event
            ENDIF
        ELSE
            RETURN .F.  && No Events
        ENDIF
    ENDDO

RETURN .F.

The SQL Server version uses a Stored Procedure to perform this task, locking down a selected row via a SERIALIZABLE transaction.

Lots of uses - get to it!

In this article I've shown you the basic concepts behind building an asynchronous handler for Web requests. Asynchronous processing comes in handy in many places, and it also allows you a way to scale processing off to other machines. The class I provided can be used beyond these types of requests for any message-based scenario that needs to pass messages between two applications. The mechanism used for this class is generic and can be applied to a variety of applications.

So, take a look at how you can apply these concepts to your applications to improve performance, scalability and user experience. Let's get to it!