Skip to main content
Last updated August 09, 2011 17:20, by m_potociar
Feedicon  

Server-side asynchronous request processing

Discussion

Server-side asynchronous request processing can serve two main purposes:

  1. decouple a business operation execution from the container executor service, thus freeing the container worker threads and making them available for serving new client requests (e.g. Servlet 3.0, Atmosphere)
  2. support non-blocking request invocation on the caller (client) side (e.g. EJB 3.1)

For the purposes of JAX-RS API, providing support for decoupling the resource method execution from the container executor service is the main goal. Optionally, we may want to explore the API support for non-blocking request invocation (e.g. using HTTP 202 or 303 with a specific link to query the response from), but this is not in the primary scope of the JAX-RS server-side asynchronous request processing model proposal. Further discussions in this document will focus solely on the asynchronous resource method execution, which is typically used in the two main use cases:

  • UC1: computing the response takes a long time (typically involving multiple calls to other services and resources)
  • UC2: response is provided as a result of an asynchronous event

In order to accomplish the asynchronous HTTP request processing, the user needs to somehow tell the underlying infrastructure to keep the connection that delivered the request open even after the initial (target) request processing method has returned. Then later, from a different execution context, the user needs to be able to locate the dormant connection and use it for sending a response back to the client.

Async support in Servlet 3.0

The approach used in Servlet 3.0 evolves mostly around a programmatic API. I say mostly, because in order to enable async request processing, a user needs to declare the async processing support using @WebServlet(url="..." asyncSupported=true) annotation. Once this is done, all the subsequent steps are conveyed using a programmatic API, as illustrated by the servlet example.

First, the user invokes HttpServletRequest.startAsync(req, resp) to tell the Servlet container that the underlying connection should remain open when the doGet() method returns. This method also returns an instance of AsyncContext that primarily serves as a connection reference. User then handles the async request processing using the standard Java Concurrency API primitives. Once the response is available, the AsyncContext instance is used to route the response to the suspended connection using one of the AsyncContext API methods (in our example AsyncContext.dispatch(...)).

Async RESTful services with Atmosphere

Project Atmosphere is a framework providing asynchronous features for web components shielding the user from the underlying techniques it supports (Comet, Servlet 3.0, etc.). Atmosphere provides an integration module for Jersey which defines API for asynchronous request processing in JAX-RS resource classes. For simplicity, I will refer to this module as "Atmosphere" in the text from now on. In contrast with Servlet 3.0, Atmosphere heavily relies on annotations. There is a @Suspend annotation for marking a resource method that should not close the underlying connection after it returns. Then there are annotations for marking methods that are supposed to resume connections return responses - @Resume (for a method that resumes a single connection) and @Broadcast (for a method that typically resumes multiple connections using the same response). Additional features, such as resume message scheduling, resume message filters or whether the resumed connection is closed once the resume message are sent can also be further configured with the annotations. The example here illustrates a simple Atmosphere JAX-RS resource emulating a web-bases chat server.

A more fine-grained targeting of the resume message can be achieved by returning a Broadcastable from a method annotated with @Resume or @Broadcast. The important point however is, that Atmosphere implicitly uses the data returned by the resume/broadcast method as the ultimate resume message. This however leads to a questionable functionality coupling when the @Resume or @Broadcast are applied to JAX-RS annotated methods, a typical programming model. The same response is used both as an Atmosphere resume message as well as a JAX-RS http response.

Motivating Example

From the comparison between Servlet 3.0 and Atmosphere approaches, the Atmosphere approach seems more high-level and better aligned with the existing server-side JAX-RS API. At the same time, the final server-side async API for JAX-RS should tackle the problematic functionality coupling in a method response. While using the annotation when telling the framework to suspend the underlying connection seems like a viable choice, resuming a connection through a programmatic API seems superior to the annotation-based approach a it decouples the "resume" functionality from the data returned by the resuming method.

In order to support broadcasting, let's introduce a concept of an injectable AsyncRequestContext used to group all the suspended connections according to 3 scopes: application, URI and request.

  • Application scope: can access all suspended connections
  • URI scope: can access all suspended connections that originated on a specific URI
  • Request scope: can access a single connection object for a specific request

With the use of AsyncRequestContext, the above mentioned Atmosphere example can be easily rewritten to a more advanced topic-sensitive version. Alternatively, to suspend the request the @Suspend annotation can be replaced using async context, as illustrated in this version of the Chat Resource example. As for the long running operations, the async support might look like this. Notice that the return type of the startTask(...) method is void. The actual response type will be inferred from the resume message data.

Variation using a simplified API

It should be possible to replace the need for @AsyncScope by:

  • making the request scope implicit
  • using @Path annotation to specify the URI-scoped contexts
  • introducing specialised annotations for the less-common types of context scopes, e.g. @ApplicationScoped or @ResourceClassScoped

Also, since the AsyncRequestContext.suspend(...) method does not have to be related to a particular context instance and it's scope, because the scoping information is fully described by the request URI, the suspend(...) method can be made static.

With these changes, the examples above would read:

@Path("/chat/{topic}")
public class ChatResource {
    /**
     * Async context associated with a particular "/chat/{topic}". The
     * the value of {topic} is substituted with the value from each particular 
     * request.
     * For example, for a request to "/chat/topicA", the injected URI-scoped 
     * context would apply to all suspended requests to URI "/chat/topicA"
     */
    @Context @Path("/chat/{topic}") private AsyncRequestContext ctx;
    /**
     * Returns "Connected" as result of this request, but
     * suspends the connection instead of closing it.
     * The async context scope is not relevant for suspending.
     */
    @GET
    public String get() {
        AsyncRequestContext.suspend();
        return "Connected";
    }
    /**
     * Broadcast message to all connections suspended on the 
     * particular "/chat/{topic}" URI (e.g. "/chat/jaxrs") and 
     * returns "Message delivered" as result of this request.
     */
    @POST
    public String say(String msg) {
        ctx.broadcast(msg); 
        return "Message delivered";
    }    
}
As for the long running operations, the async support would look like this:
@Path("/task")
public class ServerTaskResource {
    /**
     * Async context associated with individual request.
     */
    @Context private AsyncRequestContext ctx;
    /**
     * Fork thread that uses ctx defined above. Suspends
     * the response using annotation.
     */
    @POST
    @Suspend
    public void startTask(String task) {
        Executors.newSingleThreadExecutor().submit(new Runnable() {
            @Override
            public void run() {
                try {
                    Thread.sleep(2000);
                } catch (InterruptedException e) {
                    // falls through
                }
                ctx.broadcast("Task completed", true);  // forces resume
            }
        });
    }
}

The following example shows how to scope broadcasted messages using properly defined URI namespace and multiple async context instances:

@Path("/chat")
public class ChatResource {
    @Context @Path("/chat/topics/{topic}") private AsyncRequestContext ctx;
    @Context @Path("/chat") private AsyncRequestContext syadminCtx;
    @GET
    @Path("topics/{topic}")
    public String listenOnTopic() {
        AsyncRequestContext.suspend();
        return "Connected";
    }
    @POST
    @Path("topics/{topic}")
    public String postToTopic(String msg) {
        ctx.broadcast(msg); 
        return "Message delivered";
    }    

    @POST
    @Path("adminConsole")
    public String adminMessage(String msg) {
        sysadminCtx.broadcast(msg); 
        return "Admin message delivered to all topics";
    }    
}

Solution Requirements

This section is designed to track the solution requirements based on the initial discussion above and any further discussion with EG.

  • R001: User must be able to suspend a request processing.
  • R002: User must be able to asynchronously locate the suspended request (connection) and resume it with a response.
  • R003: User must be able to resume multiple suspended request at once using a single response message (broadcasting).
    • Additional message restrictions may apply, such as serializability or cloneability
    • This requirement relates to RA001

Additional requirements, not considered for the final solution in JAX-RS 2.0 timeframe. These requirements should however be possible to implement by custom extensions and the proposed API should not inhibit any of these:

  • RA001: User must be able to send data to a suspended request without ultimately resuming the request and closing the underlying connection.

Open topics

Following is the list of the currently tracked open topics:

  • Auto-resuming connections
    • e.g. after N messages sent, via a timeout or some other mechanism
    • potential solution would be to specify this as part of the @Suspend annotation or AsyncRequestContext.suspend(...) method call
  • When doing HTTP streaming (i.e. sending multiple messages over a suspended connection like in chat), how should the traditional pipeline of filters/handlers and MBR/MBW be executed?
  • Should we suspend requests explicitly in the scope of a particular context, or implicitly on the request URI?
  • How to ensure the broadcasted messages are properly processed by the filters and handlers, that may be different for each resumed request?

References

 
 
Close
loading
Please Confirm
Close