Long Polling Tomcat with Spring

"Ooh err Missus" as comedian Frankie Howerd would have said, but enough of British innuendo and double entendre because Long Polling Tomcat isn't some kind of sexual deviance with next door's moggy, it's a technique (or more of a hack) that's been developed as a result of Steve Jobs's refusal to support Adobe Flash Player on the iPhone and iPad. The thing is, using Flash Player as part of a web application was a really good way of supporting the Publish and Subscribe paradigm as it was able to cater for those scenarios that require live updates, such as live stock prices, news updates and changes to betting odds, whereas straight forward HTTP, with its request/reply paradigm, is a good way of supporting static pages. A good number of companies put a lot of effort in to developing applications that used Flash in order to provide their users with realtime data. When Apple announced that iOS would not support Adobe Flash they were left high and dry without an iPhone solution and to get back into the mobile market I imagine that a good number of them went for long polling.

So, what is a long poll? Well, it isn't a tall guy from Warsaw, the idea is to mimic the Publish and Subscribe pattern. The scenario goes like this:

1. The browser requests some data from the server.
2. The server doesn't have that data available and allows the request to hang.
3. Sometime later the response data is available and the server completes the request.
4. As soon as the browser receives the data, it displays it and then promptly requests an update.
5. The flow now loops back to point 2.

I suspect that the Guys at Spring aren't too keen on the term 'long poll' either as they refer to this technique more formally as asynchronous request processing

In looking at my Long Poll or Asynchronous Request Processing flow above you can probably guess what will happen to your server. Each time you force the server to wait for data to become available, you tie up some of its valuable resources. If your web site is popular and comes under heavy load, then the number of resources consumed waiting for updates rockets and consequentially your server may run out and crash.

In February and March I wrote a series of blogs on the Producer Consumer pattern and this seemed like the ideal situation where long polling would come in useful. If you've not read my Producer Consumer pattern blogs take a look here

In that original scenario I said that a "TV company sends a reporter to every game to feed live updates into a system and send them back to the studio. On arriving at the studio the updates will be placed in a queue before being displayed on the screen by a Teletype."

As times have changed the TV company wants to replace the old Teletype with a web application that displays the match updates in something like realtime.

In this new scenario the president of the TV company hires our friends at Agile Cowboys inc to sort out the update. To make things easier he gives them the source code for the Message, Match, and MatchReporter classes, which are reused in the new project. Agile Cowboys' CEO hires of a couple of new developers for the job: a specialist in JavaScript, JQuery, CSS and HTML to do the front end and a Java guy for the Spring MVC Webapp stuff.

The front end specialist comes up with the following JavaScript polling code:

var allow = true;
var startUrl;
var pollUrl;


function Poll() {

  this.start = function start(start, poll) {

    startUrl = start;
    pollUrl = poll;
   
    if (request) {
      request.abort(); // abort any pending request
    }

    // fire off the request to MatchUpdateController
    var request = $.ajax({
      url : startUrl,
      type : "get",
    });

    // This is jQuery 1.8+
    // callback handler that will be called on success
    request.done(function(reply) {

      console.log("Game on..." + reply);
      setInterval(function() {
        if (allow === true) {
          allow = false;
          getUpdate();
        }
      }, 500);
    });
   
    // callback handler that will be called on failure
    request.fail(function(jqXHR, textStatus, errorThrown) {
      // log the error to the console
      console.log("Start - the following error occured: " + textStatus, errorThrown);
    });
   
   
  };
 
  function getUpdate() {
   
    console.log("Okay let's go...");
                                 
    if (request) {
      request.abort();  // abort any pending request
    }
   
    // fire off the request to MatchUpdateController
    var request = $.ajax({
      url : pollUrl,
      type : "get",
    });

    // This is jQuery 1.8+
    // callback handler that will be called on success
    request.done(function(message) {
      console.log("Received a message");
     
      var update = getUpdate(message);
      $(update).insertAfter('#first_row');
    });
   
    function getUpdate(message) {

      var update = "<div class='span-4  prepend-2'>"
            + "<p class='update'>Time:</p>"
            + "</div>"
            + "<div class='span-3 border'>"
            + "<p id='time' class='update'>"
            + message.matchTime
            + "</p>"
            + "</div>"
            + "<div class='span-13 append-2 last' id='update-div'>"
            + "<p id='message' class='update'>"
            + message.messageText
            + "</p>"
            + "</div>";
      return update;
    };
   

    // callback handler that will be called on failure
    request.fail(function(jqXHR, textStatus, errorThrown) {
      // log the error to the console
      console.log("Polling - the following error occured: " + textStatus, errorThrown);
    });

    // callback handler that will be called regardless if the request failed or succeeded
    request.always(function() {
      allow = true;
    });
  }; 
};

The class, called Poll, has one method, start(), which takes two arguments. The first one is used by the browser to subscribe to the match update data feed, whilst the second is the URL that's used to poll the server for updates. This code is called from the JQuery ready(…) function.

$(document).ready(function() {
 
  var startUrl = "matchupdate/subscribe";
  var pollUrl = "matchupdate/simple";
  
  var poll = new Poll();
  poll.start(startUrl,pollUrl);
 });

When the start() method is called it makes an Ajax request to the server to subscribe to the match updates. When the server replies with a simple "OK" the request.done(…) handler starts a 1/2 second timer by calling setInterval(…) with an anonymous function as an argument. This function uses a simple flag 'allow' that if true allows the getUpdate() method to get called. The allow flag is then set to false to ensure that there are no reentrancy problems.

The getUpdate(…) function makes another Ajax call to the server using the second URL argument described above. This time the request.done(…) handler grabs the match update and converts it into HTML and inserts it after the 'first_row' div to display it on the screen.

Getting back to the scenario and the CEO of Agile Cowboys Inc wants to impress his new girlfriend, so he buys her a Porsche 911. Now he can't pay for it using his own cash as his wife will find out what's going on, so he pays for it with a chunk of the cash from the TV company deal. This means that he can only afford a graduate trainee to write the server side code. This graduate may be inexperienced, but he does reuse the Message, Match, and MatchReporter classes in order to provide match updates. Remember that a Queue and a Match are injected into the MatchReporter. When the MatchReporter.start() method is called, it loads the match and reads the update messages where it checks their timestamps and adds them to the queue at the appropriate moment. If you want to see the code for the MatchReporter, Match etc, take a look at the original blog.

The graduate trainee then creates a simple Spring match update controller

@Controller()
public class SimpleMatchUpdateController {

  private static final Logger logger = LoggerFactory.getLogger(SimpleMatchUpdateController.class);

  @Autowired
  private SimpleMatchUpdateService updateService;

  @RequestMapping(value = "/matchupdate/subscribe" + "", method = RequestMethod.GET)
  @ResponseBody
  public String start() {
    updateService.subscribe();
    return "OK";
  }

  /**
   * Get hold of the latest match report - when it arrives But in the process
   * hold on to server resources
   */
  @RequestMapping(value = "/matchupdate/simple", method = RequestMethod.GET)
  @ResponseBody
  public Message getUpdate() {

    Message message = updateService.getUpdate();
    logger.info("Got the next update in a really bad way: {}", message.getMessageText());
    return message;
  }
}

The SimpleMatchUpdateController contains two very simple methods. The first one, start(), simply calls the SimpleMatchUpdateService to subscribe to the match updates, whilst the second, getUpdate(), asks the SimpleMatchUpdateService for the next match update. Looking at this you can probably guess that all the work is done by the SimpleMatchUpdateService.

@Service("SimpleService")
public class SimpleMatchUpdateService {

  @Autowired
  @Qualifier("theQueue")
  private LinkedBlockingQueue<Message> queue;

  @Autowired
  @Qualifier("BillSkyes")
  private MatchReporter matchReporter;

  /**
   * Subscribe to a match
   */
  public void subscribe() {
    matchReporter.start();
  }

  /**
   *
   * Get hold of the latest match update
   */
  public Message getUpdate() {

    try {
      Message message = queue.take();
      return message;
    } catch (InterruptedException e) {
      throw new UpdateException("Cannot get latest update. " + e.getMessage(), e);
    }
  }

}

The SimpleMatchUpdateService also contains two methods. The first, subscribe(), tells the MatchReporter to start putting updates into the queue. The second, getUpdate(), removes the next update from the Queue and returns it to the browser as JSON for display.

So far so good; however, in this case the queue is implemented by an instance of LinkedBlockingQueue. This means that if there's no update available when the browser makes its request then the request thread will block in the queue.take() method, tying up a valuable server resources. When an update is available queue.take() returns and sends the Message to the browser. To the inexperienced graduate trainee all seems well and the code goes live. The following Saturday it's the start of the Football Premiership (soccer if you're in the US), one of the busiest weekends of the sporting calendar and a very large number of users want the latest info on the big game. Of course the server runs out of resources, is unable handle the load and constantly crashes. The president of the TV company is not too happy about this and summons the CEO of Agile Cowboys to his office. He makes it crystal clear that blood will flow if this problem is not fixed. The CEO of Agile Cowboys realises his mistake and, after an argument with his girlfriend, takes back the Porsche. He then emails a Java/Spring consultant and offers him the Porsche if he'll come and fix the code. The Spring consultant can't turn down such an offer and accepts. This is mainly because he knows that the Servlet 3 specification addresses this issue by allowing a ServletRequest to be put into asynchronous mode. This frees the server resources, but keeps the ServletResponse open, allowing some other third party thread to complete the processing. He also knows that the Guys at Spring have come up with a new technique in Spring 3.2 called the "Deferred Result" that's designed for these situations. In the meantime the Agile Cowboys CEO's ex-girlfriend, still upset about losing her Porsche, emails his wife telling her all about her husband's affair...

As this blog is turning into an episode of Dallas I think its time to end. So, will the code get fixed in time? Will the Java/Spring Consultant spend too much time driving his new Porsche? Will the CEO forgive his girlfriend? Will his wife divorce him? For the answers to these questions and more information on Spring's DeferredResult technique tune in next time…

You may have noticed that there's another HUGE hole in the sample code, namely that fact that there can only be one subscriber. As this is only sample code and I'm talking about long polling and not implementing Publish and Subscribe, the problem is rather 'off topic'. I may (or may not) fix it later.

---

The code that accompanies this blog is available on Github at: https://github.com/roghughe/captaindebug/tree/master/long-poll