Interacting With The Web: The XMLHttpRequest Object

This article was written and submitted by an external developer. The Google Desktop Team thanks Benjamin Schirmer for his time and expertise.

Benjamin Schirmer, Gadget Developer
October 2007

• Introduction
• The XMLHttpRequest Object
• Creating Synchronous Requests
• Creating Asynchronous Requests
• How To Use The Response Data
• Use Scenario: Loading Image With ResponseStream
• Extended Use Scenario: Uploading Files
• Conclusion
• Author Bio

Introduction

As you've surely seen, Google Desktop gadgets are capable of delivering website content straight to the user's desktop. Whether gathering the latest news or the user's online data, these gadgets need a way to communicate over the Internet. This is where the XMLHttpRequest object comes in. It allows you to easily establish a connection with a website to request or send information. This article is an in-depth look at all the functionality that XMLHttpRequest provides.

The XMLHttpRequest Object

Before we start, you should be familiar with the attributes and methods of the XMLHttpRequest object:

Let's start with the attributes. They are mostly used to inquire the status of pending requests and the XMLHttpRequest object itself.

readyState

The readyState attribute tells you the state of the XMLHttpRequest object. The possible values for this attribute are:

• 0 - The XMLHttpRequest object is uninitialized
• 1 - open() was called successfully
• 2 - The request was successfully sent
• 3 - Currently receiving data
• 4 - The request completed successfully

onreadystatechange

This attribute is meant to be set to a callback function. The function will be called whenever the readyState of the XMLHttpRequest object changes. If you are making a lengthy request, this function may be called multiple times with a readyState of 3. As soon as the call completes, the callback function will be called with a readyState of 4.

status

If the readyState of the XMLHttpRequest object is 3 (receiving) or 4 (loaded), this attribute will contain the HTTP status code for the request. For example, if the request finished successfully, this is usually 200.

statusText

Similar to the status attribute, but it will contain the current HTTP status text. For example, status code 404 usually responds with "Not Found" and 200 returns "OK".

responseXML

If the server responded with a document in the XML format, you can use the responseXML attribute to parse the XML document tree. Check out the Simple XML Parsing article if you need help parsing XML.

responseText

The responseText attribute is a string containing the entire document returned by the request.

responseStream

If you are requesting binary data, then the responseStream attribute contains the response data in a binary stream.

The XMLHttpRequest object provides a few powerful methods for managing requests.

One of the most important methods is open(), which initializes the XMLHttpRequest object. The parameters you send to this method are essential for controlling the behavior of the XMLHttpRequest object.
The open() method has a total of 5 parameters, but only the first two are required.

request method (required)

The first parameter is the HTTP method you want to use for the request. The most common ones are "GET" or "POST", but other methods exist such as "HEAD", "PUT", or "DELETE".

complete URI (required)

The second parameter is the complete URI to the page where the request is sent. You can not use a relative path. This URI must be absolute and must be resolvable.

asynchronous (optional)

This is an optional boolean parameter. If you set this parameter to true, the request is asynchronous and will not block. If false, the request will be synchronous and will not return until it completes.

username (optional)

If the webpage uses HTTP Authentication (Figure 1), you use this parameter to provide the username credential.

password (optional)

The password parameter is usually used together with the username parameter for HTTP Authentication (Figure 1). It provides the password credential to the page.

The send() method executes the request. It accepts one optional parameter, which can be a string or a document object. The parameter is only necessary when you want to transmit information in the request, which is usually the case for POST or PUT requests. The structure of this transmitted data varies based on the selected HTTP request method. There will be an example of this later on.

If you are using a synchronous request, the send() method returns only after the request completes. This may freeze your gadget if a request takes too long. You should always try to use an asynchronous request.

If your request requires special headers, you can set them using the setRequestHeader method. This method has two required parameters. The first one is the name of the HTTP header, and the second parameter is the value.

req.setRequestHeader("content-type", "text/xml");

When your request has finished, you can read the response headers in addition to the content of the document. To access the headers, you can use getAllResponseHeaders(), which returns all the headers in one string. Another method, getResponseHeader(), allows you to request the content for a specific header:

var contentType = req.getRequestHeader("content-type");

The XMLHttpRequest object also provides an abort() method. You can use this method to cancel a request.

abort() can only be used in asynchronous requests (remember a synchronous request blocks).

Creating Synchronous Requests

You now should be familiar with all the attributes and methods the XMLHttpRequest object provides, but how do you use them? This section will show you how to create simple, synchronous requests using the XMLHttpRequest object. Let's start with a very basic request:

var req = new XMLHttpRequest();
req.open("GET", "http://www.google.com/webhp", false);  // The last parameter determines whether the request is asynchronous.
req.send();

if (req.status == 200) {
  gadget.debug.trace("Google Webpage Loaded");
}

Example 1: Basic synchronous GET request

This small example doesn't do much. It opens a connection to http://www.google.com/webhp and loads the webpage. If it finishes correctly, it displays the text "Google Webpage Loaded" in the debug console of the Google Desktop Gadget Designer. Now let's add a query parameter to our request. The Google website uses the parameter q for the search query. We send a query for "google desktop gadgets":

var query = "google desktop gadgets";
var req = new XMLHttpRequest();
req.open("GET", "http://www.google.com/webhp?hl=en&q="+escape(query), false);
req.send();

if (req.status == 200) {
  gadget.debug.trace("Search Completed");
}

Example 2: Synchronous GET request with parameters

Since this is a GET request, the parameters are appended to the URL. For URL escaping, we pass the query value through the JavaScript escape function. When the request completes, the gadget shows the response text in the debug console. I have also added a parameter to specify English language content. As you can see, the first parameter is delimited with a ?, while additional parameters are separated by &. Keep in mind that there is a restriction on the length of the URI. If you want to submit significant amounts of data, you must use a POST request. Let's pretend Google accepts the POST method. The request would look like this:

var query = "google desktop gadgets";
var req = new XMLHttpRequest();
req.open("POST", "http://www.google.com/webhp", false);
req.send("&hl=en&q="+escape(query));

if (req.status == 200) {
  gadget.debug.trace("Search Completed");
}

Example 3: Synchronous POST request with parameters

As promised earlier, this is an example of the send() method with post data. To assemble the data, you just create a string combining all the "key=value" pairs using an &. There is usually no limit to the length of the post string. You can even use it to transmit lengthy blog posts or computer files. Your only limiting factor is time, as a large upload might take a long while. If the request is synchronous, execution will halt until the request completes. To state the obvious, this would greatly impact the usability of your gadget. This is where asynchronous requests come in.

Creating Asynchronous Requests

Asynchronous requests, as you probably figured out by now, do not stop the code execution. Figure 2 shows a sequence diagram. The top sequence is of a synchronous request while the lower one demonstrates an asynchronous request. In the synchronous example, the requestData() function runs until send() has completely finished and returned. For an asynchronous request, the requestData() method returns immediately after the request is sent out. The XMLHttpRequest object will then notify the callback function assigned to onreadystatechange whenever the state of the request changes. It thereby notifies us as soon as our request is completed (when the readyState is 4 - remember?)

For a very basic asynchronous example we create a new request based on Example 1.

var req = new XMLHttpRequest();
req.open("GET", "http://www.google.com/webhp", true);
req.onreadystatechange = function() {
    if (req.readyState == 4) {
      if (req.status == 200) {
        gadget.debug.trace("Google Webpage Loaded");
      }
    }
  };
req.send();

Example 4: Basic asynchronous GET request

How To Use The Response Data

In Example 4 we have assigned an anonymous function to the onreadystatechange attribute. This function will be called whenever the readyState changes. We first check if the readyState is indeed 4, indicating that the request has loaded. Then we check if the status is "OK" (status code 200). In this very basic example, we display a trace message in the debug log.

Let's send out a request to Google and count how many results a query returns.

function sendRequest() {
  var req = new XMLHttpRequest();
  req.open("GET", "http://www.google.com/search?hl=en&q=benjamin+schirmer", true);
  req.onreadystatechange = function() {
      if (req.readyState == 4) {
        if (req.status == 200) {
          printResultCount( req.responseText );
        }
      }
    };
  req.send();
}

function printResultCount(data) {
  var str = data.match(/of about <b>(.+?)<\/b> for <b>(.+?)<\/b>/i);
  gadget.debug.trace( str[1]+" results found" );
}

Example 5: Asynchronous GET request to Google which displays result count

Since responseText is a regular string object, you can use all the usual JavaScript string functions. Using the responseText attribute is only one of several ways to access the response data. If the request was made to XML data, such as an RSS or ATOM feed, you can use the Simple XML Parser to parse the data into JavaScript objects.

Another alternative is responseStream, which holds the binary data of the request. This is useful when loading binary files such as images.

Use Scenario: Loading Image With ResponseStream

Using responseStream to load an image is very simple.

function ajaxPicture(url)
{
  var req = new XMLHttpRequest();
  req.open("GET", url, false); 
  req.send(null);
  if(req.status == 200) {
    return req.responseStream;
  }
}

Example 6: Function to load the image from an URL

The function ajaxPicture() creates a GET request to the specified URL and returns the responseStream. To use this function, we call it with the URL of our image and assign the returned image to the src attribute of our image.

  image.src = ajaxPicture( "http://www.google.com/intl/en_ALL/images/logo.gif" );

Extended Use Scenario: Uploading Files

Another use case for the XMLHttpRequest object is uploading a file to a website. This is a bit more complex since it requires custom request headers and a specially formatted post string. Let's tackle this step by step.

The very first helper function loads a file into a string. This example works with a text file, so we can get by with the filesystem object.

function loadFile( filename ) {
  var fp = system.filesystem.OpenTextFile( filename, 1, false);
  var s = fp.ReadAll();
  fp.close();
  return s;
}

The next routine is called uploadFile and has two parameters: the URL of the page that receives the upload and the complete path to the local file.

function uploadFile( url, filename ) {
  var boundaryString = "AaBbCcX30";
  var boundary = "--"+boundaryString;

  var fileData = loadFile( filename );

  var postContent = "\r\n"+boundary+"\r\n"+
          "Content-Disposition: form-data; name=\"comment\"\r\n"+
          "\r\n"+
          "Comment is another input\r\n"+
          boundary+"\r\n"+
          "Content-Disposition: file; name=\"uploadedfile\"; filename=\"test.txt\"\r\n"+
          "Content-Type: text/plain\r\n"+
          "\r\n"+
          "%FILECONTENT%\r\n"+
          boundary+"\r\n";
  postContent = postContent.replace("%FILECONTENT%", fileData);
  gadget.debug.trace( postContent );

  var req = new XMLHttpRequest();
  req.open("POST", url, true); 
  req.setRequestHeader("Content-Type", "multipart/form-data; boundary=" + boundaryString);
  req.onreadystatechange = function() {
    gadget.debug.trace("ReadyState: "+req.readyState);
    if (req.readyState == 4) {
      if (req.status == 200) {
        gadget.debug.trace( req.responseText );
      }
    }
  };
  req.send(postContent);
}

Example 7: Uploading a text file to a server

The uploadFile method first assembles the boundary string. This string will be used to separate the post string into different sections. Our example will create a post string containing a comment field and the content of the uploaded file. Notice I used a placeholder called %FILECONTENT%, which is later filled using JavaScript's replace. The boundary string is also sent and indicated to the server in the content-type header. This header specifies the content type of the data. In our case it is multipart/form-data as that is the standard for file uploads on the Web. If you were to send an XML file, the content type would be text/xml.

Conclusion

As you can see, the XMLHttpRequest object is a very useful and versatile tool. You can use it to retrieve simple information from the Web or to perform complex tasks such as file uploads or SOAP communication. I hope this article has helped you in understanding how the XMLHttpRequest object works.

Author Bio

Benjamin Schirmer holds a diploma in engineering from Albstadt-Sigmaringen University and is an enthusiastic gadget programmer. He loves to explore new technologies. In his spare time, he likes watching movies, TV series, and going out with his friends. In the future, he wants to be a Googler himself. You can visit his gadgets page for a list of all his cool Google Desktop gadgets.

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.