XML Reference Guide

I hear a lot of complaining about SOAP, and about WSDL, about how it's too complicated, and too hard to use, and how the world would somehow be magically better if SOAP had never been invented. Look at Yahoo! Web Services, they say, they use REST, they're much simpler than Google's SOAP-based system.

Well, the complainers are right about one thing: Yahoo! Web Services does use REST. And if all you want is to request the data and look at it, then yes, it's simpler. But I like to do more than look at my data.

So having said all that, maybe you've got it in your head that using Google's API is going to be complicated, even difficult.

Well, you're wrong.

In fact, when I first set out to put together the sample application for this section, I'd intended for it to be a Java GUI application. I thought that might be a nice example of the fact that you can use these APIs to take data off the web. It wasn't long before I realized that actually using the API -- at least, the Java implementation of it that Google provides -- was only a tiny straightforward fraction of the code. Putting together the GUI was many times more complex in comparison.

So in the spirit of keeping things simple, let's build a small command-line application that requests the top 10 web pages for a particular query. Along the way we'll discuss the various options, and by the time we're through you'll have a good idea of how to incorporate this functionality into your own applications.

(You'll also see how easy it can be to use the kind of API you can generate from WSDL file. This API isn't directly generated, but it's the kind of thing that WSDL lets you do.)

First, let's talk about how this works in general terms. In order to display the items for a particular search, you need to execute these 5 steps:

1. Create a "search" object.
2. Set its credentials
3. Set the query you want to run
4. Execute the search
5. Examine each record returned by the search, retrieving the appropriate data

Let's see it in action. Before you start, make sure to download the Google API from http://www.google.com.

First, we'll create the basic application:

public class GoogleWatch {

   public static void main(String args[]){

        String keyword = args[0];
        System.out.println("You entered "+keyword);
 
   }

}

This application runs from the command line and takes a single argument, the query you want to run. (We'll talk more about the actual queries in a few moments.)

Next, we'll create the search and set it's credentials:

import com.google.soap.search.*;

public class GoogleWatch {

   public static void main(String args[]){

        String keyword = args[0];

        GoogleSearch siteSearch = new GoogleSearch();
        siteSearch.setKey("181gQtaOdlLD+83qvh6PnssQR5WlaB9d");

   }

}

Note that in order to compile and run this application, you will need to make sure that the classpath includes the googleapi.jar file downloaded with the SDK.

The actual key you're including with the setKey() method is your Google license key.. If you don't already have one, you'll need to go to http://www.google.com/apis and request one. This key is how Google limits you to 1000 queries a day.

Next we set the query:

import com.google.soap.search.*;

public class GoogleWatch {

   public static void main(String args[]){

        String keyword = args[0];

        GoogleSearch siteSearch = new GoogleSearch();
        siteSearch.setKey("181gQtaOdlLD+83qvh6PnssQR5WlaB9d");

        try {

           siteSearch.setQueryString(keyword);

        } catch (GoogleSearchFault f) {
           System.out.println("The call to the Google Web APIs failed:");
           System.out.println(f.toString());
        }


   }

}

Note that you don't need to learn a whole new means of building queries, the web services API uses the exact same queries the web interface does. So I could simply search for sites that reference "Darth Vader," with a simple query of "Darth Vader" or I can search for sites that link to "The Darth Side," a fantastic piece of fan fiction, with a query of:

link:http://darthside.blogspot.com/

I could even look for syndicated feeds referencing the wayward Jedi:

darth vader (filetype:rss OR filetype:atom OR filetype:rdf)

Notice the boolean-type rules still apply, even though we're not using the web interface.

Now we're ready to perform the actual search:

import com.google.soap.search.*;

public class GoogleWatch {

   public static void main(String args[]){

        String keyword = args[0];

        GoogleSearch siteSearch = new GoogleSearch();
        siteSearch.setKey("181gQtaOdlLD+83qvh6PnssQR5WlaB9d");

        try {
           siteSearch.setQueryString(keyword);
           GoogleSearchResult siteResults = siteSearch.doSearch();
        } catch (GoogleSearchFault f) {
           System.out.println("The call to the Google Web APIs failed:");
           System.out.println(f.toString());
        }


   }

}

The search returns a GoogleSearchResult object, which contains not just the results themselves, but also information about actual the search. For example, we can find out approximately how many results there are for a particular query:

import com.google.soap.search.*;

public class GoogleWatch {

   public static void main(String args[]){

        String keyword = args[0];

        GoogleSearch siteSearch = new GoogleSearch();
        siteSearch.setKey("181gQtaOdlLD+83qvh6PnssQR5WlaB9d");

        try {
            siteSearch.setQueryString(keyword);
            GoogleSearchResult siteResults = siteSearch.doSearch();
            int estimatedSites = siteResults.getEstimatedTotalResultsCount();
            System.out.println("There are "+estimatedSites+" matches for the query "+keyword.");
        } catch (GoogleSearchFault f) {
           System.out.println("The call to the Google Web APIs failed:");
           System.out.println(f.toString());
        }


   }

}

Note that although you can set the starting record for a search -- for example, siteSearch.setStart(50) - you can only retrieve results 1 through 1000. Anything farther down the list is not available through this interface.

The GoogleSearchResult object also holds other useful information, such as searchComments, and searchTips, which provide information intended for the end user, estimateIsExact, which is just what it sounds like, and directoryCategories, which is an array of Open Directory Project categories that match this particular search.

Then there are the results themselves. They are actually returned as an array of GoogleSearchResultElement objects, retrievable using the getResultElements(),method:

import com.google.soap.search.*;

public class GoogleWatch {

   public static void main(String args[]){

        String keyword = args[0];

        GoogleSearch siteSearch = new GoogleSearch();
        siteSearch.setKey("181gQtaOdlLD+83qvh6PnssQR5WlaB9d");

        try {
            siteSearch.setQueryString(keyword);
            GoogleSearchResult siteResults = siteSearch.doSearch();
            int estimatedSites = siteResults.getEstimatedTotalResultsCount();
            GoogleSearchResultElement[] sites = new GoogleSearchResultElement[10];
            sites = siteResults.getResultElements();
            if (sites.length == 0){
                System.out.println("No results found for query '"+keyword+"'");
            } else {
                for (int i = 0; i < sites.length; i++) {
                    GoogleSearchResultElement thisSite = sites[i];
                    String title = thisSite.getTitle();
                    String snippet = thisSite.getSnippet();
                    String url = thisSite.getURL();
                    System.out.println(title);
                    System.out.println(snippet);
                    System.out.println(url);
                    System.out.println("----------------------------");
                }
           }
        } catch (GoogleSearchFault f) {
           System.out.println("The call to the Google Web APIs failed:");
           System.out.println(f.toString());
        }


   }

}

For each result, we can pull specific information. For example, we've pulled the title, snippet, and url for each result. If we run the application, we'll see the results:

>java GoogleWatch "Darth Vader"
Star Wars: Databank | <b>Vader</b>, <b>Darth</b>
Character profile from the official Star Wars site.
http://www.starwars.com/databank/character/darthvader/
----------------------------
<b>Darth</b> <b>Vader</b> - Star Wars.com
The official site of David Prowse, the actor who played <b>Darth</b> <b>Vader</b

>. Includes<br>  character information, biography, and store.
http://www.darthvader-starwars.com/
----------------------------
STAR WARS: Injuries of <b>Darth</b> <b>Vader</b>
My father and <b>Darth</b> <b>Vader</b> fell into the crater and my father was i
nstantly killed.<br>  <b>...</b> At least one major fact has changed: Anakin Sky
walker and <b>Darth</b> <b>Vader</b> <b>...</b>

http://www.theforce.net/swtc/injuries.html
----------------------------
<b>Darth</b> <b>Vader</b> - Wikipedia, the free encyclopedia
<b>Darth</b> <b>Vader</b> was born Anakin Skywalker and is the father of twins L
uke Skywalker<br>  <b>...</b> He is given the name "<b>Darth</b> <b>Vader</
b>" in 19 BBY. Throughout the six films, <b>...</b>

http://en.wikipedia.org/wiki/Darth_Vader
----------------------------
WNC: <b>Darth</b> <b>Vader</b> Grotesque
<b>Darth</b> <b>Vader</b> Drawing Word of the competition was spread nationwide
through National<br>  Geographic World Magazine. The third-place winner was Chri
stopher Rader, <b>...</b>

http://www.cathedral.org/cathedral/discover/darth.shtml
----------------------------
Star Wars - Star Wars <b>Darth</b> <b>Vader</b> Voice Changer Mask
<b>...</b> speak into the helmetà your voice will sound just like <b>Darth</b> <

b>Vader</b>Æs! Quick tips<br>  to help achieve the best performance with the <b>
DARTH</b> <b>VADER</b> VOICE CHANGER <b>...</b>

http://www.hasbro.com/pl/page.viewproduct/product_id.15397/dn/starwars/default.c
fm
----------------------------
'boards - Screening Room
<b>Darth</b> <b>Vader's</b> heart of evil is no match for Hollywood executiv
es'. Credits:.<br>  Agency: Mother Creatives: Yan Elliot, Luke Williamson Pr
oducer: Angela Eleini <b>...</b>

http://www.boardsmag.com/screeningroom/commercials/1659/
----------------------------
The Morning News - <b>Darth</b> <b>Vader</b> Made Me Cry, by Matthew Baldwin
From the first moment you see <b>Darth</b> <b>Vader</b> choke that rebel fighter
, you know heÆs<br>  <b>...</b> <b>Darth</b> <b>Vader</b> was on the second or t
hird floorùI donÆt remember which, <b>...</b>

http://www.themorningnews.org/archives/stories/darth_vader_made_me_cry.php
----------------------------
AtomFilms - <b>Darth</b> <b>Vader's</b> Psychic Hotline
<b>Darth</b> <b>Vader's</b> Psychic Hotline United States | 4:59 | John E. H
udgens <b>...</b> One phone<br>  call solves them all with <b>Darth</b> <b>Vader

's</b> Psychic Hotline no matter whether <b>...</b>
http://www.atomfilms.com/af/content/darth_vaders_psychic
----------------------------
James Earl Jones
Voice of <b>Darth</b> <b>Vader</b>; "3rd Rock from the Sun" (1996) TV
Series (uncredited) .<br>  <b>...</b> <b>Darth</b> <b>Vader</b>; The Greatest Th
ing That Almost Happened (1977) (TV) . <b>...</b>

http://www.imdb.com/name/nm0000469/
----------------------------

Notice that the results include the "bold" tag (<b></b>) around the actual keyword, so if you were displaying this information on a web page, you could easily tell why the result was included. Other information available for each result includes:

• summary: the Open Directory Sumary, if there is one
• cachedSize: Google frequently saves a cached copy of pages it indexs. If there is such a copy, this property shows you how big it is
• relatedInformationPresent: True or false, this property tells you whether Google can provide related pages for this URL (via the related: search term)
• directoryCategory and directoryTitle: Information on how and where this URL is listed in the Open Directory Project, if applicable

Overall, the Google Web Service API is a great way to incorporate Google results into your web page or application, whether you use the API provided here or use another method, such as using XSLT to transform the data for your own purposes.