Google's Custom Search allows you to create custom search engines based on Google's solid foundation of searching capabilities. A Google Custom Search Engine is an easy way to provide a search capability for your websites or blogs. Once created, the custom search engine can be consumed by adding a search box on any of the web pages or by linking to the home page of the custom search engine. At times, however, we may need some advanced way to consume our search engine. Say, for example, we want to programmatically perform search operations against our custom search engine or want to retrieve the search results in a Windows application. Fortunately Google has come up with JSON/ATOM Custom Search API, which can be used to accomplish just that. As the name suggests, the custom search API returns search results in JSON or ATOM format, depending upon your choice, and can be invoked using the Representational State Transfer (REST) programming model.

Though the JSON/ATOM Custom Search API is simple to use and primarily rely on GET requests, wrapping this API in a custom control would make it even easier to use. That way, without having to learn the JSON/ATOM Custom Search API, anyone can just drag and drop the custom control, set a few properties, and consume the custom search engine. In this article and the next in a two-part series, we are going to do just that.

Before You Begin

Note that as of this writing the JSON/ATOM Custom Search API is in Google Labs and, as such, is subject to change. Also note that to use this API you need to have the following things:

  • a Google Custom Search Engine. You can create one by visiting www.google.com/cse and following the step-by-step instructions found there. We won't cover those basics in this article.
  • an API key. Calling JSON/ATOM Custom Search API requires an API key. You can obtain one using Google API Console.
  • Every custom search engine is assigned a unique ID. You will need this ID while making REST calls.

Understanding Google's JSON/ATOM Custom Search API

As a control author we should be aware of how Google's JSON/ATOM Custom Search API works. From our code (client side or server side), we will be sending a GET request to the following URL:

https://www.googleapis.com/customsearch/v1?parameters

You can find the complete list of query string parameters in Google's online documentation for the JSON/Atom Custom Search API. We will be using the query string parameters listed in Figure 1 in our custom control.

Query string parameter Description
Figure 1: Query string parameters of Google’s REST based custom search API
key The API key as obtained from Google API console
cx The ID of the custom search engine that you wish to use.
alt Data format for search results. Possible values are json (default) and atom.
callback Name of a JavaScript function that will be called when search results are returned.
lr Language restriction for the search results. By default no restriction is placed.
safe Search safety level. Possible values are high, moderate and off (default). See online documentation for more details.
filter Governs turning on or off the duplicate content filter. Possible values are 0 (off) and 1 (on). Default is 1.
q The search criteria.

A typical client-side call to invoke the above REST service will look like this:

<script
src="https://www.googleapis.com/customsearch/v1?
key=XXXX&cx=XXXX&q=XXXX&callback=MyHandler">
</script>

Once the results are retrieved, the client-side callback function (MyHandler in above example) will be called, and the results in JSON/ATOM format are passed to it as a parameter. The complete details of the result format for JSON and ATOM can be found in the online documentation. For our example we need the pieces of data, as listed in Figure 2.

JSON object property ATOM tag name Description
Figure 2: List of JSON object properties and ATOM tags
title <title> The title of the search result item in plain text.
htmlTitle <title> The title of the search result item in HTML.
link <link> The full URL to which the search result item is pointing.
displayLink <link> A short version of search result’s full URL
snippet <summary> A short snippet from the search result item in plain text.
htmlSnippet <summary> A short snippet from the search result item in HTML.

Now that we know the basics of using the Google JSON/ATOM custom search API, let's focus on wrapping the API in an ASP.NET custom server control.

The Custom Control

We will create an ASP.NET custom server control (Google REST API Control) complete with designer support and a smart tag that allows us to consume the JSON/ATOM Custom Search API. The control will be invisible at runtime. Figures 3 and 4, respectively show the look and feel of the control at design time along with its smart tag.

Figure 3: Google REST API custom server control at design time

Figure 4: Smart-tag of Google REST API custom server control

The Google REST API custom control will have the properties and methods as listed in Figure 5.

Property / Method Description
Figure 5: Properties and methods of Google REST API control
Url Indicates the Google URL for making REST calls. Though this URL is fixed we still make it as a property to accommodate any changes at later time.
ApiKey An API Key required to use Google REST API.
CustomSearchEngineId A unique ID of the custom search engine that you wish to use for searching.
SearchCriteria Any search criteria string that you normally enter in search box.
LanguageFilter By default search returns matching content without any language restriction. You can restrict the results to a particular language using this property.
SafeSearchOption Governs safety level of the results. Possible values are Off, High and Medium.
FilterDuplicateResults By default duplicate content is filtered but you can change the default using this property.
SearchResultFormat The format in which search results are returned. Possible formats are JSON and ATOM.
ClientCallbackName If you wish to deal with search results from client side script rather than server side code then you can specify a JavaScript function that handles the results.
GetSearchResults() This method returns search results as a string. The string can contain JSON or ATOM data depending on the format specified using the SearchResultFormat property.
GetSearchResultsAsSyndicationItems() Returns search results as ATOM feed items.
GetClientScriptBlock() Returns a client side <script> block whose src attribute points to Google search service along with required query string parameters. Useful only if ClientCallbackName property is used.

Creating a Custom Server Control Project

First we need to create a new project of type "ASP.NET Server Control". Figure 6 shows the new project dialog of Visual Studio.

Figure 6: Creating a new ASP.NET Server Control project

Give the project a name as GoogleRestApiControlLib. Then define two enumerations—SearchResultFormats and SafeSearchOptions. The former enumeration represents the data format for search results, namely JSON and ATOM. The later enumeration indicates the safe-searching levels, namely High, Medium, and Off. Figure 7 shows these enumerations.

Figure 7: Defining required enumerations
public enum SearchResultFormats
{
 JSON,
 ATOM
}

public enum SafeSearchOptions
{
 High,
 Medium,
 Off
}

Now add a new class in the project and name it GoogleRestApiControl. The GoogleRestApiControl class represents the main custom control class and inherits from WebControl base class. Figure 8 shows the skeleton definition of GoogleRestApiControl.

Figure 8: Skeleton of GoogleRestApiControl class
namespace GoogleRestApiControlLib
{
 [DefaultProperty("ApiKey")]
 [ToolboxData("<{0}:GoogleRestApiControl
  runat=server></{0}:GoogleRestApiControl>")]
 public class GoogleRestApiControl:WebControl
 {
  ...
 }
}

Notice that the GoogleRestApiControl class is decorated with [DefaultProperty] and [ToolboxData] attributes. The [DefaultProperty] attribute specifies the name of the property that will be act as a default property for the control. Specifying ApiKey as the default property means that when you select the GoogleRestApiControl on the web form designer and open its Properties window, ApiKey property will be selected for editing by default. The [ToolboxData] attribute controls the server control tag name when the control is dragged and dropped from the Visual Studio toolbox.

The GoogleRestApiControl class has all the properties as listed in Figure 5. We won't discuss all the properties here since all of them essentially store or retrieve property value to and from ViewState. Figure 9 shows the code for several of the properties.

Figure 9: Some properties of GoogleRestApiControl class
[Bindable(true)]
[Category("API")]
public string ApiKey
{
 get
 {
  return ViewState["ApiKey"] as string;
 }
 set
 {
  ViewState["ApiKey"] = value;
 }
}
[Bindable(true)]
[Category("Search")]
public SearchResultFormats SearchResultFormat
{
get
 {
  if (ViewState["SearchResultFormat"]==null)
  {
   return SearchResultFormats.ATOM;
  }
  else
  {
   return (SearchResultFormats)
    ViewState["SearchResultFormat"];
  }
 }
 set
 {
  ViewState["SearchResultFormat"] = value;
 }
}
...

Notice the use of the [Bindable] and [Category] attributes. The former attribute indicates that a property can be used for binding, and the latter attribute specifies the category of the property. Both of these attributes are used to enhance design-time experience and don't affect the actual working of our control. For example, using [Category] attribute as shown in Figure 9 helps the Visual Studio Properties window to group the control properties, as shown in Figure 10.

Figure 10: Properties grouped based on [Category] attribute.

Searching Operation

Now that we have coded the properties of the GoogleRestApiControl, let's focus on implementing the actual searching operation. The three methods of GoogleRestApiControl control—GetSearchResults(), GetSearchResultsAsSyndicationItems(), and GetClientScriptBlock()—deal with the searching operation. They do so by making a REST call to the Google JSON/ATOM Custom Search API.

The GetSearchResults() method is intended to be used by our server-side code to search a custom search engine programmatically. Figure 11 shows the complete code for the GetSearchResults() method.

Figure 11: GetSearchResults() method
public string GetSearchResults()
{
 string targetUrl = this.Url +
  "?key=" + this.ApiKey +
  "&cx=" + this.CustomSearchEngineId +
  "&alt=" + this.SearchResultFormat.ToString() +
  "&q=" + HttpContext.Current.Server.UrlEncode(this.SearchCriteria) +
  "&filter=" + (this.FilterDuplicateResults ? 1 : 0);
 if (LanguageFilter != "" && LanguageFilter != null)
 {
  targetUrl += "&lr=" + LanguageFilter;
 }
 if (SafeSearchOption != SafeSearchOptions.Off)
 {
  targetUrl += "&safe=" + SafeSearchOption;
 }
 WebRequest request = null;
 WebResponse response = null;
 Stream responsestream = null;
 StreamReader reader = null;
 request = WebRequest.Create(targetUrl);
 response = request.GetResponse();
 responsestream = response.GetResponseStream();
 reader = new StreamReader(responsestream);
 string text = reader.ReadToEnd();
 reader.Close();
 responsestream.Close();
 return text;
}

The GetSearchResults() method first forms the URL as expected by Google's JSON/ATOM custom search API. Notice how various query string parameters are formed based on the property values. Especially notice the "q" query string parameter. Since we are writing this method in a class library, ASP.NET objects such as Server won't be available directly. We need to use the HttpContext class to access the Server object and then use UrlEncode() method to encode the search criteria.

The code then programmatically makes a GET request to the URL we just formed using Create() method of the WebRequest class. The response stream is accessed using GetResponseStream() method of the WebResponse class. Finally, the StreamReader class reads the complete response stream as a string using ReadToEnd() method. The string thus obtained is returned to the calling code.

The GetSearchResultsAsSyndicationItems() method uses the GetSearchResults() method discussed earlier and returns search results as a list of SyndicationItem objects. Figure 12 shows the complete code for this method.

Figure 12: GetSearchResultsAsSyndicationItems() method
public List<SyndicationItem> GetSearchResultsAsSyndicationItems()
{
 string results = GetSearchResults();
 byte[] byteArray = Encoding.ASCII.GetBytes(results);
 MemoryStream stream = new MemoryStream(byteArray);
 XmlTextReader reader = new XmlTextReader(stream);
 Atom10FeedFormatter atom = new Atom10FeedFormatter();
 atom.ReadFrom(reader);
 stream.Close();
 reader.Close();
 return atom.Feed.Items.ToList();
}

Note that the GetSearchResultsAsSyndicationItems() method will work as expected only when the search result format is ATOM. The method first retrieves the search results as a string using GetSearchResults() method. The Atom10FeedFormatter class found in the System.ServiceModel.Syndication namespace serializes ATOM feeds. The ReadFrom() method of the Atom10FeedFormatter class expects an XmlReader from which the data is retrieved. Since the GetSearchResults() method returns a string, we first put it into a MemoryStream and wrap it in an object of type XmlTextReader. The reader is then passed to the ReadFrom() method. The Feed property of Atom10FeedFormatter class exposes the Items collection. Each element of the Items collection is of type SyndicationItem. The ToList() method converts the elements of Items collection into a generic list of SyndicationItem objects.

The REST calls to the Google's JSON/ATOM custom search API can be invoked from client-side JavaScript, too. The GetClientScriptBlock() method essentially returns a JavaScript <script> block that can be emitted when the page is rendered (with or without post back). Figure 13 shows the complete code for the GetClientScriptBlock() method.

Figure 13: GetClientScriptBlock() method
public string GetClientScriptBlock()
{
 string targetUrl = this.Url +
 "?key=" + this.ApiKey +
 "&cx=" + this.CustomSearchEngineId +
 "&alt=" + this.SearchResultFormat.ToString() +
 "&q=" + HttpContext.Current.Server.UrlEncode(this.SearchCriteria) +
 "&filter=" + (this.FilterDuplicateResults ? 1 : 0) +
 "&callback=" + this.ClientCallbackName;
 if (LanguageFilter != "" && LanguageFilter != null)
 {
  targetUrl += "&lr=" + LanguageFilter;
 }
 if (SafeSearchOption != SafeSearchOptions.Off)
 {
  targetUrl += "&safe=" + SafeSearchOption;
 }
 string script = "<script type='text/javascript' src='" +
 targetUrl + "'></script>";
 return script;
}

Notice the lines at callouts A and B. The URL is almost identical to the previous methods we developed, but we have also added "callback" query string parameter. Since the <script> block will be executed on the client side, there must be some function that handles the returned search results. The "callback" query string parameter specifies the name of this client-side function. The code then forms a <script> element whose src attribute points to the URL we just formed. The <script> block is then returned to the calling code.

More to Come

This completes the core functionality of the GoogleRestApiControl control. In the upcoming part 2 article in this series, we will revisit the GoogleRestApiControl to add smart tag and a few other designer features. Check the DevProConnections website soon for part 2!

Bipin Joshi (bipinjoshi@yahoo.com) has been programming since 1995 and has worked with .NET since its inception. He has authored or co-authored several books and numerous articles about .NET technologies. Bipin blogs about yoga and technology at bipinjoshi.com.