A handy technique that makes it easier to programmatically implement Google's Custom Search
Google's Custom Search allows you to create custom search engines based on Google's solid foundation of searching capabilities. A Google Custom Search Engine is an easy way to provide a search capability for your websites or blogs. Once created, the custom search engine can be consumed by adding a search box on any of the web pages or by linking to the home page of the custom search engine. At times, however, we may need some advanced way to consume our search engine. Say, for example, we want to programmatically perform search operations against our custom search engine or want to retrieve the search results in a Windows application. Fortunately Google has come up with JSON/ATOM Custom Search API, which can be used to accomplish just that. As the name suggests, the custom search API returns search results in JSON or ATOM format, depending upon your choice, and can be invoked using the Representational State Transfer (REST) programming model.
Though the JSON/ATOM Custom Search API is simple to use and primarily rely on GET requests, wrapping this API in a custom control would make it even easier to use. That way, without having to learn the JSON/ATOM Custom Search API, anyone can just drag and drop the custom control, set a few properties, and consume the custom search engine. In this article and the next in a two-part series, we are going to do just that.
Before You Begin
Note that as of this writing the JSON/ATOM Custom Search API is in Google Labs and, as such, is subject to change. Also note that to use this API you need to have the following things:
- a Google Custom Search Engine. You can create one by visiting www.google.com/cse and following the step-by-step instructions found there. We won't cover those basics in this article.
- an API key. Calling JSON/ATOM Custom Search API requires an API key. You can obtain one using Google API Console.
- Every custom search engine is assigned a unique ID. You will need this ID while making REST calls.
Understanding Google's JSON/ATOM Custom Search API
As a control author we should be aware of how Google's JSON/ATOM Custom Search API works. From our code (client side or server side), we will be sending a GET request to the following URL:
You can find the complete list of query string parameters in Google's online documentation for the JSON/Atom Custom Search API. We will be using the query string parameters listed in Figure 1 in our custom control.
Figure 1: Query string parameters of Google's REST-based custom search API
A typical client-side call to invoke the above REST service will look like this:
Once the results are retrieved, the client-side callback function (MyHandler in above example) will be called, and the results in JSON/ATOM format are passed to it as a parameter. The complete details of the result format for JSON and ATOM can be found in the online documentation. For our example we need the pieces of data, as listed in Figure 2.
Figure 2: List of JSON object properties and ATOM tags
Now that we know the basics of using the Google JSON/ATOM custom search API, let's focus on wrapping the API in an ASP.NET custom server control.
The Custom Control
We will create an ASP.NET custom server control (Google REST API Control) complete with designer support and a smart tag that allows us to consume the JSON/ATOM Custom Search API. The control will be invisible at runtime. Figures 3 and 4, respectively show the look and feel of the control at design time along with its smart tag.
Figure 3: Google REST API custom server control at design time
Figure 4: Smart-tag of Google REST API custom server control
The Google REST API custom control will have the properties and methods as listed in Figure 5.
Figure 5: Properties and methods of Google REST API control
Creating a Custom Server Control Project
First we need to create a new project of type "ASP.NET Server Control". Figure 6 shows the new project dialog of Visual Studio.
Figure 6: Creating a new ASP.NET Server Control project
Give the project a name as GoogleRestApiControlLib. Then define two enumerations—SearchResultFormats and SafeSearchOptions. The former enumeration represents the data format for search results, namely JSON and ATOM. The later enumeration indicates the safe-searching levels, namely High, Medium, and Off. Figure 7 shows these enumerations.
Now add a new class in the project and name it GoogleRestApiControl. The GoogleRestApiControl class represents the main custom control class and inherits from WebControl base class. Figure 8 shows the skeleton definition of GoogleRestApiControl.
Notice that the GoogleRestApiControl class is decorated with [DefaultProperty] and [ToolboxData] attributes. The [DefaultProperty] attribute specifies the name of the property that will be act as a default property for the control. Specifying ApiKey as the default property means that when you select the GoogleRestApiControl on the web form designer and open its Properties window, ApiKey property will be selected for editing by default. The [ToolboxData] attribute controls the server control tag name when the control is dragged and dropped from the Visual Studio toolbox.
The GoogleRestApiControl class has all the properties as listed in Figure 5. We won't discuss all the properties here since all of them essentially store or retrieve property value to and from ViewState. Figure 9 shows the code for several of the properties.
Notice the use of the [Bindable] and [Category] attributes. The former attribute indicates that a property can be used for binding, and the latter attribute specifies the category of the property. Both of these attributes are used to enhance design-time experience and don't affect the actual working of our control. For example, using [Category] attribute as shown in Figure 9 helps the Visual Studio Properties window to group the control properties, as shown in Figure 10.
Figure 10: Properties grouped based on [Category] attribute
Now that we have coded the properties of the GoogleRestApiControl, let's focus on implementing the actual searching operation. The three methods of GoogleRestApiControl control—GetSearchResults(), GetSearchResultsAsSyndicationItems(), and GetClientScriptBlock()—deal with the searching operation. They do so by making a REST call to the Google JSON/ATOM Custom Search API.
The GetSearchResults() method is intended to be used by our server-side code to search a custom search engine programmatically. Figure 11 shows the complete code for the GetSearchResults() method.
The GetSearchResults() method first forms the URL as expected by Google's JSON/ATOM custom search API. Notice how various query string parameters are formed based on the property values. Especially notice the "q" query string parameter. Since we are writing this method in a class library, ASP.NET objects such as Server won't be available directly. We need to use the HttpContext class to access the Server object and then use UrlEncode() method to encode the search criteria.
The code then programmatically makes a GET request to the URL we just formed using Create() method of the WebRequest class. The response stream is accessed using GetResponseStream() method of the WebResponse class. Finally, the StreamReader class reads the complete response stream as a string using ReadToEnd() method. The string thus obtained is returned to the calling code.
The GetSearchResultsAsSyndicationItems() method uses the GetSearchResults() method discussed earlier and returns search results as a list of SyndicationItem objects. Figure 12 shows the complete code for this method.
Note that the GetSearchResultsAsSyndicationItems() method will work as expected only when the search result format is ATOM. The method first retrieves the search results as a string using GetSearchResults() method. The Atom10FeedFormatter class found in the System.ServiceModel.Syndication namespace serializes ATOM feeds. The ReadFrom() method of the Atom10FeedFormatter class expects an XmlReader from which the data is retrieved. Since the GetSearchResults() method returns a string, we first put it into a MemoryStream and wrap it in an object of type XmlTextReader. The reader is then passed to the ReadFrom() method. The Feed property of Atom10FeedFormatter class exposes the Items collection. Each element of the Items collection is of type SyndicationItem. The ToList() method converts the elements of Items collection into a generic list of SyndicationItem objects.
Notice the lines at callouts A and B. The URL is almost identical to the previous methods we developed, but we have also added "callback" query string parameter. Since the <script> block will be executed on the client side, there must be some function that handles the returned search results. The "callback" query string parameter specifies the name of this client-side function. The code then forms a <script> element whose src attribute points to the URL we just formed. The <script> block is then returned to the calling code.
More to Come
This completes the core functionality of the GoogleRestApiControl control. In the upcoming part 2 article in this series, we will revisit the GoogleRestApiControl to add smart tag and a few other designer features. Check the DevProConnections website soon for part 2!
Bipin Joshi (firstname.lastname@example.org) has been programming since 1995 and has worked with .NET since its inception. He has authored or co-authored several books and numerous articles about .NET technologies. Bipin blogs about yoga and technology at bipinjoshi.com.