Having neato “single-page” websites using ajax navigation is the fun thing to do with websites lately, but it can pose a problem with search-engine indexing and SEO. The problem stems from search engines not executing JavaScript on webpages, and because the JavaScript won’t run, your pages won’t be indexed.

But never fear, Google came up with a scheme to index pages that are normally hidden behind an ajax request. To summarize, by placing a hashbang (#!) in your URL, Google will swap that with a query string identifier of ?_escaped_fragment_= when trying to crawl your pages.

For example, let’s take the Account Registration page built into ASP.NET MVC 3. When creating a new project, on the home page there exists a link to this page: http://mysite.com/account/register. Now, when converting the site to be a single-page application (using ajax to navigate), we need to change the URL that retrieves this page to http://mysite.com/#!/account/register. You then use some sort of JavaScript solution (like Ben Alman’s jQuery BBQ plugin) to handle requests based on the hashchange event.

When Google tries to index the Account Registration page, it will change the URL (since the browser will not send the URL hash to the server for processing) to this: http://mysite.com/?_escaped_fragment_=/account/register. They try to follow the URL, and now it’s up to the server to fetch and return the full page to Google for indexing. I will show you my solution to solving this in ASP.NET MVC 3.

With Global Action Filters in MVC, we have the ability to intercept all responses before it goes to a Controller Action for execution. By doing this, we can check to see if Google’s fragment exists in the URL and reroute as needed. We will start by creating a new class:

1
2
3

public class AjaxCrawlableAttribute : ActionFilterAttribute
{
}

By inheriting from ActionFilterAttribute, we are now able to register our class as a Global Action Filter in Global.asax, but we will handle that later. Let’s keep going by intercepting some requests:

1
2
3
4
5
6

public class AjaxCrawlableAttribute : ActionFilterAttribute
{
  public override void OnActionExecuting(ActionExecutingContext filterContext)
  {
  }
}

By overriding the OnActionExecuting method, we are telling ASP.NET we want to intercept the request BEFORE the request is sent to the Controller for processing. Now we can start to work on the magic:

1
2
3
4
5
6
7
8
9
10
11
12

public class AjaxCrawlableAttribute : ActionFilterAttribute
{
  private const string Fragment = "_escaped_fragment_";

  public override void OnActionExecuting(ActionExecutingContext filterContext)
  {
      var request = filterContext.RequestContext.HttpContext.Request;

      if (string.IsNullOrWhiteSpace(request.QueryString[Fragment]))
          return;
  }
}

First up, we grab the current Request and check the query string to see if it contains Google’s magical fragment identifier. Since the method is going to execute on every request (even child requests), we want to exit this method the moment we know we are not dealing with a search engine.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

public class AjaxCrawlableAttribute : ActionFilterAttribute
{
  private const string Fragment = "_escaped_fragment_";

  public override void OnActionExecuting(ActionExecutingContext filterContext)
  {
      var request = filterContext.RequestContext.HttpContext.Request;

      if (string.IsNullOrWhiteSpace(request.QueryString[Fragment]))
          return;

      var parts = request.QueryString[Fragment].Split(new[] {'/'}, StringSplitOptions.RemoveEmptyEntries);
      var routeValues = new AjaxRoute();

      if (parts.Length > 0)
          routeValues.Controller = parts[0];

      if (parts.Length > 1)
          routeValues.Action = parts[1];

      if (parts.Length > 2)
          routeValues.Id = parts[2];
  }
}

Adding a little more code here, I take the value of the query string parameter and split it on / so that we can have the parts of the URL that determine what action to go to. I have created another class, AjaxRoute, to be a container of these values:

1
2
3
4
5
6

public class AjaxRoute
{
  public string Controller { get; set; }
  public string Action { get; set; }
  public string Id { get; set; }
}

This class has very specific properties, because the names of the properties will be used to determine what route values we need for the routing. Since for this example we are only using the default MVC route, these are the only route parameters we need to set in order to get to the right Controller Action.

To finish this implementation off, we only need one more line, and it’s the line that tells MVC to reroute to the controller matching the route values we have set:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

public class AjaxCrawlableAttribute : ActionFilterAttribute
{
  private const string Fragment = "_escaped_fragment_";

  public override void OnActionExecuting(ActionExecutingContext filterContext)
  {
      var request = filterContext.RequestContext.HttpContext.Request;

      if (string.IsNullOrWhiteSpace(request.QueryString[Fragment]))
          return;

      var parts = request.QueryString[Fragment].Split(new[] {'/'}, StringSplitOptions.RemoveEmptyEntries);
      var routeValues = new AjaxRoute();

      if (parts.Length > 0)
          routeValues.Controller = parts[0];

      if (parts.Length > 1)
          routeValues.Action = parts[1];

      if (parts.Length > 2)
          routeValues.Id = parts[2];

  filterContext.Result = new RedirectToRouteResult(new RouteValueDictionary(routeValues));
  }
}

public class AjaxRoute
{
  public string Controller { get; set; }
  public string Action { get; set; }
  public string Id { get; set; }
}

Setting the filterContext result will now reroute to the proper Controller and return the full view that Google is looking for. One last order of business is to register our attribute as a Global Action Filter, thereby making it execute on every request. Add the following line in the RegisterGlobalFilters method in Global.asax:

1
2
3
4
5
6

public static void RegisterGlobalFilters(GlobalFilterCollection filters)
{
  filters.Add(new HandleErrorAttribute());
  // Add this line here:
  filters.Add(new AjaxCrawlableAttribute());
}

And there you have it! Now every time Google crawls your ajaxified single-page website, it can index each page separately successfully.