SEO in the World of Web 2.0

Wouldn’t it be great if all the whiz-bang Web 2.0 interactive elements like Asynchronous JavaScript and XML (AJAX), widgets, Flash, RSS feeds, podcasts, video blogs and so forth were all search engine optimized?

Unfortunately, that’s not the case. In fact, many of these technologies are inherently unfriendly to search engine spiders. So, if you intend to harness Web 2.0 technologies for increased conversion, improved usability and greater customer engagement, you’d better read on or you’ll end up missing the boat when it comes to better search engine rankings.

The discipline of search engine optimization (SEO) is evolving to better meet the challenges presented by a Web 2.0 world. Specific SEO tactics exist to expose content trapped within AJAX, Flash, RSS, audio and video. The bad news is that the major search engines still can’t cope with these elements.

So, the onus is on you to render them search engine friendly. Let’s start with Flash, a technology with which many of us already are familiar. Some search engines, including Google, have rudimentary means of extracting content and links from Flash. Nonetheless, any content or navigation embedded within a Flash movie will, at best, rank poorly in comparison to a static, HTML-based counterpart, and at worst, not even make it into the search engine’s index.

Flash Not Friendly

Google’s view on Flash is that it doesn’t provide a user-friendly experience. Flash is wholly inaccessible to the vision-impaired, unrenderable on devices such as mobile phones and PDAs, and can’t be accessed without broadband connectivity.

In particular, Google frowns on navigational elements presented exclusively in Flash. Given this stance, Google isn’t likely to make big improvements on how it crawls, indexes and ranks Flash files anytime soon. So, it’s in your hands to either replace those Flash elements with a more accessible alternative like CSS/DHTML or to employ a Web design approach known as “progressive enhancement,” whereby designs are layered in a concatenated manner to provide an alternative experience for non-Flash users. This way, all users, including search engine spiders, will be able to access your content and functionality.

An example of progressive enhancement in action can be found at Amazon.com’s “Create Your Own Ring” on the Web. Simply turn off the JavaScript capabilities in your browser and build your ring — with or without the Flash interaction. All customers are equally served (http://tinyurl.com/yrh6o7).

Problems With AJAX

AJAX poses similar problems to spiders as Flash does. That’s because AJAX also relies on JavaScript — that’s what the “J” in AJAX stands for, after all. Search engine spiders can’t execute JavaScript commands (or Java either, for that matter). AJAX can be used to pull data seamlessly in the background onto an already loaded Web page, sparing the user from the “click-and-wait” frustrations associated with more conventional Web sites. It’s a great timesaver for users, but the additional content that’s pulled in via AJAX is invisible to the spiders unless it’s preloaded into the page’s HTML and simply hidden from the user via CSS.

Here, progressive enhancement renders a non-JavaScript version of the AJAX application for spiders and JavaScript-incapable browsers. A low-tech alternative to progressive enhancement is to place an HTML version of your AJAX application within noscript tags.

Other options include rendering static HTML pages from product searches, as the vertical shopping engine Become.com does. Google’s guidelines warn that your search result pages must provide value to users to warrant inclusion in its index. So, extra care must be taken if employing this approach.

Widgets, the mini applications webmasters are encouraged to place on their sites to pull data from an external source, also are in most cases inaccessible to search engine spiders. Most widgets are built in search engine unfriendly Flash or AJAX.

A well-loved widget in the blogosphere is Eurekster’s Swicki (http://swicki.eurekster.com), which you may have seen in the sidebars of popular blogs like TechCrunch’s (www.techcrunch.com) “What’s Popular” buzzcloud. Under our tutelage, Eurekster made its widget more search engine friendly and reaped the benefits with a huge influx of search-referred traffic.

Unlike Flash and AJAX, RSS is inherently search engine friendly. That’s because an RSS feed is an XML file, and XML is text-rich with semantic markup. But RSS isn’t well-supported in traditional Web search. It is within certain vertical engines, such as Google Blog Search and Technorati, but those aren’t the mainstream search engines that catalogers care about.

Over time, the major engines will get better at interpreting the XML and using the extracted data for ranking purposes. For instance:
• counting the links contained within the feed as votes; and,
• taking the anchor text of those links into account for ascertaining the linked page’s keyword theme.

In the meantime, prepare for this inevitability by making one or more full-text — not summary — feeds available on your site through text links and by including within those feeds at least 20 items (not just the default 10). Each item should have a keyword-rich title and keyword-rich text links contained within the product’s HTML-encoded content.

RSS Fuels Podcasts

We have RSS to thank for the podcasting revolution. It’s the RSS feed with “enclosures” of audio or video files that makes it a podcast, not the fact that it has audio or video files available for download. For those catalogers producing podcasts, use the ID3 tags of your MP3 files to incorporate show notes, images and a link to your podcast feed. Then, syndicate your feeds via multiple venues for optimal exposure on the Web.

Some Web 2.0 technologies already are primed for search engines straight out of the box. Blogs and wikis are two such technologies. Google seems to love blogs — the rich textual content, the widespread interlinking between blogs, the intricate network of internal links, the frequent updates, the semantic markup, etc.

By then applying specific blog optimization tactics — such as using “sticky” posts to add keyword-rich intro copy to category pages, incorporating topical tag clouds and tag pages into your blog, and overriding the automatically generated title tags with custom-written ones — you can really make your blog hum. Wikis tend to do fairly well in search engines, too, because wikis are text-rich, frequently updated, and heavily linked internally. But they’re also more susceptible to spam, and the content is only as good as the contributors.

As you head down the road of Web 2.0, just remember that user-friendly doesn’t readily translate into search engine friendly without some assistance. But know that help is available when you embrace this brave new World Wide Web.

Stephan Spencer is president and founder of Netconcepts, a Web design and consulting firm specializing in search engine, optimal Web sites and applications. You can reach him at sspencer@netconcepts.com.