- Find Out Where You Stand
- Eliminate Spider Traps
- Create Spider Paths
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Create Spider Paths
Now that you've learned all about removing spider traps, let's look at a more positive approach: creating a site that the spiders love to crawl. You can do that by creating well-worn paths through your site that help spiders find every page easily.
Your site may already have the best example of a spider path—a site map. Small sites can provide site maps with direct links to every page, but sites of more than 100 pages need to link to category hub pages that fan out to the rest of your site. Sites with tens or hundreds of thousands of pages may require the site map itself to cross several pages. IBM's site map, for example, consists of three pages, one each for "Products," "Services & Solutions," and "Support & Downloads."
A special kind of site map for multinational Web sites is a country map, which lists links to the country Web sites that comprise the global domain. Procter & Gamble's country map leads the spider to each of the country sites. Site maps are essential for spiders (and human visitors, too), but are not the only kind of spider path.
Inclusion programs are special kinds of paths designed only for spiders. Paid inclusion programs were popular a few years ago, allowing Web site owners to pay a search engine to include its pages in the index (and update those pages frequently), but only Yahoo! (with its Site Match program) offers one today. Google recently announced a beta program for free inclusion, called Sitemaps , which provides several methods for letting Google know which pages to index—even telling Google which pages are more important than others.
Getting as many pages as possible into Google's index (as well as Yahoo!'s and others) is the most basic step to being found by search engines, but it isn't all you need to do. Once you've got your pages indexed, you must determine what words searchers are using to find them and then optimize your content so that Google shows your pages in its search results. For tips on these and other steps in driving search traffic to your site, check out the new IBM Press book, Search Engine Marketing, Inc.