Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
References
[1] In 2003, search engines performed more than a half-billion searches a day. See Danny Sullivan, "Searches Per Day," Search Engine Watch, Feb. 25, 2003.
[2] See Jason Lee Miller, "Left, Right, or Center? Can a Search Engine Be Biased?" WebProNews.com, May 10, 2005.
[3] There is a broad perception that search engines present search results passively and neutrally. See Leslie Marable, "False Oracles: Consumer Reaction to Learning the Truth About How Search Engines Work," Consumer Reports WebWatch, June 30, 2003; Maureen O’Rourke, "Defining the Limits of Free-Riding in Cyberspace: Trademark Liability for Metatagging," 33 Gonzaga Law Review 277 (1998).
[4] See, e.g., C. Edwin Baker, Advertising and a Democratic Press (Princeton University Press, 1994).
[5] See, e.g., "Does Google Ever Manipulate Its Search Results?" Google.com ("The order and contents of Google search results are completely automated. No one hand picks a particular result for a given search query, nor does Google ever insert jokes or send messages by changing the order of results."); "Does Google Censor Search Results?" Google.com ("Google does not censor results for any search terms. The order and content of our results are completely automated; we do not manipulate our search results by hand."); "Technology Overview," Google.com ("There is no human involvement or manipulation of results...."); "How Can I Improve My Site’s Ranking?" Google.com ("Sites’ positions in our search results are determined automatically based on a number of factors, which are explained in more detail at http://www.google.com/technology/index.html. We don’t manually assign keywords to sites, nor do we manipulate the ranking of any site in our search results.") See also Complaint at ¶¶ 37–38, 52–56, KinderStart.com LLC v. Google, Inc., Case No. C 06-2057 RS (N.D. Cal. Mar. 17, 2006), giving other examples of Google’s claims to be passive. Note that Google has subsequently revised some of these cited pages after its censorship controversy in China.
[6] See generally Abbe Mowshowitz and Akira Kawaguchi, "Bias on the Web," Comm. ACM, Sept. 2002, at 56 (distinguishing "indexical bias" and "content bias").
[7] See Judit Bar-Ilan, "Expectations Versus Reality—Search Engine Features Needed for Web Research at Mid-2005," 9 Cybermetrics 2 (2005).
[8] For example, many search engines ignore metatags. See Eric Goldman, "Deregulating Relevancy in Internet Trademark Law," 54 Emory Law Journal 507, 567–68 (2005). Search engines also incorporate only portions of very large files. See Bar-Ilan, supra note [7]; "Why Doesn’t My Site Have a Cached Copy or a Description?" Google.com (describing how some pages are "partially indexed"); "Has Google Dropped Their 101K Cache Limit?" ResearchBuzz!, Jan. 31, 2005 (discussing how historically Google indexed only the first 101KB of a document).
[9] See "My Site’s Listing Is Incorrect and I Need it Changed," Google.com. Google’s automated descriptions have spawned at least one lawsuit by a web publisher who believed the compilation created a false characterization. See Seth Fineberg, "Calif. CPA Sues Google Over ’Misleading’ Search Results," Acct. Today, Apr. 19, 2004, at 5, available at WebCPA.
[10] See Jagdeep S. Pannu, "Anchor Text Optimization," WebProNews.com, Apr. 8, 2004.
[11] For example, the first search result in Google and Yahoo! for the keyword "miserable failure" is President George W. Bush’s home page because so many web sites have linked to the biography using the term "miserable failure." See Tom McNichol, "Your Message Here," N.Y. Times, Jan. 22, 2004, at G1. This algorithmic vulnerability has spawned a phenomenon called "Google bombing," in which web sites coordinate an anchor text attack to distort search results intentionally. See John Hiler, "Google Time Bomb," Microcontent News, Mar. 3, 2002.
[12] See, e.g., Stefanie Olsen, "Search Engines Delete Adware Company," CNET News.com, May 13, 2004 (Google and Yahoo kicked WhenU.com out of their indexes for allegedly displaying different web pages to searchers and search engine robots, a process called "cloaking").
[13] This is the heart of KinderStart’s allegations against Google. See Complaint, KinderStart.com LLC v. Google, Inc., Case No. C 06-2057 (N.D. Cal. Mar. 17, 2006). Although the complaint’s allegations about Google’s core algorithmic search may not be proven, Google does liberally excise sources from Google News. For example, Google claims that "news sources are selected without regard to political viewpoint or ideology," see "Google News," Google.com, but Google dropped a white supremacist news source from Google News because it allegedly promulgated "hate content." See Susan Kuchinskas, "Google Axes Hate News," Internetnews.com, Mar. 23, 2005.
[14] See Eric Goldman, "Google’s Human Algorithm," Technology & Marketing Law Blog, June 5, 2005 (Google hires students to review search results manually for quality purposes).
[15] See Search King, Inc. v. Google Tech., Inc., No. CIV-02-1457-M, at 4 (W.D. Okla. Jan. 13, 2003). ("Google knowingly and intentionally decreased the PageRanks assigned to both SearchKing and PRAN.") This manual adjustment has also been alleged in the recent KinderStart lawsuit. See Complaint, KinderStart.com L.L.C. v. Google, Inc., Case No. C 06-2057 RS (N.D. Cal. Mar. 17, 2006).
[16] See "MSN Blockades phpBB Searchers," trimMail’s Email Battles, Jan. 18, 2006.
[17] See "An Explanation of Our Search Results" Google.com.
[18] See Jennifer Laycock, "Ask.com Actively Censoring Some Search Phrases," Search Engine Guide, June 23, 2006. On Aug. 1, 2006, I was unable to replicate these results.
[19] Google scientology.site:xenu.net and scroll to the bottom of the page (or click here and scroll to the bottom of the page). See also Chris Sherman, "Google Makes Scientology Infringement Demand Public," Search Engine Watch, Apr. 15, 2002.
[20] See Danny Sullivan, "KinderStart Becomes KinderStopped in Ranking Lawsuit Against Google," Search Engine Watch, July 14, 2006.
This duality, if it ends up leading to the dissemination of false information, could also create some legal liability. See KinderStart v. Google, No. 5:06-cv-02057-JF (N.D. Cal. motion to dismiss granted July 13, 2006), pointing out the potential inconsistency of Google’s position that PageRank is Google’s subjective opinion but an objective reflection of its algorithmic determinations.
[21] See "iProspect Search Engine User Behavior Study," iProspect, Apr. 2006 (62% of searchers click a search result on the first results page); Jakob Nielsen, "The Power of Defaults," Jakob Nielsen’s Alertbox, Sept. 26, 2005 (citing a study by Cornell professor Thorsten Joachims that the first search result gets 42% of clicks and the second search result gets 8%; further, when the first two search results are switched, the first search result gets 34%—meaning that positioning dictated searcher behavior); Nico Brooks, "The Atlas Rank Report: How Search Engine Rank Impacts Traffic," Atlas Institute Digital Marketing Insights, June 2004 (the first-ranked search result may get ten times as many clicks as the tenth-ranked search result).
[22] See Michael Totty and Mylene Mangalindan, "Web Sites Try Everything To Climb Google Rankings," Wall Street Journal Online, Feb. 26, 2003 (subscription required).
[23] See Lucas D. Introna and Helen Nissenbaum, "Shaping the Web: Why the Politics of Search Engines Matters," Info. Society, July–Sept. 2000, at 169.
[24] See "Our Search: Google Technology," Google.com.
[25] See Niva Elkin-Koren, "Let the Crawlers Crawl: On Virtual Gatekeepers and the Right To Exclude Indexing," 26 University of Dayton Law Review 179, 188 (2001); Frank Pasquale, "Rankings, Reductionism, and Responsibility," Seton Hall Public Law Research Paper No. 888327, at 25, Feb. 25, 2006; Trystan Upstill et al., "Predicting Fame and Fortune: PageRank or Indegree?" Proceedings of the 8th Australasian Document Computing Symposium, Dec. 15, 2003 (showing that BusinessWeek Top Brand, Fortune 500, and Fortune Most Admired companies get disproportionately high PageRank).
[26] See Introna and Nissenbaum, supra note [23]; Matthew Hindman et al., "’Googlearchy’: How a Few Heavily Linked Sites Dominate Politics on the Web," Mar. 31, 2003.
[27] See "Egalitarian Engines," Economist, Nov. 17, 2005 ("[T]here is a widespread belief among computer, social, and political scientists that search engines create a vicious circle that amplifies the dominance of established and already popular web sites"); see also Junghoo Cho and Sourashis Roy, "Impact of Search Engines on Page Popularity," WWW 2004, May 2004; Upstill, supra note [25]. But see Santo Fortunato et al., "The Egalitarian Effect of Search Engines," Nov. 2005 (questioning the consequences of the "rich-gets-richer" effect).
[28] See Cho and Roy, supra note [27]; but see Filippo Menczer et al., "Googlearchy or Googlocracy?" IEEE Spectrum, Feb. 2006 (providing empirical evidence suggesting that "search engines direct more traffic than expected to less-popular sites").
[29] See Search King Inc. v. Google Tech., Inc., No. CIV-02-1457-M, at 3 n.2 (W.D. Okla. Jan. 13, 2003): "Google’s mathematical algorithm is a trade secret, and it has been characterized by the company as ’one of Google’s most valuable assets’"; Stefanie Olsen, "Project Searches for Open-Source Niche," CNET News.com, Aug. 27, 2003.
[30] See Introna and Nissenbaum, supra note [23].
[31] See id.; Eszter Hargittai, "Open Portals or Closed Gates? Channeling Content on the World Wide Web," 27 Poetics 233 (2000); cf. "Cass Sunstein, Republic.com" 170-72 (2001), advocating publicly funded "deliberative domains."
[32] See Kevin J. O’Brien, "Europeans Weigh Plan on Google Challenge," International Herald Tribune, Jan. 18, 2006 (discussing a European initiative called Quaero, which is intended to break the American hegemony implicit in Google’s dominant market position); Graeme Wearden, "Japan May Create Its Own Search Engine," ZDNet News, Dec. 21, 2005.
[33] See Search King, Inc. v. Google Tech., Inc., No. CIV-02-1457-M (W.D. Okla. Jan. 13, 2003); KinderStart.com LLC v. Google, Inc., No. C 06-2057 RS (N.D. Cal. dismissed July 13, 2006); Langdon v. Google, Inc., No. 1:06-cv-00319-JJF (D. Del. complaint filed May 17, 2006); Roberts v. Google, No. 1-06-CV-063047 (Cal. Superior Ct. complaint filed May 5, 2006); Datner v. Yahoo! Inc, Case No. BC355217 (Cal. Superior Ct. complaint filed July 11, 2006). [Note: This list updated as of July 24, 2006.]
[34] As Google said in its response to the KinderStart lawsuit, "Plaintiff KinderStart contends that the judiciary should have the final say over [search engines’] editorial process. It has brought this litigation in the hopes that the Court will second-guess Google’s search rankings and order Google to view KinderStart’s site more favorably." Motion to Dismiss at 1, KinderStart.com LLC v. Google, Inc., No. C 06-2057 RS (N.D. Cal. May 2, 2006).
[35] See Sandeep Pandey et al., "Shuffling a Stacked Deck: The Case for Partially Randomized Ranking of Search Engine Results,"; cf. Sunstein, supra note [31] (explaining that web sites should be forced to link to contrary views as a way of increasing exposure to alternative viewpoints).
[36] See Pasquale, supra note [25], at 28–30 (proposing that the link be displayed as an asterisk to the search results).
[37] Every Internet venue accepting user-submitted content inevitably gets attacked by unwanted content. If left untended, the venue inexorably degrades into anarchy. See, e.g., "Step-by-Step: How to Get BILLIONS of Pages Indexed by Google," Monetize Blog, June 17, 2006, (Google indexed more than five billion "spam" pages from a single spammer before manually de-indexing the sites); Alorie Gilbert, "Google Fixes Glitch that Unleashed Flood of Porn," CNET News.com, Nov. 28, 2005 (describing how Google Base, a venue for user-submitted content, was overtaken by pornographers: "[T]he amount of adult content on Google Base was staggering considering Google only launched the tool a week ago"); Josh Quittner, "The War Between alt.tasteless and rec.pets.cats," Wired, May 1994, at 46 (describing how a group of anarchists, for fun, took over a USENET newsgroup about pets).
[38] See Mowshowitz and Kawaguchi, supra note [6], at 60 (market forces are the best way to counter adverse effects of search engine bias).
[39] See "Our Philosophy," Google.com ("The perfect search engine...would understand exactly what you mean and give back exactly what you want"); Chris Sherman, "If Search Engines Could Read Your Mind," Search Engine Watch, May 11, 2005.
[40] Searchers routinely use a very small number of keywords to express their search interests. See iProspect.com, Inc., "iProspect Natural SEO Keyword Length Study," Nov. 2004 (stating that 88% percent of search engine referrals are based on only one or two keywords); see also Declan Butler, "Souped-Up Search Engines," Nature, May 11, 2000, at 112, 115 (citing an NEC Research Institute study showing that up to 70% of searchers use only a single keyword as a search term); Bernard J. Jansen et al., "Real Life Information Retrieval: A Study of User Queries on the Web," 32 SIGIR Forum 5, 15 (1998), stating that the average keyword length was 2.35 words; one-third of searches used one keyword and 80% used three keywords or fewer; Jakob Nielsen, "Search: Visible and Simple," Jakob Nielsen’s Alertbox, May 13, 2001 (stating that the average keyword length was 2.0 words).
[41] See Kim Peterson, "Microsoft Learns to Crawl," Seattle Times, May 2, 2005 (MSN Search "learned that the arcane searches were the make-or-break moments for web searchers. People weren’t just happy when a search engine could find answers to their most bizarre, obscure, and difficult queries. They would switch loyalties."); Bob Tedeschi, "Every Click You Make, They’ll Be Watching You," N.Y. Times, Apr. 3, 2006.
[42] In addition to the recent launch of major new search engines by providers such as MSN, the open-source software community is developing Nutch to allow anyone to build and customize his or her own web search engine; see also Olsen, "Project Searches for Open-Source Niche," supra note [29].
While there are multiple major search engines, the market may still resemble an oligopoly; a few major players (Google, Yahoo!, MSN, Ask Jeeves) have the lion’s share of the search engine market. However, this observation may construe the search engine market too narrowly. Many types of search providers compete with the big mass-market search engines, ranging from specialty search engines (such as Technorati) to alternative types of search technology (adware, for instance) to non-search information retrieval processes (link navigation and such). Ultimately, every search engine competes against other search engines and these other search/retrieval options.
[43] See Rahul Telang et al., "An Empirical Analysis of Internet Search Engine Choice," Aug. 2002. On file with author. For example, search engines use the same basic interface (a white search box), and searchers rarely use advanced search features that might require additional learning time at other search engines.
[44] See Grant Crowell, "Understanding Searcher Behavior," Search Engine Watch, June 14, 2006 (citing a Kelsey Research study showing that 63% of searchers used two or more search engines); Vividence, Inc., "Google Wins Users’ Hearts, But Not Their Ad Clicks," press release, May 25, 2004 (stating that up to 47% of searchers try another search engine when their search expectations are not met).
[45] See Rahul Telang et al., "The Market Structure for Internet Search Engines," 21 J. Mgmt. Info. Sys. 137 (2004), describing how searchers sample heterogeneous ranking algorithms, which support a diversity of search engines; Mário J. Silva, "The Case for a Portuguese Web Search Engine," (describing the value of a Portuguese-oriented search engine); Jeffrey McMurray, "Social Search Promises Better Intelligence," Associated Press, July 9, 2006 (discussing niche search engines that draw on social networking); cf. Jakob Nielsen, "Diversity Is Power for Specialized Sites," Jakob Nielsen’s Alertbox, June 16, 2003 (describing how specialized sites will flourish on the Internet).
[46] See, e.g., Susan L. Gerhart, "Do Web Search Engines Suppress Controversy?" First Monday, Jan. 2004. Gerhart argues that search engines don’t adequately prioritize search results that expose controversies about the search topic. However, her argument assumes that controversy-related information has value to consumers, an assumption that deserves careful evaluation.
[47] See Eric Goldman, "A Coasean Analysis of Marketing," 2006 Wisconsin Law Review (forthcoming).
[48] See Susan P. Crawford, "Shortness of Vision: Regulatory Ambition in the Digital Age," 74 Fordham Law Review 695 (2005), discussing the shortcomings of regulatory intervention in organic information systems.
[49] See James Pitkow et al., "Personalized Search," Comm. ACM, Vol. 45:9 (Sept. 2002) at 50–51.
[50] See Michael Kanellos, "Microsoft Aims for Search on Its Own Terms," CNET News.com, Nov. 24, 2003 (quoting a Microsoft researcher as saying, "If the two of us type a query [into a search engine], we get the same thing back, and that is just brain dead. There is no way an intelligent human being would tell us the same thing about the same topic"); David H. Freedman, "Why Privacy Won’t Matter," Newsweek, Apr. 3, 2006; "Personalization of Placed Content Ordering in Search Results," U.S. Patent App. 0050240580 (filed July 13, 2004).
[51] See Pitkow, supra note [49], at 50.
[52] "What’s Personalized Search?" Google.com.
[53] See Jaime Teevan et al., "Personalizing Search via Automated Analysis of Interests and Activities," SIGIR ’05; Terry McCarthy, "On the Frontier of Search," Time, Aug. 28, 2005: "Search will ultimately be as good as having 1,000 human experts who know your tastes scanning billions of documents within a split second" (quoting Gary Flake, Microsoft Distinguished Engineer).
[54] See Kevin Lee, "Search Personalization and PPC Search Marketing," Clickz News, July 15, 2005.
[55] Personalized algorithms have other potentially adverse consequences, such as creating self-reinforcing information flows. See Sunstein, supra note [31]. For a critique of these consequences, see Goldman, "Coasean Analysis," supra note [47].
[56] See generally Goldman, "Deregulating Relevancy," supra note [8].
[57] See Danny Sullivan, "Death of a Meta Tag," Search Engine Watch, Oct. 1, 2002.