- Searching for the Next Generation of Quants
- From Big Data's Past to Its Future
- Characterizing Big Data
- Is Big Data a Strategy?
- Data Versus Insights
- Data and Value
- Value for Value
This chapter is from the book
This chapter is from the book
This chapter is from the book
From Big Data’s Past to Its Future
The two sides of the Big Data coin may go back as far as the origin of the term. According to The New York Times story by Steve Lohr, author Erik Larson wrote a piece for Harper’s Magazine in 1989 that was subsequently reprinted in The Washington Post.6 In it, Larson discusses the direct marketing industry and its practice of merging different facets of consumer data. Decades before data privacy and transparency came into the vernacular, Larson suggests that the “keepers of big data” will “track you for the rest of your consuming life—pitch you baby toys when you’re pregnant, condos when you’re fifty.”7
Interestingly enough, Lohr doesn’t ascribe credit for the term “big data” to Larson. Instead, he argues that credit should go to John Mashey, chief scientist for Silicon Graphics in the 1990s. When queried about the use of the term, Mashey told Lohr that “I was using one label for a range of issues, and I wanted the simplest, shortest phrase to convey that the boundaries of computing keep advancing.” Lohr’s logic in giving Mashey credit for coining the term was that it “should go to someone who was aware of the computing context.”
Regardless of to whom you feel credit is due, this exercise into the ancestry of one of today’s most common phrases within technology and business circles offers a curious perspective. Larson exposes the potential benefits and risks associated with compiling disparate pieces of data. Like a jigsaw puzzle, each individual piece of data does not reveal much about the person who produced it. But assemble enough of these pieces, and the full picture begins to emerge. The image that Larson describes is one based on mortgage and tax records, consumer surveys, and records from the Census Bureau. Compared to what we’re capable of seeing today, what Larson envisioned may appear to look like a five-year-old’s finger painting tacked onto a refrigerator, hanging next to a digital photograph taken with the latest digital SLR camera on the market today. The possibilities that he described pointed in the direction of the path that we eventually traveled. However, we have gone leaps and bounds beyond what many would have imagined in the past.
For that, credit is certainly due to technology. Dramatic increases in processing power have paved the way for sophisticated marketing analysis to be conducted on desktop computers instead of mainframes. Rather than speaking in terms of census tracts, we can now talk in terms of individual households or, better yet, members of the household. Retail purchase records from scanners in grocery stores, once heralded as a major advance for the marketing profession, pale in comparison to the amount of data produced by consumers visiting websites and making purchases online. None of what we are talking about today would be possible were it not for technological advances. At the same time, credit should be given to those who had the foresight and saw potential opportunities to put such granular data to use. These two forces, working in parallel, heralded the age of Big Data.