- An Interesting Problem
- An Uninteresting Problem
- A Knowledge Question
- The Engineer Growth Curve
- Slope of Productivity
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
An Uninteresting Problem
The key words here are "well enough." Not everything fits neatly into the dichotomy of "good enough" and "not quite good enough." A library or technology may do several things well, but miss one very important thing.
SQL databases are often cited as an example of this symptom. They perform very well when used as prescribed; they're durable and provide all kinds of mature data-manipulation capabilities, such as multi-table transactions, multi-part indexes, and numerous other useful features. However, they don't handle very large amounts of data well. The database is often capped by the power of the single machine on which it sits. Consequently, distributed datastores started popping up all over the placeβCassandra, HBase, and Voldemort among them.
This is the case where "Not Invented Here" was a genuine disadvantage. The nature of very large data storage, in the early days of the Internet, was that nobody had really solved how one database could manage it efficiently and functionally. The famous BigTable paper by Google touched off this revolution of research in distributed datastores. Each group built one to suit the group's specific purpose. CouchDB offers a distributed key/value-like store, using JavaScript to interface with the data. HBase (and subsequently Yahoo!) provided a pure Java version, adhering closely to the original BigTable paper and integrating with the other members of the Hadoop family of products. Google's own evolution, Megastore, was designed to handle multi-table transactions in a distributed and fault-tolerant manner, which satisfied a different need altogether.