- What You Will Learn
- The Challenge of Big Data
- Today's Big Data Explosion
- Background for This Book
- Why the Focus on Database Sharding?
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
The Challenge of Big Data
Have you ever experienced a database performance problem? It’s a silly question, I know. Everyone who has worked with databases for any amount of time has seen performance issues. In fact, it’s so common that database performance is almost an oxymoron. The very nature of database technology is to get slower over time, sometimes dramatically so. Why? Because databases only tend to get bigger through increased usage, and of course with more data the database management system (DBMS) has to do more work to keep up with application demands. Add to the mix an explosive high-transaction volume and you have the perfect recipe for something we all dread: slow database performance. In fact, sometimes it’s downright horrible database performance.
What about the memorable experience of a “database down” on a live production system? This can easily be your worst day on the job. Sometimes, when you get that call, you wish you could just forget it and not even go to work. Not only are big, growing databases hard to keep fast, they are very painful to repair when something bad happens. I’ve seen applications that were down for days due to extended database recovery times, and I’m sure you have heard of some notorious all-too-public examples of this as well. So an all-important part of a Big Data success is, in fact, planning for failure—ensuring you are prepared, with a robust, high-availability (HA) runtime environment and a solid disaster recovery (DR) plan for a worst-case scenario (they do happen).
What can you do about it? That’s what this book is all about: what you can do to make your database tier fast, scalable, and reliable, delivering the performance that your application needs. The focus of this portion of the book is not on any particular DBMS or database technology, but rather the answers to these questions—questions that apply to any database and any DBMS technology:
- What are the primary causes of database slow-down?
- How do you keep your database tier reliable and operational (especially in demanding, 24X7 environments)?
- Are there short-term, quick fixes you can implement to ease the pain?
- What are the best ways to scale your database to accommodate extreme growth in transaction volume and data sizes?
- How do you pick the best database technology (often technologies) for your application needs?
The answers to these questions and more are covered throughout the text. The aim is to arm you with the important concepts and facts common to all DBMS types, so that you can make the best decisions possible in managing your own data explosion, and come out on top.
Please note that in subsequent parts of the series I will provide a tour and overview of many common and leading-edge DBMS engines, both conceptually by category as well as specific solutions you can use in your Big Data arsenal. With this information, combined with the fundamentals covered in Part I, my hope is to assist you in preparing for your own Big Data adventure—commensurate with much success and reward.