The Promise and Perils of Distributed Systems
This chapter is from the book
This chapter is from the book
This chapter is from the book
The Limits of a Single Server
In this book, we will discuss distributed systems. But what exactly do we mean when we say “distributed systems”? And why is distribution necessary? Let’s start from the basics.
In today’s digital world, the majority of our activities rely on networked services. Whether it’s ordering food or managing finances, these services run on servers located somewhere. When using cloud services like AWS, GCP, or Azure, these servers are managed by the respective cloud providers. They store data, process user requests, and perform computations using the CPU, memory, network, and disks. These four fundamental physical resources are essential for any computation.
Consider a typical retail application functioning as a networked service, where users can perform actions such as adding items to their shopping cart, making purchases, viewing orders, and querying past orders. The capacity of a single server to handle user requests is ultimately determined by the limitations of four key resources: network bandwidth, disks, CPU, and memory.
The network bandwidth sets the maximum data transfer capacity over the network at any given time. For example, with a network bandwidth of 1Gbps (125MB/s) and 1KB records being written or read, the network can support a maximum of 125,000 requests per second. However, if the record size increases to 5KB, the number of requests that can be passed over the network decreases to only 25,000.
Disk performance depends on several factors, including the type of read or write operations and how well disk caches are used. Mechanical disks are also affected by hardware features such as rotational speed and seek time. Sequential operations usually have better performance than random ones. Moreover, the performance is influenced by concurrent read/write operations and software-based transactional processes. These factors can significantly affect the overall throughput and latency on a single server.
Figure 1.1 Resources of computation
Likewise, when the CPU or memory limit is reached, requests must wait for their turn to be processed. When these physical limits are pushed to their capacity, this results in queuing. As more requests pile up, waiting times increase, negatively impacting the server’s ability to efficiently handle user requests.
The impact of reaching the limits of these resources becomes evident in the overall throughput of the system, as illustrated in Figure 1.2.
)
Figure 1.2 Drop in throughput with increase in requests
This poses a problem for end users. As the system is expected to accommodate an increasing user base, its performance actually degrades.
To ensure requests are served effectively, you have to divide and process them on multiple servers. This enables the utilization of separate CPUs, networks, memory, and disks to handle user requests. In our example, the workload should be divided so that each server handles approximately five hundred requests.