- 6.1 About Ethernet
- 6.2 About Hubs, Switches, and Routers
- 6.3 About TCP/IP
- 6.4 About Packets
- 6.5 About Remote Procedure Calls (RPCs)
- 6.6 Slop
- 6.7 Observing Network Traffic
- 6.8 Sample RPC Message Definition
- 6.9 Sample Logging Design
- 6.10 Sample Client-Server System Using RPCs
- 6.11 Sample Server Program
- 6.12 Spinlocks
- 6.13 Sample Client Program
- 6.14 Measuring One Sample Client-Server RPC
- 6.15 Postprocessing RPC Logs
- 6.16 Observations
- 6.17 Summary
- Exercises
This chapter is from the book
This chapter is from the book
This chapter is from the book
6.3 About TCP/IP
The TCP/IP design allows packets to be routed not just within a single building but anywhere in the world that is connected to the global Internet. This routing sends packets across various media—not just Ethernet links, but also long-haul dedicated fibers, radio links to satellites, WiFi connections within houses, and many more kinds of sub-networks. The complex dynamics and delays of long-distance communication are beyond the scope of this book; we will concentrate just on the complex-enough dynamics and delays of Ethernet connections within a single building.
For a message from machine A to machine B, sending software on A for guaranteed-delivery protocols such as TCP keeps track of packets sent that do not arrive and retransmits them. Packets are therefore not guaranteed to arrive in the order originally sent, so receiving software further tracks them and reassembles messages in receive buffers. This tracking is done by the receiving TCP software on B sending back an ACK indication to A for one or more received packets. ACKs can be sent in short packets of their own but are usually piggybacked as part of other packets already going back B⇒A. Senders have a limited number of multiple packets outstanding—sent but not yet acknowledged. When this limit is reached, the sender must wait until some ACKs arrive. If a packet ACK does not arrive within some configured timeout period, the sender is responsible for retransmitting that packet.
We are using TCP/IPv4 to send RPC messages between servers, with each message possibly requiring many packets. Our remote procedure calls depend on the guaranteed delivery mechanism of TCP to deliver an entire message with the pieces in proper order.
It is unlikely in our little sample server cluster that we will see packets dropped and retransmitted because of hardware errors, but we will soon try to create enough network congestion to force some packet drops because of overloaded switch buffering. To try to protect the rest of any building network from also becoming overloaded when we do saturation experiments, it is best for our lab machines to connect directly with each other through their own local switch, as described earlier.
TCP establishes a reliable connection to carry a pair of byte streams between two programs on two machines, one stream in each direction. These are the bi-directional connections shown in Figure 6.2. Each machine is specified by its IP address and the specific program by its port number. Two-byte port numbers range from 0..65535, but ports below 1024 are restricted to specific uses. We will use ports 12345..12348 on our sample servers for RPC traffic. (Our lab machines may also have a software firewall that closes traffic on most other ports.)
){kind=link}
Once a connection is established, there is a stream of data available in each direction between the two machines. A machine can send an (almost) arbitrary number of bytes at once, and the TCP software deals with breaking up long messages into multiple packets, or packing multiple short messages or fragments of longer messages into single packets. The communication model is just a stream of bytes, so a given RPC message may start and end in the middle of packets.
On the other end, a machine can request receiving an (almost) arbitrary number of bytes into a buffer, but the number actually delivered at once can be less than the buffer size. The receive calls normally return any data available, up to the requested amount, rather than waiting for receipt of the full amount requested. This design allows the receiving software some flexibility in managing buffers and in managing how long to wait for data (or what else to do in the meantime). The receiving logic thus must be prepared to do multiple receive calls to get all the pieces of a single complete message and must also be prepared to receive multiple messages and partial messages at each call.