This chapter is from the book
This chapter is from the book
This chapter is from the book
Starsiege: Tribes
Starsiege: Tribes is a sci-fi first-person shooter that was released at the end of 1998. At the time of release, it was well regarded as a game featuring both fast-paced combat and a comparatively massive number of players. Some game modes supported 128 players over either a LAN or the Internet. To gain some perspective on the magnitude of the challenge in implementing such a game, keep in mind that during this time period, the vast majority of players with an Internet connection used a dial-up service. At best, these dial-up users had a modem capable of speeds up to 56.6 kbps. In the case of Tribes, it actually supported users with modem speeds of only 28.8 kbps. By modern standards, these are extremely slow connection speeds. Another factor was that dial-up connections also had relatively high latency—a latency of several hundred milliseconds was rather common.
It may seem that a networking model designed for a game with low bandwidth constraints would be irrelevant in the modern day. However, it turns out that the model used in Tribes still has a great deal of validity even today. This section summarizes the original Tribes networking model—for a more in-depth discussion, refer to the article by Frohnmayer and Gift referenced at the end of this chapter.
Do not be concerned if some of the concepts covered in this section don’t entirely make sense right now. The intent is that by looking at a networked multiplayer game’s architecture at a high level, you will gain an appreciation for the numerous technical challenges faced and decisions to be made. All the topics touched on in this section are covered in much greater detail throughout the remainder of this book. Furthermore, one of the sample games built throughout this book, RoboCat Action, ultimately uses a model similar to the Tribes networking model.
One of the first choices made when engineering a networked game is to choose a communications protocol, or an established convention by which data is exchanged between two computers. Chapter 2, “The Internet,” covers how the Internet works and the commonly used protocols. Chapter 3, “Berkeley Sockets,” covers a ubiquitous library used to facilitate communication via these protocols. For the sake of the current discussion, the only thing you need to know is that, for efficiency reasons, Tribes uses an unreliable protocol. This means that data sent over the network is not guaranteed to be received by the destination.
However, using an unreliable protocol can be problematic when a game needs to send information that is important to all the players in the game. Thus, the engineers needed to consider the different types data they wanted to send out. The developers of Tribes ultimately separated their data requirements into the following four categories:
- Non-guaranteed data. As one might expect, this is data that the game designates as nonessential to the game. When bandwidth-starved, the game can choose to drop this data first.
- Guaranteed data. This data guarantees both arrival and ordering of the data in question. This is used for data deemed critical by the game, such as an event signifying when a player has fired a weapon.
- “Most recent state” data. This type of data is for cases where only the most recent version of the data is of importance. One example is the hit points of a particular player. A player’s hit points 5 seconds ago are not terribly relevant if the game knows what their hit points are right now.
- Guaranteed quickest data. This data is given the highest priority in order to transmit as quickly as possible with guaranteed delivery. An example of this type of data is player movement information, which is typically relevant for a very short period of time, and thus should be transmitted quickly.
Many of the implementation decisions made in the Tribes Networking Model center on providing these four types of data transmission.
Another important design decision was to utilize a client-server model instead of a peer-to-peer model. In a client-server model, players all connect to a central server, whereas in a peer-to-peer model, every player connects to every other player. As discussed in Chapter 6, “Network Topologies and Sample Games,” a peer-to-peer model requires O(n2) bandwidth. This means that the bandwidth grows at a quadratic rate based on the number of users. In this case, with n being as high as 128, using peer-to-peer would lead to very little bandwidth per player. To avoid this issue, Tribes instead implemented a client-server model. In this configuration, the bandwidth requirements of each player remain constant, while the server must handle only O(n) bandwidth. However, this meant that the server needed to be on a network that would allow for several incoming connections—the type of connection that only a company or university might have owned at the time.
Next, Tribes split up their networking implementation into several different layers—one can think of this as a “layer cake” of the Tribes Networking Model. This is illustrated in Figure 1.1. The remainder of this section briefly describes the composition of each of these layers.
)
Figure 1.1 The main components of the Tribes Networking Model
Platform Packet Module
A packet is a formatted set of data sent over a network. In the Tribes model, the platform packet module is the lowest layer. It is the only layer in the model that is platform-specific. In essence, this layer is a wrapper for the standard socket APIs that can construct and send various packet formats. The implementation of this layer might look rather similar to the systems implemented in Chapter 3, “Berkeley Sockets.”
Since Tribes utilized an unreliable protocol, the developers needed to add some mechanism to handle the data they decided needed to be guaranteed. Similar to the approach discussed in Chapter 7, “Latency, Jitter, and Reliability,” Tribes implemented a custom reliability layer. However, this reliability layer is not handled by the platform packet module; instead the higher level managers such as the ghost manager, move manager, or event manager are responsible for adding any reliability.
Connection Manager
The job of the connection manager is to abstract the connection between two computers over the network. It receives data from the layer above it, the stream manager, and transmits data to the layer below it, the platform packet module.
The connection manager level is still unreliable. It does not guarantee delivery of data sent to it. However, the connection manager does guarantee a delivery status notification—that is to say, the status of a request passed to the connection manager can be verified. In this way, it is possible for the level above the connection manager (the stream manager) to know whether or not particular data was successfully delivered.
The delivery status notification is implemented with a sliding window bit field of acknowledgments. Although the original Tribes Networking Model paper does not contain a detailed discussion regarding the implementation of the connection manager, an implementation of a similar system is discussed in Chapter 7, “Latency, Jitter, and Reliability.”
Stream Manager
The primary job of the stream manager is to send data to the connection manager. One important aspect of this is determining the maximum rate of data transmission that is allowed. This will vary depending on the quality of the Internet connection. An example given in the original paper is where a user on a 28.8-kbps modem might have their packet rate set to 10 packets per second with a maximum size of 200 bytes per packet, for approximately 2 kB of data per second. This rate and size is sent to the server upon connection of the client, in order to ensure that the server does not overwhelm the client’s connection with too much data.
Since several other systems will ask the stream manager to send data, it is also the duty of the stream manager to prioritize these requests. The move, event, and ghost managers are given the highest priority when in a bandwidth-bound scenario. Once the stream manager decides on what data to send, the packets are dispatched to the connection manager. In turn, the higher-level managers will be informed by the stream manager regarding the status of delivery.
Because of the set interval and packet size enforced by the stream manager, it is very much possible for a packet to be dispatched with multiple types of data in it. For example, a packet may have some data from the move manager, some data from the event manager, and some data from the ghost manager.
Event Manager
The event manager maintains a queue of events that are generated by the game’s simulation. These events can be thought of as a simple form of a remote procedure call or RPC, a function that can be executed on a remote machine. RPCs are discussed in Chapter 5, “Object Replication.”
For example, when a player fires a weapon, this would likely cause a “player fired” event to be sent to the event manager. This event can then be sent to the server, which will actually validate and execute the weapon firing. It is also the purview of the event manager to prioritize the events—it will try to write as many of the highest priority events as possible until any of the following conditions are true: the packet is full, the event queue is empty, or there are currently too many active events.
The event manager also tracks the transmission records for each event marked as reliable. In this way, it is very simple for the event manager to enforce reliability. If a reliable event is unacknowledged, then the event manager can simply prepend the event to the event queue and try again. Of course, there will be some events that are marked as unreliable. For these unreliable events, there is no need to even track their transmission records.
Ghost Manager
The ghost manager is perhaps the most important system in terms of supporting up to 128 players. At a high level, the job of the ghost manager is to replicate or “ghost” dynamic objects that are deemed relevant to a particular client. In other words, the server sends information about dynamic objects to the clients, but only the objects that the server thinks the client needs to know about. The game’s simulation layer is responsible for determining what a client absolutely needs to know and what a client ideally should know. This adds an inherent prioritization to game objects in the world: “need to know” objects are the highest priority, while “should know” objects are lower priority. In order to determine whether or not an object is relevant to a particular client, there are several different approaches that can be employed. Chapter 9, “Scalability,” covers some of these approaches. In general, determining object relevancy is very game-specific.
Regardless of how the set of relevant objects is computed, the job of the ghost manager is to transmit object state from server to client for as many relevant objects as possible. It’s very important that the ghost manager guarantees that the most recent data is always successfully transmitted to all of the clients. The reason for this is that the game object information that is ghosted will often contain information such as health, weapons, ammo count, and so on—all cases where the most recent data is the only information that matters.
When an object becomes relevant (or “in scope”), the ghost manager will assign some information to the object, which is appropriately called a ghost record. This record will include items such as a unique ID, a state mask, the priority, and status change (whether or not the object has been marked as in or out of scope).
For transmission of the ghost records, the objects are prioritized first by status change and then by the priority level. Once the ghost manager determines the objects that should be sent, their data can be added to the outgoing packet using an approach similar to what is covered in Chapter 5, “Object Replication.”
Move Manager
The responsibility of the move manager is to transmit player movement data as quickly as possible. If you’ve ever played a fast-paced multiplayer game, you are likely cognizant of the fact that accurate movement information is extremely important. If the information regarding a player’s position is slow to arrive, it could result in players shooting at where a player used to be instead of where a player is, which can result in frustrating gameplay. Quick movement updates can be an important way to reduce the perception of latency on the part of player.
The other reason the move manager is assigned a high priority is because input data is captured at 30 FPS. This means there is new input information available 30 times per second, so the latest data is sent as quickly as possible. This higher priority also means that, when move data is available, the stream manager will always first add any pending move manager data to an outgoing packet. Each client is responsible for transmitting their move information to the server. The server then applies this move information in its simulation of the game, and acknowledges the receipt of the move information to the client who sent it.
Other Systems
There are a few other systems in the Tribes model, though these are less critical to the overarching design. For example, there is a datablock manager, which handles transmission of game objects that are relatively static in nature. This differs from the relatively dynamic objects that are handled by the ghost manager. An example for this might be a static vehicle such as a turret—the object doesn’t really move, but it exists to serve a purpose when a player interacts with it.