This chapter is from the book
This chapter is from the book
This chapter is from the book
3.7 Implementing Utilities
Utility classes and methods can encapsulate efficient, reliable, general-purpose implementations of concurrency control constructs in a way that lets them be used almost as if they were part of the language proper. These classes can capture clever, complex, error-prone constructions and efficiently exploit special cases, packaging the results so that programs using them can be written more simply, reliably, and often with better performance. It is worth the development effort to arrive at such classes only once, and only when justified by real design concerns.
This section illustrates some techniques used in the construction of common utilities. All of them rely on general design and implementation tactics described previously in this book, but also introduce a few additional specialized constructions that typically arise only when building support classes.
The section starts by illustrating how to package acquire-release protocols under a common interface. This is followed by an example showing how to apply joint action design techniques to split classes into parts for the sake of obtaining necessary concurrency control, and then recombining them to improve efficiency. Finally, it discusses how to isolate waiting threads in order to manage notifications.
3.7.1 Acquire-Release Protocols
As discussed in 2.5.1 and 3.4.1, many concurrency control constructs conform to an acquire-release protocol that can be encompassed under the simple interface:
interface Sync {
void acquire() throws InterruptedException;
void release();
boolean attempt(long msec) throws InterruptedException;
}
Supporting this interface under a given semantics (for example, locks, semaphores, latches) requires that the internal state representations that drive waits and notifications be managed by the Sync objects, not the classes using them. Additionally, all control must be placed within the exported methods; it cannot be strewn around other methods in client classes, and it is a bad idea to introduce other methods that clients must use in a special, non-standard way to obtain desired behavior.
Most of the resulting issues and concerns can be illustrated with a sample implementation of the basic Semaphore class discussed in 3.4.1. Implementations of other Sync classes follow similar patterns. (In fact, as shown in 3.4.1, classes such as Mutex can in turn be defined using semaphores.)
Both at the conceptual and representational level, a semaphore maintains a count of the number of permits that it manages. The basic idea is that an acquire should wait (if necessary) until there is at least one permit, and that a release should increment the number of permits and provide notifications to any waiting threads. Here are some other observations and choices that lead to an implementation:
Since all waiting threads are waiting for permits, and since a release adds one permit, we can use notify rather than notifyAll, leading to cheaper notifications. Also, the extra-notify-on-interrupt technique described in 3.2.4.2 is available to avoid lossage when threads are interrupted at just the wrong time.
Because this is intended to be a general-purpose class, we should play it safe and use long (not int) for counts. This avoids all practical possibility of value overflow and costs almost nothing compared to monitor overhead.
To maintain responsiveness, we should check to make sure that the current thread has not been interrupted before acquiring any locks. This minimizes windows of vulnerability for client threads getting stuck waiting for locks when they should be cancelling themselves (see 3.1.2). It also provides a more uniform guarantee that InterruptedException will be thrown if the thread enters in an interrupted state, rather than having the exception thrown only if the thread happens to block on the internal wait.
class Semaphore implements Sync {
protected long permits; // current number of available permits
public Semaphore(long initialPermits) {
permits = initialPermits;
}
public synchronized void release() {
++permits;
notify();
}
public void acquire() throws InterruptedException {
if (Thread.interrupted()) throw new InterruptedException();
synchronized(this) {
try {
while (permits <= 0) wait();
--permits;
}
catch (InterruptedException ie) {
notify();
throw ie;
}
}
}
public boolean attempt(long msecs)throws InterruptedException{
if (Thread.interrupted()) throw new InterruptedException();
synchronized(this) {
if (permits > 0) { // same as acquire but messier
--permits;
return true;
}
else if (msecs <= 0) // avoid timed wait if not needed
return false;
else {
try {
long startTime = System.currentTimeMillis();
long waitTime = msecs;
for (;;) {
wait(waitTime);
if (permits > 0) {
--permits;
return true;
}
else { // check for time-out
long now = System.currentTimeMillis();
waitTime = msecs - (now - startTime);
if (waitTime <= 0)
return false;
}
}
}
catch(InterruptedException ie) {
notify();
throw ie;
}
}
}
}
}
3.7.2 Delegated Actions
Joint action designs can be used to address a potential source of inefficiency in guarded methods in which different threads in a wait set are waiting for different logical conditions. A notifyAll intended to alert threads about one condition also wakes up threads waiting for completely unrelated conditions. Useless signals, and the resulting “thundering herds” of context switches can be minimized by delegating operations with different wait conditions to different helper objects.
We achieved this effect with almost no effort using semaphores in 3.4.1. Here, we will proceed from the ground up, potentially achieving better performance by exploiting the special properties of particular design problems. The techniques here are worth using only when a design problem is amenable to optimizations that can be applied only to bare waiting and notification mechanics.
Splitting classes with state-dependent actions extends ideas seen in 2.4.2 for splitting objects with respect to locks, as well as some from the States as Objects pattern (see Design Patterns). However, the design space is restricted to a narrow range of constructions because of constraints including:
Since helpers must access common state, you cannot fully isolate each helper along with its own self-contained representation. Independent access to common representations across helpers requires appropriate synchronization.
Each of the helpers that might affect guard conditions for another must provide it with effective notifications while avoiding liveness problems.
Synchronization of helper methods involving wait mechanics must avoid nested monitor problems (3.3.4).
3.7.2.1 Design steps
A general approach to these constraints is first to decompose the Host class into its smallest possible pieces: one class for the shared state representation and one each per kind of helper. You can then deal with the resulting coordinated joint action design problem. Finally, you can organize the pieces into useful classes:
Define a class, say Representation, to hold fields used across more than one method. This is just a record-style class with non-private fields, allowing arbitrary accesses and updates to be performed within special synchronized blocks.
Define a Helper class for each set of functionality that shares the same wait conditions. Each Helper class requires instance variables referencing the host and the representation (this reference may be indirect via the host).
Define the Host class as a pass-through: Each public Host method should be an unsynchronized forwarding method. Also, define unsynchronized methods designed to be called by helpers whenever they change states in ways that may affect other helpers. Relay the associated notify or notifyAll calls. (Alternatively, these notifications can be sent directly among helpers.) The host should also initialize all helper objects in its constructor.
Each helper method must avoid liveness failures while still preserving safety. In particular:
If the condition checks involve the shared representation, they must be performed while both the representation and helper are locked.
The representation lock must be released before entering any wait, but the lock on the helper must be retained to avoid missed signals (see 3.2.4 in which waits are started after notifications have already occurred.
Notification relays must be initiated without synchronization to avoid potential deadlocks.
)
A generic helper method can take the form:
void doM() throws InterruptedException {
for(;;) { // wait loop
synchronized(this) { // check->wait must lock this
synchronized(rep) { // check->act must lock rep
boolean canAct = inRightState(rep);
if (canAct) {
update(rep); // the guarded action
break;
}
} // release rep lock before wait
wait(); // fall-through if !canAct
} // release lock before signal
}
host.signalChange();
}
3.7.2.2 Bounded buffers
As our last examples of BoundedBuffer, we will create delegated versions that also exploit special characteristics of the underlying data structure and algorithm to obtain better performance. The final result is just slightly faster than previous versions, but serves to exemplify techniques.
First, we need to split up helper objects to do put and take. Delegation designs normally require a helper class per method. But here, we can get away with only one helper class (with two instances) by exploiting an observation about ownership transfers. As noted in 2.3.4, the single operation exchange can be used for both put-style and take-style transfers. For example, exchange(null) accomplishes a take. The buffer-based version of exchange replaces the old value with the argument at the current array slot and then circularly advances to the next array position.
It is convenient to define the helper class Exchanger as an inner class to gain access to the host and the array serving as the shared representation. We also need a slot counter variable to indicate when an exchange operation must stall because there are no more items. For the helper doing put, the counter starts off at capacity; for take, it starts off at zero. An exchange operation can proceed only if the number of slots is greater than zero.
Each successful exchange operation decrements the count. Waits on zero counts can be released only by the helper performing the complementary operation, which must provide a notification. This is implemented by issuing an addedSlotNotification to the other exchanger, as relayed through the host.
Another special consideration in this particular design leads to another minor economy. Even though the data array must be shared across the two helpers, it does not need synchronization protection so long as put and take are the only operations supported. This can be ensured by declaring the host class final. We can make do without a synchronization lock because, in this algorithm, any executing put must be operating on a different array slot than the one being accessed by any executing take. Additionally, the outer synchronizations suffice to preclude memory visibility problems (see 2.2.7). In contrast, the BoundedBufferWithSemaphores class requires locking around array operations because it does not otherwise restrict at most one put or take to occur at any given time.
As a further performance enhancement, notifications here use notify, since the conditions for its use (discussed in 3.2.4.2) are met: (1) Each waiting task in each helper is waiting on the same logical condition (non-emptiness for take, and non-fullness for put). (2) Each notification enables at most a single thread to continue — each put enables one take, and each take enables one put. (3) We can re- notify to deal with interruptions.
And to squeeze another bit of efficiency out of this, it is simple here to (conservatively) track whether there are any waiting threads, and issue notify only if there can be threads that need notifying. The performance effect of this tactic varies across JVM implementations. As notify operations become increasingly cheap, the minor bookkeeping overhead here to avoid calls becomes decreasingly worthwhile.
final class BoundedBufferWithDelegates {
private Object[] array;
private Exchanger putter;
private Exchanger taker;
public BoundedBufferWithDelegates(int capacity)
throws IllegalArgumentException {
if (capacity <= 0) throw new IllegalArgumentException();
array = new Object[capacity];
putter = new Exchanger(capacity);
taker = new Exchanger(0);
}
public void put(Object x) throws InterruptedException {
putter.exchange(x);
}
public Object take() throws InterruptedException {
return taker.exchange(null);
}
void removedSlotNotification(Exchanger h) { // relay
if (h == putter) taker.addedSlotNotification();
else putter.addedSlotNotification();
}
protected class Exchanger { // Inner class
protected int ptr = 0; // circular index
protected int slots; // number of usable slots
protected int waiting = 0; // number of waiting threads
Exchanger(int n) { slots = n; }
synchronized void addedSlotNotification() {
++slots;
if (waiting > 0) // unblock a single waiting thread
notify();
}
Object exchange(Object x) throws InterruptedException {
Object old = null; // return value
synchronized(this) {
while (slots <= 0) { // wait for slot
++waiting;
try {
wait();
}
catch(InterruptedException ie) {
notify();
throw ie;
}
finally {
--waiting;
}
}
--slots; // use slot
old = array[ptr];
array[ptr] = x;
ptr = (ptr + 1) % array.length; // advance position
}
removedSlotNotification(this); // notify of change
return old;
}
}
}
3.7.2.3 Collapsing classes
Synchronization splitting of all kinds can be accomplished in two ways. In the case of lock- splitting (2.4.2), you can either create new helper classes and forward operations from the host, or you can just keep the methods in the host but invoke them under synchronization of Objects that conceptually represent the different helpers.
The same principle holds when splitting state-dependent actions. Rather than delegating actions to helpers, you can keep the methods in the host class, adding Objects that conceptually represent the helpers. Objects used solely for synchronization serve as locks. Those used for waiting and notification serve as monitors — places to put threads that need to wait and be notified.
Combining helpers into a host class makes the host class more complex but also potentially more efficient, due to short-circuited method calls and the like. Performing such simplifications along the way, we can define a more concise, slightly more efficient, and surely more frightening version of BoundedBuffer:
final class BoundedBufferWithMonitorObjects {
private final Object[] array; // the elements
private int putPtr = 0; // circular indices
private int takePtr = 0;
private int emptySlots; // slot counts
private int usedSlots = 0;
private int waitingPuts = 0; // counts of waiting threads
private int waitingTakes = 0;
private final Object putMonitor = new Object();
private final Object takeMonitor = new Object();
public BoundedBufferWithMonitorObjects(int capacity)
throws IllegalArgumentException {
if (capacity <= 0)
throw new IllegalArgumentException();
array = new Object[capacity];
emptySlots = capacity;
}
public void put(Object x) throws InterruptedException {
synchronized(putMonitor) {
while (emptySlots <= 0) {
++waitingPuts;
try { putMonitor.wait(); }
catch(InterruptedException ie) {
putMonitor.notify();
throw ie;
}
finally { --waitingPuts; }
}
--emptySlots;
array[putPtr] = x;
putPtr = (putPtr + 1) % array.length;
}
synchronized(takeMonitor) { // directly notify
++usedSlots;
if (waitingTakes > 0)
takeMonitor.notify();
}
}
public Object take() throws InterruptedException {
Object old = null;
synchronized(takeMonitor) {
while (usedSlots <= 0) {
++waitingTakes;
try { takeMonitor.wait(); }
catch(InterruptedException ie) {
takeMonitor.notify();
throw ie;
}
finally { --waitingTakes; }
}
--usedSlots;
old = array[takePtr];
array[takePtr] = null;
takePtr = (takePtr + 1) % array.length;
}
synchronized(putMonitor) {
++emptySlots;
if (waitingPuts > 0)
putMonitor.notify();
}
return old;
}
}
3.7.3 Specific Notifications
Instead of treating the little helper Objects in classes such as BoundedBufferWithMonitorObjects as the culmination of design efforts, you can treat them as tools for implementing any design problem amenable to solution via split monitors. The Specific Notification pattern devised by Tom Cargill takes precisely this tactic.
The basic idea is to put tasks to sleep via waits in monitors — ordinary Objects (or more typically, instances of simple classes that help with bookkeeping) used solely for their wait sets. One monitor is used for each task or set of tasks that must be individually notified. In most cases, this requires one monitor per thread; in others, a group of threads that should all be awakened at once can use the same monitor. These monitors serve similar purposes to the condition queues that are natively supported in some monitor-based concurrent programming languages (see 3.4.4). The main difference is that, without native support, these helper monitors must be dealt with more carefully to avoid nesting problems.
Specific notifications may be useful whenever you need threads to wait and the notification policy does not dynamically depend on the properties of the threads. Once a thread is put in its wait set, it is impossible to access it in any way other than to wake it up. Among the common applications to which these constraints apply are:
Supporting specific scheduling policies through the use of an explicit queue (for example FIFO, LIFO, priority).
Dividing incoming tasks into different queues depending on the method they are waiting to perform. This can be used to extend techniques based on conflict sets (see 3.3.2).
However, while it may be tempting to combine support for scheduling constraints such as FIFO with constraints based on logical state or execution state, interactions between these two applications usually lead to both conceptual and logistical problems. For example, you need to consider cases where thread A should be enabled before thread B because it arrived earlier, but thread B is logically able to proceed while thread A is not. This may necessitate elaborate apparatus to requeue threads, manage locking orders, and arbitrarily handle corner cases.
3.7.3.1 Design steps
The main design steps are specializations of those described in 3.7.2.1. Create or modify a class, say Host, as follows:
-
For each thread or set of threads that needs specific notification, create an object serving as a monitor. These monitors may be arranged in arrays or other collections, or dynamically created during execution.
-
Set up bookkeeping in the classes serving as monitors to manage waiting and notification operations and their interactions with time-out and interruption policies. As shown in the WaitNode class in 3.7.3.2, this usually entails maintaining a released field to remember if a waiting thread has been released due to notification, interruption, or time-out. These classes may then support methods, say doWait, doTimedWait, doNotify, and doNotifyAll, that perform reliable waiting and notification and deal with interrupts and time-outs in the desired fashion. If you cannot add bookkeeping to the classes serving as monitors, then these matters need to be addressed in the Host class methods.
-
In each Host method in which tasks are to be suspended, use monitor.doWait() with the appropriate monitor object. This code must avoid nested monitor problems by ensuring that the wait is performed within code regions that are not synchronized on the host object. The simplest and most desirable form is:
boolean needToWait; // to remember value after synch exit synchronized (this) { needToWait = ...; if (needToWait) enqueue(monitor); // or any similar bookkeeping } if (needToWait) monitor.doWait(); -
In each method in which tasks are to be resumed, use monitor.doNotify(), also handling the consequences of time-out or interruption.
3.7.3.2 FIFO semaphores
Specific notifications can be used to implement the kinds of First-In-First-Out semaphore classes discussed in 3.4.1.5. FIFO semaphores can in turn be used to build other utilities that rely on FIFO properties.
The following FIFOSemaphore class (a streamlined version of one in util.concurrent) is defined as a subclass of the generic Semaphore class from 3.7.1. The FIFOSemaphore class maintains a linked WaitQueue holding WaitNodes, each serving as a monitor. An acquire operation that cannot immediately obtain a permit enqueues a new monitor object that enters a wait. The release operation dequeues the oldest waiting node and notifies it.
A released field in each WaitNode helps manage interactions between notifications and interruptions. During a release, any monitor that has aborted due to interruption is skipped over. Conversely, an interrupted wait first checks to see if it has been notified in addition to being interrupted. If so, it must roll forward, ignoring the exception but resetting interrupt status (see 3.1.2) to preserve cancellation status. (An unshown doTimedWait method can be implemented similarly, by setting released status upon time-out.) The potential for interruptions at inconvenient times accounts for the retry loop in release.
The interactions among FIFOSemaphore, WaitQueue, and WaitNode ensure the necessary atomicity while avoiding nested monitor problems. They also demonstrate some of the arbitrariness of decisions surrounding support of FIFO policies. We can promise only that the semaphore is FIFO with respect to an arbitrary start point and end point. The start point commences with the synchronized(this) in acquire. The end point normally occurs upon release from a wait due to notify. Two threads entering acquire might obtain the lock in different orders from their arrivals, for example if the first one is scheduled out by the JVM before it hits the synchronized(this) statement. Similarly, a thread released before another might finally return to its caller after the other. Especially on multiprocessors, the class provides as strong a guarantee as users of the class should expect.
The scheduling rules can be changed by substituting a different kind of queue here; for example one based on Thread.getPriority. However, it is trickier to adapt this class to handle semantic restrictions based on execution or logical state. Most semantic restrictions require notified or interrupted threads to acquire additional locks. This would introduce complications to the scheme here that exploits the fact that awakened threads need not access the main lock. These would need to be resolved in an application-specific manner.
class FIFOSemaphore extends Semaphore {
protected final WaitQueue queue = new WaitQueue();
public FIFOSemaphore(long initialPermits) {
super(initialPermits);
}
public void acquire() throws InterruptedException {
if (Thread.interrupted()) throw new InterruptedException();
WaitNode node = null;
synchronized(this) {
if (permits > 0) { // no need to queue
--permits;
return;
}
else {
node = new WaitNode();
queue.enq(node);
}
}
// must release lock before node wait
node.doWait();
}
public synchronized void release() {
for (;;) { // retry until success
WaitNode node = queue.deq();
if (node == null) { // queue is empty
++permits;
return;
}
else if (node.doNotify())
return;
// else node was already released due to
// interruption or time-out, so must retry
}
}
// Queue node class. Each node serves as a monitor.
protected static class WaitNode {
boolean released = false;
WaitNode next = null; // to arrange in linked list
synchronized void doWait() throws InterruptedException {
try {
while (!released)
wait();
}
catch (InterruptedException ie) {
if (!released) { // interrupted before notified
// Suppress future notifications:
released = true;
throw ie;
}
else { // interrupted after notified
// ignore exception but propagate status:
Thread.currentThread().interrupt();
}
}
}
synchronized boolean doNotify() { // return true if notified
if (released) // was interrupted or timed out
return false;
else {
released = true;
notify();
return true;
}
}
synchronized boolean doTimedWait(long msecs)
throws InterruptedException {
// similar
}
}
// Standard linked queue class.
// Used only when holding Semaphore lock.
protected static class WaitQueue {
protected WaitNode head = null;
protected WaitNode last = null;
protected void enq(WaitNode node) {
if (last == null)
head = last = node;
else {
last.next = node;
last = node;
}
}
protected WaitNode deq() {
WaitNode node = head;
if (node != null) {
head = node.next;
if (head == null) last = null;
node.next = null;
}
return node;
}
}
}
3.7.4 Further Readings
Techniques for implementing elementary locks using, for example, Dekker's algorithm and ticket-based algorithms are presented in the concurrent programming texts by Andrews and others listed in 1.2.5. However, there is no reason to base general-purpose concurrency control utilities on such techniques rather than on built-in synchronized methods and blocks.
The Specific Notification pattern was first described in:
Cargill, Thomas. “Specific Notification for Java Thread Synchronization”, Proceedings of the Pattern Languages of Programming Conference, 1996.
An alternative account of refining notifyAll constructions using specific notifications can be found in:
Mizuno, Masaaki. “A Structured Approach for Developing Concurrent Programs in Java”, Information Processing Letters, 1999.
Further examples and extensions of the techniques described in this section may be found in the online supplement.