Processing Input and Output

This chapter is from the book

Core Java for the Impatient, 3rd Edition

Learn More Buy

This chapter is from the book

This chapter is from the book 

Core Java for the Impatient, 3rd Edition

Learn More Buy

9.5 Serialization

In the following sections, you will learn about object serialization—a mechanism for turning an object into a bunch of bytes that can be shipped somewhere else or stored on disk, and for reconstituting the object from those bytes.

Serialization is an essential tool for distributed processing, where objects are shipped from one virtual machine to another. It is also used for fail-over and load balancing, when serialized objects can be moved to another server. If you work with server-side software, you will often need to enable serialization for classes. The following sections tell you how to do that.

9.5.1 The Serializable Interface

In order for an object to be serialized—that is, turned into a bunch of bytes—it must be an instance of a class that implements the Serializable interface. This is a marker interface with no methods, similar to the Cloneable interface that you saw in Chapter 4.

For example, to make Employee objects serializable, the class needs to be declared as

public class Employee implements Serializable {
    private String name;
    private double salary;
    ...
}

It is appropriate for a class to implement the Serializable interface if all instance variables have primitive or enum type, or contain references to serializable objects. Many classes in the standard library are serializable. Arrays and the collection classes that you saw in Chapter 7 are serializable provided their elements are.

In the case of the Employee class, and indeed with most classes, there is no problem. In the following sections, you will see what to do when a little extra help is needed.

To serialize objects, you need an ObjectOutputStream, which is constructed with another OutputStream that receives the actual bytes.

var out = new ObjectOutputStream(Files.newOutputStream(path));

Now call the writeObject method:

var peter = new Employee("Peter", 90000);
var paul = new Manager("Paul", 180000);
out.writeObject(peter);
out.writeObject(paul);

To read the objects back in, construct an ObjectInputStream:

var in = new ObjectInputStream(Files.newInputStream(path));

Retrieve the objects in the same order in which they were written, using the readObject method.

var e1 = (Employee) in.readObject();
var e2 = (Employee) in.readObject();

When an object is written, the name of the class and the names and values of all instance variables are saved. If the value of an instance variable belongs to a primitive type, it is saved as binary data. If it is an object, it is again written with the writeObject method.

When an object is read in, the process is reversed. The class name and the names and values of the instance variables are read, and the object is reconstituted.

There is just one catch. Suppose there were two references to the same object. Let’s say each employee has a reference to their boss:

var peter = new Employee("Peter", 90000);
var paul = new Manager("Barney", 105000);
var mary = new Manager("Mary", 180000);
peter.setBoss(mary);
paul.setBoss(mary);
out.writeObject(peter);
out.writeObject(paul);

When reading these two objects back in, both of them need to have the same boss, not two references to identical but distinct objects.

In order to achieve this, each object gets a serial number when it is saved. When you pass an object reference to writeObject, the ObjectOutputStream checks if the object reference was previously written. In that case, it just writes out the serial number and does not duplicate the contents of the object.

In the same way, an ObjectInputStream remembers all objects it has encountered. When reading in a reference to a repeated object, it simply yields a reference to the previously read object.

class Person // Not serializable
class Employee extends Person implements Serializable

9.5.2 Transient Instance Variables

Certain instance variables should not be serialized—for example, database connections that are meaningless when an object is reconstituted. Also, when an object keeps a cache of values, it might be better to drop the cache and recompute it instead of storing it.

To prevent an instance variable from being serialized, simply tag it with the transient modifier. Always mark instance variables as transient if they hold instances of nonserializable classes. Transient instance variables are skipped when objects are serialized.

9.5.3 The readObject and writeObject Methods

In rare cases, you need to tweak the serialization mechanism. A serializable class can add any desired action to the default read and write behavior, by defining methods with the signature

@Serial private void readObject(ObjectInputStream in)
    throws IOException, ClassNotFoundException
@Serial private void writeObject(ObjectOutputStream out)
    throws IOException

Then, the object headers continue to be written as usual, but the instance variables fields are no longer automatically serialized. Instead, these methods are called.

Note the @Serial annotation. The methods for tweaking serialization don’t belong to interfaces. Therefore, you can’t use the @Override annotation to have the compiler check the method declarations. The @Serial annotation is meant to enable the same checking for serialization methods. Up to Java 17, the javac compiler doesn’t do that checking, but it might happen in the future. Some IDEs check the annotation.

A number of classes in the java.awt.geom package, such as Point2D.Double, are not serializable. Now, suppose you want to serialize a class LabeledPoint that stores a String and a Point2D.Double. First, you need to mark the Point2D.Double field as transient to avoid a NotSerializableException.

public class LabeledPoint implements Serializable {
    private String label;
    private transient Point2D.Double point;
    ...
}

In the writeObject method, first write the object descriptor and the String field, label, by calling the defaultWriteObject method. This is a special method of the ObjectOutputStream class that can only be called from within a writeObject method of a serializable class. Then we write the point coordinates, using the standard DataOutput calls.

@Serial before private void writeObject(ObjectOutputStream out) throws IOException {
    out.defaultWriteObject();
    out.writeDouble(point.getX());
    out.writeDouble(point.getY());
}

In the readObject method, we reverse the process:

@Serial before private void readObject(ObjectInputStream in)
        throws IOException, ClassNotFoundException {
    in.defaultReadObject();
    double x = in.readDouble();
    double y = in.readDouble();
    point = new Point2D.Double(x, y);
}

Another example is the HashSet class that supplies its own readObject and writeObject methods. Instead of saving the internal structure of the hash table, the writeObject method simply saves the capacity, load factor, size, and elements. The readObject method reads back the capacity and load factor, constructs a new table, and inserts the elements.

The readObject and writeObject methods only need to save and load their data. They do not concern themselves with superclass data or any other class information.

The Date class uses this approach. Its writeObject method saves the milliseconds since the “epoch” (January 1, 1970). The data structure that caches calendar data is not saved.

@Serial private static final ObjectStreamField[] serialPersistentFields

9.5.4 The readExternal and writeExternal Methods

Instead of letting the serialization mechanism save and restore object data, a class can define its own mechanism. For example, you can encrypt the data or use a format that is more efficient than the serialization format.

To do this, a class must implement the Externalizable interface. This, in turn, requires it to define two methods:

public void readExternal(ObjectInputStream in)
    throws IOException
public void writeExternal(ObjectOutputStream out)
    throws IOException

Unlike the readObject and writeObject methods, these methods are fully responsible for saving and restoring the entire object, including the superclass data. When writing an object, the serialization mechanism merely records the class of the object in the output stream. When reading an externalizable object, the object input stream creates an object with the no-argument constructor and then calls the readExternal method.

In this example, the LabeledPixel class extends the serializable Point class, but it takes over the serialization of the class and superclass. The fields of the object are not stored in the standard serialization format. Instead, the data are placed in an opaque block.

public class LabeledPixel extends Point implements Externalizable {
    private String label;

    public LabeledPixel() {} // required for externalizable class

    @Override public void writeExternal(ObjectOutput out)
            throws IOException {
        out.writeInt((int) getX());
        out.writeInt((int) getY());
        out.writeUTF(label);
    }

    @Override public void readExternal(ObjectInput in)
            throws IOException, ClassNotFoundException {
        int x = in.readInt();
        int y = in.readInt();
        setLocation(x, y);
        label = in.readUTF();
    }
    ...
}

9.5.5 The readResolve and writeReplace Methods

We take it for granted that objects can only be constructed with the constructor. However, a deserialized object is not constructed. Its instance variables are simply restored from an object stream.

This is a problem if the constructor enforces some condition. For example, a singleton object may be implemented so that the constructor can only be called once. As another example, database entities can be constructed so that they always come from a pool of managed instances.

You shouldn’t implement your own mechanism for singletons. If you need a singleton, make an enumerated type with one instance that is, by convention, called INSTANCE.

public enum PersonDatabase {
    INSTANCE;

    public Person findById(int id) { ... }
    ...
}

This works because enum are guaranteed to be deserialized properly.

Now let’s suppose that you are in the rare situation where you want to control the identity of each deserialized instance. As an example, suppose a Person class wants to restore its instances from a database when deserializing. Then don’t serialize the object itself but some proxy that can locate or construct the object. Provide a writeReplace method that returns the proxy object:

public class Person implements Serializable {
    private int id;
    // Other instance variables
    ...

    @Serial private Object writeReplace() {
        return new PersonProxy(id);
    }
}

When a Person object is serialized, none of its instance variables are saved. Instead, the writeReplace method is called and its return value is serialized and written to the stream.

The proxy class needs to implement a readResolve method that yields a Person instance:

class PersonProxy implements Serializable {
    private int id;

    public PersonProxy(int id) {
        this.id = id;
    }

    @Serial private Object readResolve() {
        return PersonDatabase.INSTANCE.findById(id);
    }
}

When the readObject method finds a PersonProxy in an ObjectInputStream, it deserializes the proxy, calls its readResolve method, and returns the result.

9.5.6 Versioning

Serialization was intended for sending objects from one virtual machine to another, or for short-term persistence of state. If you use serialization for long-term persistence, or in any situation where classes can change between serialization and deserialization, you will need to consider what happens when your classes evolve. Can version 2 read the old data? Can the users who still use version 1 read the files produced by the new version?

The serialization mechanism supports a simple versioning scheme. When an object is serialized, both the name of the class and its serialVersionUID are written to the object stream. That unique identifier is assigned by the implementor, by defining an instance variable

@Serial private static final long serialVersionUID = 1L; // Version 1

When the class evolves in an incompatible way, the implementor should change the UID. Whenever a deserialized object has a nonmatching UID, the readObject method throws an InvalidClassException.

If the serialVersionUID matches, deserialization proceeds even if the implementation has changed. Each non-transient instance variable of the object to be read is set to the value in the serialized state, provided that the name and type match. All other instance variables are set to the default: null for object references, zero for numbers, and false for boolean values. Anything in the serialized state that doesn’t exist in the object to be read is ignored.

Is that process safe? Only the implementor of the class can tell. If it is, then the implementor should give the new version of the class the same serialVersionUID as the old version.

If you don’t assign a serialVersionUID, one is automatically generated by hashing a canonical description of the instance variables, methods, and supertypes. You can see the hash code with the serialver utility. The command

serialver ch09.sec05.Employee

displays

private static final long serialVersionUID = -4932578720821218323L;

When the class implementation changes, there is a very high probability that the hash code changes as well.

If you need to be able to read old version instances, and you are certain that is safe to do so, run serialver on the old version of your class and add the result to the new version.

@Serial private void readObjectNoData() throws ObjectStreamException

9.5.7 Deserialization and Security

During deserialization of a serializable class, objects are created without invoking any constructor of the class. Even if the class has a no-argument constructor, it is not used. The field values are set directly from the values of the object input stream.

Bypassing construction is a security risk. An attacker can craft bytes describing an invalid object that could have never been constructed. Suppose, for example, that the Employee constructor throws an exception when called with a negative salary. We would like to think that no Employee object can have a negative salary as a result. But it is not difficult to inspect the bytes for a serialized object and modify some of them. This way, one can craft bytes for an employee with a negative salary and then deserialize them.

A serializable class can optionally implement the ObjectInputValidation interface and define a validateObject method to check whether its objects are properly deserialized. For example, the Employee class can check that salaries are not negative:

public void validateObject() throws InvalidObjectException {
    System.out.println("validateObject");
    if (salary < 0)
        throw new InvalidObjectException("salary < 0");
}

Unfortunately, the method is not invoked automatically. To invoke it, you also must provide the following method:

@Serial private void readObject(ObjectInputStream in)
        throws IOException, ClassNotFoundException {
    in.registerValidation(this, 0);
    in.defaultReadObject();
}

The object is then scheduled for validation, and the validateObject method is called when this object and all dependent objects have been loaded. The second parameter lets you specify a priority. Validation requests with higher priorities are done first.

There are other security risks. Adversaries can create data structures that consume enough resources to crash a virtual machine. More insidiously, any class on the class path can be deserialized. Hackers have been devious about piecing together “gadget chains”—sequences of operations in various utility classes that use reflection and culminate in calling methods such as Runtime.exec with a string of their choice.

Any application that receives serialized data from untrusted sources over a network connection is vulnerable to such attacks. For example, some servers serialize session data and deserialize whatever data are returned in the HTTP session cookie.

You should avoid situations in which arbitrary data from untrusted sources are deserialized. In the example of session data, the server should sign the data, and only deserialize data with a valid signature.

A serialization filter mechanism can harden applications from such attacks. The filters see the names of deserialized classes and several metrics (stream size, array sizes, total number of references, longest chain of references). Based on those data, the deserialization can be aborted.

In its simplest form, you provide a pattern describing the valid and invalid classes. For example, if you start our sample serialization demo as

java -Djdk.serialFilter='serial.*;java.**;!*' serial.ObjectStreamTest

then the objects will be loaded. The filter allows all classes in the serial package and all classes whose package name starts with java, but no others. If you don’t allow java.**, or at least java.util.Date, deserialization fails.

You can place the filter pattern into a configuration file and specify multiple filters for different purposes. You can also implement your own filters. See https://docs.oracle.com/en/java/javase/17/core/serialization-filtering1.html for details.