This chapter is from the book
This chapter is from the book
This chapter is from the book
9.3 HTTP Connections
You can read from a URL by using the input stream returned from URL.getInputStream method. However, if you want additional information about a web resource, or if you want to write data, you need more control over the process than the URL class provides. The URLConnection class was designed before HTTP was the universal protocol of the Web. It provides support for a number of protocols, but its HTTP support is somewhat cumbersome. When the decision was made to support HTTP/2, it became clear that it would be best to provide a modern client interface instead of reworking the existing API. The HttpClient provides a more convenient API and HTTP/2 support.
In the following sections, I provide a cookbook for using the HttpURLConnection class, and then give an overview of the API.
9.3.1 The URLConnection and HttpURLConnection Classes
To use the URLConnection class, follow these steps:
Get an URLConnection object:
URLConnection connection = url.openConnection();
For an HTTP URL, the returned object is actually an instance of HttpURLConnection.
If desired, set request properties:
connection.setRequestProperty("Accept-Charset", "UTF-8, ISO-8859-1");If a key has multiple values, separate them by commas.
To send data to the server, call
connection.setDoOutput(true); try (OutputStream out = connection.getOutputStream()) { // Write to out }If you want to read the response headers and you haven’t called getOutputStream, call
connection.connect();
Then query the header information:
Map<String, List<String>> headers = connection.getHeaderFields();
For each key, you get a list of values since there may be multiple header fields with the same key.
Read the response:
try (InputStream in = connection.getInputStream()) { // Read from in }
A common use case is to post form data. The URLConnection class automatically sets the content type to application/x-www-form-urlencoded when writing data to a HTTP URL, but you need to encode the name/value pairs:
URL url = ...;
URLConnection connection = url.openConnection();
connection.setDoOutput(true);
try (var out = new OutputStreamWriter(
connection.getOutputStream(), StandardCharsets.UTF_8)) {
Map<String, String> postData = ...;
boolean first = true;
for (Map.Entry<String, String> entry : postData.entrySet()) {
if (first) first = false;
else out.write("&");
out.write(URLEncoder.encode(entry.getKey(), "UTF-8"));
out.write("=");
out.write(URLEncoder.encode(entry.getValue(), "UTF-8"));
}
}
try (InputStream in = connection.getInputStream()) {
...
}
9.3.2 The HTTP Client API
The HTTP client API provides another mechanism for connecting to a web server which is simpler than the URLConnection class with its rather fussy set of stages. More importantly, the implementation supports HTTP/2.
An HttpClient can issue requests and receive responses. You get a client by calling
HttpClient client = HttpClient.newHttpClient();
Alternatively, if you need to configure the client, use a builder API like this:
HttpClient client = HttpClient.newBuilder()
.followRedirects(HttpClient.Redirect.ALWAYS)
.build();
That is, you get a builder, call methods to customize the item that is going to be built, and then call the build method to finalize the building process. This is a common pattern for constructing immutable objects.
Follow the same pattern for formulating requests. Here is a GET request:
HttpRequest request = HttpRequest.newBuilder()
.uri(new URI("https://horstmann.com"))
.GET()
.build();
The URI is the “uniform resource identifier” which is, when using HTTP, the same as a URL. However, in Java, the URL class has methods for actually opening a connection to a URL, whereas the URI class is only concerned with the syntax (scheme, host, port, path, query, fragment, and so on).
When sending the request, you have to tell the client how to handle the response. If you just want the body as a string, send the request with a HttpResponse.BodyHandlers.ofString(), like this:
HttpResponse<String> response
= client.send(request, HttpResponse.BodyHandlers.ofString());
The HttpResponse class is a template whose type denotes the type of the body. You get the response body string simply as
String bodyString = response.body();
There are other response body handlers that get the response as a byte array or a file. One can hope that eventually the JDK will support JSON and provide a JSON handler.
With a POST request, you similarly need a “body publisher” that turns the request data into the data that is being posted. There are body publishers for strings, byte arrays, and files. Again, one can hope that the library designers will wake up to the reality that most POST requests involve form data or JSON objects, and provide appropriate publishers.
In the meantime, to send a form post, you need to URL-encode the request data, just like in the preceding section.
Map<String, String> postData = ...;
boolean first = true;
var body = new StringBuilder();
for (Map.Entry<String, String> entry : postData.entrySet()) {
if (first) first = false;
else body.append("&");
body.append(URLEncoder.encode(entry.getKey(), "UTF-8"));
body.append("=");
body.append(URLEncoder.encode(entry.getValue(), "UTF-8"));
}
HttpRequest request = HttpRequest.newBuilder()
.uri(httpUrlString)
.header("Content-Type", "application/x-www-form-urlencoded")
.POST(HttpRequest.BodyPublishers.ofString(body.toString()))
.build();
Note that, unlike with the URLConnection class, you need to specify the content type for forms.
Similarly, for posting JSON data, you specify the content type and provide a JSON string.
The HttpResponse object also yields the status code and the response headers.
int status = response.statusCode(); HttpHeaders responseHeaders = response.headers();
You can turn the HttpHeaders object into a map:
Map<String, List<String>> headerMap = responseHeaders.map();
The map values are lists since in HTTP, each key can have multiple values.
If you just want the value of a particular key, and you know that there won’t be multiple values, call the firstValue method:
Optional<String> lastModified = headerMap.firstValue("Last-Modified");
You get the response value or an empty optional if none was supplied.
TIP
To enable logging for the HttpClient, add this line to net.properties in your JDK:
jdk.httpclient.HttpClient.log=all
Instead of all, you can specify a comma-separated list of headers, requests, content, errors, ssl, trace, and frames, optionally followed by :control, :data, :window, or :all. Don’t use any spaces.
Then set the logging level for the logger named jdk.httpclient.HttpClient to INFO, for example by adding this line to the logging.properties file in your JDK:
jdk.httpclient.HttpClient.level=INFO