This chapter is from the book
This chapter is from the book
This chapter is from the book
1.5 Fetching a URL
For many applications, access to information from the Internet is as important as access to the local file system. Go provides a collection of packages, grouped under net, that make it easy to send and receive information through the Internet, make low-level network connections, and set up servers, for which Go’s concurrency features (introduced in Chapter 8) are particularly useful.
To illustrate the minimum necessary to retrieve information over HTTP, here’s a simple program called fetch that fetches the content of each specified URL and prints it as uninterpreted text; it’s inspired by the invaluable utility curl. Obviously one would usually do more with such data, but this shows the basic idea. We will use this program frequently in the book.
gopl.io/ch1/fetch
// Fetch prints the content found at a URL.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"os"
)
func main() {
for _, url := range os.Args[1:] {
resp, err := http.Get(url)
if err != nil {
fmt.Fprintf(os.Stderr, "fetch: %v\n", err)
os.Exit(1)
}
b, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
fmt.Fprintf(os.Stderr, "fetch: reading %s: %v\n", url, err)
os.Exit(1)
}
fmt.Printf("%s", b)
}
}
This program introduces functions from two packages, net/http and io/ioutil. The http.Get function makes an HTTP request and, if there is no error, returns the result in the response struct resp. The Body field of resp contains the server response as a readable stream. Next, ioutil.ReadAll reads the entire response; the result is stored in b. The Body stream is closed to avoid leaking resources, and Printf writes the response to the standard output.
$ go build gopl.io/ch1/fetch $ ./fetch http://gopl.io <html> <head> <title>The Go Programming Language</title> ...
If the HTTP request fails, fetch reports the failure instead:
$ ./fetch http://bad.gopl.io fetch: Get http://bad.gopl.io: dial tcp: lookup bad.gopl.io: no such host
In either error case, os.Exit(1) causes the process to exit with a status code of 1.
Exercise 1.7: The function call io.Copy(dst, src) reads from src and writes to dst. Use it instead of ioutil.ReadAll to copy the response body to os.Stdout without requiring a buffer large enough to hold the entire stream. Be sure to check the error result of io.Copy.
Exercise 1.8: Modify fetch to add the prefix http:// to each argument URL if it is missing. You might want to use strings.HasPrefix.
Exercise 1.9: Modify fetch to also print the HTTP status code, found in resp.Status.