Software System Logging
Mark Seemann helps programmers make source code easier to maintain. In this excerpt from his book, he outlines what you should be logging while building software systems.
From the book
From the book
I once worked with a team that had found just the right amount of logging. We were developing and maintaining a suite of REST APIs. Each API would log details1 of each HTTP request and the HTTP response it returned. It would also log all database interactions, including input arguments and the entire result set returned by the database.
I don't recall that there was a single defect that we couldn't track down and understand. It was just the right amount of logging. Most development organisations log too much. Particularly when it comes to instrumentation, I often see examples of 'overlogging'. When logging is done to support future troubleshooting, you can't predict what you're going to need, so it's better to log too much data than too little. Or, at least, that's the rationale for 'overlogging'.
It'd be even better to log only what you need. Not too little, not too much, but just the right amount of logging. Obviously, we should call this Goldilogs.
How do you know what to log? How do you know that you've logged everything that you'll need, when you don't know your future needs?
The key is repeatability. Just like you should be able to reproduce builds and repeat deployments, you should also be able to reproduce execution. If you can replay what happened when a problem manifested itself, you can troubleshoot it. You need to log just enough data to enable you to repeat execution. How do you identify that data?
Consider a line of code like this listing 1. Would you log this?
int z = x + y;
Listing 1: Would you log this?
It might make sense to log what x and y are, particularly if these values are run-time values (e.g. entered by a user, the result of a web service call, etc.). You might do something like listing 2.
Would you ever log the result, though, like in listing 3?
Log.Debug($"Adding {x} and {y}.");
int z = x + y;
Listing 2: Logging input values might make sense.
Log.Debug($"Adding {x} and {y}.");
int z = x + y;
Log.Debug($"Result of addition: {z}");
Listing 3: Does it make sense to log the output of addition?
There's no reason to log the result of the calculation. Addition is a pure function; it's deterministic. If you know the inputs, you can always repeat the calculation to get the output. Two plus two is always four.
The more your code is composed from pure functions, the less you need to log. This is one of the many reasons that referential transparency is so desirable, and why you should favour the functional core, imperative shell style of architecture.
Log all impure actions, but no more.
Log everything that you can't reproduce. That includes all non-deterministic code like getting the current date, time of day, generating a random number, reading from a file or database, et cetera. It also includes everything that has side effects. All else you don't need to log.
Of course, if your code base doesn't separate pure functions from impure actions, you'll have to log everything.
1 Except sensitive information like JSON Web Tokens, which we redacted.