Software Documentation: Identifying Authoritative Knowledge

This chapter is from the book

Living Documentation: Continuous Knowledge Sharing by Design

Learn More Buy

This chapter is from the book

This chapter is from the book 

Living Documentation: Continuous Knowledge Sharing by Design

Learn More Buy

Guided Tours and Sightseeing Maps

It is easier to quickly discover the best of a new place with a guided tour or a sightseeing map. In a city you have never visited before, you can explore randomly, hoping to bump into something interesting. This is something I love to do during an afternoon within a longer stay, to get a feel of the place. However, if I have only one day and I want to quickly enjoy the best of the city, I take a guided tour with a theme. For example, I have excellent souvenirs from a guided tour of the old sky-scrapers in Chicago, where the guide knew how to get us into the historical lobbies to enjoy the low light that was typical of early light bulbs. One year later, I enjoyed an architecture boat tour of Chicago, from the river, which is another way to really grasp the city. In Berlin, I booked a tour dedicated to Berlin’s street art, which was eye opening. For me, the same street art I see every day without really noticing gains another dimension when put in a context with one hint from a guide.

But guided tours start at a fixed hour on a few days a week only, often take a few hours, and may be expensive. If you happen to pass through a city on the wrong day, you are out of luck. But you can still get a tourist map or printed guided tours. And, of course, there is probably an app for that! Plenty of apps provide guided tours and sightseeing maps, classified by themes such as attractions, eat, drink, dance, and concerts. In Chicago, the Society of Architecture offers free architecture tours on leaflets, too. And the Internet is full of resources to help plan a visit, such as “Top 20 List of Must-See Highlights,” “Itineraries to Help You Plan Your Visit,” and “101 Things to Do in London.”

The process of becoming familiar with a code base can be similar to the process of becoming familiar with a city. The best way for someone to discover it is with another human—a colleague. But if you want to provide an alternative to a human guide, you can take inspiration from the tourism industry and provide itineraries of guided tours and sightseeing maps. This tourism metaphor comes from Simon Brown, who writes the blog “Coding the Architecture” and also wrote the book Software Architecture for Developers, Volume 2.

One important thing to realize is that all the tourism guidance in a city is highly curated: Only a very small subset of all the possible content of the city is presented, for various reasons ranging from the historical importance of different landmarks to more money-related reasons.

One important difference between a code base and a city is that a code base can change more frequently than most cities. As a result, the guidance must be provided in such a way that the work to keep it up-to-date is minimized; of course, automation is a good option here.

Therefore: Provide curated guides of a code base, each with a big theme. Augment the code with extra metadata about the guided tour or a sightseeing map, and set up an automated mechanism to publish as often as desired an updated guide from these metadata. A sightseeing map or a guided tour based on tags in the code is a perfect example of the augmented code approach.

If the code base does not change much, a guided tour or a sightseeing map can be as simple as a bookmark with a list of the selected places of interest, each with a short description and a link to its location in the code. If the code is on a platform like GitHub, it is easy to link to any line of code directly. This bookmark can be created in HTML, Markdown, JSON, a dedicated bookmark format, or any other form you like.

If the code base changes frequently or may change frequently, a manually managed bookmark would require too much effort to keep up-to-date, so you might choose dynamic curation instead: Place tags on the selected locations in the code and rely on the search features of the IDE to instantly display the bookmarks. If needed, you can add metadata to the tags to enable reconstruction of the complete guided tour, simply by scanning the code base.

You might be worrying that adding tags about sightseeing maps or guided tours into the code will pollute the code—and you are right. These tags are not really about the tagged element intrinsically but about how it is used, so use this approach sparingly.

Consider your code base as a beautiful wilderness in the mountains where you go hiking. It is a protected area, and there are red-and-white hiking trail signs painted directly on the stones and on the trees. This paint does pollute the natural environment in a small way, but we all accept it because it’s very useful and degrades the landscape only a limited amount.

Creating a Sightseeing Map

To create a sightseeing map, you first create a custom annotation or attribute, and then you put it on the few most important places that you want to emphasize. To be effective, you should keep the number of places of interest low—ideally 5 to 7 and certainly no more than 10.

It may well be that one of the most difficult decision here is to name each annotation or attribute. Here are some naming suggestions:

• KeyLandmark or Landmark

• MustSee

• SightSeeingSite

• CoreConcept or CoreProcess

• PlaceOfInterest, PointOfInterest, or POI

• TopAttraction

• VIPCode

• KeyAlgorithm or KeyCalculation

For the approach to be useful, you also need to make sure everybody knows about the tags and how to search them.

A Sightseeing Map Example in C# and Java

Say that in creating a custom attribute, you decide to put it into its own assembly to be shared by other Visual Studio projects (which also means you don’t want anything to be specific to any particular project there). Here is how the attribute might look in C#:

1  public class KeyLandmarkAttribute: Attribute
2  {
3  }

You can now immediately use this attribute to tag your code:

1  public class Foo
2  {
3     [KeyLandmark("The main steps of enriching the Customer
4     Purchase from the initial order to a ready-to-confirm
5     purchase")]
6     public void Enrich(CustomerPurchase cp)
7     {
8        //... interesting stuff here
9     }
10 }

Java and C# are very similar. Here’s the same example, now in Java:

1  package acme.documentation.annotations;
2
3  /**
4 * Marks this place in the code as a point of interest
worth listing on a sightseeing map.
5 */
6
7  @Retention(RetentionPolicy.RUNTIME)
8  @Documented
9  public @interface PointOfInterest {
10
11  String description() default "";
12  }

And now we can use it as follows:

1  @PointOfInterest("Key calculation")
2  private double pricing(ExoticDerivative ...){
3  ...

An alternative naming could look like this:

1  @SightSeeingSite("This is our secret sauce")
2  public SupplyChainAllocation optimize(Inventory ...){
3  ...

In C# you would use the custom attribute as follows:

1 public class CoreConceptAttribute : Attribute
2
3 [CoreConcept("The main steps of enriching the Customer
4 Purchase from the initial order to the ready to ship
5 Shipment Request")]

The wording is up to you, and you can use one generic annotation with a generic name like PointOfInterest and add the parameter Key calculation to tell precisely what it is about. Alternatively, you could decide to create one annotation for each kind of point of interest:

1  @KeyCalculation()
2  private double pricing(ExoticDerivative ...){
3  ...

Creating a Guided Tour

In the example shown in this section, the idea is to take a newcomer by the hand along the complete chain of processing of an incoming transaction, from the event listener on a message queue down to storing the outgoing report to the database. Note that even though it strictly separates the domain logic and the infrastructure logic, this guided tour spans both business logic elements with elements of the underlying infrastructure in order to give a complete picture of a complete execution path.

This guided tour currently has six steps, each of which is anchored on a code element that can be a class, a method, a field, or a package.

This example uses the custom annotation @GuidedTour with some parameters:

• The name of the guided tour: This is optional if there is only one tour, or if you prefer one annotation by guided tour, like @QuickDevTour.

• A description of the step in the context of this tour: This is in contrast to the Javadoc comment on the element, which describes the element for what it is and not necessarily for how it is used.

• A rank: The rank can be expressed as a number or anything comparable, and it is used to order the steps when presenting them to the visitor.

Here’s an example of a guided tour:

1 /**
2 * Listens to incoming fuel card transactions from the
3 * external system of the Fuel Card Provider
4 */
5 @GuidedTour(name = "Quick Developer Tour",
6     description = "The MQ listener which triggers a full
7 chain of processing", rank = 1)
8 public class FuelCardTxListener {

It then goes through other steps, until the last one:

1 @GuidedTour(name = "Quick Developer Tour",
2     description = "The DAO to store the resulting
3     fuel card reports after processing", rank = 7)
4 public class ReportDAO {
5
6 public void save(FuelCardTransactionReport report){
7 ...

If you wanted to provide a simple selection of points of interest only for an audience of developers, you could stop here and rely on the user to do a search of the custom annotation to get the IDE to present the tour as a whole:

1  Search results for 'flottio.annotations.GuidedTour'
6 References:
2
3  flottio.fuelcardmonitoring.domain - (src/main/java/l...)
4  - FuelCardMonitoring
5  - monitor(FuelCardTransaction, Vehicle)
6  - FuelCardTransaction
7  - FuelCardTransactionReport
8
9  flottio.fuelcardmonitoring.infra - (src/main/java/l...)
10 - FuelCardTxListener
11 - ReportDAO

The recap is all here, but it is not pretty, and there is no ordering. This could be enough for a small list of the main landmarks that a developer can explore in any order desired, though, so do not discount the value of the integrated approach, as it is much simpler and may be more convenient than more sophisticated mechanisms.

However, this first case is not enough for a guided tour that is meant to be visited in order, from start to finish. So the next step is to create a living document out of it so that it is a living guided tour.

Creating a Living Guided Tour

Going further than in the preceding section, you can create a little mechanism to scan the code base to extract the information about each step of the guided tour and produce a synthetic report of the guided tour in the form of a ready-to-follow and ordered itinerary.

FuelCardTxListener

The MQ listener which triggers a full chain of processing.

Listens to incoming fuel card transactions from the external system of the fuel card provider.

FuelCardTransaction

The incoming fuel card transaction.

A transaction, between a card and a merchant, as reported by the fuel card provider.

FuelCardMonitoring

The service that takes care of all the fuel card monitoring.

Monitoring of fuel card use to help improve fuel efficiency and detect fuel leakages and potential driver misbehaviors.

monitor(transaction, vehicle)

The method that does all the potential fraud detection for an incoming fuel card transaction.

1  public FuelCardTransactionReport monitor(FuelCardTransaction
2  transaction, Vehicle vehicle) {
3     List<String> issues = new ArrayList<String>();
4
5     verifyFuelQuantity(transaction, vehicle, issues);
6     verifyVehicleLocation(transaction, vehicle, issues);
7
8  MonitoringStatus status
9     = issues.isEmpty() ? VERIFIED : ANOMALY;
9  return new FuelCardTransactionReport(
10    transaction, status, issues);
11 }

FuelCardTransactionReport

The report for an incoming fuel card transaction.

The fuel card monitoring report for one transaction, with a status and any potential issue found.

ReportDAO

The DAO to store the resulting fuel card reports after processing.

Note that in this guided tour, each title is a link to the corresponding line of code on GitHub. When the point of interest is a method (like the monitor() method), I include its block of code verbatim from GitHub, for convenience. In a similar fashion, when the point of interest is a class, I might include an outline of the nonstatic fields and the public methods if I find it convenient and relevant to the focus of the guided tour.

This living guided tour document is generated in Markdown, for convenience. Then a tool like Maven site (or sbt or any other similar tool) could do the rendering to a web page or in any other format. An alternative, as shown here, is to use a Java-Script library to render the Markdown in the browser, which requires no additional toolchain.

An alternative to using strings in the guided tour annotations would be to use enums, which take care of naming, descriptions, and ordering at the same time. However, this moves the descriptions of each step of the guided tour from the annotated code to the enum class, as you can see here:

1 public enum PaymentJourneySteps {
2    REST_ENDPOINT("The single page app call this endpoint with
the id of the shopping cart"),
3    AUTH_FILTER("The call is being authenticated"),
4    AUDIT_TRAIL("The call is audit-trailed in case of dispute
and to comply to regulation"),
5
6    PAYMENT_SERVICE("Now enter the actual service to perform
the job"),
7
8    REDIRECT("The response from the payment is sent through a
redirect");
9
10 private final String description;
11 }

This enum is then used as the value in the annotation:

1  @PaymentJourney(PaymentJourneySteps.PAYMENT_SERVICE)
2  public class PaymentService...

The Implementation of the Guided Tour

In Java you can use a Doclet-like library called QDox to do the implementation grunt work, which allows you to access the Javadoc comments. If you don’t need Javadoc, then any parser and even pain reflection could work.

QDox scans every Java file in src/main/java, and from the collection of parsed elements, you can do the filtering by annotation. When a Java element (class, method, package, and so on) has the custom GuidedTour annotation, it is included in the guided tour. You can extract the parameters of the annotation and also extract the name, Javadoc comment, line of code, and other information (including the code itself, when necessary). You can then turn all that into fragments of Markdown for each step, stored in a map sorted by the step rank criteria. This way, when the scan is done, you can render the whole document by concatenating all the fragments in the rank ordering.

Of course, the devil is in the details, and this kind of code can quickly grow hairy, depending on how demanding you are with respect to the end result. Scanning code and traversing the Java or C# metamodel is not always nice. In the worst case, you could even end up with a visitor pattern. I expect that more mainstream adoption of these practices will lead to new small libraries which will take care of most of the grunt work for common use cases.

A Poor Man’s Literate Programming

A guided tour is reminiscent of literate programming but in reverse: Instead of having prose with code, a guided tour has code with prose. For a sightseeing map, you only have to select the points of interest and group them by big themes. For a guided tour, you need to devise a linear ordering of the code elements. In literate programming, you also tell a linear story that progresses through the code and ends up with a document explaining the reasoning and the corresponding software at the same time.

A guided tour or sightseeing map is not just a documentation concern but also a way to encourage continuous reflection on your own work as you do it. It would therefore be a good idea to document a guided tour as soon as you are building the early walking skeleton of the application. This way, you will benefit from the thoughtful effect of doing the documentation at the same time of doing the work.