Text Processing with Objective-C, Part 1
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
My latest book, Objective-C Phrasebook, is appearing in shops about now. A phrasebook contains a lot of code snippets for solving very specific problems. In the next two articles, I'm going to take a look at a small number of the Objective-C patterns that are covered in the book, in the context of a real program.
As usual, I wrote this book using a domain-specific language implemented in LaTeX, containing lots of semantic markup. For example, each section in a phrasebook begins with a short code listing. I hate code listings in books that don't work, so each of these is in a separate source file for testing and is imported into the text during typesetting by a directive like this:
\startsnippet{stringIterateSlow.m}{7}{12}
This inserts lines 7-12 from the stringIterateSlow.m example into a dark gray box, uses the LaTeX listings package to do lexical highlighting, and adds a caption 'From stringIterateSlow.m' in italic at the bottom. The PDF for Safari Books Online and similar places uses the standard LaTeX output. The version that goes to the printer has the syntax highlighting colors changed to shades of gray and adds crop boxes, but apart from that is mostly the same.
Producing the ePub edition is a bit more tricky. ePub is basically XHTML wrapped up in a zip container with some extra metadata, so the real problem is generating XHTML from LaTeX. There are a few packages for doing this, the most popular of which is tex4ht, but the output is unsatisfying.
The problem is that they generate something that looks like the PDF or DVI output, rather than clean semantic [X]HTML markup. For example, a \section{} or \chapter{} directive in the LaTeX source should become some kind of heading tag, like <h1> or <h2>, in the HTML. In the code listings, I'd like keywords flagged with a class indicating that they are keywords so the entire thing can be styled with CSS. I couldn't find a tool that produced output that I was happy with, so I decided to write one.
Parsing LaTeX
The first problem was parsing the LaTeX source. This is one of those problems that is impossible in theory but possible in practice. LaTeX is a set of macros on top of TeX, and TeX is a very strange language. A TeX file is a program that looks a lot like a Turing Machine tape. The TeX interpreter reads the current character, updates the state table (including some registers), and continues based on the current state.
It's the sort of thing that only a theoretical computer scientist would design: an elegant mathematical model and a colossal pain in real-world use. Parsing it is impossible because it's a program, even when it looks like a markup language. You can change how anything is handled—including the escape sequences. tex4ht works around this by running the program and capturing the drawing commands that it executes, which is why it generates HTML that looks like the LaTeX output, but doesn't contain any of the semantic markup. Things like the \chapter{} tag in LaTeX are just macros that expand to some commands for drawing the string and adding it to the table of contents.
Fortunately, my LaTeX code separates content and presentation. All of the macro definitions and typesetting instructions are in the document class definition file or the preamble, while the text of the chapters contains just semantic markup. This makes the problem much simpler—I don't have to parse TeX; I have to parse a domain-specific language that TeX can also parse.
This language is quite simple. For the most part, it's simply plain text with some LaTeX-style commands and environments.