Laying Out Text
Once you have a sequence of glyph runs, the next step is to lay them out in some kind of container. Before we look at how Cocoa does this, let's go back a few more years and looking at how the TeX language, which heavily influenced most modern typesetting systems, handles layout.
In TeX, everything is a box. The simplest boxes are individual glyphs, which are assembled into words, which are assembled into paragraphs, which are assembled into pages. The algorithm details are quite complex (Donald Knuth's PhD students have published several papers on the implementation details), but the general approach is the same at all levels. You're trying to fit a set of boxes optimally into a larger box, often backtracking and trying again if you go wrong.
One of the first tasks for a typesetter is finding the optimal place to insert line breaks. The simplest option is to group characters into words, insert words into a line until you've gone past the end, backtrack, and mark the line break. In reality, of course, selecting line breaks is a lot more complicated. Most proportional fonts have a little bit of slack in the kerning. You can move characters closer together or further apart by a (very) small amount without making them look ugly. You can also increase the spacing slightly between words. Finally, you can hyphenate words. Stretching or compressing words alters the amount of space that a run of glyphs takes, and adding hyphens increases the number of possible places for line breaks. Breaking lines with a hyphen isn't as good as breaking them at a word boundary, but looks better than having lines of very different lengths. A good typesetter needs to take all of this strategy into account.
The NSTypesetter class, used by NSLayoutManager, is responsible for all of these tasks. As of OS X 10.4, this job includes support for hyphenation. Hyphenation is very difficult to do correctly. Most early typesetting systems cheated, using a massive list of correct places to insert hyphens into words. Working out where hyphens go algorithmically is more difficult, not least because the rules differ between languages. In British English, for example, the decomposition is based on the root words, while in American English they're based on phonemes. Other languages have different rules.
NSTypesetter allows you to control the hyphenation. You can adjust the hyphenation factor, using the -setHyphenationFactor: method to control the penalty for inserting a hyphen when breaking a line. If you need finer control, you can subclass NSTypesetter and override -hyphenationFactorForGlyphAtIndex: to adjust the hyphenation factor for individual glyphs.