#typesetting

Rule-based typesetting with Ink

The plan for Ink took a bit of a turn a few nights ago. Erlang’s pattern matching was on my mind (having read about it earlier that evening) when I came across a passage from Mitchell’s Book Typography on house rules:

The following are examples of the authors’ own house rules:

• Speech to be indicated by single quotation marks (‘quote’ not “quote”)
• Circa shortened to italic c. with no word-space (c.1895 not c. 1895)
• Use multiplication symbol, not ‘x’ for dimensions (24 × 36 not 24 x 36)
• Letter-space strings of capital letters (ABCD not ABCD)

The two ideas came together and I saw that declarative typesetting (rule-based typesetting) could be a much nicer way to typeset.

For example: if you want to use the multiplication symbol for dimensions (× instead of x), you usually have to edit your source file (InDesign, TeX, etc.), find matching instances, and change them. It’s a one-time thing, a permanent transformation.

If, on the other hand, you had a rule that said “find any ‘x’ characters between numbers and transform them to ‘×’), then you could leave your source file alone and let the rule do the work for you instead. Using rules like this — textual and stylistic transformations applied at compile time — seems far more reusable, shareable, and easier to use.

From there, the Ink language morphed almost completely from how I was envisioning it earlier (TeX with nicer syntax, basically) to this new thing, inspired by Erlang, XSLT/XPath, CSS, Inform, and more.

A few quick notes before we get to the examples:

• I’m leaning very much towards a template/data separation, like in Django and Mustache and other template engines popular in web frameworks.
• For this rule-based thing to work well, you have to be able to set general rules but also fix specific cases where the rule doesn’t apply, or where it doesn’t make sense to write a general rule. At the moment I’m leaning towards having those specific fixes be rules as well, rather than tagging the source file. See the second rule listed in Exceptions below for an initial stab at this idea.
• Ink will be three languages — High Ink (or just Ink), which is the rule-based language shown below; Medium Ink, a tagged version of the source text with all the rules applied; and Low Ink (har har), a page description language that gets compiled to PDF.
• Splitting it up like this allows for extra flexibility — it would be relatively easy, for example, to write a compiler that takes Medium Ink and outputs HTML/CSS or EPUB or what have you. I don’t know that that would actually be a good idea, but it’s more possible this way. Splitting it up also makes it more manageable.
• Rephrased, the Ink-to-Medium-Ink compilation involves applying the rules intelligently. Medium-Ink-to-Low-Ink compilation involves the typesetting itself — line breaks, page breaks, etc. Low-Ink-to-PDF compilation will be easiest, translating the Low Ink code to PDF code.
• This morning I came across Jon Gold’s post on declarative design tools, with somewhat similar ideas. I like the direction he’s gone in with the combinations — it’s a nice workflow. We could do something in that vein here, with syntax to output a bunch of variations (typefaces, sizes, leading, etc.) with minimal effort.

Examples (gist)

Note: these are all first-draft thoughts on how to do this kind of a thing. Syntax is very much not set in stone at all — rule/endrule vs. rule { }, selector syntax, whether to use regular expressions or something simpler, etc.

rule
    size 6x9";
    # alternates
    size letter;
    size 210x297mm;
    size a4;

    font Arno Pro, 10/13pt;

    margin 1";
    inner-margin .75"; # overrides earlier margin value
endrule

# Named rule (for use later)
rule @times
    find \dx\d : replace \1 × \1
endrule @times

rule @year-labels
    # Find "a.d." and turn on the smcp OpenType feature
    find a.d. : feature smcp;
    # alternate way
    find [b.c. | b.c.e. | c.e. | a.d.] : feature smcp;
endrule

# Exceptions
rule
    # If a.d. is found in heading style text, don't run @year-labels on it
    find a.d. (style=heading) : ignore @year-labels;

    # Find a specific "m.a.d." and don't run @year-labels on it
    # This selector language needs a lot of work
    find /chapter:4/paragraph:2/word:[m.a.d.] : ignore @year-labels;

    # Usually, though, you'd want to revise the general rule like this
    # Only run the rule if it's by itself (word boundaries) and not a heading
    find \wa.d.\w (style!=heading) : feature smcp;
endrule

rule @paragraph-indents
    # Indent first line of all paragraphs 1.25em;
    find %paragraph : initial-indent 1.25em;

    # Override for first paragraph of a section/chapter
    find %paragraph:nth(1) : initial-indent 0;

    # Paragraphs following tables aren't indented
    find %table + %paragraph : initial-indent 0;

    # Hanging indents for paragraphs with hanging tag
    find %paragraph.hanging : hanging-indent .125";
endrule

# Tracking/kerning
rule
    # Find sequential uppercase and bump tracking up
    find [A-Z]+ | tracking 50;

    # Find V followed by a and kern -25
    find Va | kern -25;
endrule

# Coptic (named Unicode range)
range @coptic
    u+2c80 .. u+2cff;
    u+03e2 .. u+03ee;
    u+03ef; # separated for demo purposes
endrange

rule @coptic-font
    # Any characters in the Coptic range should be set in Antinoou
    find range @coptic : font Antinoou, 24/28pt, dlig;
endrule

# Unicode properties
rule @numbers
    # Replace any numbers with old-style figures
    find [ unicode.Nd | unicode.No ] : feature onum;
endrule

# Masters
master @a
    frame ...; # incomplete, but this part would have text frames
endmaster

# Apply masters
rule
    # All pages get master @a by default
    page * : master @a;

    # Remove master for pages i-iv
    page i-iv : master none;

    # Ignore @paragraph-indents rule on page 9
    page 9 : ignore @paragraph-indents;

    # Set data to be put into masters
    data @main, @index;

    # Variable used for running heads
    $title War and Peace;

    # Set running heads
    # (@inside-header etc. are frames in the master)
    page.odd : @inside-header $pagenum;
    page.odd : @outside-header $section; # set in data
    page.even : @center-header $title;
endrule

# Data
data @main
    transform @ingest;
    include preface.txt;
    include chapters/chapter*.txt;
enddata

data @index
    include index.json;
enddata

# Transform
# Used for an initial transform if necessary
transform @ingest (text)
    // JavaScript or other scripting language
    // These aren't great examples, though
    var response = text.replace(/CHAPTER/, "\n\nChapter");
    response = response.replace(/\wteh\w/, "the");
    return response;
endtransform

# Styles
# I don't have an example yet of how to use this, but imagine
# something ala InDesign or CSS
style @heading1
    font Warnock Pro;
    size 18/24pt;
    onum;
    -smcp; # turn off small caps
    space-before 1.24em;
    space-after 1.24em;
endstyle

Going forward

I’m open to feedback on all of this, of course, so feel free to comment or get in touch with me.

Ink

As mentioned on Twitter, I’ve decided to write my own typesetting engine, called Ink. Apparently I’m crazy.

The details are still very much in the air, but here are some quick notes:

• Written in Rust (for speed)
• Programmatic (sort of like TeX)
• Scripting language for extensibility (JavaScript or Lua or Python)
• Intended for use in typesetting book interiors, covers, and charts
• Possibly some kind of template/data division
• Full OpenType feature support (shaping via HarfBuzz)
• Custom PDF generation library (inkpdf)

Reasons for doing this insane thing:

• PlotDevice only runs on OS X and I want the source of my language charts to be usable on other platforms.
• I’d like to open source the books I typeset, so InDesign isn’t a great solution.
• TeX is powerful and well-seasoned and all, but it’s not exactly pleasant or easy to work with, especially for the kind of stuff I do.
• I’ll learn a lot and have fun while I’m at it.

The initial roadmap, not necessarily in order:

• Write inkpdf in Python (which I think will be a better fit for the charts anyway)
• Get familiar with HarfBuzz
• Learn Rust
• Port inkpdf to Rust
• Plan out the Ink language (I’ve started on this and it’s looking promising)
• Figure out how scripting is going to work and embed the interpreter

I’ll document the process on this blog, of course. First steps: reading the PDF spec and figuring out how to make PDFs by hand.

(For those who’ve been reading for a while, Ink was also the name of my static blog engine. That’s now ink-static, and at some point I’ll either retire it completely or change the name to something unrelated.)

Quad

Matthew Butterick posted a link today to Quad, his in-development typesetting engine, written in Racket. It’s an attempt to take the best ideas from both LaTeX and web browsers and build a modern, flexible typesetting engine.

The syntax (at least as it stands right now) is naturally very LISPy, and the examples are fairly low-level, but I’m quite interested to see where things go. While I haven’t done much typesetting lately, I’ve been itching to do more in LaTeX and less in InDesign, so that my source files are plain text and not locked into a proprietary format. And some things are more easily done in code. Also, I’ve wanted to share the source files for my work (as I’ve started doing with the PlotDevice sources for my language charts), but putting InDesign files in a GitHub repo just feels wrong. And InDesign isn’t exactly cheap, either.

So, LaTeX for now, and possibly Quad once it’s matured a bit.