I’ve been hand-coding PDFs in Vim, reading the PDF spec to learn how things work. It’s fascinating. My first, extremely simple PDF:
%PDF-1.4
1 0 obj << /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj << /Type /Page /Parent 2 0 R /Resources 4 0 R /MediaBox [0 0 500 800] /Contents 6 0 R >>
endobj
4 0 obj << /Font << /F1 5 0 R >> >>
endobj
5 0 obj << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Length 44 >>
stream
BT /F1 24 Tf 175 720 Td (Hello World!) Tj ET
endstream
endobj
xref
0 7
0000000000 65535 f
0000000010 00000 n
0000000059 00000 n
0000000116 00000 n
0000000220 00000 n
0000000263 00000 n
0000000333 00000 n
trailer << /Size 7 /Root 1 0 R >>
startxref
427
%%EOF
It’s not as bad as it looks, I promise. (I’m doing PDF 1.4 because CreateSpace doesn’t seem to support higher versions of the spec.)
Anyway, I’ve been reading through chapter 5 of the spec, learning how text works in PDF. I’ve learned how to modify character spacing with Tc, word spacing with Tw, leading with TL, and individual glyph positions with TJ (not sure yet if I can change vertical positioning or not). I’ve also learned how to change the text color. It’s all been fairly straightforward.
As part of this, I’ve used Hex Fiend (an OS X hex editor) to pry apart some simple PDFs I made with PlotDevice, to see how things were encoded. The streams themselves are generally compressed through Flate compression (opposite of deflate, har har), and I found this script to easily decode the streams:
#!/usr/bin/env python
import zlib
import sys
input = sys.argv[1]
output = sys.argv[2]
with open(input, 'rb') as f:
buffer = f.read()
decomp = zlib.decompress(buffer)
with open(output, 'w') as f:
f.write(decomp)
I copied each stream in hex from Hex Fiend, pasted it into a file, ran the Python script on it, and it would output decoded text to a new file.
Things I don’t know/understand yet, which are legion:
How to encode Unicode (I’m not to this point of the spec yet, but I believe it involves CID fonts and using cmaps to map glyph codes or something like that).
How to take a font name and, in a cross-platform way, get the path to the font file so I can embed it and also use it with HarfBuzz.
How to take the output of HarfBuzz (a list of glyphs with position coordinates for each) and use that in positioning the glyphs in the PDF. I believe HarfBuzz will handle parsing the OpenType features of the font, but I’m not positive on that. I did get HarfBuzz Python bindings working, though, and I plan to play around with it soon.
Whether I need to use FreeType at all. I might need it for font metrics, but HarfBuzz might give me everything I need there.
When typesetting multiple lines, I don’t know whether it’s best to use the PDF built-in support (T* and TL and such), or to set each line manually as its own text object. The built-in support seems better, though I don’t know if that limits what’s possible.
At some point soon — I think when I start embedding fonts — doing this by hand in Vim will stop being as feasible, and at that point I’ll start writing Python to manage the PDF creation process for me. For now, though, it’s easier to just edit the PDF manually.
Updated the Latin declensions chart with a fix for fructus, which was incorrectly feminine (it’s masculine). Thanks to Chih-cheng Yuan for the heads up!
As mentioned on Twitter, I’ve decided to write my own typesetting engine, called Ink. Apparently I’m crazy.
The details are still very much in the air, but here are some quick notes:
Written in Rust (for speed)
Programmatic (sort of like TeX)
Scripting language for extensibility (JavaScript or Lua or Python)
Intended for use in typesetting book interiors, covers, and charts
Possibly some kind of template/data division
Full OpenType feature support (shaping via HarfBuzz)
Custom PDF generation library (inkpdf)
Reasons for doing this insane thing:
PlotDevice only runs on OS X and I want the source of my language charts to be usable on other platforms.
I’d like to open source the books I typeset, so InDesign isn’t a great solution.
TeX is powerful and well-seasoned and all, but it’s not exactly pleasant or easy to work with, especially for the kind of stuff I do.
I’ll learn a lot and have fun while I’m at it.
The initial roadmap, not necessarily in order:
Write inkpdf in Python (which I think will be a better fit for the charts anyway)
Get familiar with HarfBuzz
Learn Rust
Port inkpdf to Rust
Plan out the Ink language (I’ve started on this and it’s looking promising)
Figure out how scripting is going to work and embed the interpreter
I’ll document the process on this blog, of course. First steps: reading the PDF spec and figuring out how to make PDFs by hand.
(For those who’ve been reading for a while, Ink was also the name of my static blog engine. That’s now ink-static, and at some point I’ll either retire it completely or change the name to something unrelated.)
I’ve spent the last few months avoiding the heck out of writing the novel (Edge of Magic). The plot changed several times — completely changed, that is — and then I abandoned it and started on a short story which I then also abandoned.
The good news, however, is that I’ve written a thousand words a day every day solid since the beginning of December. The habit is securely in place.
Tonight I realized that for some reason, when I’m working on a story I can’t do other projects at the same time (art, typesetting, coding, etc.). Whether it’s because I use those other projects as avoidance or something else, I don’t know. But during a writing project — at least the planning stage, not sure yet whether it applies to the drafting stage — I need to put all my other projects on the back burner. That gives me enough mental space to let the story grow.
The plan right now: finish the aforementioned short story (featuring Dagh and María Bonita — I don’t have a title yet), then decide which novel I’m actually going to write, outline it, and write it as fast as I can, to get it done before self-doubt slashes my tires.
One problem with the novel, by the way, was that Makrannan’s character was never very clear or interesting to me. He never came alive to me. Thus his part of the book was always weak, even after several re-outlinings. With this current story, however, Dagh and María Bonita feel much more alive to me, and it’s making all the difference. (And hopefully that will make the story much more worth reading than my earlier stories.)
Goal: finish the first draft of the Dagh story by May 21, and release the edited story on my site by May 31.
Quick note: yesterday I pulled all my stuff off Redbubble (things seemed shady over there) and uploaded my religious art to Society6. Prints are still available from Imagekind as well.
I’ve missed learning new programming languages, so I’m now teaching myself Haskell for fun. I’ll post my progress as I go along.
Initial impressions based off what I know at this point: Haskell is very functional. The code looks different to my imperative-language eyes, though I’m intrigued by how short Haskell code often seems to be. Oh, and there are monads, though right now I have no idea what those are.
I’m excited to delve deeper into a pure functional language — I’ve done some limited functional programming in Python, and I have a cursory knowledge of Lisp, but I haven’t written anything real in a functional language before. (And I do plan to write a real project in Haskell. Not sure what it’ll be yet.)
Yesterday’s generative art piece, called “Circlecells” (these names are amazing, I know):
Detail:
The code is on GitHub. I also posted a few process pics to Twitter yesterday. There are also three other accompanying pieces in the set generated with different seeds: 02, 03, 04.
Explanation:
There’s a 20×20 grid which gets populated with an initial seed population of living/dead cells. (I get a random value between 0 and 5; if it’s 0 or 1, the cell is alive.)
The lines are drawn from any living cells to any immediately neighboring living cells.
The size of each circle is dependent on how many living neighbors the cell has.
The initial round is drawn in light tan, then the grid is run through a modified Conway’s Game of Life (any cells with 2, 3, or 5 living neighbors are alive in the next round).
Two more iterated rounds are drawn, one in a slightly darker tan and the last in dark red. (Drawing is done with the multiply blending mode.)
I textured the piece in Photoshop afterwards, using some Kyle T. Webster brushes — add canvas, add noise, encaustic grit, a couple others that I can’t remember.
