Dot grid paper
I’ve added dot grid paper to the note paper page, in a handful of sizes:
Colophon: I made these with JavaScript/Node, generating SVGs that I then converted into PDFs with Inkscape.
Blog: #pdf
New daily goal charts
Inspired by Robert A. Caro’s planning calendar, I’ve redesigned my daily goal charts:
This time the chart doesn’t have a specific year baked in, so it’s reusable. (And there’s a variation for leap years.) It’s freely available as PDFs in both portrait and landscape. Currently just letter size, though maybe someday I’ll start including A4 and other sizes.
Colophon: I made these charts with HTML (it’s just a table), CSS, JavaScript (on page rather than via Node), and Firefox. The font is Avenir Next.
Just added diagonal graph paper to the note paper page. It looks like this:
Why would you use this? No idea. I haven’t been able to come up with any good use cases for it, but the itch needed to be scratched so here we are.
Sidenote: I made the original lined/grid paper with PlotDevice but decided to use SVG instead this time, converting the files to PDF with Inkscape.
Historia Calamitatum
After what feels like a long absence from bookmaking, I’ve gotten back into it and have a new release: Historia Calamitatum, available as a PDF.
The book is a medieval autobiography by Peter Abélard, a Catholic philosopher who lived in the eleventh and twelfth centuries in France.
Some notes on the making, for those who like that sort of thing:
- I used paged.js for the typesetting, so I was editing HTML and CSS files instead of wrangling InDesign or Affinity Publisher or LaTeX. It’s a different workflow, to be sure (lots of reloading in Chrome and then finding my spot again), but overall I love having the source files be plain text.
- The line-breaking algorithm isn’t as nice as InDesign’s. Had to finagle the
word-spacingandletter-spacingproperties a bit to fix some more egregious spots. (At the same time, I wasn’t fixated on making the spacing perfect. Nor did I fix the hyphenation stacks, because they don’t bother me. I’m clearly becoming a bit more relaxed about typesetting rules as I get older.) - For the typeface I went with IM Fell DW Pica, which is no doubt anachronistic but I like the feeling it gives the book.
- I proofed the PDFs in the Documents app on my iPad. Much nicer than printing the whole thing out (which I used to do, years ago).
- I made the cover using Cirque with textures applied in Affinity Photo.
Update on Press (the PDF compiler). I haven’t worked on it at all lately, but I wanted to document its current state for history’s sake, and as part of working in public. (I’ve also been sitting on this post for over a year.)
Back in 2017 I did end up re-architecting Press to use Low Ink as an intermediate page description language. In the process, Low Ink changed from a JSON-based idea to this:
:page 11x8.5in
:bleedbox x=0.125in y=0.125in w=5.75in h=8.75in
:fontmap family=helv weight=regular style=normal standard=Helvetica
:yinvert
:push
:translate x=72 y=72
# ascender
:push
:translate x=0 y=1040
:strokecolor hex=#999
:linewidth 0.25pt
:line x1=0 y1=0 x2=1080 y2=0
:stroke
:push
:fillcolor hex=#999
:font family=helv size=14pt
:text x=1085 y=-3 text="ascender"
:pop
:pop
# filled glyph
:push
:translate x=1320 y=240
:fillcolor hex=#000
:moveto x=0 y=0
:pathto x=400 y=300 cx1=120 cy1=300 cx2=140 cy2=300
:pathto x=320 y=200 cx1=540 cy1=300 cx2=320 cy2=180
:lineto x=350 y=350
:lineto x=450 y=250
:lineto x=150 y=0
:moveto x=200 y=200
:lineto x=200 y=250
:lineto x=250 y=250
:lineto x=250 y=200
:lineto x=200 y=200
:fill
:pop
It was intended to be a fairly low-level wrapper on the PDF format, with the idea being that other libraries/apps would provide more ergonomic abstractions on top of it.
I initially used Python because Press started out as a library, but with the pivot to a compiler model, I think Go or Rust would probably end up being a better choice (Rust would make integrating HarfBuzz a bit easier, at any rate).
Potential improvements
To my 2021 eyes, the language design isn’t particularly elegant. I like that the parameters are named (clarity), but for most of the commands there aren’t actually that many parameters, because many of the settings that would normally be parameters are separate commands. For parameters that are clearly unambiguous, the names hamper readability. For example, I think something like this might be better:
:line 0,0 to 1080,0
:fillcolor #345
I’ve also thought that push and pop could potentially be clearer as curly braces, and that the initial colons aren’t really necessary:
{
translate 0,1040
strokecolor #999
linewidth 0.25pt
line 0,0 to 1080,0
stroke
{
fillcolor #999
font 14pt helv
text 1085,-3 "ascender"
}
}
The future
My initial reason for building Press was to have an easy, programmable cross-platform way to create language chart PDFs (so I could move away from PlotDevice/DrawBot), and what I’ve realized (acknowledging that I haven’t really been making language charts in recent years) is that there are some other, better options now.
One that seems decent is SVG, converted to PDF by way of Inkscape. Initial tests here seem like it would probably work fine.
Another promising option that I admittedly haven’t looked into very much yet is Paged.js. HTML and CSS are already great for declarative typesetting, and the more I’ve thought about programmatic typesetting, the more this model seems to be the future I’d want to work with (and not just because of parity with web, though that makes it much more compelling).
tl;dr I don’t see myself continuing on with Press, so we may as well call a mortem on it.
Progress on Press has been a bit slower lately. I’ve fixed most of the errors I discovered by running the exported PDFs through the 3-Heights PDF validator. I also refactored the code and reorganized the package per Kenneth Reitz’s advice.
I’ve implemented initial support for embedding subsetted fonts (doing the subsetting via fontTools.subset), and while the fonts (including uninstalled fonts) display fine on my macOS box, the PDFs don’t validate properly and the fonts don’t show at all on iOS, which means the embedding isn’t actually working right. Current suspects include the /Differences array (which I’m not generating properly yet) and the CMap (which I haven’t implemented at all yet). I still have to implement ToUnicode as well, so that copying and pasting does what it should, but I’m fairly certain that isn’t what’s causing the fonts to not embed properly.
I’m also trying to figure out color spaces. In general I believe I want the output to be either DeviceRGB or DeviceCMYK, with some way of specifying an output intent, and also an option for the user to embed an ICC profile if they want. I’m part of the way there.
Anyway, the font stuff is far more complicated than I expected going in, but I’m still making progress, and I’m learning a lot.
Press can now generate PDFs. For example:
from press import Press
with Press('press-demo.pdf', size=Press.LETTER, margin=1*Press.INCH) as p:
# Top and bottom borders
p.pen(0.25)
p.line(p.page_min_x, p.page_min_y, p.page_max_x, p.page_min_y)
p.line(p.page_min_x, p.page_max_y, p.page_max_x, p.page_max_y)
# Rotated colored rectangle
p.push()
p.pen(15)
p.stroke(hsl=(0, 0.5, 0.5))
p.fill('#fd0')
p.translate(p.page_min_x + 4*Press.INCH, p.page_max_y - 4*Press.INCH)
p.rotate(45)
p.rect(0, 0, 2*Press.INCH, 2*Press.INCH)
p.pop()
# Lines of varying thickness
p.translate(p.page_min_x + 1*Press.INCH, p.page_min_y + 1*Press.CM)
for i in range(1, 20):
p.pen(i / 2)
p.line(0, 0, 30, 0)
p.translate(0, 20)
That code generates the following PDF (linked):
I’m working on text/font support now, which is by far the most complicated thing about this project.
Since there isn’t a clean, cross-platform way to select a font via code, I’ve decided to use font maps (inspired by @font-face in CSS):
fontmap = {
'paths': [ './fonts', '/Library/Fonts', ],
'Minion Pro': [
{ 'weight': 300, 'italics': False, 'filename': 'MinionPro-Regular.otf', },
{ 'weight': 300, 'italics': True, 'filename': 'MinionPro-It.otf', },
{ 'weight': 600, 'italics': False, 'filename': 'MinionPro-Bold.otf', },
{ 'weight': 600, 'italics': True, 'filename': 'MinionPro-BoldIt.otf', },
],
}
with Press('output.pdf', size=Press.LETTER, fontmap=fontmap) as p:
p.font('Minion Pro', size=24, weight=300, italics=True, dlig=True, smcp=True, tracking=50)
p.align(Press.LEFT)
p.text("This is a test.", 50, 50)
Font maps are admittedly extra work, but they do have some advantages as well: you can use fonts you haven’t installed, for example, and you can specify exactly which font files you want to use. And I can’t see any good way around the lack of a cross-platform font selection mechanism (meaning, a way to pass in ‘Minion Pro’ with specific weight and styles, and get a font filename in return).
Anyway, I’m in the middle of reading the PDF spec on CIDFonts and CMaps. It’s … complicated. It makes my head hurt. But it’ll be awesome when it’s done.
I’ve renamed inkpdf to Press (as in printing press).
I reached the point where creating the PDF manually is no longer feasible, so I’ve been working on getting Press to a point where I can implement the PDF generation. The basic structure is in place, sans the PDF part. (That’s next.)
Here’s what a Press script looks like right now:
from press import Press
p = Press('output.pdf', width=6*Press.INCH, height=11*Press.INCH,
margin=1*Press.INCH)
# Horizontal borders at top and bottom of page
p.stroke('#000')
p.pen(1.0)
p.line(p.page_min_x, p.page_min_y, p.page_max_x, p.page_min_y)
p.line(p.page_max_x, p.page_min_y, p.page_max_x, p.page_max_y)
# Page 2
p.page(2)
p.layer('base')
p.stroke(rgb=(1, 0, 0))
p.line(150, 150, 300, 300)
p.layer('fg')
p.stroke(hsl=(0, 0.5, 0.8))
p.line(300, 300, 450, 150)
# Go back and add another line to page 1
p.page(1)
p.stroke('#025')
p.line(p.page_min_x, p.page_min_y, p.page_min_x, p.page_max_y)
p.save() # this doesn't work yet
You can also do something like this:
with Press('output2.pdf', size=Press.LETTER,
margin=(0.5*Press.INCH, 1.0*Press.INCH),
inner_margin=0.5*Press.INCH,
outer_margin=1.25*Press.INCH,
bleed=.125*Press.INCH) as p:
p.line(50, 50, 250, 50)
# And so on
(Context manager, inner/outer margin, bleed, built-in paper sizes.)
Up next: adding more primitives, designing the font selection mechanism, getting it to generate an actual PDF, embedding fonts, using arbitrary Unicode code points, integrating HarfBuzz, etc.
Making PDFs by hand
I’ve been hand-coding PDFs in Vim, reading the PDF spec to learn how things work. It’s fascinating. My first, extremely simple PDF:
%PDF-1.4
1 0 obj << /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj << /Type /Page /Parent 2 0 R /Resources 4 0 R /MediaBox [0 0 500 800] /Contents 6 0 R >>
endobj
4 0 obj << /Font << /F1 5 0 R >> >>
endobj
5 0 obj << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Length 44 >>
stream
BT /F1 24 Tf 175 720 Td (Hello World!) Tj ET
endstream
endobj
xref
0 7
0000000000 65535 f
0000000010 00000 n
0000000059 00000 n
0000000116 00000 n
0000000220 00000 n
0000000263 00000 n
0000000333 00000 n
trailer << /Size 7 /Root 1 0 R >>
startxref
427
%%EOF
It’s not as bad as it looks, I promise. (I’m doing PDF 1.4 because CreateSpace doesn’t seem to support higher versions of the spec.)
Anyway, I’ve been reading through chapter 5 of the spec, learning how text works in PDF. I’ve learned how to modify character spacing with Tc, word spacing with Tw, leading with TL, and individual glyph positions with TJ (not sure yet if I can change vertical positioning or not). I’ve also learned how to change the text color. It’s all been fairly straightforward.
As part of this, I’ve used Hex Fiend (an OS X hex editor) to pry apart some simple PDFs I made with PlotDevice, to see how things were encoded. The streams themselves are generally compressed through Flate compression (opposite of deflate, har har), and I found this script to easily decode the streams:
#!/usr/bin/env python
import zlib
import sys
input = sys.argv[1]
output = sys.argv[2]
with open(input, 'rb') as f:
buffer = f.read()
decomp = zlib.decompress(buffer)
with open(output, 'w') as f:
f.write(decomp)
I copied each stream in hex from Hex Fiend, pasted it into a file, ran the Python script on it, and it would output decoded text to a new file.
Things I don’t know/understand yet, which are legion:
- How to encode Unicode (I’m not to this point of the spec yet, but I believe it involves CID fonts and using cmaps to map glyph codes or something like that).
- How to take a font name and, in a cross-platform way, get the path to the font file so I can embed it and also use it with HarfBuzz.
- How to take the output of HarfBuzz (a list of glyphs with position coordinates for each) and use that in positioning the glyphs in the PDF. I believe HarfBuzz will handle parsing the OpenType features of the font, but I’m not positive on that. I did get HarfBuzz Python bindings working, though, and I plan to play around with it soon.
- Whether I need to use FreeType at all. I might need it for font metrics, but HarfBuzz might give me everything I need there.
- When typesetting multiple lines, I don’t know whether it’s best to use the PDF built-in support (
T*andTLand such), or to set each line manually as its own text object. The built-in support seems better, though I don’t know if that limits what’s possible.
At some point soon — I think when I start embedding fonts — doing this by hand in Vim will stop being as feasible, and at that point I’ll start writing Python to manage the PDF creation process for me. For now, though, it’s easier to just edit the PDF manually.
A slightly different kind of ebook
Turns out reading PDFs of old books (from Google Books, Internet Archive, etc.) on my iPhone works out reasonably well. For example:
On the left is the fully zoomed out page. Indoors, I’m able to read it without too much difficulty, though my eyes do thank me when I zoom in (as on the right). The problem with zooming, however, is that navigating to the next page then requires more swiping, and, at least in iBooks, you have to zoom in again every time you turn the page.
After a bit of this, I got to wondering what it would be like to typeset an iPhone-sized PDF, designed specifically to be read on a phone. Here’s how it turned out (and this is a proof of concept, nothing too polished):
The pages are set at 7.573×4.267″, which I arrived at by taking 1136×640 (iPhone screen dimensions in pixels) and dividing by 150. Arbitrary, but it worked out well enough. And the text is at 16 points on the left and 18 on the right. (Also arbitrary, but dependent on the page size, of course.)
The PDFs:
The main advantage to a foolhardy scheme like this is full typographic control — margins, fonts, layout (important for poetry), tracking, etc., all without worrying about limitations of ebook readers. I could try to do something about widows and orphans, for instance, though I didn’t do that with this proof of concept.
The downside is that it’s custom-tailored to the dimensions of the iPhone 5S, and on other devices it wouldn’t fit as perfectly. Not necessarily a dealbreaker, though.
Is it worth pursuing? No idea. One of these days I’ll set a full book this way and try reading it on my phone to see how it compares.