Some quick thoughts about the project space I see myself working in (meaning personal coding projects that aren’t the productivity tools I mentioned before), both now and for the foreseeable future. To be honest, it’s mostly a roadmap for myself, posted here as part of working in public.
Bookmaking tools
One of the areas in the project space is bookmaking tools: tools that help with making either print books or ebooks. What I’ve worked on in that area (and some of these are still in progress or in the future):
- Press — low-level typesetting (PDF compiler)
- Ink — higher-level typesetting
- Curves — programmatic type design
- Typlate — type design templates
- md2epub/Caxton — ebook compiler
- epubdiff — ebook differ
- Fledge — text processing shell
- Storybook — writing tool (covered under the productivity tools, yes, but I feel it fits in here)
Creativity tools
The next area, somewhat related, is creativity tools: tools for making art, music, etc. I do realize that there’s a bit of overlap between the two areas — art can be used in books, for example. This is not a rigorous taxonomy.
What I’ve worked on:
- Trill — music composition REPL
- Grain — command-line tool for texturing art
While I haven’t done much in this area so far, the intersection of software and art has been calling to me more lately. I expect creativity tools to become much more of a focus for me, probably even more so than the bookmaking tools.
Human-Computer Interaction
Last but not least, HCI. My master’s thesis is in this area, and much of my other work also touches on it in limited ways. (What I mean by that, I think, is that with projects like Trill, Curves, and Press, the parts that have most interested me are the interfaces. Also, those interfaces have been textual in these particular cases, but I’m also interested in other kinds of UIs.) So I plan to start building more proofs of concept and interface experiments — like the spatial interface ideas I mentioned several weeks ago.
I’m getting a bit of a nostalgia kick reading through the Standard Ebooks process. I haven’t made anything with them (though they do good work and I’m reading two of their editions right now), but years ago — in a former life, it seems — I used to make ebook editions of old books.
As far as I can tell, the first ebook I made was Chesterton’s The Ball and the Cross, which I typed up by hand and posted to Project Gutenberg. Around that time I worked on a handful of other books for PG, including Henry Sweet’s An Icelandic Primer, which was much more involved (Old Icelandic characters, tables, etc.) and incredibly fun.
After that I worked on several more books as part of the Mormon Texts Project and also started making EPUB and Kindle editions of other books (like the 1812/1815 edition of Grimms’ fairy tales and George MacDonald’s The Light Princess). Those were quite fun, too.
Somewhere around five or so years ago I stopped, partly from working on other things, partly from repetitive strain injuries. (Even with Vim macros to help, there’s still a multitude of repetitive keystrokes in cleaning up texts, at least for me.) With reading about Standard Ebooks and writing this post, though, I’m tempted to get back into it. I built Fledge years ago as an attempt to script away more of the repetitive work, and I suspect wiser use of both it and Vim might be enough to minimize the RSI.
On a related note, I’ve been wanting to rewrite md2epub. It’s a decent-enough Python script that takes Markdown files and turns them into an EPUB, and it’s worked well. But it’s an old codebase, and I don’t like the name anymore, and it could be faster, and I have a few ideas on how to make it more ergonomic, so I’m planning to dub it Caxton and rewrite it in Go or Rust. (Primarily so I’ll have an easier way to make EPUB editions of my fiction.) This part is the most likely to actually happen, I think.
Introducing Fledge, a file processing language/shell written in Python. I wrote it for two reasons: first, as an experiment with an idea that I’ll explain in a moment, and second, because I can’t for the life of me ever remember how to do batch find-and-replace in text files on the command line. (And yes, I could have written a script to help me with the latter. In fact, I did. I just can’t ever remember what it’s called. I … may have issues.)
Anyway, the idea is that you select the files first, then execute one or more actions on them, rather than the normal way round. For example:
with my-book.text
split on /CHAPTER/ to chapter-%%.text
replace /CHAPTER/Chapter/
trim
That splits my-book.text out on a regex to chapter-01.text, chapter-02.text, etc., does a batch find-and-replace on all those chapter files, and then trims them of whitespace at the beginning and end of each file. The alternative would be something like this, assuming you had split, replace, and trim scripts:
$ split my-book.text "/CHAPTER/" "chapter-%%.text"
$ replace chapter-*.text "/CHAPTER/Chapter/"
$ trim chapter-*.text
So, instead of having to specify which files you’re working on at every step of the way, with Fledge you just do it at the beginning and that selection is used for whatever subsequent actions you perform.
Some more examples
with *.jpg where size != 0
rename page-%%%.text starting with 0
This gets all the non-zero image files in the directory and renames them sequentially to page-000.jpg, page-001.jpg, etc.
alias {mtp} ~/Documents/mtp
with **/*.jpg
convert to png
deselect DSC*
move to {mtp}
This finds all the JPEG files recursively under the current directory, converts them to PNG, deselects everything that starts with DSC, and moves the rest of the PNGs to ~/Documents/mtp.
More about Fledge
The fl script can either execute a script of Fledge actions contained in a file or, if you run it without arguments, it loads an interactive shell.
There are a number of built-in Fledge actions (listed in the README), mostly geared towards working with text and image files, but you can write custom actions (which live in ~/.fledge/actions) to do pretty much anything. (Keeping in mind that in general these would be things you want to do to a large number of files.)
How I’m using it
I use it to help prep image scans of journals and personal histories and stuff to upload to Unbindery for transcription (which, incidentally, I’ve been thinking about paring down to a much simpler, more focused app aimed at family history transcription). I also use it for ebook production (splitting Project Gutenberg files into chapters to be made into EPUBs, doing batch substitutions, etc.) and for Mormon Artist (resizing and renaming images, mostly).
So far, I like it. It’s still very much a work in progress (as I write this I’m realizing rename probably should be rename to for consistency), but I can actually remember these commands. And for situations where I’m doing a lot of sequential actions on multiple files (as in ebook production), it’s nice not having to re-specify everything every step.