Another entry in the tedious series about my personal productivity tools.
Ditto is my transcription app. It’s a Python app using FastAPI for the web parts.
Overview
The goal with Ditto is to take scans (usually of journals) and make it easy to transcribe them a page at a time. Same idea as Unbindery, though scaled way, way down: single-user instead of crowdsourced, a simplified workflow, and it only supports one project at a time so I don’t spread my limited transcription time too thin. It looks like this:
And there’s an inverted mode, which initially seemed like a great idea (dark mode, basically) but I never actually use it, I think because it’s harder to read:
The transcriptions are stored as plain text files in the same directory as the corresponding images.
How I use Ditto
I spend a few minutes transcribing journals each morning as part of my daily routine. (As I finish each volume, I pull the transcribed text files off my server, concatenate them, do some minor formatting, and then import them into Vinci.)
The future
When I made Ditto, one of my goals was to make it work well on mobile so I could have a portable transcription station anywhere I went. It has a responsive design that does work on a phone, but the experience is currently slightly awkward (lots of panning), so I tend to only use it on my laptop. At some point I’d like to try to fix that.
Other than that, though, I’m happy with it. It works well. And it’s small — 340 lines of code. (Which makes me inordinately happy. Small tools are the best.)
For the first year doing the Mormon Texts Project, our process went like this: Volunteers would tell me they were ready for a batch. I would send them a range of five page numbers (145-149, for example) and a text file containing the unproofed text for those pages. (I would also track this in a Google spreadsheet.) The volunteer would then go to the book in Google Books, open the text file in a text editor, and proof the text. After they finished, they would email the text file back to me and I would assign them a new batch. Rinse and repeat.
Too much overhead. So I started working on Unbindery, a web app to automate almost all of this. This is what it looked like a couple weeks ago (a functional but completely unpolished bare-bones app):
I was kind of disappointed about the project — it was moving incredibly slowly, I didn’t really care about it anymore, etc. — and I had made up my mind to let it quietly fall by the wayside.
Then I was at the temple two weeks ago and got a clear impression that I wasn’t going to get off the hook that easily, and that I needed to keep going with MTP and finish Unbindery. With that impression came some inspiration on how to polish the app, and I’ve been working on it since then. Here’s what it looks like now:
For the past week we’ve been using it for MTP work, and since then our productivity has totally skyrocketed. Here are my educated guesses on why:
Smaller chunks. Volunteers proof one page at a time instead of five. It’s easier and, because it’s easier, volunteers proof more than before.
Progress bars. Since volunteers can see their progress visually, there’s more of a drive to keep proofing so they can make the black bar go all the way to the end.
Scoreboard and leaderboard. Volunteers get points every time they finish proofing a page. With the leaderboard, there’s competition, and already I’ve seen an increase in productivity because of this (at least subconsciously).
Convenience. Since Unbindery is a web app, volunteers can proof pages anywhere, instead of having to download/upload text files and all that.
There’s also a new feature I added today, where volunteers can see how much of a given project they proofed:
This way there’s more of a sense of ownership to the work. We’ll see how it goes.
Like I said a few days ago, we went from taking eight months per book to a few days per book. My volunteers have proofed 373 pages in the last week, and I’m now scrambling to get enough books into Unbindery so we don’t run out. That’s a very different problem from the stagnation I had a month or two ago. (Yes, it’s a good problem to have.)
I’ve been spending more time on the Mormon Texts Project (we’re almost done with The Life of Heber C. Kimball, by the way), and I’ve realized that having a nice integrated system for assigning and editing pages would make things much easier. Enter Unbindery:
My friend Rikker and I started Unbindery a few years ago, but it petered out before it got off the ground, and there it languished until a month or two ago. Since then, I’ve gotten the core up and running and it’s now usable enough to start doing actual MTP work with it. (Taking OCRed text and cleaning it up, that is.) I still have a lot of polish left to do, though.
It’s written in PHP and Javascript and I’ll be releasing it on GitHub in the near future, once I clean up the source a bit.
