Ben Crowder

Blog: #tools

10 posts :: tag feed :: archive :: about the blog

Some WIP experimentation with art.

Brief backstory: when I’m doing my minimalist religious art, I usually sketch an idea out first by hand or in Paper on my phone, then mock them up in Illustrator to iterate on the concept. Once it’s satisfactory, I move to execution, either painting the piece in Procreate or using some of the brushes in Illustrator to get a more organic look. And finally I texture the image in Photoshop.

A couple months ago I got interested in exploring alternatives to Illustrator and Photoshop for both execution and texturing processes. And me being me, I wanted to try doing it in code, just to see what it was like. (Some things are easier in code, though I don’t know how often that would actually be the case with these.)

Note: this is still very much a WIP, and who knows if I’ll end up using any of it or not. But here’s the current state of things.

SVG

After reading somewhere that SVG has turbulence and displacement filters, I realized I could potentially use those for the execution part of the process, to distress the edges enough to make things more interesting. (And hopefully to be less repetitious than the Illustrator brushes I use.)

I put together an initial test using a few different settings, and it turned out a bit better than I expected. A sample of the code:

<filter id="person1Filter">
    <feTurbulence type="turbulence" baseFrequency="0.5" numOctaves="2" result="turb1" />
    <feDisplacementMap in2="turb1" in="SourceGraphic" scale="3" xChannelSelector="R" yChannelSelector="G" result="result1" />
    <feTurbulence type="turbulence" baseFrequency="0.05" numOctaves="2" result="turb2" />
    <feDisplacementMap in2="turb2" in="result1" scale="3" xChannelSelector="R" yChannelSelector="G" />
</filter>

<style type="text/css">
    .person1 {
        fill: #a34130;
        filter: url(#person1Filter);
    }
</style>

<g id="person-1">
    <circle class="person1" cx="200" cy="250" r="30" />
    <polygon class="person1" points="225,270 205,500 350,500" />
</g>

And this is what it looks like:

svg-test.png

The background rectangle, the red figure, and the white figure all have different turbulence and displacement values. The red figure uses two sets of turbulence and displacement filters, which worked out fairly well, I think.

I used Inkscape render it out to a high-res PNG, since Illustrator wasn’t able to handle the filters. Eventually, if I keep going down this path, I’d hopefully be able to find a command-line tool that can do the rendering. (Maybe Inkscape has a headless option.)

Overall, this path seems promising. I don’t know that I’d use it all the time, but for certain things it may be handy. I still need to look into sane ways to round corners, and it seems that the other filters (dilation/erosion, convolution, etc.) may be helpful, too.

Grain

I’ve begun writing a Python script called Grain for texturing the final art image. The goal here is to see if I can streamline the process at all, and to see if this idea even works. Grain takes as input a text-based input file that looks like this:

:image test1-texture.jpg
:blend screen
:opacity 0.05
:x -100

:image textures#random
:blend soft-light
:opacity 0.1

:pattern roughdots
:blend soft-light
:opacity 0.2

:image textures#2019-05-21 17.28.14.jpg
:blend soft-light
:opacity 0.01

:image test1-base.png

Each block is a layer. Grain starts with the bottom layer (the executed base image) and goes up from there, adding each layer on top with the specified blending mode and opacity.

The :pattern roughdots command would generate procedural dots (not implemented yet), and the textures# bit in the :image command calls is a shortcut to my folder with texture photos.

So far, the results are disappointing. While the layering does currently work, it isn’t yet producing anything remotely publishable. I think there might be some discrepancies between blending modes in pyvips and in Photoshop. Hard to tell.

And, less importantly, it’s a little slow — partially from using high-res images, partially from Python. If the idea ends up working, I’ll most likely port this to Rust or Go, and probably also have scale things down for the exploration phase of texturing (with a final high-res export at the end).

I’ll keep tinkering with it from time to time and we’ll see how it goes.

Slash intro

Another entry in the patience-testing series talking about my personal productivity tools.

Slash is the engine that runs this blog. It’s just a Python app running Django, but calling it an engine is too satisfying for me to stop anytime soon. The name comes from the ubiquitous forward slash in URLs.

Overview

Slash has an internal frontend with some post management pages (see below) along with a small API which is used by Blackbullet (my current website engine, separate from the blog) to pull posts into my site template. The API also publishes the RSS and JSON feeds, which Blackbullet passes right through.

The dashboard lists current drafts and, lower on the page where you can’t see it in this screenshot, recently published posts:

slash-1.png

Disclaimer: there’s no guarantee that these particular post drafts will ever see the light of day. I often put ideas in and then decide later that they’re not worth blogging about.

Here’s the post edit page, which is very much a work in progress (last week I added the visual tag controls, since adding tags via the metadata textbox made it impossible to tell whether I’d used a tag before or not):

slash-2.png

It’s spartan but works for me.

Payload syntax

Other than the notebook specifier, the syntax is pretty much the same as Vinci’s. Posts are written in Markdown. Metadata is specified with the initial-colon syntax.

One thing I realize I forgot to mention in the Vinci post is that in both apps I have a shortcut syntax for including images that looks like this:

(( readers-edition.png | class=border | url=/book-of-mormon-readers-edition/ ))

I have a page for uploading images to a date-named folder — year and month — and this syntax relies on the image being in the matching folder for the post. A small bit of convention to make things simpler.

How I use Slash

On my laptop, I open Slash when needed. On my phone, I have it saved to my homescreen as a PWA.

Other than that, I use it the way you’d expect — I write blog posts (usually directly in Slash, but occasionally in Gate or Quill), I edit them, I publish them. Months later I finally notice the typos. It’s not too exciting.

The future

As of now, the plan is to replace both Blackbullet and Slash with a new, simpler, consolidated Slash, using plain text for the backend and probably moving to FastAPI. Since I’m in the middle of planning the rewrite right now (and since I’m now working in public), you’ll see posts about it soon.

Ditto intro

Another entry in the tedious series about my personal productivity tools.

Ditto is my transcription app. It’s a Python app using FastAPI for the web parts.

Overview

The goal with Ditto is to take scans (usually of journals) and make it easy to transcribe them a page at a time. Same idea as Unbindery, though scaled way, way down: single-user instead of crowdsourced, a simplified workflow, and it only supports one project at a time so I don’t spread my limited transcription time too thin. It looks like this:

ditto-1.png

And there’s an inverted mode, which initially seemed like a great idea (dark mode, basically) but I never actually use it, I think because it’s harder to read:

ditto-2.png

The transcriptions are stored as plain text files in the same directory as the corresponding images.

How I use Ditto

I spend a few minutes transcribing journals each morning as part of my daily routine. (As I finish each volume, I pull the transcribed text files off my server, concatenate them, do some minor formatting, and then import them into Vinci.)

The future

When I made Ditto, one of my goals was to make it work well on mobile so I could have a portable transcription station anywhere I went. It has a responsive design that does work on a phone, but the experience is currently slightly awkward (lots of panning), so I tend to only use it on my laptop. At some point I’d like to try to fix that.

Other than that, though, I’m happy with it. It works well. And it’s small — 340 lines of code. (Which makes me inordinately happy. Small tools are the best.)

Vinci intro

Another entry in the snore-inducing series talking about my personal productivity tools.

Vinci is my journal/log app, a private blog of sorts. It’s a Python app running in Django.

Overview

Vinci has notebooks which contain entries. Like most blogs, entries are displayed in reverse chronological order. It looks like this, except I usually write in English, har har:

vinci-1.png

Editing an entry is a modal fullscreen panel, with the main textbox at top, the metadata textbox under that, and some controls at the bottom.

vinci-2.png

On save, Vinci splices the metadata and the text together and runs it through the payload processor.

Payload syntax

Vinci uses the text-based payload idea like Liszt. Its payload syntax looks like this:

/projects
:tags foo, bar

Worked on the [foo project](/leaf/3290) for a while. Ran into a few issues.

The first line is the notebook specifier. The second line (and this could have been anywhere, didn’t have to be at the top) has a command with some parameters. And the rest is Markdown.

There’s also a variant syntax where the first line can look like this: /projects/tag/tag2/tag3. I’ve started using that a little more often.

Because of how I like to write in my journal, I’ve also set up a special case for my /journal notebook, where adding an entry will either create a new entry for the day (if there isn’t one yet) or append to the existing entry, so there’s just one entry for each day. (I use Gate or Quill to jot down a paragraph and then append it quickly and easily.)

How I use Vinci

I use Vinci a lot. I maintain my personal journal in it, along with logs for most areas of my life — work, school, writing, projects, church, etc. Sometimes I create notebooks for specific projects, other times I use a higher-level notebook (like /projects) and use tags instead. At some point I’ll probably consolidate.

I have a /thinking notebook where once each morning I think through my current tasks/projects and write out what I need to do for each. Writing things down makes a world of difference for me, across the board.

Each morning I also spend a few minutes reading one of my past journal entries, as mentioned earlier today. (A while back I scanned all my paper journals and I’ve been slowly transcribing them — we’ll get to Ditto soon — and importing them into Vinci.) Lately I’ve been reading through my 2004 college entries. My undergraduate years were great, but I am very, very glad I’m not in that phase of life anymore.

Lastly, I reference these notebooks fairly frequently. (When did we replace our dishwasher, what did I last work on for that project I haven’t worked on in months, etc.)

The future

Vinci currently uses Whoosh for full-text indexing, but it’s unsupported and hasn’t been working as well for me lately. Several months ago I realized that if all my notebook entries are stored in plain text files, I can just use ripgrep or ag for fast and accurate searching, with the further benefit that in the event of my untimely demise, everything would be fairly easy for my family to copy out and preserve. (I really like plain text, can you tell?)

To that end, I started writing a new, smaller app in Go called Leaf. It’s going well, but I’m tempted to switch to FastAPI. Not sure yet if I will or not. (I’ve enjoyed learning Go and have used it on several small projects now, but I’m also thinking more about long-term maintenance across all these apps, and using a single stack would simplify things for me.)

I’m also thinking about adding a small CLI (in the web interface) that would make entry management easier — moving all entries with a specific tag to their own notebook, for example. Truth be told, I don’t know that entry management happens frequently enough to warrant a CLI, but I’m intrigued by the idea of putting a CLI in the interface, and if it goes as well as I hope it will, I see myself doing that in more of my tools. (Right now I see it as an extension of the search interface. Where right now I type dishwasher to search for that keyword, I’d eventually also be able to type something like :move /projects#cardiff /cardiff.)

As with Liszt, I’m also looking forward to moving to a lighter, simpler codebase. Vinci has a moderate amount of vestigial functionality that needs to go.

Liszt intro

First in the series introducing my personal productivity tools. Buckle up, this is going to be nerdy. And long.

Liszt is my todo list app. Named after the composer, though I regret it a little since I butcher his name by pronouncing it lisht to differentiate it from the ordinary list. Heresy. The next version will have a better name, though. It’s a Python app running on Django.

Disclaimer: I don’t think this app is perfect. (Or any of the other tools I’ll talk about, for that matter.) I’ll describe things as they are, acknowledging here that there’s a lot of room for improvement.

Overview

This is what the dashboard looks like, populated with some dummy data:

liszt-1.png

And on mobile, where the controls move to the bottom for easier access:

liszt-1mobile.png

The top bit is my stats panel, with the data pulled in from my other tools’ APIs. Daily writing counts, daily words left, total words written on the novel (all three from Storybook), daily pages left (from Bookshelf), and daily goals left (from Momentum). I’ll cover those tools later.

There’s also a slide-in menu on the side, with my most commonly used top-level lists:

liszt-2.png

Double-clicking on a list item opens up an edit panel, which also allows me to move the item to another list with some commonly used lists included as buttons (this whole panel is kind of clunky and needs improvement):

liszt-7.png

Payload syntax

The basic idea behind Liszt (and this is common to many of my apps) is the text-based payload, which enables some nice cross-app integrations (more on this when I cover Gate and Quill). Adding items looks like this:

liszt-3.png

A Liszt payload (the text entered into the add tray) has one item per line, with optional blank lines and optional list specifiers. If there’s no specifier, it’ll assume the current list if there is one, otherwise it’ll default to ::inbox. (I use the double-colon prefix to mark lists, with a slash to specify sublists.) Here’s a fuller example of the syntax, again with dummy data:

Process email :5
Review the merge requests :10

::work/notifications
Read up on Python futures ::: https://docs.python.org/3/library/concurrent.futures.html
Refactor [::work/notifications/refactor]
Write up the design doc

The first two items (which have belt-mode durations, I’ll explain those in a minute) would go into the ::today list (which is the dashboard list). The last three items would go into the ::work/notifications list, as pictured here:

liszt-3b.png

Of these, the first line sets a subtitle by putting it after the triple-colon marker. I use this all the time.

The second line is a symlink of sorts, pulling in the top item from the linked list (different meaning here) and showing it in place, with the Refactor text shown as the subtitle. I used to use this more often but haven’t as much lately.

You can also see that this list has a child list (::work/notifications/refactor).

Belt mode

liszt-4.png

If an item has a duration marker included, that triggers belt mode (ala conveyer belts), as evidenced by the new bar at the top of the screen in the image above.

Brief backstory: I initially wrote an Electron app called Belt that did the same sort of thing, then a few months ago ported it to Go as a menubar app. Shortly after I finished that, I realized it would make much more sense in Liszt and brought the functionality in.

And what is that functionality? It’s just an easy way to time tasks from the list. When I hit Start, it switches into belt mode (also changing the favicon so I can tell that it’s running and turning on focus mode so I can only see the task I’m actively working on):

liszt-5.png

When the timer runs out, it plays a sound and brings up a panel allowing me to continue on to the next item in the list, stop belt mode, or add more time to the timer. (There are also keyboard shortcuts for all of this.)

liszt-6.png

How I use Liszt

On my laptop, I have Liszt open in Firefox as a pinned tab. On my phone, I have it saved to my homescreen as a PWA.

Every morning I go through the main lists and move the items I’m going to work on to the today list. I then work out of the today list the rest of the day, opening it often.

I use belt mode most days, primarily to help me stop avoiding tasks I don’t really want to work on (but that still need to be done).

The future

Lately I’ve grown enamored of the idea of storing data as plain text files in directories, rather than using an actual database like Postgres or Mongo. There are plenty of apps where this doesn’t make sense, but for personal, small tools, it works nicely, so I’m planning to migrate Liszt off SQLite to plain text, and I am very excited about it. Yes, I am that kind of a person.

While Django is fine (we use it at work and I love it), I’m planning to move to FastAPI, which I’m already using for Ditto and Arc. It’s a bit faster and feels more lightweight. I think in my mind I mostly use Django because of the ORM and admin; once I’ve given that up, the baby goes out with the bathwater.

I’m also looking forward to simplifying things, removing vestigial functionality, and sanding down as many of the friction points as I can.

Over the last several years I’ve built a number of personal productivity tools (almost all of them web apps) that I’ve never written about here. In the spirit of working in public, that’s about to change.

There are around fourteen of them, though, so I’ll be spacing them out a little, with other posts interspersed here and there for the sake of our sanity. Also, while you’ll be hearing about them in a relatively short timeframe, remember that they weren’t written all at once, and that the older ones have been iterated on for a long time.

I generally won’t be releasing the source code, FYI. I still fiddle with these apps on the regular, and feeling an obligation to maintain stability for outside users would put a severe damper on that. Sorry. That said, the ideas are all free for the taking, and I’m happy to answer questions. (I feel I should lower the expectations here. These apps aren’t amazing or groundbreaking. Consider them small curiosities.)

I’ll have the first post in the series up soon.

Edit: I’ll list them here as I post them.

By way of the Glue chat comic (interesting ideas, by the way), I came across John Palmer’s excellent posts on spatial interfaces and spatial software. Mmm. As a productivity nerd and a hobbyist toolmaker (there are probably ten or so personal tools I’ve been using over the past several years that I’ve never blogged about, but that’ll change soon), I’m very, very interested in these kinds of ideas. Particularly in using 3D interfaces for productivity. I’ve thought a bit about doing that in VR, but I hadn’t really considered building tools in 3D even outside of VR.

No concrete ideas yet, just little threads leading off to various areas that I’m interested in exploring (todo lists, website/blog engines, writing/outlining tools, etc.).

At this point the plan is to use Three.js for prototyping some ideas to see if it’s viable. My initial sense is that for the kinds of tools I’m interested in, I’ll probably need a good way to render/edit text from within Three.js, and it looks like rendering to a canvas and then using the canvas as a texture will work. Anyway, more to come.

A thought on writing novels

As I’m now starting to get more serious about writing novels, I recently made a list of the next few books I want to write after the one I’m working on, and I ran into an unexpected side effect: knowing what the next few items in the queue are has somehow made writing a novel feel far more doable. It’s now a task that has an ending, rather than being something with no end in sight.

Sidenote: I’m not sure how much I’ll talk here about the novel in progress, at least not until I finish a full draft, but I do plan to talk about tools and process. (For example, I’ll write more about this later, but I’m using Vim with a Git repo and a post-commit hook that generates a print-ready proof PDF of the full book via TeX whenever I commit.)

Writing and retooling

I’m currently working on a short story (working title is “A Glass Darkly”), outlining the whole thing before I write it. For a while I thought I might be a discovery writer, but I’ve learned that I do better if I have a solid plan. Figuring out the structure of the story beforehand — the overall arc, the individual scene arcs, the character arcs, etc. — seems to help me write better stories, and not to panic when I get to the middle and usually have no idea what should come next. Structure is my friend.

Anyway, I’m wrapping up the outline (figuring out the middle, as it happens), and should have a first draft banged out a week or so after that. This time, rather than releasing it on my site immediately after I finish it, I’m going to start submitting it to magazines, something I haven’t done before. Racking up a few rejection slips will be a good thing for me as a writer, I think. (Sidenote: I’m very much on the fence re: trying to get my fiction traditionally published vs. self-publishing it. But that’s a topic for another post.)

Tool-wise, I put together a Google spreadsheet for tracking daily word counts, but I realized that when I’m not drafting — when I’m outlining — it doesn’t make sense. Time spent (plus deadlines) is a better metric. So I’ve revised my spreadsheet to track minutes instead.

I’ve also been itching to have a better place to do the actual outlining and writing. This may just be my neverending toolmaking itch, but I think it’ll help me be more productive. I want to write an outline, then flesh it out in place into a list of scenes, then flesh that out into an actual draft. A simplified version of the snowflake method, basically. Forest, trees, branches, leaves.

While you could do this fairly easily in Word or any other writing app, some of the other bits I want (easily moving between levels of abstraction, drag and drop reordering of scenes, etc.) might not be as easy, so to work through the ideas and figure out what I really need, I’m working on a new web app called Storybook. I built an initial prototype a few months ago, but it isn’t very good, for a number of reasons (it isn’t mobile-friendly, it’s too cluttered, etc.). I’m in the middle of rethinking how it should work, and I think I’ve got a better handle now on how the UI should work.

A final word: back in my younger days, I would build tools and then stop using them after a short time. I’m not entirely sure what changed, but now the tools I build stick with me a lot longer, and I can easily see the productivity gains from using them. Toolmaking does take time away from whatever it is the tool is supposed to help me with (writing, in this case), but I’ve found it to be well worth the investment.