mdlbear: (technonerdmonster)

In comments on "Done Since 2018-07-15" we started having a discussion of mirroring and cross-posting DW blog entries, and in particular what my plans are for implementing personal blog sites that mirror all or some of a -- this -- Dreamwidth journal.

Non-techie readers might conceivably want to skip this post.

Where I am now:

Right now, my blog posting process is, well, let's just say idiosyncratic. Up until sometime late last year, I was posting using an Emacs major mode called lj-update-mode; it was pretty good. It had only two significant problems:

  1. It could only create one post at a time, and there was no good way to save a draft and come back to it later. I could live with that.
  2. It stopped working when DW switched to all HTTPS. It was using an obsolete http library, and noone was maintaining either of them.

My current system is much better.

  1. I run a command, either make draft or, if I'm pretty sure I'm going to post immediately, make entry. I pass the filename, without the yyyy/mm/dd prefix, along with an optional title. If I don't pass the title I can add it later. The draft gets checked in with git; I can find out when I started by using git log.
  2. I edit the draft. It can sit around for days or months; doesn't matter. It' an ordinary html file except that it has an email-like header with the metadata in it.
  3. When I'm done, I make post. Done. If I'm posting a draft I have to pass the filename again to tell it which draft; make entry makes a symlink to the entry, which is already in a file called yyyy/mm/dd-filename.html. It gets posted, and committed in git with a suitable commit message.

You can see the code in MakeStuff/blogging on GitHub. It depends on a Python client called charm, which I forked to add the Location: header and some sane defaults like not auto-formatting. Charm is mostly useless -- it does almost everything using a terminal-based text editor. Really? But it does have a "quick-post" mode that takes metadata on the command line, and a "sync" mode that you can use to sync your journal with an archive. Posts in the archive are almost, but not quite, in the same format as the MakeStuff archive; the main difference is that the filenames look like yyyy/mm/dd_HHMM. Close, but not quite there.

There's another advantage that isn't apparent in the code: you can add custom make targets that set up your draft using a template. For example, my "Done since ..." posts are started with make done, and my "Computer Curmudgeon" posts are started with make curmudgeon. There are other shortcuts for River and S4S posts. I also have multiple directories for drafts, separated roughly by subject, but all posting into the same archive.

Where I want to go:

Here's what I want next:

  • The ability to post in either HTML or markdown -- markdown has a great toolchain, including the ability to syntax-color your code blocks.
  • The ability to edit posts by editing the archived post and uploading it. Right now it's a real pain to keep them in sync.
  • A unified archive, with actual URLs in the metadata rather than just the date and time in the filename.
  • The ability to put all or part of my blog on different sites. I really want the computer-related posts to go on (usually shortened to in my notes), and a complete mirror on (
  • Cross-links in both directions between my sites and DW.

How to get there:

Here's a very brief sketch of what needs to be done. It's only vaguely in sequence, and I've undoubtedly left parts out. But it's a start.

Posting, editing, and archiving

  • Posting in HTML or markdown is a pretty easy one; I can do that just by modifying the makefiles and (probably) changing the final extension from .html to .posted so that make can apply its usual dependency-inference magic.
  • Editing and a unified archive will both require a new command-line client. There aren't any. There are libraries, in Ruby, Haskell, and Javascript, that I can wrap a program around. (The Python code in charm doesn't look worth saving.) I wanted to learn Ruby anyway.
  • The unified archive will also require a program that can go back in time and match up archived posts with the right URLs, reconcile the two file naming conventions, and remove the duplicates that are due to archiving posts both in charm and MakeStuff. Not too hard, and it only has to be done once.
  • It would be nice to be able to archive comments, too. The old ljbackup program can do it, so it's feasible. It's in Perl, so it might be a good place to start.

Mirror, mirror, on the server...

This is a separate section because it's mostly orthogonal to the posting, archiving, etc.

  • The only part of the posting section that really needs to be done first is the first one, changing the extension of archived posts to .posted. (That's because make uses extensions to figure out what rules to apply to get from one to another. Remind me to post about make some time.)
  • The post archive may want to have its own git repository.
  • Templating and styling. My websites are starting to show their age; there's nothing really wrong with a retro look, but they also aren't responsive (to different screen sizes -- that's important when most people are reading websites on their phones), or accessible (screen-reader friendly and navigable by keyboard; having different font sizes helps here, too). Any respectable static site generator can do it -- you may remember this post on The Joy of Static Sites -- but the way I'm formatting my metadata will require some custom work. Blosxom and nanoblogger are probably the closest, but they're ancient. I probably ought to resist the temptation to roll my own.

Yeah. Right.

Another fine post from The Computer Curmudgeon.

mdlbear: (technonerdmonster)

It's time to do a little planning. As you may remember from the previous post in this series, there are some projects I want to work on. (I also need to find a job, but that's a not completely separate issue. If anyone needs an expert Java programmer or a git expert, let me know.) The ones I want to concentrate on today are the apps, specifically the checklist app and the setlist app. The first major decision about each of them is which framework to base them on.

I really want to learn both React (with React Native on the back end), and Elm (with Electron on the back end), and I think it makes the most sense to write -- or at least start -- the checklist app first, and use React for it.

Here's my reasoning:

  1. React: React is by far the more popular of the two frameworks, so a lot of jobs ask for React experience. One reason it's more popular is that it's basically just a Javascript library -- programs look like Javascript with a little bit of HTML and HTML-like tags embedded in it. It's easy to learn (not that that's really a problem for me -- see below), and there are a lot of starter kits and tutorials around.
  2. Checklist app: It's pretty clear that the Checklist app will have a much wider audience, so it makes sense to do that first. It will also be easier to monetize (possibly as a freemium app, with the free version stand-alone and the premium version tied to a back-end service).
  3. The Setlist app, which includes a lyrics viewer and playlist generator as well, is likely to start out using my rather unusual music toolchain, and would actually be more useful (and get a lot more traffic) as a front end to the and Song pages. It makes sense for it to start out as part of a website rather than as a stand-alone app.
  4. Elm is a pure functional language (I love functional languages, which is why it would be easy for me) that is closer to Haskell than to Javascript.

Next steps:

  1. Make a place in my working tree for projects. Try not to give in to the temptation to completely refactor the whole hierarchy.
  2. Pull down and install a React starter kit and some kind of Elm starter kit.
  3. Set up the projects' git repos and working trees.


2018-04-13 06:26 am
mdlbear: (technonerdmonster)

It's been just short of a year since I retired, and I don't have a whole lot to show for it in the way of programming, apart from a little work on MakeStuff. OK, a fair amount. And it has an actual user now. But still.

Somewhere around the New Year I started making a(nother) list of potential projects that I wanted to work on in my retirement. As these things do, it got out of hand -- at last count there were 86 unfinished items in it. Time to start something.

There are a few constraints. I can't start any of the woodworking projects yet, because the contents of the garage are in storage waiting for the remodeling to get finished; that includes all of the woodworking tools. So there's that. Same for the recording projects -- my good microphones have either been boxed, or vanished altogether in the last move.

More than half the "projects" on the list are ideas for articles or blog posts -- there are forty or so of those. I should start picking them off, one or two every week, but they're not really projects.

What remains is mostly software: programs (the young people call them "apps" these days) and work on my websites. These are also areas where I have a lot to learn, and where I can develop skills that will be useful if I want -- or need -- to do some consulting. And there's another factor: there's really no difference between a web app and a mobile app! Not any more: with React Native and Electron, one can now build stand-alone cross-platform applications using web front-end frameworks -- they basically bundle a stripped-down browser and a trivial server with your web"site", which is often just a Single-Page Application (SPA). And with languages like Elm that compile into Javascript,...

I can haz apps

One app I want to write will be for managing checklists. (There's an existing app called Checkmate -- my first choice for a name; grumble -- that looks worth mining for ideas.) Beyond being able to have multiple, named lists, I want timing information so that one can ask questions like "how long ago did I last take my pain medication, and is it safe to take another dose?" That needs to work for both scheduled items, and "as-needed", floating items that can start their timer going at any time. More like a combination checklist and reminder system. I'd also like to be able to track the time it takes to go through a checklist, both so that I know when to start if I'm getting ready for something, and so that I know how much I'm improving with practice. Eventually it would be nice to link this to both a website and an Alexa skill -- the website will be easy, since almost the entire app will be usable as the front end.

It would be nice to have a combined lyrics viewer and a setlist planner. (I used to have a setlist planner, but it was in Perl and kept its state in the HTTP query string - bad news for caching and sort of search-engine pessimization.) There would be some overlap with the checklist app, since they both involve going through a list of things in sequence, with associated times. It would be especially useful on a tablet for performances, but it would be most effective combined with a website that hosts lyrics and music, which brings us to...

What a mangled web

Another thing I've been looking at is "Responsive Web Design" -- making websites that adjust smoothly between tiny mobile devices and large-screen desktops. This has long been a design requirement of mine anyway -- almost all my sites do this, but they do it by simply not having much of a layout, and they look bad both on very large screens and very small ones. It's time to take this to the next level by adding responsive CSS and mobile.

There are several websites that need work:,,, and at least. The first one is by far the simplest; just songs, concerts, (proposed) albums, and a gig schedule. adds writing and software projects, but there's still a lot of overlap.

It would make sense to do the others using different responsive design frameworks, just to get experience with a few of the options., in particular, is my "portfolio" site; it's also the only one that has a sidebar at the moment. It might be a good idea to turn into a GitHub Pages site. is my "commercial" site, and it would make sense to use a CMS like Joomla or Drupal for it.

(I have some other sites, e.g. and, but they're simple enough to simply copy the CSS from one of the others. The Interesting.Places sub-site would be worth some attention.)

Now, here's my plan...

The underlying reason for picking this particular set of projects is to market myself as a blogger, consultant, and developer, in hopes of making a little money on the side. That suggests that I should start with the checklist app, and probably start the site makeovers by moving to a GitHub site (which would give me an obvious home for projects like MakeStuff and my development-focussed blogging). On the other hand, making over would probably teach me more about responsive design, especially if I make into a GitHub site. It might make more sense to keep as a separate site, and build the GitHub site from scratch.

In any case, my main blogging site will remain here on Dreamwidth; most likely I'll just cross-post development-related blog entries to (and GitHub, if it's separate). Or would it make more sense to keep all of the blogging concentrated here?

Comments? Ideas? Suggestions? Over to you folks.

Most Popular Tags


RSS Atom

Style Credit

Page generated 2019-04-23 02:23 am
Powered by Dreamwidth Studios