Making it

[mdlbear]

"If you want something to get done, ask a busy person to do it."

I haven't been accomplishing much lately, so obviously I'm not busy enough.

There are several seemingly-unrelated projects going on in the household at the moment: I'm starting my next album, the chaoswolf is starting an HTML class, the servers are getting re-organized, and people have been after the flower_cat to write a cookbook. Meanwhile I've been thinking about writing my blog locally and mirroring it up to LJ.

They're all more closely related than one might think.

You see, I'm a geek. I think nothing of writing a big pile of Makefile templates and Perl scripts to cobble an album, a songbook, and multiple websites together from the same set of sources. The Cat is emphatically not a geek, she wants to be able to type recipes in, maybe to a text editor or a blog client, and have them magically assembled into a cookbook. And a website, of course. Hmmm.

Meanwhile, I've been dissatisfied with the available "content management systems" (a category which includes blogs and wikis as well as monstrous enterprise document management systems like Alfresco (which is pretty damned cool, but then I like monsters, not just ponies and monkeys. But I digress). Partly this is because most of them don't let me use my own version-control tools and my own editor, and essentially none of them take advantage of make.

Not even ikiwiki and blosxom, which probably come closest to what I've been working on.

I like my tools. They pull together metadata from a variety of places, including (for example) song lyric files, recording projects, and local metadata files, and build CD masters, ripped files, playlists, annotated track listings, and multiple web pages. They could do that for cookbooks, too. Or blogs.

What make is, fundamentally, a program for working out dependencies and following rules to run the commands that rebuild whatever depends on the files that changed since the last time you did it. This is just what ikiwiki and blosxom do, only they do it with massive Perl scripts that rumble over your directory tree and figure out what's been changed and follow rules to... Right. Why reinvent make?

The tools have to allow varying degrees of participation: public comments similar to blogs, but also more intense collaboration on editing and recording. I'm not going to be shipping pre-order bonus disks this time around: I want to sell subscriptions to the development process.

... So that's the plan: to refactor my CD, concert, and web tools so that they work for assembling books and blogs as well, publish to hardcopy as well as on multiple websites, and do it in a way that's extensible (with plug-ins), collaborative, and simple enough to be used by non-geeks.

I'm probably going to need a lot of help with that last part.

Comments

[technoshaman.livejournal.com]

simple enough to be used by non-geeks.

I'm probably going to need a lot of help with that last part.

Would be happy to provide suggestions for you, and maybe even some engineering design... I think I still remember enough Human Factors to do this right... and besides, we have the perfect guinea pig in flower_cat. She knows just enough to be dangerous, and you're within shouting distance (and she has my cell number).

First thing we gotta do, is that RSS is mandatory. If we're gonna do this for blogs, it's gotta be LJ-readable, else we lose much of our audience.

Second thing is, we have to figure out a GUI way to tweak the config file. Anything on the command line is gonna confuse non-geeks.

And third thing is, I gotta go sleep before I start typing with my nose here. But I'm sure I'll have more suggestions as we go.

[mdlbear]

Absolutely, on the RSS. You'll be able to hang a blog and its RSS feed anywhere in the directory tree, e.g. vaguely Wikipedia-style discussions, only real blogs. But also able to mirror to LJ and other blogs. So, depending on a post's tags and its position in the tree, it might get mirrored to mdlbear, healthy_fen, and somewhere out on blogger. In other cases I might just post a pointer, the way ohiblather does.

It should rarely be necessary to hand-tweak makefiles -- I do it now because I can, but it can be a lot more automatic than it is. I have a few examples. The main config file is pretty simple, just name=value pairs; we should be able to come up with a Javascript-based editor for them. The editor for non-local users will be one of the Javascript HTML editors.

Metadata files at the moment look like either mail headers (name:value) or LaTeX (\name{value}). The advantage of mail headers is that they greatly simplify post-by-email.

[webmaven.livejournal.com]

"Why reinvent make?"

Probably because many *geeks* find it impossibly arcane to use? make scripts are something that get written once and rarely touched again in many FLOSS projects because no-one can understand them. Many geeks therefore consider make a necessary evil. This gets make reinvented even in contexts that match it's ostensible purpose.

In the circles I develop in (python web apps) for example, buildout recipes are usually used as an alternative to make to take advantage of setuptools, distutils, and easy_install. But still no-one uses it inside a CMS.

Consider that there is an impedance mismatch between running make and it's equivalents (ie. rebuilding everything that has changed) and the desired functionality of many CM systems, which in some contexts need to do things like publish all changes immediately (ie. without waiting for someone to push the big red button), and in others want someone else to approve a change (or set of changes) first, or leave a draft item unpublished indefinitely, or publish something at a future date, etc. In some cases, different projects need different workflows, so it needs to be configurable, even by an administrator, without writing or editing scripts.

[mdlbear]

And sometimes make gets used under the covers; for example a Debian Rules file is an executable makefile. There will definitely be a simple mechanism for splicing in subtrees with different build and publication rules, and even different version-control systems, simply because you want to be able to publish open-source projects. (There are a couple of Linux distributions and other projects that do this kind of thing in a build tree, but I've never seen it in a CMS.)

The big CM systems I've seen all have rudimentary version control, but it's not particularly compatible with the VC systems used in software development.

I already handle drafts and selective publication pretty well -- had to put that in early because I keep my entire filkbook in one directory; some of the songs are under construction, and some aren't mine to begin with so I have to keep them off the web. And over in the album area there's this little problem of bandwidth that keeps me from immediately pushing half a gig of .wav files every time something changes... The setup for that kind of thing isn't as automatic as it needs to be.