Words by c.z.robertson

Funky Caching and re-inventing Make

2002-11-15 10:48:17 UTC

Phil Ringnalda talks about Funky Caching (via Kryogenix). I've been thinking about this issue while working on Catkin. Catkin is pretty much tied to the Movable Type model at the moment, and I'll release a version 1 before I make any fundamental changes. But for version 2 and beyond I've been thinking about doing some sort of funky caching.

I figure that it actually needs to be something more than just caching as Phil Ringnalda describes it. A few weeks ago I figured out that what I was doing was essentially re-inventing Make.

In case you're unfamiliar with it, Make is a tool that builds files selectively by looking at the files they depend on. For example, myProg is built from compiling main.c and linking in libOne.o and libTwo.o. libOne.o is built from libOne.c and libOne.h, and similarly libTwo.o is built from libTwo.c and libTwo.h. Now, if libTwo.h is modified, running "make myProg" will cause Make to look at the timestamps on the files and conclude that libTwo.o has to be rebuilt, and then myProg will have to be rebuilt. It can tell that libOne.o won't have to be rebuilt because none of its dependencies have been modified since it was last built.

So that's what I've been considering for a future version of Catkin. Trivially speaking, the html page for an entry depends on the xml file for that entry and the template for html pages of entries.

Actually we can be more sophisticated than that. Catkin currently has a file called index.xml which helps in rebuilding the front page, archive, and rss files. A Make-like system could also have intermediate files like this.

I can see one difficulty with this that I couldn't solve just by using Make. My index.xml file is usually not rebuilt from scratch. It gets read, modified, and re-written when a new entry or comment is made in such a way that it stays in sync with the entry xml files without re-reading all the xml files that wouldn't have changed. With a purely Make-style system I'd have to re-read all those entry xml files whenever one of them was modified. Unfortunately that's one of the most costly parts of rebuilding.