Here’s what I’ve got for Making Light since March 1: original post date, abbreviated name of the post, and the number of comments I have actual copies of.

In addition, I have a 131-comment version of the Clay Shirky post, but in fact I know there were at least 254 comments; if anyone has a 254-comment version, please do send it on. (Apologies if you did already. I may have lost it. Processing all this stuff on the fly has been a challenge.)

I also have the recently-posted comments to old threads “Worldcongoing,” “New Magics,” and “Abi on catz.” I do not to have the recently-posted comments to “Darwin fish found”; the same apologies apply as in the previous paragraph.

Patrick here, again. It’s Sunday morning in Brooklyn.

Last night, Hosting Matters set us up with a new server (larger and more powerful, and all to ourselves, I gather), and we uploaded the complete March 1 backup of our home directory. This is why there’s been a March 1 version of Making Light’s home page visible for the last twelve hours or so at the usual location. Up until a few minutes ago, this was a version of the front page that looked all right, except that links to individual-post pages-with-comments didn’t work, and the Sidelights and Particle sections were empty, along with every other sidebar section that works by pulling data in from a file external to the main index page.

We also uploaded the March 1 export from the site’s MySQL database, and while we slept, Hosting Matters verified that the database export seemed to be okay, and re-created the database and the user. They suggested that we do a full “rebuild” from inside Movable Type, which should recreate all the internal links.

I’ve just now been trying to do that. The first thing I tried was a full site rebuild, but it crapped out and reported “internal server error” somewhere in the midst of rebuilding the second hundred individual-post pages. (A full site rebuild does individual-post pages first, then monthly archive pages, then index pages last.) The next thing I tried was an “index pages only” rebuild, which yielded the version of the front page that’s now visible, with the entire center column empty. I then tried an “individual archive pages only” rebuild, which once again failed in the middle of the second hundred pages. Finally, I made another attempt at a full site rebuild, and it failed (“Internal Server Error – The server encountered an internal error or misconfiguration and was unable to complete your request”) just after the 300-pages-rebuilt mark.

Rebuilds that fail to complete have been a constant bane of our Movable Type installation, but in the past we’ve dealt with them by simply trying again another day. Does anyone have a better idea that will get the site (at least, the March 1 instantiation of the site) up and functional now?

(Noted and of interest: You can get at the individual archive pages that are linked from the “recent comments” sidebar; you can even load the “last 4000 comments” page and get at lots and lots of individual archive pages from that. But as far as I know, until I get a full rebuild to happen, the middle column of the front page is going to stay empty.)

[Patrick here, not Abi.]

As luck would have it, I had to run off to band practice very shortly after the crisis began. I’m back. And I’ve had a life-giving, brain-restoring sandwich. As soon as I post this I’m going to start reading the email, and comments below, from our incredibly generous and helpful readers.

As mentioned previously, we do have a complete backup, downloaded with wget, of the site as of March 1, 2008. Hosting Matters has now set up new server space for us, and presently I’ll be trying to upload the backup; if everything works, Making-Light-as-of-March-1 will appear at the usual URL. Much more likely, however, is that I’ll have a zillion finicky permission problems that I’ll need someone’s help with before we get to that point.

At this point we have HTML of all the missing posts, and quite a lot of the missing comments, but many more comments remain to be excavated. Please, if you have been using an RSS reader to follow ML comments–generally or in specific threads–consider that you may actually have copies of comments from the last two months in your reader’s cache folders. Please take a look.

More to come. My actual suspicion is that once we get the March 1 version of ML restored, the best way to re-attach the missing material is going to be by editing and manipulating the MySQL database in the guts of the site. I know very little about this, and the small amount that’s been explained to me by helpful readers is pretty much worn away by time, but it seems to me that in principle it should be possible to automate, or at least semi-automate, the processes of:

(1) stripping away extraneous cruft from various saves of ML content in HTML and XML form, and–

(2) –wedging it all into the appropriate fields and tables in the MySQL database, so that–

(3) –all of ML reappears, right up to the present, along with all the–

[…]books we bought in college and sold for half-price unread
And sacks and sacks of earring backs lost under someone’s bed
And baseball cards and army men and model planes galore
And every tiny plastic high-heel Barbie ever wore

(Thank you, Austin Lounge Lizards. Music here, if you’re willing to sign up for a free trial of Rhapsody.)

Of course, knowing that something is theoretically possible doesn’t mean it’s practical, but I bring it up for discussion nonetheless.