Ink, turpentine, paper, water

For at least 1500 years, Japanese artists have practiced suminagashi, the art of marbling paper with ink floating on water. The marbler uses brushes to place alternating drops of black calligraphy ink and turpentine on the surface of a full basin, then lays a sheet of paper down to capture the resulting patterns. They look like clouds, or smoke, or the grain of twisted trees. Each pattern is unique, unlike in Western marbling, where the creator can reproduce essentially the same design many times.

Ink, turpentine, water, paper. It seems so simple.

And it is very simple, but only after you accept one thing: you are not in control of the outcome. The ink goes where it wills, and the marbler can only follow. There are tricks to give the pattern an overall direction, such as controlling the amount of ink and turpentine or gently blowing over the surface of the water. But the heart of suminagashi is trusting what you can’t predict or control.

I recently read George Oates’s essay about the ways that Flickr created its community: Community: From Little Things, Big Things Grow on A List Apart. Two particular paragraphs really jumped out at me:

Embrace the idea that people will warp and stretch your site in ways you can’t predict—they’ll surprise you with their creativity and make something wonderful with what you provide.

There’s no way to design all things for all people. When you’re dealing with The Masses, it’s best to try to facilitate behavior, rather than to predict it. Design, in this context, becomes more about showing what’s *possible* than showing what’s *there*.

Flickr’s history has proven her right. There are any number of wildly varying communities on the site, many of them either accidentally or deliberately experimental. Flickr groups are even cited as a case study in Here Comes Everybody, Clay Shirkey’s recent book on online community dynamics.

And now it’s our turn.

Last year, my company (MediaLab, which makes a library search software package called Aqua Browser Library) released our new social library software: My Discoveries.

The essence of My Discoveries is this: allow users to add information to the library catalog. Let them tag things, make lists of related items, fill in ratings, write reviews. Then let others see what they’ve done. Turn the patron’s interaction with the library’s catalog into a conversation with the catalog, and with each other.

I’ve been involved in both the design and testing. One of the core principles we’ve kept in mind throughout the process is that we cannot predict what people will do with it1. Designing and testing in the light of that kind of uncertainty is very different, and much more interesting, than working to a known, restricted usage profile. It affects everything we do, from what characters are allowed in list names to which statistics we want to gather. How does one design metrics to detect the unpredictable?

Tags, lists, ratings, reviews. It seems so simple.


  1. Of course, we are not so naive as to think that all the new ideas that people come up with for My Discoveries will be good ones. I moderate a web community in my spare time, so I know how bad things can get. As a result, I have put a lot of attention into the administrative interface—and I expect do more on it in the future. If we give users room to innovate, we have to give librarians the wherewithal to detect and clean up misbehavior.

Little Brother

(To the tune of “How can I keep from singing?”)

My high school days were simple once
But now that time is ending.
I’ve learned how much I have to lose
And what is worth defending.
My freedom and my privacy
Depend on one another.
And those who threaten either one
Will deal with Little Brother.

Encryption guards my web of trust
Against the infiltration
Of DHS officials who
Would pry for information.
The Xnet grows with leaps and bounds
No outside force can smother
The message spreads from peer to peer:
We all are Little Brother.

The army trucks and prison cells
That caught us and confined us
Stripped all the innocence away
That we had thought defined us.
But now we know how strong we are
When we work with each other
So anyone who’s watching us:
Watch out for Little Brother.

Originally posted on Making Light

Run down the Jolly Roger, Run up the Union Jack

Making Light is back up, substantially populated with the lost data. Our saintly datameisters are still filling in the cracks, but we have active threads again.

So thank you, everyone, for behaving so nicely here, but let’s move the Making Light discussions back to their natural home. I’ll sweep up and fold up the guest beds, and restore normal evilrooster-type behavior here over the next few days.

Rebuilding the threads

Here’s what I’ve got for Making Light since March 1: original post date, abbreviated name of the post, and the number of comments I have actual copies of.


3/1 Who’s surprised? 66
3/3 All come singing 69
3/3 Can you read this 53
3/4 Greyhawk 253
3/11 Collect underpants 265
3/13 Open thread 103 936
3/16 Just do it 38
3/16 Literary divination 106
3/18 Arthur C. Clarke 177
3/20 Bigger laser 174
3/28 Divided by errors 34
3/28 Open thread 104 931
3/30 London photograph 204
3/31 Deep value 434
4/1 Amsterdam 70
4/2 Pity the Times 167
4/4 Forty years gone 70
4/6 Heads they win 320
4/6 Employ the scythe 126
4/9 SFWA deadline 25
4/11 Future of publishing 32
4/12 Book by its cover 37
4/13 Bury my acorns 87
4/13 Goose-stepping (actually 469) 468
4/14 Open thread 105 906
4/16 Housekeeping 7
4/16 Newsweek 245
4/17 Little Brother 180
4/22 Penn for Hillary 124
4/23 Font game 125
4/23 TNH in San Francisco 18
4/25 Indistinguishable from parody 186
4/26 Clapton 107
4/26 Feeling the heat 31
4/26 SFWA election 45
4/26 TNH in the Observer 105
4/27 Open thread 106 288

In addition, I have a 131-comment version of the Clay Shirky post, but in fact I know there were at least 254 comments; if anyone has a 254-comment version, please do send it on. (Apologies if you did already. I may have lost it. Processing all this stuff on the fly has been a challenge.)

I also have the recently-posted comments to old threads “Worldcongoing,” “New Magics,” and “Abi on catz.” I do not to have the recently-posted comments to “Darwin fish found”; the same apologies apply as in the previous paragraph.

Who’s been saved

Teresa here. I’ve figured out how to make a Google doc universally readable. Please understand that vast amounts of what we’ve been sent is still being logged. This just means you get to see it happening.

There are now two spreadsheets. One is the general spreadsheet Abi built yesterday. I can’t make it visible because Abi owns it, and I didn’t ask her about it before she left for the day. The other is a new one I just put together. It lists individual users known to have posted on or later than 01 March 2008, and notes whether their comments have been saved, and by whom. (If you’re working on that project and want to be able to enter data, write and ask. Our addresses are in their usual spot.)

Here’s the link:

Individual commenters: saved or unsaved.

It’s too bad Mike Ford isn’t here to write “Sinners in the Hands of an Angry Google.”

Needed: Movable Type mavens

Patrick here, again. It’s Sunday morning in Brooklyn.

Last night, Hosting Matters set us up with a new server (larger and more powerful, and all to ourselves, I gather), and we uploaded the complete March 1 backup of our home directory. This is why there’s been a March 1 version of Making Light’s home page visible for the last twelve hours or so at the usual location. Up until a few minutes ago, this was a version of the front page that looked all right, except that links to individual-post pages-with-comments didn’t work, and the Sidelights and Particle sections were empty, along with every other sidebar section that works by pulling data in from a file external to the main index page.

We also uploaded the March 1 export from the site’s MySQL database, and while we slept, Hosting Matters verified that the database export seemed to be okay, and re-created the database and the user. They suggested that we do a full “rebuild” from inside Movable Type, which should recreate all the internal links.

I’ve just now been trying to do that. The first thing I tried was a full site rebuild, but it crapped out and reported “internal server error” somewhere in the midst of rebuilding the second hundred individual-post pages. (A full site rebuild does individual-post pages first, then monthly archive pages, then index pages last.) The next thing I tried was an “index pages only” rebuild, which yielded the version of the front page that’s now visible, with the entire center column empty. I then tried an “individual archive pages only” rebuild, which once again failed in the middle of the second hundred pages. Finally, I made another attempt at a full site rebuild, and it failed (“Internal Server Error – The server encountered an internal error or misconfiguration and was unable to complete your request”) just after the 300-pages-rebuilt mark.

Rebuilds that fail to complete have been a constant bane of our Movable Type installation, but in the past we’ve dealt with them by simply trying again another day. Does anyone have a better idea that will get the site (at least, the March 1 instantiation of the site) up and functional now?

(Noted and of interest: You can get at the individual archive pages that are linked from the “recent comments” sidebar; you can even load the “last 4000 comments” page and get at lots and lots of individual archive pages from that. But as far as I know, until I get a full rebuild to happen, the middle column of the front page is going to stay empty.)

What you need to do *Right Now*; also, an interim status report

Hi, everybody. Teresa here. You may have already read earlier, incomplete versions of this post, which I published prematurely in order to get the THINGS TO DO RIGHT NOW part out fast. I said in them that you’d know you’d seen the complete version when the last word in it was dinosaur. That word is there now.

Onward.

First: apologies if you’ve written to us and haven’t heard back. We’ve been wrestling with the task of getting the pre-March 2008 part of Making Light back online. For a bit there, it looked like we might have lost the whole thing, not just March and April. That was exciting. Fortunately, Patrick found his alternate backup of the MySQL database that’s the difference between losing two months and losing seven years. As Erik Olson said at that point,

Good. We’re now at “this really sucks.”
Not “well, fuck, Maude, better sell the cattle.”

Thanks to many people who shall receive fulsome thanks once this settles down, we’ve reconstituted all the front-page entries, plus Sidelights and Particles. What we don’t have all of are the comments.

THIS IS IMPORTANT: collect your own comments NOW from Google or other search engines, then help collect others. Do it as soon as you can, because Google has already overwritten some caches with versions that end at 01 March. If you find this has happened to the Google cache of your own comments–for instance, Xopher, Niall McAuley, CHip, and NC Hanger have already gotten nailed–try the caches at Yahoo, MSN Live Search, or other search engines.

If you’ve been reading the comments at Making Light via RSS feed, please check and see what you have cached. It’s possible you have the only surviving version of some of our missing comments. We’re particularly on the prowl for new comments that got posted to old threads. We know to look for complete runs of comments from (f.i.) “All come singing” or “The rather difficult font game”; what’s far less obvious is the need to collect recent runs of comments in old threads like “Introduction to New Magics“.

Abi’s chart of what we do and don’t have isn’t up to date. That’s because Abi went to sleep (she’s six time zones east of Plymouth Rock), and Patrick and I are still figuring out exactly what we have on hand. She’ll be gone all day tomorrow, but we figure we’ll have the chart updated by tomorrow morning EST. If you’re doing intensive work salvaging comments and need more up-to-date information, send us your email address and we’ll try to reset the permissions so you can see our working spreadsheet in Google docs.

I’ve finally gotten in touch with Jim Macdonald. He’d been away most of today, and had no idea what was going on. The bits of Making Light’s database we have least hope of recovering are unfinished articles the editors had saved as draft posts in Movable Type. We’ve all lost some, but Jim had been working on the big final post in his Trauma and You series. He took the news very calmly, though it’s possible he was simply too tired to get upset. He says he’ll just have to rewrite the article from memory:

I’d gotten through traumatic amputations, degloving, and avulsions, and was about to start on incisions and lacerations.

When last seen, Jim was running searches on other people’s names. His suggested search string is “Comments posted to Making Light by [name of poster]”. If that doesn’t work, try something else. If that works, come back and tell us what it was.

(UPDATE: Jim has done as much comment-salvaging as he can do tonight. In the comment thread for this entry you’ll find his list of people he knows to have posted comments during the lost months, but whose body of comments he hasn’t salvaged. If you have time, please consider running some of those searches. If you know of other names, post a list of your own–Jim’s list is by no means complete. While you may wind up duplicating someone else’s efforts, you may also save the comments of people who haven’t gotten the word in time to do it themselves. Also, read the whole thread. There’s useful information in it. End of update: tnh, 0300 EDT.)

Salvaging the data is only the first part of the project. Once we’ve collected it, it’ll have to be reprocessed into proper MySQL format and grafted back onto the main database. We’ve had several offers of help, but if you want to add to them, please feel free.

In the midst of all this effort to salvage the missing months, we’re feeling awed and humble about the amount of help we’re getting. We’ve said all along that its readers are the best thing about Making Light, but that’s never seemed more true than it does right now.

(“Hey, look! It’s a dinosaur!”)

Hello, I’m back

[Patrick here, not Abi.]

As luck would have it, I had to run off to band practice very shortly after the crisis began. I’m back. And I’ve had a life-giving, brain-restoring sandwich. As soon as I post this I’m going to start reading the email, and comments below, from our incredibly generous and helpful readers.

As mentioned previously, we do have a complete backup, downloaded with wget, of the site as of March 1, 2008. Hosting Matters has now set up new server space for us, and presently I’ll be trying to upload the backup; if everything works, Making-Light-as-of-March-1 will appear at the usual URL. Much more likely, however, is that I’ll have a zillion finicky permission problems that I’ll need someone’s help with before we get to that point.

At this point we have HTML of all the missing posts, and quite a lot of the missing comments, but many more comments remain to be excavated. Please, if you have been using an RSS reader to follow ML comments–generally or in specific threads–consider that you may actually have copies of comments from the last two months in your reader’s cache folders. Please take a look.

More to come. My actual suspicion is that once we get the March 1 version of ML restored, the best way to re-attach the missing material is going to be by editing and manipulating the MySQL database in the guts of the site. I know very little about this, and the small amount that’s been explained to me by helpful readers is pretty much worn away by time, but it seems to me that in principle it should be possible to automate, or at least semi-automate, the processes of:

(1) stripping away extraneous cruft from various saves of ML content in HTML and XML form, and–

(2) –wedging it all into the appropriate fields and tables in the MySQL database, so that–

(3) –all of ML reappears, right up to the present, along with all the–

[…]books we bought in college and sold for half-price unread
And sacks and sacks of earring backs lost under someone’s bed
And baseball cards and army men and model planes galore
And every tiny plastic high-heel Barbie ever wore

(Thank you, Austin Lounge Lizards. Music here, if you’re willing to sign up for a free trial of Rhapsody.)

Of course, knowing that something is theoretically possible doesn’t mean it’s practical, but I bring it up for discussion nonetheless.

Making Light Entries: The Master List

Below is an initial list of postings that have been active recently, to the best of my memory. I have noted caches that I have URLs for, kindly provided by people in these threads. I am digging through my own caches. If you have better caches of data, please send them to me (abi at the domain we’re on), Patrick, and Teresa (their initials at panix.com). We will update this as we go.

(Why, yes, there is a spreadsheet!)

Making Light

Post No Date Title Highest Known No Latest Complete Comment Savior
April Archives (all April posts) Doctor Science
010186 Apr-27 Where do people find the time? 251 #251 ::: Serge ::: (view all by) ::: May 02, 03:54 PM: Nick Fagerlund
010184 Apr-27 Open thread 106 11 Abi has the email notifications
010178 Apr-26 Eric Clapton, White Power enthusiast 73 #10 ::: will shetterly ::: (view all by) ::: April 26, 11:18 PM: Doctor Science
010177 Apr-26 Teresa in the Observer 54 #13 ::: Angelle ::: (view all by) ::: April 26, 10:53 PM: Doctor Science
010176 Apr-26 Feeling the Heat 26
010175 Apr-26 SFWA election results 45 #45 ::: Dave Bell ::: (view all by) ::: April 30, 02:51 AM: Doctor Science
010174 Apr-25 Indistinguishable from parody 186 #186 ::: Clifton Royston ::: (view all by) ::: May 02, 06:52 PM: Doctor Science
010173 Apr-24 The Rather Difficult Font Game 123 #123 ::: David Goldfarb ::: (view all by) ::: April 29, 06:46 AM: Doctor Science
010168 Apr-23 Live in San Francisco, it’s TNH! 18 #18 ::: pat greene ::: (view all by) ::: April 24, 06:54 PM: Doctor Science
010167 Apr-22 NBC News calls Penn for Hillary 124 #124 ::: Matthew Austern ::: (view all by) ::: April 28, 11:44 PM: Doctor Science
010157 Apr-17 Little Brother 180 #180 ::: Bruce Cohen (SpeakerToManagers) ::: (view all by) ::: April 30, 06:13 AM: Doctor Science
010154 Apr-16 Newsweek invents an alarming trend 245 #235 ::: Ginger ::: (view all by) ::: April 29, 05:02 PM: Doctor Science
010151 Apr-16 Housekeeping 7 #7 ::: paul ::: (view all by) ::: April 17, 03:23 PM: Doctor Science
010146 Apr-14 Open thread 105 906 #215 ::: Bruce Cohen (SpeakerToManagers) ::: (view all by) ::: April 18, 01:48 AM: Doctor Science
010143 Apr-13 Could lead to goose-stepping 469 #150 ::: Lee ::: (view all by) ::: April 15, 02:18 AM: Doctor Science
010142 Apr-13 Bury my acorns at Wounded Knee 87 #87 ::: Stefan Jones ::: (view all by) ::: April 21, 2008, 12:24 AM: Abi
010134 Apr-12 A book by its cover 37 #37 ::: Claire ::: (view all by) ::: April 19, 2008, 11:34 PM: Abi
010133 Apr-11 Future of Publishing, Part 5,271,009 32 #32 ::: Dave Bell ::: (view all by) ::: April 17, 2008, 09:17 AM: Abi
010129 Apr-09 Don’t Miss the Deadline 25 #25 ::: Greg London ::: (view all by) ::: April 11, 2008, 11:13 PM: Abi
010125 Apr-06 Heads they win; tails we lose 320 175 Carol Witt
010123 Apr-06 Some must employ the scythe 126 #126 ::: abi ::: (view all by) ::: April 15, 2008, 03:53 PM: Abi
010115 Apr-04 Pity the Times 167 #167 ::: Terry Karney ::: (view all by) ::: April 16, 2008, 12:22 PM: Abi
010113 Apr-04 Forty years gone 70 #70 ::: rea ::: (view all by) ::: April 08, 2008, 07:20 PM: Abi
010108 Apr-01 Amsterdam 70 #70 ::: Bill Higgins finds more spam ::: (view all by) ::: April 25, 2008, 08:48 AM: Abi
March Archives (all March posts) Doctor Science
010104 Mar-31 Deep Value 434 165 Carol Witt
010103 Mar-30 The photograph that terrorized London 204 203 Carol Witt
010097 Mar-28 Open thread 104 931 239 Carol Witt
010101 Mar-28 Divided by common errors 34 #34 ::: Edward Oleander ::: (view all by) ::: April 05, 2008, 01:39 AM: Abi
010083 Mar-20 Going to need a bigger laser 174 #174 ::: Paul A. ::: (view all by) ::: March 31, 2008, 08:08 PM: Abi
010077 Mar-18 Arthur C. Clarke, 1917-2008 177 #177 ::: Dave Bell ::: (view all by) ::: April 06, 2008, 08:14 PM: Abi
010069 Mar-16 Just do it 38 #38 ::: Fragano Ledgister ::: (view all by) ::: April 10, 2008, 06:36 AM: Abi
010066 Mar-16 Literary Divination, A Parlour Game 106 #106 ::: heresiarch sees, um, spam? ::: (view all by) ::: March 26, 2008, 03:25 AM: abi
010046 Mar-13 Open thread 103 936 222 Carol Witt
010044 Mar-11 Phase one: collect underpants 265 #265 ::: Laurie ::: (view all by) ::: April 21, 2008, 08:18 AM: Abi
010028 Mar-04 Greyhawk’s flags at half-staff 253 200 Carol Witt
010025 Mar-03 Can you read this? 53 #53 ::: Robert N Stephenson ::: (view all by) ::: March 07, 2008, 08:21 PM: Abi
010012 Mar-03 All come singing 69 #69 ::: The Constructivist ::: (view all by) ::: March 21, 2008, 04:37 PM: Abi
007399 Darwin fish found 386 204
005451 Worldcongoing 326 183
009050 Abi Sutherland, on Catz 574 293

Particles

Teresa has been looking at this and will report

Sidelights

Vide supra

Nielsenhayden.com

Placeholder for anything from this page, in particular the new narcolepsy postings

Techie Thread: Making Light is Down

See Patrick’s LiveJournal post for details, but the short version is that we’ve lost everything from March 1.

This thread is for co-ordinating solutions to the problem. What we need:

  1. Any caches of Making Light since March 1. If you have tabs open with ML, please save it, and send to Patrick and Teresa (their initials @panix.com), please.
  2. Any other saved versions: please post what you have here
  3. Offers of assistance in processing these saves
  4. Wisdom in the ways of LJ feeds: apparently there is a makinglight one, which has data back to April 13th
  5. Movable Type and blogging gods and gurus: suggestions, ideas
  6. Time machines you can loan us
  7. Suppliers for magical pixie dust