Posts

Posted on

On refusing to be terrorized

When it comes to security, I’m firmly in Bruce Schneier’s “refuse to be terrorized” camp. I’m sick of the “we must do something” panic reaction that follows even minimal terrorist threats, and the security theatre it leads to. It’s not about the inconvenience it causes me and other passengers; it’s not even about the ridiculous costs that have to be borne by companies and taxpayers in order to implement measures that will do nothing to prevent a determined terrorist carrying out a attack — all they have to do, after all, is change their target.

Bruce Schneier:

It’s magical thinking: If we defend against what the terrorists did last time, we’ll somehow defend against what they do next time. Of course this doesn’t work. We take away guns and bombs, so the terrorists use box cutters. We take away box cutters and corkscrews, and the terrorists hide explosives in their shoes. We screen shoes, they use liquids. We limit liquids, they sew PETN into their underwear. We implement full-body scanners, and they’re going to do something else. This is a stupid game; we should stop playing it.

Edward Hasbrouck:

What’s most striking about the government’s response to this unsuccessful bombing attempt is the complete lack of any rational relationship between the actions that have been taken and are being proposed, any analysis of which of these and similar tactics did or did not contribute to the success or failure of the Christmas Day attack on Northwest Airlines flight 253, and any likelihood that they would make future attempts at terrorism less likely to succeed.

No, what bothers me is the fact that distorting public policy, removing individual liberties, and instituting a culture of irrationality and fear is exactly what the terrorists wanted in the first place.

Our job is to think critically and rationally, and to ignore the cacophony of other interests trying to use terrorism to advance political careers or increase a television show’s viewership.

So what can I personally do to refuse to be terrorized? Unfortunately, airlines and airports have a monopoly on air travel, and national governments have a monopoly on border control, so I can’t refuse to play the security game whenever I go abroad. I would if I could, because personally I feel more at risk from the security measures than I do from terrorist action. The risk of being subject of an attempted terrorist attack (much less a sucessful one) is, after all, less than being struck by lightning.

For one thing, I can help point out when people take the right approach. For example, The Register:

First: It is completely impossible to prevent terrorists from attacking airliners.

Second: This does not matter. There is no need for greater efforts on security.

Third: A terrorist set fire to his own trousers, suffering eyewateringly painful burns to what Australian cricket commentators sometimes refer to as the “groinal area”, and nobody seems to be laughing. What’s wrong with us?

I can also call bullshit on articles like this one on Travolution in which a tiny, poorly worded sidebar poll on a travel industry blog gets blown up to represent that a majority of all travellers are in favour of body scanners that would not have flagged the completely pants bomber in the first place.

But ranting about this stuff on my blog reminds me of the depressing inevitability of the war in Iraq in late 2002 and early 2003. No matter how much we, the people, protested — and the lead-up to the war we saw the biggest anti-war protests in history — our governments still went blithely ahead. I may have high hopes for the decade ahead, but an end to the ridiculous security theatre that plagues us is beyond even my wildest dreams.

Posted on

How to detect a page request from Safari 4’s Top Sites feature

While reading Jeremy Keith’s blog entry “Safari Askew” I remembered that I had looked at this just before Christmas, and found an answer.

The problem is to do with Safari 4’s “Top Sites” feature, which shows you a pretty grid of thumbnails for the sites you visit most regularly (or that you have pinned in place). The interesting thing is that these thumbnails are live (ish) previews of what those pages currently look like. If you don’t happen to have a Top Sites page open in a tab, and if Safari considers that the current thumbnail is sufficiently out of date, it will automatically go and retrieve the latest version.

Safari 4's Top Sites feature

This can cause headaches for site owners, because in order to show the actual state of the page, Safari relies on full page requests: it downloads all the HTML, CSS, images, and JavaScript for the page, and then displays everything exactly as if the user were viewing the page in a standard tab. Adverts are rendered, page tracking scripts are executed, and to the server it looks just like a regular page hit. This can lead to the site recording unnecessary actions, and your site analytics being all messed up.

At Skyscanner, for example, we noticed this because Google Analytics was showing an unusually high number of Safari users (8.5%) with an abnormally high bounce rate (the proportion of sessions where users view a single page, then walk away with no further interaction): Safari 4 users were twice as likely to bounce as other browsers. Useless sessions generated by Top Sites were the problem.

As Jeremy noted, the user agent that Safari 4 reports for a Top Sites request is exactly the same as for a normal page request. Fortunately, there is a way to distinguish the two types of request: in the current version of Safari 4 (4.0.4) the Top Sites request for the base page (but not its JS/CSS/image resources) carries an additional HTTP header, namely “X-Purpose: preview“.

An easy way to verify this is to use an HTTP debugging proxy like Fiddler or Charles to watch what happens when Top Sites makes a request — see the screen grabs below:

Normal and Top Sites HTTP requests from Safari 4

If your pages are dynamically generated, you can adjust your server-side code to examine the HTTP headers of the incoming request, and take appropriate action if this is a “preview” request. Here’s some sample PHP code:

<?php
if ($_SERVER["HTTP_X_PURPOSE"] == "preview") {
	echo "preview";
} else {
	echo "normal";
}
?>

("X-Purpose" is not a standard HTTP header, and you won’t find “HTTP_X_PURPOSE” in the PHP documentation. It’s the CGI specification that specifies how HTTP headers should be handled: they should be made into an environment variable with an “HTTP_” prefix followed by the header name, with dashes replaced by underscores. Hence, the value of the "X-Purpose" header is placed in the "HTTP_X_PURPOSE" environment variable, and retrieved as $_SERVER["HTTP_X_PURPOSE"].)

If all you’re looking to do it fix your site stats in Google Analytics, then you should just make sure that you don’t write out the GA tracking code for preview requests. If you are concerned about excessive load on your servers, unwanted user actions, or spurious advert impressions, you can take more aggressive action, perhaps by rendering a lightweight version of the page. An extreme possibility I considered was generating a completely different version of the page, specifically designed to look good in the thumbnail format of the Top Sites preview page:

Safari 4 Top Sites with custom preview thumbnail: PROBABLY A BAD IDEA

However, doing this runs counter to the notion that these thumbnails represent previews, and I don’t know how your users would react. More importantly, Google might consider this cloaking, and come round your house in the middle of the night with a baseball bat. Just because it’s possible, doesn’t mean it’s a good idea…

Posted on

2009 in review: books

2009 was the year of the re-read for me, and in particular the series re-read. I love diving into a series and living in the same world as the characters for weeks on end. At the start of the year I couldn’t find any new series I wanted to start on, so I went back to some old favourites:

  • Roger Zelazny’s Amber series
  • Neil Gaiman’s Sandman books
  • Lois MacMaster Bujold’s Vorkosigan series
  • Julian May’s Pliocene and Galactic Milieu series

I have read the first three series several times before already, and they are always fresh, and always fun. For a change, I read the Vorkosigan books in chronological order (although I started with Shards of Honor, rather than Falling Free), and in rapid succession. The early Miles books (The Warrior’s Apprentice and The Vor Game are still my favourites, and it turns out that Diplomatic Immunity isn’t as bad as I remembered it.

The Julian May books were less of a pleasure. If it weren’t for the “I’ve got this far so might as well keep going” sunk-costs argument, I would (should) have stopped about fifty pages into The Golden Torc. I remember waiting eagerly for the publication of Diamond Mask and Magnificat in the mid-90s, but now I just find the characters overly self-absorbed and melodramatic; I don’t think I’ll be re-reading them again.

Lois McMaster Bujold - The Sharing Knife, volume one: Beguilement
Lois McMaster Bujold - The Sharing Knife, volume two: Legacy

Staying with Lois McMaster Bujold, 2009 saw the publication of the fourth and final book in her Sharing Knife series. I bought the first book, Beguilement when it was released in 2006, but didn’t read it at the time. I knew it was only the first part of a single big story, and having been burned by Peter F. Hamilton’s Pandora’s Star the previous year, I didn’t want to start on the tale, only to have to wait three years to get to its end. But when the fourth part, Horizon was published in February, I grabbed it and started from the beginning.

Lois McMaster Bujold - The Sharing Knife, volume three: Passage

The Sharing Knife is a fantasy romance — not a combination that normally leaps off the shelves at me. But I trust Bujold to write characters I care about and a story that interests me regardless of my genre prejudices, and that is exactly what she has done here. The world she has created has more of an American frontier feel to it rather than a pseudo-medieval European vibe: prairies and riverside trading towns rather than dark forests and castles. Lakewalkers patrol the land on the lookout for malices, creatures that rise from the earth and feed on the life energy of everything nearby, and are capable of devastating entire towns. The malices are an ancient menace, but thanks to Lakewalker efforts over the centuries, a rare one now. So rare that many farmers don’t even believe in them any more, which causes tension when Lakewalkers come to town and require their aid. Because of the apparent safety, and the particular magical means by which Lakewalkers fight the malices, farmers have grown likely to fear and mistrust them, and call them witches, rather than heed their warnings.

Lois McMaster Bujold - The Sharing Knife, volume four: Horizon

The main characters are Fawn, a young farmer woman who has run away from home, and Dag, a grizzled Lakewalker veteran. They fall in love despite the large cultural gap that separates them. The books follow them as, outcast from their families and communities, they try find a place for themselves in the world, and to somehow reconcile these two parallel but highly interdependent societies.

Oh, and fight evil! The book covers may look all sweet and pastoral, but the plot is driven forward by the ongoing and very real threat of the malices. While I wouldn’t characterize the books as adventure stories, there is no lack of action. Bujold strikes a masterful balance between the clash-of-cultures love story and edge-of-your-seat thrills, and I can highly recommend the whole series.

(Just make sure you read the first two books, Beguilement and Legacy together, because it is really one book split in two — even the covers are two halves of a single piece. Three and four make up “part two”, but they are a looser pairing.)

Steven Brust - Jhereg

Throughout November and December I also caught up on Steven Brust’s Vlad Taltos series (including an advance copy of Iorich thanks to Patrick at Tor). I know that I had read Jhereg before, and I’m pretty sure that parts of Yendi were familiar; what I can’t figure out is why the hell I would have read the first two books, but not gone on to devour the rest of the series, because it’s awesome. They are (for the most part) hard-boiled fantasy detective novels, not dissimilar in tone to some of my favourite contemporary detective series (Spenser, Elvis Cole, Myron Bolitar, et al.) The “detective” may be a witchcraft-using assassin, and his sidekick a miniature dragon (jhereg) on his shoulder, but the mean streets of the city are still mean, and when tough talk fails, knocking a few heads together often shakes loose the information required.

That only really describes the surface appearance of this series, though; the underlying world is rich and complex, full of Gods, Great Weapons, ancient sorcery, discrimination, social unrest, and peasant revolutions. Another thing that appeals to me about the books is that they are short, in the 200–300 page range. Steven Brust says what he wants to say, and then moves on. I like that. (Unfortunately his writing style in the other Dragaeran books has the opposite effect on me. I haven’t been able to get past the first few pages of The Phoenix Guards.)

Joe Navarro - What Every Body Is Saying

Finally, some non-fiction. I have been interested in non-verbal communication since I picked up a copy of Allan Pease’s Body Language in the mid-eighties. In fact, this is one of the reasons I dislike working remotely. Phone calls strip out all the visual cues I use to pick up on the mood of the office, and to gauge unvoiced concerns in a meeting.

Joe Navarro is a former FBI agent with a background in interrogation and deception detection, and What Every Body Is Saying pays a lot of attention to reading the signals people emit during conversation. Navarro emphasizes that this is not about being able to tell whether someone is lying or telling the truth (although that does make for good TV), but rather about learning whether someone is confident or holding back, relaxed or stressed. When you understand that, you can try to guide the conversation to figure out why they are feeling that way.

Interestingly, this has significant use in usability testing. One of the tenets of usability testing is that you pay attention to what people do, rather than what they say. People are often reluctant to criticize, preferring instead to tell white lies about things they dislike. By studying their posture, gestures, and expressions during a test, and in post-test conversations, you can gain a much better understanding of their emotional state, and where problems lie.

This is definitely one of my top picks of the year. It’s very clearly written, and sprinkled with good illustrations, and not only does it make an excellent introduction to the subject of non-verbal testing, but it also rewards repeated reading and bite-sized dippings-into. Very excellent.

Posted on

One of the most important things I read in 2009

Dean Allen commenting over on Zeldman’s site during the discussion of why he shut down Favrd:

No fit of pique for me, actually. More like a gradual aha, with a slight wince and sigh at the end. I’ve spent the past year or so reading and writing and doing my level best to chip away at 40 years of belief in the logical fallacy that one’s identity meaning – self-worth, self-image, whatever you want to call it – can accurately be measured in the thoughts of others. Much as you and I may enjoy being encouraged through recognition and praise and dislike being saddened by rejection or indifference (god knows we’re taught to right from the outset by caregivers: good boy, pretty picture, heckuva job Brownie), deriving personal value from these transactions in the absence of a well-formed internal frame of reference through which you can decide on your own what does and doesn’t work, and subsequently accept the opinions of others as feedback, is just plain faulty thinking, of the sort that makes otherwise capable, centred people all loopy and weird.

I never used Favrd; I haven’t spent a lot of time thinking about the rights and wrongs of shutting it down so abruptly. What I have spent a lot of time thinking about is my own sense of self and self-worth. I intend to spend more time on this in 2010.

Posted on

To blog or to tweet?

Yesterday Anil Dash wrote a great piece about being on Twitter’s “Suggested User List.” I tweeted it this morning, as did many other people. This afternoon Anil tweeted:

Thanks to all who retweeted my piece about Twitter’s suggested list. (I miss the days when people responded with their own blog posts, tho.)

Yeah, I miss those days too. Twitter (which I will use as shorthand for any short messaging service) makes it very easy to post a quick thought, a status update, or a link to something interesting. Much of blogging used to be like that, but the blog postings used to be longer, with more context. I rarely posted one-line blog entries or simple links, because I felt that a blog entry should have some kind of story — a thread, a narrative — to guide the reader to a particular conclusion or insight. This classical “essay” model of blogging has problems, though:

  • If you are locked into the classical mode, a sense of obligation to write a more elaborate piece can prevent you from writing anything.
  • Not all thoughts, insights, and amusing moments you want to share with your friends (and the rest of the world) warrant more than a brief sentence or two. Explaining an aphorism can ruin the joke.
  • And sometimes all you want to do is point to a link, or stick up a photo. In first-generation blogging tools the overhead of creating an “entry” could be more effort than the mini-message was worth.

Over the last year, I have heard many people say that they blog much less now that Twitter is around. By providing an outlet for people to express themselves quickly and efficiently (albeit in 140-character chunks), it bleeds off a certain amount of creative pressure that would otherwise have led to more and longer blog postings.

I don’t want to make this a lament on the “death of blogging”, because blogging emphatically isn’t dead; otherwise, where would we all find the cool links to tweet about? In fact, good blog articles benefit from Twitter, because it provides a fast and social notification medium in parallel to the existing mechanisms of RSS feeds and reblogging.

Furthermore, “The Tweet” is as much a new form of writing as blogging was, with its own grammar and conventions. In his article “How the Web and the Weblog have changed Writing“, Philip Greenspun talks about how blogs have given a natural home to ideas that don’t need to be padded out to a whole book or a full-length magazine article; Twitter is just the next logical step.

The web has given us an unprecedented range of tools with which to express ourselves. It is up to us to learn to use these tools, and to understand when it is appropriate to use each one.

Posted on

JavaScript: Rule 1

JavaScript development (for the web) is big right now, and in 2010 it’s only going to get bigger. There is a lot of information available on good practices for web application development with JavaScript, but if performance and maintainability matter to you, few things* are more important than Rule 1:

Source code is for humans, live code is for machines.

I first heard Christian Heilmann state this rule at @Media Ajax 2007 as “production code is for humans, live code is for machines” but I prefer the term “source code” to “production code”. The rule was important and prescient then; now I consider it the absolute foundation of any significant web application.

Development code is transformed into live code before being sent to the browser.

Rule 1 comes into play as soon as you have a project that involves any of the following:

  • More than a couple of hundred lines of JavaScript source code.
  • More than a couple of JavaScript source code files.
  • More than one developer.

In olden days (say, 2006), the JavaScript code for many web applications consisted of a single file. Comments were few and far between, and variable names were kept short because we wanted the code to be as small as possible. But this strategy does not scale. As projects get more complex, a single file filled with non-descriprive functions and one-letter variable names quickly becomes unreadable. In modern web application development there are two weapons we can use to tackle this problem: concatenation and minification.

Concatenation is the process of taking multiple JavaScript files, and combining them into a single file. Minification is the process of transforming a JavaScript source file to make it smaller. Early JavaScript minifiers just removed comments and unnecessary whitespace, but modern minifiers (YUI compressor, Dojo Shrinksafe, Google Closure Compiler) can also perform more aggresive optimizations, such as shortening variable names and removing unneeded code.

Basically, the JavaScript source code that developers write is no longer the JavaScript code that end users receive, and can see when they view the components for the page. The first benefit of this is performance: concatenating files reduces the number of HTTP requests web app requires, and minification reduces the download size (and can also reduce parsing time on the client). The second benefit comes in the form of better source code clarity, which is good for development and maintenance:

  • You can use multiple files to hold your source code, and organize them (naming convention, file hierarchy) however you want.
  • You can use lots of whitespace and comments throughout your source code.
  • You can use descriptive (long) variable and function names, because the minifier will try to reduce them to one-letter variables (identifier replacement).

A further consequence of this is that when you are writing JavaScript code, you now have to keep three audiences in mind:

  • The browser’s JavaScript interpreter. Your code MUST run in the target browser(s).
  • Humans. Your code SHOULD be clear and comprehensible to your colleagues.
  • The minifier (“compiler”). Your code SHOULD be structured to help your minifier apply its optimizations.

Personally, I am most familiar with the YUI compressor. If you choose to use it, then Nicholas Zakas’ slide deck on structuring your code for optimal optimization is required reading. In addition to helping the YUI compressor make your code even smaller, many of these tips make for tidier, more comprehensible code; and because they encourage the use of local instead of global variables, they can even make your code run faster too. Here are some of the main points:

  • Any literal value used two or more times should be stored in a local variable.
  • Any global variable used two or more times should be stored in a local variable.
  • Any property used two or more times should be stored in a local variable.
  • Try to use only one var keyword and one return statement per function.
  • Don’t use eval, with, or JScript conditional comments.

Where exactly you apply the concatenation and minification is up to you, and will depend on how your deployment pipeline is set up. You can do it as part of your build process, or you can do it at runtime — provided you have some form of caching enabled so that you don’t repeat the process for every request.

*Finally, I should note that the process of concatenation and minification is not the single most effective strategy in reducing the download size of your JavaScript components; that honour goes to gzipping, so I suppose that enabling gzip compression on your web application should, in the programming tradition of zero-based lists, be considered as “Rule 0”. But when it comes to maintaining a complex application, and delivering a compact download package, this is the way to go.