Two roads diverged, and I missed it

Two roads diverged (as they so often do)
Not in a yellow wood, nor anywhere
Where I could look down each, and muse, and stare,
Compare the leaves and how the grasses grew.
Indeed, there was no choosing when they split—
I didn’t really see the fork at all.
It’s only looking back that I recall
There was a better way, back there a bit.
I could be telling this, in ages hence,
And sigh for roads not taken, chances lost.
But pausing to regret has its own cost
In present choices missed at my expense.
What I intended once, I cannot be,
But I am all that’s possible for me.

Originally posted on Making Light

Shakespearean Firefly

In the spirit of this, a few lines that may be familiar.

Capt: A dozen years have pass’d since this took place,
And all that time hath Parliament kept hid
The secret of this world, till River here
Unearth’d it from their minds.  They feared she knew.
And right they were to dread, since many more
Among the spinning worlds would know it too.
And someone has to speak for those now dead.
For divers reasons did you join my crew
But all have come together to this place.
I’ve in the past demanded much of you.
Today I ask yet more; perhaps for all.
For this I know, as I know anything:
That they will try again.  Another world
Will be the lab for this experiment.
Or maybe they will sweep this landscape clean
And in a year or ten attempt again.
They’ll swing back like the needle to the north
To the belief that they can better men.
And I hold not to that.  Here from this grave
I will not run. I aim to misbehave.

– o0o –

There’s more to flight than buttons, albatross,
More to the pilot’s role than charts and maps.
You know the foremost rule of flying?  Aye,
I know you do, since you know what I’ll say
Before I part my lips.
Riv:                         I do, but yet
I like to hear you say it nonetheless.
Capt:  ‘Tis love.  Though you know all the math the ‘verse
Contains, if in the sky you take a ship unloved
She’ll shake you off as sure as worlds turn.
Love keeps her in the air when she should fall
And tells you that she hurts before she keens.
It makes her home.
Riv:                         The storm is getting worse.
Capt: We will endure a while, till it disperse.

The argument less fraught

In the spirit of one of the greatest xkcd cartoons of all, as given life by Patrick Nielsen Hayden:

Two threads diverged in a blog comment,
And sorry I could not argue both
And be one advocate, on I went
Researching one, and all that it meant
Unto the limits of its growth;

Then fought the other, just as keen
And having perhaps the better claim
Because it had less buzzword-sheen;
Though as for that, ’twas just as mean,
With obfuscation much the same.

And both held promise of delight
With comments not yet answered back.
Oh, I marked the first for another night!
Yet knowing how fight leads on to fight
I doubted if I should ever come back.

I shall be blogging this with a sigh
Someday ages and ages hence:
Two threads diverged in a blog, and I,
I took the one less comment-shy
And that has made all the difference.

The very unhappy path to Terminal 5

I see that the new terminal at London’s Heathrow airport is in the midst of another weekend’s disruption. Problems on the terminal’s opening weekend resulted in over 200 flights cancelled and a backlog of 28,000 bags. The chaos has already cost British Airways, the sole user of the terminal, £16m, and some estimates put the eventual cost around £50m.

Initial problems reported included the failure of either passengers or staff to find the car parks, slow security clearance for staff, consequent delayed opening of check-in desks, and multiple unspecified failures of the baggage handling systems. Once the initial failures occurred, a cascade of problems followed as passengers began to clog up the people-processing mechanisms of the terminal.

This weekend’s disruption has been blamed on “a new glitch” in the baggage handling system. I suspect that means that when they solved one set of problems they unmasked another. A spokeswoman assures us that they’re merely planning how to put an identified solution in place. Her statement doesn’t include any reference to the fact that these problems often nest, like Russian dolls, and that the new solution may uncover—or introduce—new problems.

Of course, my reaction was, “Did they test the terminal before opening it?” The errors shown include both functional errors (people can’t find the car park) and non-functional ones (the baggage system failed under load). No system is implemented bug-free, but the breadth of error type got me wondering.

Fortunately, the Beeb covered some of the testing performed before the terminal opened. Apparently, operation of the terminal was tested over a six month period, using 15,000 people. The testing started with small groups of 30 – 100 people walking through specific parts of the passenger experience. Later, larger groups simulated more complex situations. The maximum test group used was 2,250. BAA said these people would “try out the facilities as if they were operating live.”

Do 2,250 people count as a live test? Are they numerous enough to cause the sorts of problems you’re looking for in a volume test?

I plucked a few numbers off the web and passed them through a spreadsheet. T5 was designed to handle 30 million passengers per year, which comes out to an average of 82,000 per day, or 5,000-odd per hour in the 16-hour operating day (Heathrow has nighttime flight restrictions). These are wildly low numbers, because airports have to handle substantial peaks and troughs. Say that on the busiest day you get 150% of flat average, or 7,500 people per hour. Assuming 75% of the people in the terminal are either arriving from or heading toward London, and the rest are stopping over for an average of 2 hours, that’s about 9,375 passengers in the terminal at a given time.

9,375 is more than 2,250. You can,however, magnify a small sample to simulate a large one (for instance, by shutting off 2/3 the terminal to compact them into a smaller space). It’s not just a numbers game, but a question of how you use your resources.

Most of the testing documentation will of course be confidential. But I found an account of one of the big tests. I would expect that any such report was authorised by BAA, and would therefore be unrealistically rosy; they want passengers to look forward to using the new terminal. But still, the summary shocked me.

In fact the whole experience is probably a bit like the heyday of glamorous air travel – no queues, no borders and no hassle.

Any tester can translate that one. It means:

We didn’t test the queuing mechanisms, border controls, or the way the systems deal with hassled passengers.

In software terms, there is something known as the happy path, which is what happens when all goes well. The happy path is nice to code, nice to test, nice to show to management. It is, however, not the only path through the system, and all the wretched, miserable and thorn-strewn paths must also be checked. This is particularly important in any scenario where problems are prone to snowballing. (Airport problems, of course, snowball beautifully.)

Based on the account I read, these testers were set up to walk the happy path. They were not paid for their labours, but were instead fed and rewarded with gifts. I’m sure food and goodie bags were cheaper than actual pay, but they dilute the honesty of the exchange. We’re animals at heart, and we don’t bite the hand that feeds us. We like people who give us presents. Getting those people—mostly British people—to act like awkward customers, simulate jet lag or disorientation, or even report problems must have been like getting water to flow uphill.

Furthermore, look at the profile of testers mentioned: an ordinary reporter and a bunch of scouts and guides. I wish I believed that the disabled, the families with cranky children, and the non-English speakers were just at another table at breakfast. But I don’t. I suspect the test population was either self-selecting, or chosen to be easy to deal with. In either case, it didn’t sound very realistic.

It’s possible that there was another test day for people who walked the unhappy path, and that it wasn’t reported. It’s possible that they did clever things, like salt the crowd with paid actors to clog up the works and make trouble, and that our reporter simply missed those incidents.

But I’ve worked on big projects for big companies, and that’s not what I’m betting. I suspect there were very good test plans, but that for reasons of cost and timing they were deemed impractical. So compromises were sought in large meetings with mediocre biscuits. Gantt charts were redrawn late at night using vague estimates that were then taken as hard facts. Tempers were lost, pecking orders maintained. People assured each other that it would be all right on the night.

It wasn’t.

I wish I believed that the next time someone does something like this, they’ll learn the lessons from the T5 disaster. But that’s happy path thinking, and I’m a tester. I know better.