Legends of the Sun Pig

Dean Allen commenting over on Zeldman’s site during the discussion of why he shut down Favrd:

No fit of pique for me, actually. More like a gradual aha, with a slight wince and sigh at the end. I’ve spent the past year or so reading and writing and doing my level best to chip away at 40 years of belief in the logical fallacy that one’s identity meaning – self-worth, self-image, whatever you want to call it – can accurately be measured in the thoughts of others. Much as you and I may enjoy being encouraged through recognition and praise and dislike being saddened by rejection or indifference (god knows we’re taught to right from the outset by caregivers: good boy, pretty picture, heckuva job Brownie), deriving personal value from these transactions in the absence of a well-formed internal frame of reference through which you can decide on your own what does and doesn’t work, and subsequently accept the opinions of others as feedback, is just plain faulty thinking, of the sort that makes otherwise capable, centred people all loopy and weird.

I never used Favrd; I haven’t spent a lot of time thinking about the rights and wrongs of shutting it down so abruptly. What I have spent a lot of time thinking about is my own sense of self and self-worth. I intend to spend more time on this in 2010.

Yesterday Anil Dash wrote a great piece about being on Twitter’s “Suggested User List.” I tweeted it this morning, as did many other people. This afternoon Anil tweeted:

Thanks to all who retweeted my piece about Twitter’s suggested list. (I miss the days when people responded with their own blog posts, tho.)

Yeah, I miss those days too. Twitter (which I will use as shorthand for any short messaging service) makes it very easy to post a quick thought, a status update, or a link to something interesting. Much of blogging used to be like that, but the blog postings used to be longer, with more context. I rarely posted one-line blog entries or simple links, because I felt that a blog entry should have some kind of story — a thread, a narrative — to guide the reader to a particular conclusion or insight. This classical “essay” model of blogging has problems, though:

• If you are locked into the classical mode, a sense of obligation to write a more elaborate piece can prevent you from writing anything.
• Not all thoughts, insights, and amusing moments you want to share with your friends (and the rest of the world) warrant more than a brief sentence or two. Explaining an aphorism can ruin the joke.
• And sometimes all you want to do is point to a link, or stick up a photo. In first-generation blogging tools the overhead of creating an “entry” could be more effort than the mini-message was worth.

Over the last year, I have heard many people say that they blog much less now that Twitter is around. By providing an outlet for people to express themselves quickly and efficiently (albeit in 140-character chunks), it bleeds off a certain amount of creative pressure that would otherwise have led to more and longer blog postings.

I don’t want to make this a lament on the “death of blogging”, because blogging emphatically isn’t dead; otherwise, where would we all find the cool links to tweet about? In fact, good blog articles benefit from Twitter, because it provides a fast and social notification medium in parallel to the existing mechanisms of RSS feeds and reblogging.

Furthermore, “The Tweet” is as much a new form of writing as blogging was, with its own grammar and conventions. In his article “How the Web and the Weblog have changed Writing“, Philip Greenspun talks about how blogs have given a natural home to ideas that don’t need to be padded out to a whole book or a full-length magazine article; Twitter is just the next logical step.

The web has given us an unprecedented range of tools with which to express ourselves. It is up to us to learn to use these tools, and to understand when it is appropriate to use each one.

JavaScript development (for the web) is big right now, and in 2010 it’s only going to get bigger. There is a lot of information available on good practices for web application development with JavaScript, but if performance and maintainability matter to you, few things^* are more important than Rule 1:

Source code is for humans, live code is for machines.

I first heard Christian Heilmann state this rule at @Media Ajax 2007 as “production code is for humans, live code is for machines” but I prefer the term “source code” to “production code”. The rule was important and prescient then; now I consider it the absolute foundation of any significant web application.

Rule 1 comes into play as soon as you have a project that involves any of the following:

• More than a couple of hundred lines of JavaScript source code.
• More than a couple of JavaScript source code files.
• More than one developer.

In olden days (say, 2006), the JavaScript code for many web applications consisted of a single file. Comments were few and far between, and variable names were kept short because we wanted the code to be as small as possible. But this strategy does not scale. As projects get more complex, a single file filled with non-descriprive functions and one-letter variable names quickly becomes unreadable. In modern web application development there are two weapons we can use to tackle this problem: concatenation and minification.

Concatenation is the process of taking multiple JavaScript files, and combining them into a single file. Minification is the process of transforming a JavaScript source file to make it smaller. Early JavaScript minifiers just removed comments and unnecessary whitespace, but modern minifiers (YUI compressor, Dojo Shrinksafe, Google Closure Compiler) can also perform more aggresive optimizations, such as shortening variable names and removing unneeded code.

Basically, the JavaScript source code that developers write is no longer the JavaScript code that end users receive, and can see when they view the components for the page. The first benefit of this is performance: concatenating files reduces the number of HTTP requests web app requires, and minification reduces the download size (and can also reduce parsing time on the client). The second benefit comes in the form of better source code clarity, which is good for development and maintenance:

• You can use multiple files to hold your source code, and organize them (naming convention, file hierarchy) however you want.
• You can use lots of whitespace and comments throughout your source code.
• You can use descriptive (long) variable and function names, because the minifier will try to reduce them to one-letter variables (identifier replacement).

A further consequence of this is that when you are writing JavaScript code, you now have to keep three audiences in mind:

• The browser’s JavaScript interpreter. Your code MUST run in the target browser(s).
• Humans. Your code SHOULD be clear and comprehensible to your colleagues.
• The minifier (“compiler”). Your code SHOULD be structured to help your minifier apply its optimizations.

Personally, I am most familiar with the YUI compressor. If you choose to use it, then Nicholas Zakas’ slide deck on structuring your code for optimal optimization is required reading. In addition to helping the YUI compressor make your code even smaller, many of these tips make for tidier, more comprehensible code; and because they encourage the use of local instead of global variables, they can even make your code run faster too. Here are some of the main points:

• Any literal value used two or more times should be stored in a local variable.
• Any global variable used two or more times should be stored in a local variable.
• Any property used two or more times should be stored in a local variable.
• Try to use only one var keyword and one return statement per function.
• Don’t use eval, with, or JScript conditional comments.

Where exactly you apply the concatenation and minification is up to you, and will depend on how your deployment pipeline is set up. You can do it as part of your build process, or you can do it at runtime — provided you have some form of caching enabled so that you don’t repeat the process for every request.

^*Finally, I should note that the process of concatenation and minification is not the single most effective strategy in reducing the download size of your JavaScript components; that honour goes to gzipping, so I suppose that enabling gzip compression on your web application should, in the programming tradition of zero-based lists, be considered as “Rule 0”. But when it comes to maintaining a complex application, and delivering a compact download package, this is the way to go.

Fronteers is the Dutch organization for Front-end web developers. Since 2007 it has run regular meetings for members and non-members, and it also runs an annual conference that attracts speakers and delegates from all over the world. For some reason I didn’t blog about the 2008 conference, but here’s my report of the 2009 conference that took place at Felix Meritis in Amsterdam last week. It was a single-track conference with remarkably few beards present — I’d estimate that fewer than 10% of the men were sporting facial hair (even including soul patches), an extremely low figure for the tech industry.

The speaker photos in this blog post were made by Marien Van Os (marvos on flickr), used under a Creative Commons license. (And also by permission — because it’s polite to ask.) Please go and have a look at his photos on 1pictureaday.com, because they’re amazing.

Thursday 5 November

What’s New With The Mobile Web – Molly Holzschlag

Molly opened the conference with an overview of where “The Mobile Web” stands today, and what we can (and should) be doing to work with it. Her view is that the most practical strategy at the moment is to create a single site for both the mobile and desktop web (“desktop web” being the term that many of the speakers used to distinguish the “traditional” mode of accessing the web with a PC and a browser), but optimize the experience for the device. This means using things like server-side UA detection, CSS media types, capability detection, and CSS3 media queries to determine what the user is browsing with, and adjust the display accordingly. This certainly fits with existing best practices for accessibility, which emphasize progressive enhancement over graceful degradation; progressive enhancement for mobile devices can be treated as another branch of the same tree.

The Mobile Web, Or The Masochist’s Guide To Gleeful Self-flagellation – Peter-Paul Koch

Peter-Paul has recently been doing research into mobile browsers, and has published a ton of useful and interesting information on his site Quirksmode.org. Here he talked about the mobile browser landscape, highlighting the fact that — at present — you have almost no hope of building a site that will render the same on all devices.

He then went on to discuss W3C Widgets, which is a specification to allow you to package up a single page of HTML, along with CSS and JavaScript, and deploy it as a single package to a widget-capable device. The device typically uses a separate runtime to display the widget outside of a browser context, which makes it seem much more like an application than a web page. The technology to run these widgets exists already on S60 phones, and in Opera Mobile, but it is unlikely that the iPhone will (ever) implement it, which is a significant downside for developers who would like to target a uniform platform.

• Slides: The Mobile Web (or the masochist’s guide to gleeful self-flagellation)

Of Hamsters, Feature Creatures And Missed Opportunities – Chris Heilmann

Chris is always an entertaining speaker. He is passionate about the web, software development, and the technologies we have available to us today. Although the topics he covered in the first half of this presentation were fun and inspiring (don’t hang on to knowledge beyond its use-by date; share your skills; use a JavaScript library), it was the second half where he talked about YQL that really set the audience alight.

In case you haven’t come across it before, YQL is a service from Yahoo! that allows you to run SQL-like statements on the web. Not as in, “here’s a web-based interface to a back-end SQL database”, but as in, “select * from flickr.photos.search where text="Cat" limit 10” to get a list of cat pictures from Flickr as XML, or as JSON with a callback (JSONP) so that you can get the results directly from client code, with no server-side proxy needed.

(Pause for awesome.)

YQL has been publicly available for about a year now, and has been getting steadily more powerful and useful. Not only can you query Yahoo’s own services, but community tables allows anyone to create a table definition for any arbitrary web service. You can use XPath queries to select data from the HTML of an arbitrary web page. It is cool, useful, and above all stable. It is built on top of Yahoo Pipes, and apparently powers many of Yahoo’s own new services now. According to Chris, “if YQL goes down, Yahoo goes down.”

If you haven’t tried it yet, just fire up the developer console and experiment with some of the sample queries. You don’t even need a developer key to get started, and run YQL queries from your own applications.

• Slides, talk outline, live audio recording

Building And Maintaining a complex frontend codebase – Martin Savelkoul

Martin spoke about the processes and techniques they use at Mirabeau for organizing their front-end code to help with things like client hand-overs, version control, and build processes. Everything he talked about was good practice, but none of it was new to me; and the case study he showed wasn’t what I would consider “complex.”

The Future State of Layout – Stephen Hay

Stephen discussed the three competing layout modules currently under consideration for CSS3: CSS3-grid, CSS3-layout, and CSS3-flexbox. All three of these proposals allow you to specify a layout for your elements in CSS independent of the element source order — which means no more tedious mucking around with floats and negative margins to implement grid-based page designs.

Unfortunately, the only native browser implementation that exists right now is CSS-flexbox in Firefox. There is a jQuery plugin that implements the CSS3-layout functionality, which is interesting. Stephen seemed to favour this proposal, and I like it, too. Your CSS effectively contains an ascii art representation of what the layout will look like. It’s a very visual representation, and is easy to explain and understand. Combined with CSS3 media queries (different than CSS2 media types), CSS3 layout will completely change how we provide support for multiple resolutions, media, and devices in our sites.

• Slides: The Future State Of Layout

Understanding JavaScript Testing – John Resig

When you talk about JavaScript testing, what you usually mean is testing whether a piece of JavaScript works correctly on a page in a web browser. Does panel A expand when you click button B, for example. You are testing DOM interactions — with all the cross-browser fun that brings — as much as the actual algorithms.

John noted that there are many JavaScript frameworks available for running unit tests on client-side code: QUnit, JSUnit, YUITest, etc. He demonstrated that writing such a framework is not even all that hard. What is hard is scaling this kind of testing. When you have half a dozen major browsers to support, each with potentially dozens of variations, how do you effectively manage your test suite?

John has recently been working on TestSwarm, a distributed continuous integration JavaScript testing solution. You point a set of clients at a central server, and the server distributes tests for the client to run. At the moment, TestSwarm.com exists to support testing for some of the most popular JavaScript libraries (jQuery, YUI, Dojo, et al.), but you can download the code to run your own server and set up your own swarm of clients to for your own projects.

• Slides: Understanding JavaScript Testing

Even Faster Web Sites – Steve Souders

Steve Souders literally wrote the book on web site performance. (And its sequel, too.) He created YSlow, Hammerhead, SpriteMe, and writes extensively about performance on his blog. Front-end performance is a subject very close to my heart, and I was very excited that he was speaking at Fronteers this year.

The theme of his talk was Fast By Default. There is a lot of research and information available that shows how much the perceived speed of a web site depends on front-end performance, and how much of an impact this speed (or lack of it) has on the bottom line for a business. So why don’t more businesses spend more time on it? Because it’s actually still quite hard.

Steve highlighted several tools that can help with this, though. Aptimize Website Accelerator, for example, is a drop-in software solution that automatically applies optimizations like rolling up multiple JS and CSS files to reduce HTTP requests, minifying and gzipping JS and CSS resources, and spriting of CSS background images. WebPageTest.org, a web version of the AOL PageTest tool, gives extensive performance statistics and optimization tips for any page you choose to submit. ShowSlow.com uses showslow to collect and chart YSlow performance statistics over time, so you can see trends for your pages. SpriteMe, Steve’s own tool, analyzes a web page and gives you ready-made CSS sprites and sample CSS code to optimize your background images.

He also talked about progressive rendering, the techniques available for parallel downloading of script resources, and recent innovations in this area, such as reducing startup latency by putting script inside comments so that it the code isn’t parsed as soon as it is downloaded. Instead, you wait until the code is needed, remove the comment markers (/* */) and eval the whole block.

• Slides: Even Faster Web Sites

Friday 6 November

Ajax Security – Douglas Crockford

Douglas Crockford really knows how to weave a good narrative. He leads you through important and complex material with the grace of a great storyteller — his presentations often feel like documentary films. In this one, he talked about Ajax security.

The core of the security problem with “Ajax” (using that as a convenient shorthand for third-party script inclusion) is that third-party scripts have the same rights on a page as scripts from the host site. Third-party script can take over the page, and there is no way for the user to know that what they see is not part of the application. If the script is malicious by default, we call this a XSS (cross-site scripting) attack. But even well-intentioned inclusions can turn sour, as the New York Times found back in September. Basically, as soon as you include third-party script, you give up control of your page. Douglas put it succinctly as follows:

A mashup is a self-inflicted XSS attack. (Advertising is a mashup.)

(Digression: Adverts are the bane of my life. If you want to include banner ads on your site, an ad network will typically give you a piece of JavaScript to insert on your page wherever you want the ad to appear. This JavaScript will typically be some of the most hideously tortuous code you have ever seen. It will reveal a positively Lovecraftian chain of document.write() calls, eval() statements, and nonsensical browser hacks that eventually result in a 2MB Flash file being pulled over a 28.8kbps network connection from somewhere in Kazakhstan. Advert code is the closest thing modern science has to the Berryman Logical Imaging Technique. I hear that the CIA used to teach terrorist suspects JavaScript just so they could expose them to advert code, and break their wills.)

However, it turns out that by adopting an object-capabilities model, third party scripts can be secured. I didn’t understand object capabilities the first time I came across them (at @media Ajax in 2007), but this presentation made the principles clear. Douglas’s ADSafe defines a safe subset of JavaScript, and an API with which the code can get access to the DOM. The JSLint tool has an ADSafe mode that allows you to statically check a JS file and prove that it conforms.

What we need now is for advert publishers to start publishing adverts for the modern web:

• Adverts that do not expose our sites to XSS attacks (ADSafe compliant)
• Adverts that can be loaded unobtrusively instead of with document.write() calls, so that we we can avoid the performance penalties of inline code (and the hacks we sometimes use to get around them anyway)

Unfortunately, site owners are not in a position of power here. The advert publishers have code that “works”: so long as you stick to their guidelines, it loads in virtually any browser with JS enabled — at the expense of performance. They don’t own the sites on which the ads appear, so they are not the target of security vulnerabilities. As Douglas put it towards the end of the presentation:

The advertising problem is serious. The publishers must go as one to the advertising industry and demand basic standards of ad quality.

Afterwards, I was chatting with Martin Reurings and Sam Foster about this. One of the options we discussed was creating some kind of action group like the Web Standards Project (WaSP) to fight for sanity and standards in web adverts. (Might this even be grounds for another WaSP task force?) This has been going round in my head ever since, and I want to write more about it soon. If you have an interest in this subject, please leave a comment below.

• Slides: Ajax Security

Using HTML5 Features In IE8 – Pete LePage

Pete spoke about some of the features available in IE8 that were not there in previous versions, such as mutable DOM properties, native JSON, and the selectors API. Not all strictly HTML5 features, but hey. He also highlighted the SuperPreview tool (part of the Expression Web suite) that allows you to preview web pages side-by side in various browsers, e.g. Firefox and IE8. It also has the IE6 rendering engine built in to it, so you can even use it to test IE6, IE7, and IE8 on the same machine, without resorting to unsupported hacks. This is very cool, and there is even a free version available (SuperPreview for IE), which is limited to comparing pages in the different versions of IE, but which is still very useful for doing compatibility work.

• Slides: Using HTML5 Features in IE8

The Type We Want: Using Fonts On The Web – Jonathan Snook

Native web fonts are big right now. With the release of Firefox 3.5 earlier in 2009, we have reached a critical mass of browsers that support embedding of fonts with @font-face. Okay, so you have to have the font in several different formats (TTF, EOT, SVG) for it to render in all of the supportable browsers, but at least it’s possible. Well, technically, at least. There’s still that whole licensing issue.

Jonathan has done a lot of research into web fonts recently, and in this presentation he took us through all the available options, from text replacement techniques (sIFR and Cufón), right up to WOFF support in Firefox 3.6. There are many different things to consider to get @font-face fonts to work correctly (and legally) on as many browsers as possible, but the results are lovely. Give it another year or so, and we’ll be wondering how we ever built sites without them.

• Slides: The Type We Want

The Challenges And Rewards Of Writing A 120K JavaScript Application – Robbert Broersma

Xopus make a WYSIWIG XML editor for the web. It’s a big application — more than 120,000 lines of code in about 1,150 files — and Robbert talked about some of the challenges they have in getting it to perform well. For a user, the responsiveness of the application while typing is crucial: the browser cannot be allowed to freeze up while it is processing keystrokes, and the user must not have to pause typing just to let the browser catch up with what they have done. Xopus therefore makes extensive use of event handling queues, and chunked processing with setTimeout() calls to keep everything running smoothly.

Roll Your Own Effects Framework – Thomas Fuchs

Thomas Fuchs is the author of the script.aculo.us UI framework. However, for small projects sometimes a library like that can be overkill. So Thomas has developed Émile, a tiny and utterly beautiful library that provides basic but useful animation effects: colour transitions, movement, and easing. And when I say tiny, I really mean tiny. It fits into 50 lines of JavaScript code, and minified and gzipped weighs in at less than 1K.

Because it is so small, Thomas was able to guide us through it line-by line in less than an hour. He clearly explained all the principles behind the effects that the library provides, and showed precisely how to implement them all. At each step he showed the problems that could arise, and what he did in code to circumvent them. The Emile library is a delightful piece of tight, elegant code; the presentation itself was a masterpiece of education.

• Blog Post: A 50 Lines-of-Code JavaScript Animation Framework: Say Hello to Émile
• PDF: Roll Your Own JavaScript Effects Framework

Object Oriented CSS – Nicole Sullivan

CSS is a funny old thing. I’m starting to think that of the three pillars of modern front-end web development (semantic HTML, unobtrusive JavaScript, and CSS), CSS is actually the hardest to do well.

There are many CSS frameworks available that assist in setting up things like grids and typography, and many of these advocate some kind of “modular” approach, though what exactly “modular” means can vary. But all of these approaches are very hard to scale once you get up to sites (or applications) with tens, hundreds, or even thousands of different pages, layouts, or widgets. Knowing how to create cool new CSS3 gradients, rounded corners, and transitions is one thing; making that CSS re-usable, efficient, and maintainable and for a large-scale site is a whole different ball game.

Nicole Sullivan reckons that modular CSS — in this context, taking “modular” to mean CSS specific to a visible block of content on a page — is fundamentally a mistake. I have often used exactly such a modular approach specifically to isolate modules from each other: if you want to change the styling of one module, you don’t run the risk of screwing up something else. As Nicole points out, however, this approach results in your CSS expanding linearly with the number and complexity of your modules, and does not allow for effective re-use of code, or good design consistency.

Object-oriented CSS is an approach that Nicole has been writing about for some time now. It’s a pattern for designing re-usable content modules by making better use of the CSS cascade. It’s the cascade, in conjunction with carefully structured HTML, that allows us to use object-oriented techniques like inheritance encapsulation for styling our HTML. To be honest, when I first started reading about OOCSS earlier this year I was skeptical, but I have gradually been coming round to it, and am starting to appreciate its subtlety and usefulness now. I’m not sure if it is the ultimate answer to creating effective CSS, but it is definitely an important piece of the puzzle.

• Slides: Object-oriented CSS

The Future Of Web Applications – Dion Almaer & Ben Galbraith

Poor Ben was really sick with the flu, and should have been in bed rather than on stage. But he made it all the way through to the end nonetheless, and they closed the conference with one of their classic inspirational rallies, guaranteed to leave you with a smile on your face and a spring in your step.

They talked about all the things that excite them about the future of the web, such as canvas, GPU acceleration, and web workers, which will allow us to build applications that previously would only have been possible as platform native apps. “Mobile” featured heavily, of course, as it did throughout the conference. They are firmly convinced that the future lies with “devices” — not just mobile phones, but anything outside the traditional PC form factor. And I’m pretty sure they’re right.

The biggest challenge ahead is going to be keeping “the web” as a flourishing common platform, and not allowing the device space to splinter into incompatible proprietary walled gardens. This isn’t a far-future challenge, either — it’s happening right now.

Wrap-up

Fronteers 2009 was a fantastic conference. The quality of speakers and presentations was great, and the spectrum of technologies they covered was just right. There were plenty of opportunities for chat and networking, and even the lunchtime food was pretty good. (Long queues, but that’s only to be expected.) My thanks to all the organizers and volunteers who made it happen: great work, guys. I’ll definitely be there again next year.