"It's hard to argue with these folks. After all, we're talking science here, right? More than three clicks to get to a piece of information? Bad! Users must scroll? Failure! Too many "graphical design elements"? Strip 'em out! If it doesn't measure up, it must go. If it doesn't measure up, it's wrong."

Actually I would have thought that came from 9 or 10 years ago. Those comments don't sound like they come from someone who hasn't worked with a usability professional on a design project. If he has, he should have interviewed more practitioners before he hired them.

Like everything in life, usabilility evaluations must be taken in the right context. For competitive evaluations, I think that usability testing is often a bad tool. A test necessarily forces you to assume that the sites have the same goal and same business model. But it is often the case that they don't, as in the GAP v. Newport News situation.

Usability expertise is most value as part of the whole design process. It's a quality that should be built into a site, and rarely comes with hard and fast instructions. I have been conducting usability research for 5 years, and I've spent more time convincing people *not* to worry about the length of pages, or the absolute number of clicks on a path than I care to remember. A number of years ago, Jarod did a good job of convincing me that they weren't important; I have seen that demonstrated to me dozens of times.

I'm not sure who in the usability world turns their nose up at analytics- I personally eat them up but most analytics are completely unusable because of poor software. Analytics can also be misleading if you do not understand the context of the numbers. For example, you might see one page was visited 50,000 times. If you don't ask around you might now ever know that this happened due to an advertisement and that the traffic would likely drop off.

One interesting point you are getting at it that a usability study is inherently limited by what you test and who tests it. These limitations are then often left off the results, so instead of saying, "For users who knew what they wanted, the site was very usable," the statement is usually, "the site was very usable." This is a big problem.

I do think your statement about usability experts turning their nose up at web analytics is incorrect. Web analytics are a valuable quantitative tool, especially when combined with A/B testing. What it won't tell you, and where the value of usability testing comes in, is WHY some designs work and others don't and how to improve them. A good "usability expert's" toolbox comprises more than usability testing.

And, oh yeah… although the use of Comic Sans make this designer cringe, it's good to be reminded that real people don't always care about what designers think is right for them. See MySpace