Nielson / Comscore traffic numbers vs. reality
If it wasn’t clear by now, I find discussing traffic numbers pretty passe and gauche (always have). Yes, Engadget is at or near the top of the heap in tech news — but as an editor, to me, that’s not really what’s important. The numbers are an effect — not a cause — of reputation, engagement, and doing as good a job as you possibly can.
Anyhow, Saul Hansell wrote up a piece on the high-flying performance of Engadget and other blogs at AOL — the only problem is that he used Nielson numbers to back it up. For those not in the know, professional traffic metrics business like Nielson and Comscore (and to a lesser extent Alexa, Compete, etc.) make use of panels and cross-sections to estimate traffic and usage relative to the rest of the internet. Unfortunately these panel-based metrics are not only notoriously inaccurate, but they’re also frequently passed off as cold, hard numbers (as in the case of the NY Times piece, as well as a follow-up bit by Silicon Alley Reporter).
Just how far off are the numbers? As it turns out, somewhere in the range of 6x! In February of this year, over 12 million unique readers visited Engadget (not including RSS users or Engadget Chinese, Japanese, and Spanish), and in January it was over 14m. (January is always tech news’s biggest month of the year, thanks to CES and Macworld.) As usual, I’m not going to get into page views, RSS readers, and all the other methods of measurement, but the bottom line is that — specific numbers aside — in the last year and a half we did, in fact, add more readers than any time in our history, and we’re continuing on a rate of acceleration that tends to astound even me!
P.S. -And no, Saul did not get in touch to fact-check this story. He also did not update his post despite an email I sent providing him the more accurate numbers. Go figure!
Update: Saul got in touch, turns out he didn’t receive my email. His take on the numbers is in comments. Thanks for clarifying, Saul!
Co-founder of 
Hey Ryan:
I saw your comment on Bits and this related post. A few thoughts.
I’m not insensitive to the discrepancies between media rating agencies and server logs. I’ve been writing about this for a decade. Note that the 12 million user figure you cite seems to be worldwide, versus the U.S. home and office (not school, wi-fi and public terminal) estimate for Nielson. Also, server measurements have their own problems.
Of course, those differences aren’t the whole story. These ratings do have significant flaws. So do the Nielsen TV ratings and the Arbitron radio ratings, and, as we’ve seen, election exit polls. I’ve come to the conclusion that I serve my readers by quoting from Nielsen and comscore numbers. (I choose not to use Compete or Hitwise because I don’t think their methods are up to snuff. Some others here at the Times will use their numbers.)
Why? Because I think their estimates are based on a sound, internally consistent method. So it seems likely that the relative rankings have meaning. If Engadget is growing faster than Gizmodo (or vice versa) this method should, with some error, pick it up. Clearly the point of my post–that AOL dominates the list of top blogs–seems sound. I certainly didn’t draw any meaning out of 2.1 million. I was a bit impressed at 80 percent growth.
Reasonable people may disagree. But this is how I’ve approached this consistently over a decade.
Also, If you sent me an e-mail, other than the comment, I didn’t see it. I wouldn’t ignore you deliberately. Feel free to write or call any time.
Very best
Saul
I knew something was up when the article placed Gadling in front of Engadget. Not that I’m complaining or anything. ;)