The Metrics System – online traffic measurement services of Media Metrix Inc., Nielsen Net Ratings Inc., At Plan Inc
Hassan Fattah
Advertisers and e-businesses alike are questioning the effectiveness of online ad traffic measurement. But is there a better way?
The ritual starts about the middle of every month or fiscal quarter. From New York to Silicon Valley, printers and mail servers at Web sites and public relations firms churn out press releases citing each site’s traffic figures in a chorus of superlatives: “The Street.com makes the list of Media Metrix’s top gainers.” “Media Metrix and Nielsen NetRatings confirm Monster.com is the most powerful careers Web site around.” “IVillage garners 12 percent reach in September.”
Thanks to the metrics developed by Media Metrix, Nielsen NetRatings (which is part of VNU, Adweek’s parent company) and @Plan, the top three Web traffic measurement firms, it may seem like there’s always a good figure for Web publishers to talk up. Some numbers stress individual users, others celebrate audience “reach,” while still others highlight a site’s so-called “gross usage minutes.” And the variety of online metrics continues to grow as each site struggles to better define its model and its audience.
BUT DO METRICS MEASURE ANYTHING?
All of this hoopla is more than an exercise in chest beating. Indeed, big money-in the shape of advertising revenue and ad rates–rides on those numbers. Wall Street wants to see continued growth to support its investments. Partners and joint venturers want proof they’ve consummated with the right team. And most important, perhaps, advertisers want guidelines on where to spend money and where their ad dollars are being spent. Like the Nielsen ratings on TV, and Arbitron on radio, the various Web metrics have been an important-if not the only-indicator of a Web site’s relative health, growth and placement potential.
But the numbers seem to be telling a different story these days. After more than a year of extravagant online spending, scores of advertisers who’ve used the metrics are looking over the results and wondering what went wrong. For many publishers, Web advertising has proven less effective than it was supposed to be. In some cases it’s proved to be completely ineffective. And a vociferous band of media planners has begun asking a very fundamental question: Do these metrics mean anything?
“When we’ve used these research tools to predict what sites to buy, they just haven’t worked,” says Brian Monahan, president of Inrhythm Marketing and a co-founder of online agency Exile on Seventh. “The research tools don’t predict success … they’re measuring a site’s traffic in the aggregate, but that’s not what we buy. We buy blocks of impressions.”
SO WHO ARE ONLINE ADS TALKING TO?
Everyone’s used to the traditional measures–reach, clickthrough and overall traffic. But those say little about the advertiser’s audience, media planners argue. While Media Metrix and Nielsen NetRatings have built their businesses by hyping things like overall traffic, time spent on a given site and clickthrough rates, these things may say very little about the user’s lifestyle or their psychographics. In short, they say nothing about the people advertisers want to speak to.
“They’re trying to fit a square peg into a round hole,” says Tom Hespos, Internet strategist at Mezzina Brown & Partners. “Everyone’s been stuck to the cickthrough model, and that doesn’t work anymore.”
The dissatisfaction with site traffic measurement doesn’t stop with advertisers, either. Many beleaguered Web sites are grumbling that they’re not getting credit where it’s due. The well publicized case of Britannica.com, which saw its independently reported traffic figures come in far below traffic measured off its servers, underscored the fickle nature of Web traffic numbers. As Britannica’s managers see it, neither Media Metrix nor Nielsen NetRatings accounted for the gobs of international traffic to its site, though both measurement firms have disputed that assertion. Still, major variations in the numbers from each service–sometimes amounting to 40 percent or more–only intensify questions about the metrics’ efficacy or truthfulness.
“I don’t tend to put a lot of credence in the numbers themselves,” stresses Martin McClanan, CEO of Red Envelope Gifts Online, an online retailer and frequent online advertiser. The numbers may give short shrift to sections within a site that might attract an avid group of users, and tend to only speak to a run of site audience. “For us, the key thing they tell you is the relative comparisons,” says McClanan. “They can’t be used like the Nielsen ratings on TV.”
NO STANDARD METHODOLOGY
And that’s at the heart of the controversy. With no standard measure, planners must be especially careful when using the Web metrics. Media planners and even the research firms say planners must understand the strengths and weaknesses of the research to use it right. And at the heart of that are the techniques each firm uses. Much like Nielsen Media Research’s TV ratings, all three firms rely on polls of users to make their measurements. But the way they build their panels and cull their figures has a huge effect on the results.
“The methodology behind the numbers is critical to the numbers themselves,” says Tim Meadows, executive vp of products and services at Milpitas, Calif.-based Nielsen NetRatings. NetRatings, for instance, polls some 65,000 people and projects its findings to the 140 million adults living in the United States. “You can end up magnifying any skews in the data. Most people don’t take 10 minutes to sit down and understand that.”
Both Media Metrix and Nielsen NetRatings use software to track whether tens of thousands of panelists are on or off the Web, and if on, where they are. New York-based Media Metrix recruits its panel using random digit dialing and direct mail to find three groups–PC owners at home, PC users at work and users who have PCs both at home and at work. NetRatings, meanwhile, uses random digit dialing to find home and work users who’ve used the Web at least once in the past 30 days.
The essence of both systems is to capture everything that happens on the PC. Media Metrix claims its system works at the operating system level, recording whether the computer’s on or off, whether it’s being actively used, what application is being used, specific operations within the application (useful for measuring AOL’s members, for example) and more. NetRatings collects data on everything the user does after logging on to the Net.
Critics note that Media Metrix’s tendency to roll up traffic numbers for groups of related sites leaves many figures poorly defined, a fact that can be misleading when looking at tabulations. The company was expected to begin restructuring the way it does rollups and reclassify online destinations into several genres. But both services are also skewed toward people willing to download the software and be watched as they surf the Net. That might leave out users with completely different habits, although both firms maintain that people do not adjust their online behavior, witnessed by the high numbers of traffic to adult sites by panelists.
A BETTER, OR JUST DIFFERENT, WAY
In contrast, New York-based @Plan takes a completely different tack. A favorite of some planners, the service relies on randomly dialed direct phone surveys for its numbers, producing much richer types of results, they claim. The service allows users to compare certain demographic data, lifestyle and brand usage profiles of home, work and school users. But critics note that its reliance on recall has limitations, while higher than average panel turnover can be problematic. Most concerning to Mezzina Brown’s Hespos is that @Plan, a neutral observer, was recently acquired by DoubleClick, whose interest it is to sell advertising–an inherent conflict of interest, he says.
Even with all those considerations, things get a lot more complex when trying to combine Web users with TV viewers. @Plan, for one, is said to be well tailored to permit cross tabulation of TV and radio usage with Web usage. But some players have opted for their own metrics. To link all its media properties and link them to its TV audiences, Turner Broadcasting developed its own metric, gross usage minutes (GUM), based on Media Metrix, Nielsen NetRatings and proprietary research data.
“For us, GUM became a way to assess the Web and individual Web sites without looking at reach,” says Sherrill Mane, senior vp of marketing services at CNN. “We look at it as the total number of viewing opportunities that exist at a Web destination. Putting it in that nomenclature puts it in a framework traditional planners will understand.”
That says loads about efforts to measure reach on the Web. Indeed, reach is proving to be one of the Net’s bigger pipe dreams. “You can’t buy reach on the Web,” underscores Jim Meskauskas, chief Internet strategist at interactive media agency Mediasmith. “When you buy advertising on TV, you show one ad and it’s shown to everyone watching–that’s reach. On the Web, 10 different people might see 10 different ads in a given time.” As a result, Meskauskas figures people will eventually move away from the traditional metrics to completely different measures.
That won’t be too much sweat for firms like Media Metrix and Nielsen NetRating, which are deriving more revenues from businesses other than advertising and content. Media Metrix recently finalized its merger with New York-based Jupiter Research to form Jupiter Media Metrix, while NetRatings has its own analytics group. According to Doug MacFarland, president of Media Metrix, most of the firms’ new customers are e-businesses looking to benchmark their sites against their competitors.
“This is not purely about advertising. Some use it as nothing other than the 59th floor number–but there’s lots of different uses for these numbers,” MacFarland says. Everyone from pharmaceuticals players to auto companies have signed up for the Media Metrix service, largely to see whether their sites are having an effect. As a result, MacFarland notes, there’s no one correct way to use the metrics. “People want everything to be black and white. But there are also many people who want to know who’s number 16 on the list.”
If anything, MacFarland declares, that’s going to call for more metrics, not fewer ones. “When TV first came out, they measured it according to the number of homes. It took 30 years to get people to figure out it’s about the people watching at any given time. Data always begs for more data, and then it begs to be understood,” he says.
What’s more, at least some of the blame lies with ad agencies who may not have understood the medium or didn’t spend enough time trying to understand it. With everyone from startup dot-coms to established companies throwing money at ad agencies, and with many trying to figure out what works, it was easy to squander advertising dollars. “It’s like the age-old dilemma between print and TV,” says Nigel Hollis, executive vp at Millward Brown Interactive. “It’s so cheap to make the [banner] ads and get them out there that you don’t give it the same attention that you would TV.”
YOU SAY TO-MAH-TO…
But perhaps the metrics for success themselves are what’s wrong. Some planners continue to insist that Web advertising is about direct response, not branding. But a recent study by Millward Brown found that while clickthrough is a far from reliable measure of anything, other than direct response, a look at clickthroughs often ignores the effects of branding. The study found that at least one in five banners had a strong branding effect on surfers, while seven out of 10 leave a mark in aided awareness.
“Almost 55 percent of online ads are designed to build brand,” says Hollis. “If you use clickthrough as your metric, you might be missing a lot of that effect.”
So is this the end of metrics? Don’t count on it, all agree. Ironically, many critics underscore that Media Metrix and its competitors might actually have the kind of data in their database that ad planners need. So it’s left up to the ad planners to help the change.
“I think the online media planning industry needs to tell these people what kind of things they want to see,” says Hespos, “This is a do-or-die time for this industry, and we need to start proving success or else [advertisers] may start pulling their online advertising.”
Hassan Fattah is freelance writer based in New York.
COPYRIGHT 2000 BPI Communications, Inc.
COPYRIGHT 2000 Gale Group
