It's usually leaders in the industry that make good use of metrics, not followers. Reason being is that followers often try to emulate a process they have no working knowledge of, so their measurements and actionables become cosmetic. Case in point, in Baltimore, part of the Compstat system measured health of neighborhoods by counting how many basketball courts had hoop nets. It was a pretty good way to tell what was neglected and where to turn attention to.
Within years, people tasked with making actual improvements started gaming the system and driving around and installing basketball hoop nets in community courts. Let that sink in for a minute.
So yeah, I agree with you wholeheartedly. Even having all the right information in the universe, a person can still make the wrong decisions. You are right about that. If I had all the time to explain my POV, I want to tell you that metrics aren't numbers, they're patterns. Anyone looking at them as numbers is doing it wrong.
The metrics lag is just one problem with measurements I pointed out because there's not enough time and space in the universe to describe them all. I want to say that a higher sampling speed is always better pretty much everywhere, from SCADA systems in the power industry switching to synchrophasors, to news agencies observing metrics patterns on a daily basis, to cameras recording more frames.
Here's why- there are some things you can only tell by observing metrics regularly, meaning daily if not hourly because sometimes it's just passive information. From it we could tell when Libya unplugged from the internet back at the onset of the Arab Spring- that was a big deal. They just went dark instantly, overnight. We could tell when Google+ launched and the Chinese government censors forgot to block ipv6 addresses to it for a few days/weeks and we got unfiltered web traffic from China for a time. We used metrics to make staff shift adjustments because our Asian audiences woke up when our guys went home and the readers didn't want yesterday's news today.
There was a particular lead poisoning story that did very well and got tons of traffic. However, the traffic sources were viral. If you took the time to exclude non-local traffic, the pageviews were pitiful. It meant that the local community took no notice of it and we failed to bring the issue to attention of officials in our own back yard.
Steering my rant toward my original point. The leaders that make good use of metrics might just be Besos and WaPo. Reason being is that while metrics are highly coveted in new media agencies, traditional news agencies have never taken it seriously enough for decision making. To do that, you have to formally tie numbers and methodology to performance evaluation, and that's a huge step. Fighting local print guilds to make that happen is a herculean task.
Just for sake of some humor and perspective, compare the clickbaity stories you dislike to a police blotter story from print. One of my favorites from Baltimore is this one, in its entirety:
"Someone entered the rear yard of a house in the 5900 block of Johnson St. on Saturday morning and removed a tomato from a tomato plant. The tomato was valued at $3, police said."
Just to point this out, traffic always drove shaping of journalism. That's exactly how beats are established and budgeted. For example, typical metro desk stories are always popular but not always overly staffed because the news pretty much writes itself. Education takes more work, but it's not that important to audience so you pick a person or two wisely to run it, maybe throw a general assignment reporter on a few stories. Investigative pieces, for those you throw your best people and if it's newsworthy - that is, in the matter of public interest - you wastefully throw resources at it to get the story out. You can't do that for every story though, so editorial meetings are harsh.
The only difference is that for traditional print, the metrics lagged behind by months to a year (these were done by circulation analysts or media analysis companies that do this for a living, with lots of phone calls and even more interpolation). With interpolation, you can imagine how you can under- or overestimate a situation. With print interpolation, it's even worse: they would ask people if they didn't read a newspaper this week, did they at least glance at one that someone held, or left on a bench? If they said yes, they got a checkmark for a partial reach. Based on these delayed and inaccurate numbers, people sometimes got or lost their jobs. More often, they got reassigned to do something else. It's always been like that.
Now, with near-realtime feedback, you better know how a piece or a series or a section front is doing and can react to it in a more timely manner and make adjustments (ex: throw more people on the project, or pull people away from it and have them try something else). For example, ebola as a news topic is not doing so well now because it's pretty much contained, so lets get that reporter onto something else.
Don't assume that this modernization change is going to be a rowship with a whip and a piper. Besos has a decent track record on enabling journalists to do their job better, seen both from the outside and the inside. And yeah, metrics can be misread, misused and misinterpreted, but that's always been a possibility.
Within years, people tasked with making actual improvements started gaming the system and driving around and installing basketball hoop nets in community courts. Let that sink in for a minute.
So yeah, I agree with you wholeheartedly. Even having all the right information in the universe, a person can still make the wrong decisions. You are right about that. If I had all the time to explain my POV, I want to tell you that metrics aren't numbers, they're patterns. Anyone looking at them as numbers is doing it wrong.
The metrics lag is just one problem with measurements I pointed out because there's not enough time and space in the universe to describe them all. I want to say that a higher sampling speed is always better pretty much everywhere, from SCADA systems in the power industry switching to synchrophasors, to news agencies observing metrics patterns on a daily basis, to cameras recording more frames.
Here's why- there are some things you can only tell by observing metrics regularly, meaning daily if not hourly because sometimes it's just passive information. From it we could tell when Libya unplugged from the internet back at the onset of the Arab Spring- that was a big deal. They just went dark instantly, overnight. We could tell when Google+ launched and the Chinese government censors forgot to block ipv6 addresses to it for a few days/weeks and we got unfiltered web traffic from China for a time. We used metrics to make staff shift adjustments because our Asian audiences woke up when our guys went home and the readers didn't want yesterday's news today.
There was a particular lead poisoning story that did very well and got tons of traffic. However, the traffic sources were viral. If you took the time to exclude non-local traffic, the pageviews were pitiful. It meant that the local community took no notice of it and we failed to bring the issue to attention of officials in our own back yard.
Steering my rant toward my original point. The leaders that make good use of metrics might just be Besos and WaPo. Reason being is that while metrics are highly coveted in new media agencies, traditional news agencies have never taken it seriously enough for decision making. To do that, you have to formally tie numbers and methodology to performance evaluation, and that's a huge step. Fighting local print guilds to make that happen is a herculean task.