NPR's Guide to Hypothesis-Driven Design for Editorial Projects(docs.google.com)
docs.google.com
NPR's Guide to Hypothesis-Driven Design for Editorial Projects
https://docs.google.com/document/d/1Jm3jc2RGUTS_pp02eVJ1BuzfyDYGtOgnow02HamHPXE/edit
4 comments
I cannot imagine the staff it takes to pull this off. Most publications are not staffed nearly to levels that would make them able to even try to follow what's outlined here. This is pipe dream, and as someone in the industry it makes me question their staffing and funding levels, actually.
What industry are you in and why do you think that the proposed approach is not achievable with a modest staff? (Or perhaps NPR has more developers and designers than I realize.)
Online publishing, technical and editorial side. 20 years. This is an incredible level of staffing. I don't know that the payoff is worth what it takes with this 38 page process.
I'm still curious why you think this is an "incredible level of staffing". Can you quantify "incredible" a bit for those of us not in the publishing industry (perhaps there are workload levels I'm not recognizing being pretty much exclusively in the software world).
38 page process to get something done. That's insanity. Most places the editors have pre-prepped art that you slap on. There's no time or staff for careful crafting.There's not even editors to look at writers pieces before they go up anymore in most cases. Writers do their own production! MAYBE, if you have a special project, you might have ONE meeting, if not two. No one has these kinds resources to have the staff to have these kinds of meeting and these kinds of processes, not even at the big guys. You'd have to be flush with staff to even allow your staff to have enough time to entertain something like this. And again, that's so much time to spend on something -- look at all these meetings, these discussions, and then look at those results. I don't see the payoff here.
Hi there, I'm the senior dev for NPR Visuals and the person who worked on White Lies (linked above, for which we did, in fact, use this guide). As a general rule, I don't comment here. But I did think this is worth responding to, because it speaks to the degree to which newsroom developers are still not visible even within the industry, much less outside.
At this time, NPR's News Apps team is about five people, for a large national newsroom. That's actually pretty small! If you look at the kinds of teams that do this kind of work at places like the LA Times (which has both a graphics department and an award-winning Data Desk) or the NYT (last I checked, they had 40+ people on their graphics desk, plus an interactive team in editorial that works on projects like elections) or the Washington Post (big and still hiring). But many newsrooms (both print and broadcast) have at least small teams or single devs that work on big projects like this, even if they're not "flush with staff." That's because having an interactive team inside editorial is a force-multiplier for digital journalism--it lets us combine writing, video, audio, animation, and data in ways that makes the entire package better, so that hopefully people are more likely to pay for it.
And if you have a team, you need a procedure for managing their resources and producing quality work, and that's all this document is. Sure, it's 38 pages long, but a few points: A) we don't follow this letter-by-letter on every project, it's a general tool; B) there's a lot of pictures and it's well-written, which boosts the page count; and C) the audience here isn't just the team, it's also the newsroom: we need to define our processes clearly so that we can interface with journalists who don't necessarily have digital experience or understand why we're making the choices we're making.
We also need to onboard people into the culture of NPR Visuals, which has been a leader in the interactive news space for many years--these guidelines help newcomers and interns understand how we work methodically and carefully to create high-quality journalism. I joined this team, in part, because of its legacy of innovative narrative work. We're not just slapping pre-selected stock photos on a first draft and calling it a day.
It may be true that a lot of small outlets don't have the equivalent of News Apps, or the Interactives team at The Seattle Times (where I previously worked). But it's just not accurate to say that these kinds of teams and procedures don't exist, or that this is a pipe dream. My whole career, and that of my fellow data journalists, shows that's not the case.
Finally, if you're interested in seeing not just how NPR works, but also how teams produce similar rich visual projects around the world, I strongly recommend checking out the archives at Source (https://source.opennews.org/), which in many ways serves as the de facto journal for digital newsroom types. I think you'd be surprised at who you see represented there, and how strong and interesting the work is.
In conclusion, subscribe to your local paper and/or NPR Member station!
At this time, NPR's News Apps team is about five people, for a large national newsroom. That's actually pretty small! If you look at the kinds of teams that do this kind of work at places like the LA Times (which has both a graphics department and an award-winning Data Desk) or the NYT (last I checked, they had 40+ people on their graphics desk, plus an interactive team in editorial that works on projects like elections) or the Washington Post (big and still hiring). But many newsrooms (both print and broadcast) have at least small teams or single devs that work on big projects like this, even if they're not "flush with staff." That's because having an interactive team inside editorial is a force-multiplier for digital journalism--it lets us combine writing, video, audio, animation, and data in ways that makes the entire package better, so that hopefully people are more likely to pay for it.
And if you have a team, you need a procedure for managing their resources and producing quality work, and that's all this document is. Sure, it's 38 pages long, but a few points: A) we don't follow this letter-by-letter on every project, it's a general tool; B) there's a lot of pictures and it's well-written, which boosts the page count; and C) the audience here isn't just the team, it's also the newsroom: we need to define our processes clearly so that we can interface with journalists who don't necessarily have digital experience or understand why we're making the choices we're making.
We also need to onboard people into the culture of NPR Visuals, which has been a leader in the interactive news space for many years--these guidelines help newcomers and interns understand how we work methodically and carefully to create high-quality journalism. I joined this team, in part, because of its legacy of innovative narrative work. We're not just slapping pre-selected stock photos on a first draft and calling it a day.
It may be true that a lot of small outlets don't have the equivalent of News Apps, or the Interactives team at The Seattle Times (where I previously worked). But it's just not accurate to say that these kinds of teams and procedures don't exist, or that this is a pipe dream. My whole career, and that of my fellow data journalists, shows that's not the case.
Finally, if you're interested in seeing not just how NPR works, but also how teams produce similar rich visual projects around the world, I strongly recommend checking out the archives at Source (https://source.opennews.org/), which in many ways serves as the de facto journal for digital newsroom types. I think you'd be surprised at who you see represented there, and how strong and interesting the work is.
In conclusion, subscribe to your local paper and/or NPR Member station!
Thanking you sharing this additional context, it's very informative and interesting. I really appreciate what teams like yours do!
What's the ad revenue picture like? Has the industry found ways to adapt to the current landscape or are subscriptions/branded content the only way to fund a media business these days?
(I know not related to NPR)
(I know not related to NPR)
Interesting idea and, at a glance, it looks pretty well thought out. It was obviously inspired by Agile/Scrum and the software development process. I like the checklists and noticed it has a couple retrospective point, which I think is a critical part of Agile development.
A couple things I'd be interested in for follow-up:
- How does it go in practice with an actual project? Any real world examples or case studies?
- Any practices introduced here that could be carried back into the scrum / software development world?
A couple things I'd be interested in for follow-up:
- How does it go in practice with an actual project? Any real world examples or case studies?
- Any practices introduced here that could be carried back into the scrum / software development world?
I saw the guide linked to a recent indepth writeup on how the NPR Visuals Team developed the project page for their White Lies investigative podcast:
http://blog.apps.npr.org/2019/05/17/white-lies.html
An excerpt:
> In keeping with our hypothesis-driven design process, we started the project by identifying likely audiences for our digital experience, the contexts by which they’d come to the site (via a promo on social, etc.) and their expectations. With that audience exercise in mind, we decided on a few primary editorial goals for the project:
> - Provide a standalone narrative for people who only encounter the story on the web (and may or may not go on to subscribe to the podcast)
> - Intrigue readers and entice them to listen to the full podcast
> - Provide multimedia material that couldn’t fit into an audio-only experience
> - Introduce newcomers to the history surrounding the murder, including Bloody Sunday and the civil rights struggle
http://blog.apps.npr.org/2019/05/17/white-lies.html
An excerpt:
> In keeping with our hypothesis-driven design process, we started the project by identifying likely audiences for our digital experience, the contexts by which they’d come to the site (via a promo on social, etc.) and their expectations. With that audience exercise in mind, we decided on a few primary editorial goals for the project:
> - Provide a standalone narrative for people who only encounter the story on the web (and may or may not go on to subscribe to the podcast)
> - Intrigue readers and entice them to listen to the full podcast
> - Provide multimedia material that couldn’t fit into an audio-only experience
> - Introduce newcomers to the history surrounding the murder, including Bloody Sunday and the civil rights struggle
One thing I noticed the document doesn't address: what if the hypothesis is wrong? Is the project abandoned? Or is that included in the eventual product/result?
And is the product in this case an editorial story/article? First sentence suggests a "storytelling project":
> Hypothesis-Driven Design (or HDD for short) is a problem solving method that helps editorial teams take a user-centered and evidence-based approach to the discovery, design, and development of new storytelling projects.
I assume that means podcast/This American Life-style stories.
And is the product in this case an editorial story/article? First sentence suggests a "storytelling project":
> Hypothesis-Driven Design (or HDD for short) is a problem solving method that helps editorial teams take a user-centered and evidence-based approach to the discovery, design, and development of new storytelling projects.
I assume that means podcast/This American Life-style stories.
Don't know the answers to your questions, though it seems the Visuals team produces work for daily stories, not just big features like special podcasts and series. They also produce a few projects that don't seem to originate from the radio/podcast side:
- What do Red and Blue America Have in Common? Craft Breweries and more: https://blog.apps.npr.org/2018/11/19/elex-18-districts.html
- Listeners' favorite albums: https://blog.apps.npr.org/2018/01/03/all-songs-considered-po...
- Visualization of redactions in the Mueller report: https://blog.apps.npr.org/2019/04/22/reading-between-redacti...
- Previously, on Arrested Development (an interactive database of AD's jokes and gags): https://apps.npr.org/arrested-development/
- Playgrounds for Everyone (a crowdsourced guide to accessible playgrounds): http://www.playgroundsforeveryone.com/
- Book Concierge (book recommendations): https://apps.npr.org/best-books-2018/
- What do Red and Blue America Have in Common? Craft Breweries and more: https://blog.apps.npr.org/2018/11/19/elex-18-districts.html
- Listeners' favorite albums: https://blog.apps.npr.org/2018/01/03/all-songs-considered-po...
- Visualization of redactions in the Mueller report: https://blog.apps.npr.org/2019/04/22/reading-between-redacti...
- Previously, on Arrested Development (an interactive database of AD's jokes and gags): https://apps.npr.org/arrested-development/
- Playgrounds for Everyone (a crowdsourced guide to accessible playgrounds): http://www.playgroundsforeveryone.com/
- Book Concierge (book recommendations): https://apps.npr.org/best-books-2018/
Not just podcasts. Follow the link provided in GP, it's a mixed-media format (audio, text, images). It seems the "hypothesis" here is about how to present the information. Hypothesize about the format, propose an experiment, see how users react, evaluate how to proceed. It's an application of Scrum project management with multiple prototypes (hypothesis -> prototype) as experiments on how to present the information. It's not about the story's hypothesis (X happened because of Y, oops that's not how the facts pan out).
I imagine a repo of “terminated stories” kind of like how clinical trials have to register their trial before knowing the results.
Probably not practical for NPR, but I’m going to try it out for my internal collection of projects.
Probably not practical for NPR, but I’m going to try it out for my internal collection of projects.
Admittedly, I don't like a lot of NPR content, but I think they do a wonderful job of storytelling. Thanks for sharing this, it's in my reading list.
The article lists (and links to!) "Lean UX" [1] and Google Ventures' Design Sprint Methodology as inspirations.
[1] "Lean UX: Applying Lean Principles to Improve User Experience" http://shop.oreilly.com/product/0636920021827.do
[2] https://www.gv.com/sprint/
"How To Write A Technical Paper" [3][4] has: (Related Work, System Model, Problem Statement), (Your Solution), (Analysis), (Simulation, Experimentation), (Conclusion)
[3] https://news.ycombinator.com/item?id=18226543
[4] https://westurner.github.io/hnlog/#story-18225197