Nieman Journalism Lab |
- Crowdsourcing campaign spending: What ProPublica learned from Free the Files
- As YouTube renews channels, will media companies make the cut?
- The rise of the quants, clip-and-share video, and a more diverse NPR: Some of Spark Camp’s big ideas of 2012
| Crowdsourcing campaign spending: What ProPublica learned from Free the Files Posted: 12 Dec 2012 11:50 AM PST Editor’s note: Back in September, we told you about Free the Files, a project from ProPublica that tried to harness the energies of the crowd to sift through (virtual) stacks of documents regarding political ad spending at local television stations. The financial documents existed, but in a form number-crunchers couldn’t easily parse; could ProPublica readers be drafted to make them more parseable? Crowdsourcing efforts like this one have a spotty history in journalism. For every project that gains traction among a news site’s audience, there’s another that never really took off. What distinguishes the ones that succeed from the ones that don’t? With the election now past, ProPublica’s Amanda Zamora took a look back at how Free the Files fared — $1 billion in ad spending logged, almost 1,000 volunteers working on it — and what broader lessons might be drawn for other journalistic crowdsourcing. Here’s her piece, republished from ProPublica.
Nearly 1,000 people pored over the files, logging detailed ad spending data to create a public database that otherwise wouldn’t exist. We logged as much as $1 billion in political ad buys, and a month after the election, people are still reviewing documents. So what made Free the Files work? A quick backstory: Free the Files actually began last spring as an effort to enlist volunteers to visit local TV stations and request access to the “public inspection file.” Stations had long been required to keep detailed records of political ad buys, but they were only available on paper and required actually traveling to the station. In August, the FCC ordered stations in the top 50 markets to begin posting the documents online. Finally, we would be able to access a stream of political ad data based on the files. Right? Wrong. It turns out the FCC didn’t require stations to submit the data in anything that approaches an organized, standardized format. The result was that stations sent in a jumble of difficult to search PDF files. So we decided if the FCC or stations wouldn’t organize the information, we would. Enter Free the Files 2.0. Our intention was to build an app to help translate the mishmash of files into structured data about the ad buys, ultimately letting voters sort the files by market, contract amount and candidate or political group (which isn’t possible on the FCC’s web site), and to do it with the help of volunteers. In the end, Free the Files succeeded in large part because it leveraged data and community tools toward a single goal. We’ve compiled a bit of what we’ve learned about crowdsourcing and a few ideas on how news organizations can adapt a Free the Files model for their own projects. The team who worked on Free the Files included Amanda Zamora, engagement editor; Justin Elliott, reporter; Scott Klein, news applications editor; Al Shaw, news applications developer, and Jeremy Merrill, also a news applications developer. And thanks to Daniel Victor and Blair Hickman for helping create the building blocks of the Free the Files community. Know your story, set the scope
We started Free the Files as we do with any major data project — asking ourselves what story we wanted to tell (how dark money spending was impacting the election) and what information we had available to tell it (thousands of PDF files). But turning the files into something reportable would require manual review of each document, a crowdsourcing challenge compounded by the fact that we had no idea exactly how many files we would be dealing with. Every day volunteers reviewed hundreds of files, and every day we downloaded hundreds more from the FCC web site. It was like starting a race without knowing when you’d hit the finish line. So we narrowed the scope of our project to increase our chances of participation. We decided to review files in 33 swing markets, not all 50. We also limited the data elements we asked people to help us verify, paring down to the minimum necessary to tell our story. While the documents offered many interesting details about each ad buy (target demographic, cost per TV spot, date and title of each spot, etc), we focused on four key pieces of data that would help us track spending: Who bought the ad, the agency placing the ad, the contract number and the amount of the ad buy. Could we have had people transcribe every possible piece of data? Sure. Could we have exhaustively digitized 11,000 files by election day? Probably not. What are your success metrics? Track and report themWhen a visitor arrives at Free the Files, one of the first things they see are project metrics: the number of files “freed” and the total ad spending logged by our volunteers. These were our success metrics, touted each day to keep the momentum going and spur new participation. If we didn’t know when (or if) we would complete the last file, we could certainly celebrate breaking another million in ad spending. We also encouraged volunteers individually with a personalized progress report stripped across the top of the app for every logged-in user: “You have reviewed 700 files, here’s another from Cleveland, Ohio!” In addition to these public figures, the ProPublica team tracked the number of volunteers and files processed each day (scraped from the FCC site, reviewed, freed and discarded). We also kept track of how accurate our volunteers were: In order for a file to be freed, at least two volunteers had to agree on each data point. Our analytics helped us track how many people it took to free a file (an average of 2.8), measure our pace, and estimate how many files we could expect to complete. We saw participation spike after three key events: teaming up with the Huffington Post in four markets (Denver, Detroit, Miami, and Washington, D.C.); hosting a live document review session with the The News Outlet in Youngstown, Ohio; and creating an Election Day Challenge aimed at freeing all the remaining files in Las Vegas. These mini-challenges helped break a huge project into more achievable chunks and gave our crowd something to rally around.
What we would have done differently: We weren’t sure exactly what reporting data we’d need until the project got underway, and ended up building some stuff along the way that we wished we’d had from the start. For instance, we launched with an internal daily stats email and a dashboard of key metrics to track our progress. But we ended up hand-compiling a spreadsheet of document review rates and ad data for each market so we could share weekly progress reports with our volunteers. Next time, in addition to having a dashboard with aggregate metrics, we would have started out with a simple way to share all of our progress data by day. ‘Potato chip’ design
The success of Free the Files hinged in large part on the design of our app. The easier we made it for people to review and annotate documents, the higher the participation rate, the more data we could make available to everyone. Our maxim was to make the process of reviewing documents like eating a potato chip: “Once you start, you can’t stop.” We also broke out of our normal article template and used a special design for our document review page, based on the design of our Message Machine app. The page is spared any extraneous links or bits of information that could distract a person from the immediate task. The only links are in a personal progress bar and “help” section that offers guidance on interpreting the documents. The result is an addictive, game-like experience that made it fun for people to participate. We also ensured that volunteers could review documents with one click: a big green “give me a file!” button is featured at the top of the app and on the page for each ad market. We also created a widget that partners could easily embed on their websites to get their readers freeing files with one click.
What we would have done differently: We also might have personalized the “gimme” button to retrieve files from markets based on a volunteer’s location, as detected in their IP address or Facebook profile. Make it social, make it funWe leveraged social networking in three key ways to help build participation. — We used a hashtag and customized Twitter sharing to help drive participation. Hashtags are a great way to unify conversation on Twitter around a project, and we used #FreeTheFiles for both project and personalized updates. We tweeted every progress report from ProPublica with #FreeTheFiles, but also built customized share buttons into the document review tool to enable volunteers to help us spread the word and tout their own participation. And Twitter was the leading source of traffic to the app.
— We leveraged Facebook to help power a friendly leaderboard. We injected a little friendly competition into the project by tracking the top 10 volunteers leading document reviews each week and through the entire project. The leaderboard included the faces of volunteers who logged in through Facebook (nearly half of all active participants) and gravatars for everyone who logged in with email addresses. Beyond spurring competition, the leaderboard helped us recognize and reward our top volunteers. The all-time top 10 earned t-shirts for their efforts, but more importantly, we contacted them to learn more about what drove each of them to participate. After reviewing more than 3,600 files, volunteer Sarita Nemerow Eisenstark noted that she was “amazed at the volume of money flowing. To see just a $$ figure on a chart or in an article doesn’t have the impact of turning page after page after page, many with hundreds of thousand of dollars flying out.”
— We used Facebook groups to stay in touch with volunteers. In the end, the group became a practical home for our “super users,” who posted when they had questions about the ad filings and alerted us to bugs in the document review tool. As a community manager, the group gave us a way to respond to volunteers directly without overwhelming our entire Facebook and Twitter audience. And the group proved helpful for our developers, too. Based on their feedback, we added two more buttons to the document review tool to help volunteers weed out extraneous documents. The takeaway: Create a space to gather feedback from your volunteers and be responsive to their needs. Though our Facebook group was small, its dedicated members helped make the experience better for the hundreds of people who participated.
Our Free the Files challenge for the FCCAfter 10 weeks, 16,000 files freed and up to $1 billion in ad contracts logged, we’re winding down Free the Files. ProPublica’s goal was to illuminate dark money spending in the election. Our volunteers helped us do just that by creating a searchable, sortable dataset. Thanks to them, we were able to pair ad buyers in 33 markets with our PACTrack database, allowing readers to identify dark money groups and drill deeper into overall expenditures as reported to the Federal Elections Commission. And thanks in part to our readers’ efforts, we published story after story on dark money groups. But despite all of this, we still can’t get an accurate count of the money spent. The FCC’s data is just too dirty. For example, TV stations can file multiple versions of a single contract with contradictory spending amounts — and multiple ad buys with the same contract number means radically different things to different stations. But the problem goes deeper. Different stations use wildly different contract page designs, structure deals in idiosyncratic ways, and even refer to candidates and groups differently. By 2014, all broadcast TV stations will be required to post their political ad files online. But as long as they are allowed to post the kind of material they posted this year, the data will be virtually useless. So we’re ending Free the Files 2012 with a challenge to the FCC for 2014: take the lead from our 1,000 volunteers and begin requiring stations to submit at least a subset of key data points as structured data when they submit their ad files to you online. We built an app for that, it happened to work, and we’ll even share the code with you. |
| As YouTube renews channels, will media companies make the cut? Posted: 12 Dec 2012 10:31 AM PST
Now, it’s renewal season for the channels, and YouTube is saying they expect to offer new deals to just 40 percent of their partners from the first year. While YouTube’s exact criteria for choosing which channels will be renewed is not clear, the amount of time people spend watching videos could be a deciding factor. And in terms of video views, there is a gap between the top-tier of entertainment channels on YouTube and news channels. Some of the highest watched channels on YouTube, produced by Red Bull, Warner Music Group, and the WWE, pull in between 3-7 million views in a typical week, according to rankings from Deadline. Viewership of channels from the Journal and Reuters vary quite a bit from week to week, but the Journal is often in a 500,000 to 1,000,000 view range, with Reuters between 200,000 and 400,000. (The numbers are broadly similar in YouTube rankings compiled by Ad Age.) Were Reuters or the Journal not to be renewed for the project — and there’s no reason to think they will or won’t at this point — there’d be nothing stopping them from continuing to operate their channels. But they’d be doing that without funding and other support YouTube provides as part of the agreement. Editors, producers, and executives at Reuters, the Journal, and other media companies with YouTube channels were reluctant to share their renewal status, but many said the project provided a needed boost to their video efforts and helped establish a stronger foothold on the site. “The appetite [for video] is there,” said Barclay Palmer, the executive producer overseeing online video for Reuters. “It’s just a matter of which outlets can embrace it, find it, and deliver it in the most dynamic way.” When Reuters launched its channel in January, it debuted 10 shows featuring different journalists on staff. The programs varied, but many stuck with an interview or feature format. As they’ve studied their audience and received feedback, they’ve made changes to certain shows, Palmer said. Viewers will watch longer videos, he said, but online videos can’t afford to be sluggish, he said. “You got to try and do things to pull people in and get people watching,” says Palmer. Since launch, Reuters TV has racked up 8.6 million views on YouTube. Views are roughly split in half between embedded videos and YouTube.com, Palmer told me, and the average time spent viewing videos is 2:08. The fact that Reuters and other news channels are outperformed by entertainment-focused efforts isn’t surprising, Palmer said, noting that online viewing habits will mirror consumption on TV. But Palmer thinks partnerships with platforms like YouTube can be beneficial to both sides. “If (contract renewal) doesn’t happen I think this company has been excited by creating a new business in online programming,” he said. “I think we will continue no matter what. It’s been a very good partnership.” Online video has seen nothing but growth in recent years, but news companies have sometimes had trouble taking advantage of the broader gains. Regardless of cost or prior success, the reason news outlets keep trying video is because it can command premium advertising rates and is increasingly accessible on multiple devices like phones and tablets. YouTube’s channel project is a play at reaching those new audiences by making programming that, while not exactly like TV, has production values that edge closer to what viewers expect from television. The strategy YouTube took for that was to ramp up original content by offering more than $100 million in advances to channel partners. One condition of the deals is that YouTube sells and controls ads on the videos until they recoup their investment through ad revenue, which could be up to $5 million depending on each channel’s agreement. Now with the renewals on the line, YouTube likely wants to see companies that not only have a solid audience, but a sustained method of making money. A spokeswoman for YouTube declined to comment on the channel renewal agreements. The Wall Street Journal currently splits the rights to sell ads against videos with YouTube, Dow Jones spokeswoman Sara Blask told me. Mark Fishkin, vice president of digital sales and marketing for The Wall Street Journal Digital Network, said pre-roll advertising on YouTube has been something of a success. At launch, the Journal’s channel had a full compliment of ads for automotive, luxury, and consumer goods, he said. “The nice thing is, not only does the WSJ Live audience mirror the audience of the entire network, which is affluent and influential, the folks on YouTube seem to be the same upscale, 40-somethings we tend to get here,” he said. The Journal benefits from the fact that WSJ Live was already established on other platforms, meaning there was already a team in place to sell ads. But Fishkin said being on YouTube is part of the Journal’s broad digital strategy to be everywhere. That’s why you can find WSJ Live on Hulu, Roku, Apple TV, and Xbox. The Journal is now doing more than 100 hours of original video a month, Fishkin told me, so it makes sense to try to find as many ways as possible to expose that to new audienes. While Fishkin wouldn’t talk specifically about the details of the Journal’s contract renewal with YouTube, he said they’re happy with the partnership and have seen steady increases in video views. At Vox Media, home of SB Nation, The Verge, and Polygon, the YouTube channel program has been a way to scale their video efforts at an accelerated pace, said Chad Mumm, head of Vox Studios, the video production arm of the company. Similar to Reuters, Vox used its channel deal to launch a variety of shows, with the SB Nation brand in particular. On SB Nation’s channel, you’ll find interviews, discussion shows, narrative features, team reports, and commentary. Also, like Reuters, SB Nation drew on the talent it already had, within its network of sports blogs. But Mumm said one of the biggest changes they’ve made is to produce more real-time and in-the-moment video that ties to sporting events before or after they occur. “We’ve been very happy with the storytelling we’ve been able to do,” Mumm said. “We’ve seen huge growth in this network approach.” Media companies have gone through different opinions about YouTube — at times preferring to build their own video platforms (using third-party providers like Brightcove) and at times wanting to access the reach and audience of the king of online video. The financial incentives YouTube has provided — and will continue to provide for some — change that evaluation. When I asked Vox Media CEO Jim Bankoff why his company partnered with YouTube, his answer echoed a kind of long-view thinking many in the media are taking with YouTube. “YouTube will be an important part of our mix and an important platform for us,” Bankoff said. “There’s a big audience there.” Image by Josh Hikes used under a Creative Commons license. |
| Posted: 12 Dec 2012 09:25 AM PST  Editor’s Note: What were the most interesting and provocative ideas in journalism in 2012? We’re partnering with Spark Camp to produce a special end-of-year series to ask a group of smart people — Spark Camp’s alumni — that question. (Learn more about Spark Camp at the first post in this series.) Here are thoughts from: — Jonathan Stray, longtime Nieman Lab contributor and project lead for the Overview Project, an open-source document archive analysis system for investigative journalists, a project of the Associated Press and the Knight Foundation. — Meredith Artley, managing editor and vice president of CNN Digital. Before coming to CNN, Artley was executive editor of LATimes.com and editor and director of IHT.com. She’s on the boards of the Online News Association and the Global Editors Network. — John Davidow, executive editor for digital at WBUR in Boston. He is also the recipient of a 2010 Knight News Challenge award for WBUR’s OpenCourt.us, an effort to open up courtroom proceedings to the wider public. Jonathan Stray
There were enormous advances in truth-finding techniques in the 20th century — the controlled experiment, modern sociology, Bayesian statistics, the experimental demonstration of cognitive biases. Journalists typically aren’t taught these techniques, especially the quantitative ones. As both information and misinformation become far more widely available, journalists are being called to “move up the food chain” and pass judgment on what is truth. The fact-checking movement demonstrates this. But without the right training, there will be certain topics where journalists won’t be any better at knowing what is true than anyone else. Quotes from an expert will no longer cut it — not least because the expert is often speaking publicly anyway. What Silver did was make it obvious that there was a better way of knowing than the pundits. These tools for knowing probably won’t apply to most stories, but they will apply to some. When there is a need for certain kinds of understanding, I want journalists to know of the existence and the uses of these remarkable ways of figuring something out. I want them to be able to produce state-of-the-art critiques of knowledge — and equally, to be able to understand and admit when something can’t actually be known. Just as crucially, they need to be able to explain the logic of their knowledge to the audience, and listen to criticism honestly. This isn’t to say the path forward is obvious. It’s often unclear how tools from other disciplines can be applied within the demands of news production, and there will be a balance between journalists applying the tools themselves versus curating sources that do. It’s all a brand new territory to explore. Meredith Artley
Responsive design gained more steam. There was the resurgence and triumph of the animated GIF. And lots of innovations around video, like HuffPost Live — and I’m naturally partial to CNN’s debate video experience with the live clip-and-share feature. BuzzFeed had a quite a year – they seem to be having a lot of fun over there and made great hires like Kate Aurthur and Richard Rushfield out in L.A. and Dao Nguyen as director of growth. They are an organization to watch. My hope is that we’ll see more story-form experimentation in 2013. We have all these different formats in the mix nowadays — social forms, big data viz and multimedia projects, GIFs, galleries, lists, and so on — but the default storytelling vehicle for most orgs is still a newspaper-esque article plus photo or video, maybe with some comments at the bottom. We’re going to take a renewed push at this at CNN Digital this year, and I hope others join us. And with social and mobile coming on stronger than ever, some expansive thinking and action around conversational, interactive journalism is a must. We see more than ever that stories are active, evolving, connected conversations that ripple beyond the point of origin. I’m enjoying reading the Tow Center report from Emily Bell and gang on post-industrial journalism; it contains thoughts worth contemplating over the holidays and carrying into 2013, starting with summing up the past decade in our industry as: “Everybody suddenly got a lot more freedom.” John Davidow
The most-hyped idea from 2012? The problems with the iPhone 5. I think the Saturday Night Live Tech Talk segment captured perfectly the hipster hype, hysteria, and hypocrisy. It’s definitely worth another look before year’s end. |
This fall, ProPublica set out to 
When 
There were many great threads this election year, but what jumps out to me is how stunningly social the election was, with narratives and memes exploding with every soundbite.
This year, I think that the CPB-funded initiative to | You are subscribed to email updates from Nieman Journalism Lab To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
| Google Inc., 20 West Kinzie, Chicago IL USA 60610 | |