Jumat, 06 Januari 2012

Hacking consensus: How we can build better arguments online


Nieman Journalism Lab


Posted: 05 Jan 2012 11:30 AM PST
In a recent New York Times column, Paul Krugman argued that we should impose a tax on financial transactions, citing the need to reduce budget deficits, the dubious value of much financial trading, and the literature on economic growth. So should we? Assuming for a moment that you’re not deeply versed in financial economics, on what basis can you evaluate this argument? You can ask yourself whether you trust Krugman. Perhaps you can call to mind other articles you’ve seen that mentioned the need to cut the deficit or questioned the value of Wall Street trading. But without independent knowledge — and with no external links — evaluating the strength of Krugman’s argument is quite difficult.
It doesn’t have to be. The Internet makes it possible for readers to research what they read more easily than ever before, provided they have both the time and the ability to filter reliable sources from unreliable ones. But why not make it even easier for them? By re-imagining the way arguments are presented, journalism can provide content that is dramatically more useful than the standard op-ed, or even than the various “debate” formats employed at places like the Times or The Economist.
To do so, publishers should experiment in three directions: acknowledging the structure of the argument in the presentation of the content; aggregating evidence for and against each claim; and providing a credible assessment of each claim’s reliability. If all this sounds elaborate, bear in mind that each of these steps is already being taken by a variety of entrepreneurial organizations and individuals.

Defining an argument

We’re all familiar with arguments, both in media and in everyday life. But it’s worth briefly reviewing what an argument actually is, as doing so can inform how we might better structure arguments online. “The basic purpose of offering an argument is to give a reason (or more than one) to support a claim that is subject to doubt, and thereby remove that doubt,” writes Douglas Walton in his book Fundamentals of Critical Argumentation. “An argument is made up of statements called premises and a conclusion. The premises give a reason (or reasons) to support the conclusion.”
So an argument can be broken up into discrete claims, unified by a structure that ties them together. But our typical conceptions of online content ignore all that. Why not design content to more easily assess each claim in an argument individually? UI designer Bret Victor is working on doing just that through a series of experiments he collectively calls “Explorable Explanations.”
Writes Victor:
A typical reading tool, such as a book or website, displays the author’s argument, and nothing else. The reader’s line of thought remains internal and invisible, vague and speculative. We form questions, but can’t answer them. We consider alternatives, but can’t explore them. We question assumptions, but can’t verify them. And so, in the end, we blindly trust, or blindly don’t, and we miss the deep understanding that comes from dialogue and exploration.
The alternative is what he calls a “reactive document” that imposes some structure onto content so that the reader can “play with the premise and assumptions of various claims, and see the consequences update immediately.”
Although Victor’s first prototype, Ten Brighter Ideas, is a list of recommendations rather than a formal argument, it gives a feel of how such a document could work. But the specific look, feel and design of his example aren’t important. The point is simply that breaking up the contents of an argument beyond the level of just a post or column makes it possible for authors, editors or the community to deeply analyze each claim individually, while not losing sight of its place in the argument’s structure.

Show me the evidence (and the conversation)

Victor’s prototype suggests a more interesting way to structure and display arguments by breaking them up into individual claims, but it doesn’t tell us anything about what sort of content should be displayed alongside each claim. To start with, each claim could be accompanied by relevant links that help the reader make sense of that claim, either by providing evidence, counterpoints, context, or even just a sense of who does and does not agree.
Each claim could be accompanied by relevant links that help the reader make sense of that claim by providing evidence, counterpoints, context, or even just a sense of who does and does not agree.
At multiple points in his column, Krugman references “the evidence,” presumably referring to parts of the economics literature that support his argument. But what is the evidence? Why can’t it be cited alongside the column? And, while we’re at it, why not link to countervailing evidence as well? For an idea of how this might work, it’s helpful to look at a crowd-sourced fact-checking experiment run by the nonprofit NewsTrust. The “TruthSquad” pilot has ended, but the content is still online. One thing that NewsTrust recognized was that rather than just being useful for comment or opinion, the crowd can be a powerful tool for sourcing claims. For each fact that TruthSquad assessed, readers were invited to submit relevant links and mark them as For, Against, or Neutral.
The links that the crowd identified in the NewsTrust experiment went beyond direct evidence, and that’s fine. It’s also interesting for the reader to see what other writers are saying, who agrees, who disagrees, etc. The point is that a curated or crowd-sourced collection of links directly relevant to a specific claim can help a reader interested in learning more to save time. And allowing space for links both for and against an assertion is much more interesting than just having the author include a single link in support of his or her claim.
Community efforts to aggregate relevant links along the lines of the TruthSquad could easily be supplemented both by editor-curators (which NewsTrust relied on) and by algorithms which, if not yet good enough to do the job on their own, can at least lessen the effort required by readers and editors. The nonprofit ProPublica is also experimenting with a more limited but promising effort to source claims in their stories. (To get a sense of the usefulness of good evidence aggregation on a really thorny topic, try this post collecting studies of the stimulus bill’s impact on the economy.)

Truth, reliability, and acceptance

While curating relevant links allows the reader to get a sense of the debate around a claim and makes it easier for him or her to learn more, making sense of evidence still takes considerable time. What if a brief assessment of the claim’s truth, reliability or acceptance were included as well? This piece is arguably the hardest of those I have described. In particular, it would require editors to abandon the view from nowhere to publish a judgment about complicated statements well beyond traditional fact-checking. And yet doing so would provide huge value to the reader and could be accomplished in a number of ways.
Imagine that as you read Krugman’s column, each claim he makes is highlighted in a shade between green and red to communicate its truth or reliability. This sort of user interface is part of the idea behind “Truth Goggles,” a master’s project by Dan Schultz, an MIT Media Lab student and Mozilla-Knight Fellow. Schultz proposes to use an algorithm to check articles against a database of claims that have previously been fact-checked by Politifact. Schultz’s layer would highlight a claim and offer an assessment (perhaps by shading the text) based on the work of the fact checkers.
The beauty of using color is the speed and ease with which the reader is able to absorb an assessment of what he or she is reading. The verdict on the statement’s truthfulness is seamlessly integrated into the original content. As Schultz describes the central problem:
The basic premise is that we, as readers, are inherently lazy… It’s hard to blame us. Just look at the amount of information flying around every which way. Who has time to think carefully about everything?
Still, the number of statements that PolitiFact has checked is relatively small, and what I’m describing requires the evaluation of messy empirical claims that stretch the limits of traditional fact-checking. So how might a publication arrive at such an assessment? In any number of ways. For starters, there’s good, old-fashioned editorial judgment. Journalists can provide assessments, so long as they resist the view from nowhere. (Since we’re rethinking the opinion pages here, why not task the editorial board with such a role?)
Publications could also rely on other experts. Rather than asking six experts to contribute to a “Room for Debate”-style forum, why not ask one to write a lead argument and the others not merely to “respond,” but to directly assess the lead author’s claims? Universities may be uniquely positioned to help in this, as some are already experimenting with polling their own experts on questions of public interest. Or what if a Quora-like commenting mechanism was included for each claim, as Dave Winer has suggested, so that readers could offer assessments, with the best ones rising to the top?
Ultimately, how to assess a claim is a process question, and a difficult one. But numerous relevant experiments exist in other formats. One new effort, Hypothes.is, is aiming to add a layer of peer review to the web, reliant in part on experts. While the project is in its early stages, its founder Dan Whaley is thinking hard about many of these same questions.

Better arguments

What I’ve described so far may seem elaborate or resource-intensive. Few publications these days have the staff and the time to experiment in these directions. But my contention is that the kind of content I am describing would be of dramatically higher value to the reader than the content currently available. And while Victor’s UI points towards a more aggressive restructuring of content, much could be done with existing tools. By breaking up an argument into discrete claims, curating evidence and relevant links, and providing credible assessments of those claims, publishers would equip readers to form opinions on merit and evidence rather than merely on trust, intuition, or bias. Aggregation sites like The Atlantic Wire may be especially well-positioned to experiment in this direction.
I have avoided a number of issues in this explanation. Notably, I have neglected to discuss counter-arguments (which I believe could be easily integrated) and haven’t discussed the tension between empirical claims and value claims (I have assumed a focus on the former). And I’ve ignored the tricky psychology surrounding bias and belief formation. Furthermore, some might cite the recent PolitiFact Lie of the Year controversy as evidence that this sort of journalism is too difficult. In my mind, that incident further illustrates the need for credible, honest referees.
Aggregation sites like The Atlantic Wire may be especially well-positioned to experiment in this direction.
Returning once more to Krugman’s argument, imagine the color of the text signaling whether his claims about financial transactions and economic growth are widely accepted. Or mousing over his point about reducing deficits to quickly see links providing background on the issue. What if it turned out that all of Krugman’s premises were assessed as compelling, but his conclusion was not? It would then be obvious that something was missing. Perhaps more interestingly, what if his conclusion was rated compelling but his claims were weak? Might he be trying to convince you of his case using popular arguments that don’t hold up, rather than the actual merits of the case? All of this would finally be apparent in such a setup.
In rethinking how we structure and assess arguments online, I’ve undoubtedly raised more questions than I’ve answered. But hopefully I’ve convinced you that better presentation of arguments online is at least possible. Not only that, but numerous hackers, designers, and journalists — and many who blur the lines between those roles — are embarking on experiments to challenge how we think about content, argument, truth, and credibility. It is in their work that the answers will be found.
Image by rhys_kiwi used under a Creative Commons license.
Posted: 05 Jan 2012 07:30 AM PST
Twenty-nine major news organizations have signed on as investors in NewsRight, a newly launched company that aims to make it easier for publishers to license and track their content on the web.
NewsRight logo
David Westin, the president of NewsRight and former head of ABC News, says news organizations are suffering even though demand for news on the web is exploding, calling it an “imperfection in the marketplace.”
“Much of that digital growth is coming to the benefit of companies who themselves are not hiring reporters, or at least not very many reporters. They are relying on content taken from websites of the traditional news providers,” Westin said.
That’s a polite way of addressing unresolved tension between traditional news organizations and the aggregators, bloggers, and scrapers — ”some of which are perfectly legitimate, some of which are perfectly outrageous, and a fair number of which lie in between and are subject to honest disagreement,” Westin said.
The details are still being worked out, but the company will provide a platform for news organizations to license and distribute clean feeds of their content to third parties. That software includes analytics to help measure the reach of the content — and find out whether it’s being ripped off. NewsRight will provide legal guidance to publishers where necessary.
BeaconA little history: Remember THE BEACON? Back in 2009, the Associated Press took a somewhat more antagonistic approach to protecting its intellectual property on the web. We reported on the AP’s plans to build AP News Registry, “a way to identify, record and track every piece of content AP makes available to its members and other paying customers.” Part of that plan was the beacon, a little bit of JavaScript embedded into the AP’s syndicated news feeds, which helped expose people who, in the AP’s view, were scraping or, well, over-aggregating, its material. The AP took a lot of flak in the journalism universe.
The beacon is still very much alight and integrated into the NewsRight platform, which AP spun off into its own separate concern. The company is tracking more than 16,000 websites that use material from almost 900 news sites in its database, Westin said, and the software is measuring more than 160 million unique readers and four billion impressions a month.
Most of the websites that auto-scrape AP news feeds without permission don’t remove the tracking code, Westin said. To hunt down those savvy enough to remove the beacon, the tracking software also scours the web for text that matches the source material and flags anything that’s a 70-percent match or stronger.
Westin, who spent years as a litigator in Washington, said NewsRight is not Righthaven, the aggressive copyright enforcer that has all but folded. “We have not been set up first and foremost as a litigation shop,” Westin said. “Now, that doesn’t mean down the road there won’t be litigation. I hope there’s not. Some people may decide to sue, and we can support that with the data we gather, the information we gather. But…those are very expensive, cumbersome, time-consuming processes.”
NewsRight’s partner news organizations include Advance Publications, A.H. Belo, Community Newspaper Holdings, Gatehouse Media, The Gazette Company, Hearst Newspapers, Journal Communications, McClatchy, MediaNews, The New York Times Co., Scripps, and The Washington Post Co. AP remains on the NewsRight board and is a minority shareholder. Westin said NewsRight is accepting new applications for news organizations and bloggers who want to syndicate their content.
Posted: 05 Jan 2012 07:15 AM PST
It’s an emerging issue of our time and place. They know too much about us, and we know too little about what they know. We do know that what they know about us is increasingly determining what they choose to give us to read. We wonder: What are we missing? And just who is making those decisions?

Today, in 2012, those questions are more pressing in our age of news deluge. We’re confronted at every turn, at every finger gesture, with more to read or view or listen to. It’s not just the web: It’s also the smartphone and especially the tablet, birthing new aggregator products — Google Currents and Yahoo Livestand have joined Flipboard, Pulse, Zite, and AOL Editions — every month. Compare for a moment the “top stories” you get on each side-by-side, and you’ll be amazed. How did they get there? Why are they so different?
Was it some checkbox I checked (or didn’t?!) at sign-in? Using Facebook to sign in seemed so easy, but how is that affecting what I get? Are all those Twitterees I followed determining my story selection? (Or maybe that’s why I’m getting so many Chinese and German stories?) Did I tell the Times to give the sports section such low priority? The questions are endless, a ball of twine we’ve spun in declaring some preferences in our profiles over the years, wound ever wider by the intended or (or un-) social curation of Facebook and Twitter, and mutliplied by the unseen but all-knowing algorithms that think they know what we really want to read, more than we do. (What if they are right? Hold that thought.)
The “theys” here aren’t just the digital behemoths. Everyone in the media business — think Netflix and The New York Times as much as Pandora and People — wants to do this simple thing better: serve their customers more of what they are likely to consume so that they’ll consume more — perhaps buying digital subscriptions, services, or goods and providing very targetable eyes for advertisers. It’s not a bad goal in and of itself, but sometimes it feels like it is being done to us, rather than for us.
Our concern, and even paranoia, is growing. Take Eli Pariser’s well-viewed (500,000 times, just on YouTube) May 2011 TED presentation on “filter bubbles,” which preceded his June-published book of the same name. In the talk, Pariser talks about the fickle faces of Facebook and Google, making “invisible algorithmic editing of the web” an issue. He tells the story of how a good progressive like himself, a founder of MoveOn.org, likes to keep in touch with conservative voices and included a number in his early Facebook pages.
He then describes how Facebook, as it watched his actual reading patterns — he tended to read his progressive friends more than his conservative ones — began surfacing the conservative posts less and less over time, leaving his main choices (others, of course, are buried deeper down in his datastream, but not easily surfaced on that all-important first screen of his consciousness) those of like-minded people. Over time, he lost the diversity he’d sought.
Citing the 57 unseen filters Google uses to personalize its results for us, Pariser notes that it’s a personalization that doesn’t even seem personalized, or easily comparable: “You can’t see how different your search results are than your friends…We’re seeing a passing of the torch from human gatekeepers to algorithmic ones.”
Pariser’s worries have been echoed by a motley crew we can call algorithmic and social skeptics. Slowly, Fear of Facebook has joined vague grumbles about Google and ruminations about Amazon’s all-knowing recommendations. Ping, we’ve got a new digital problem on our bands. Big Data — now well-advertised in every airport and every business magazine as the new business problem of the digital age to pay someone to solve — has gotten very personal. We are more than the sum of our data, we shout. And why does everyone else know more more about me that I do?
The That’s My Datamine Era has arrived.
So we see Personal.com, a capitalist solution to the uber-capitalist usage of our data. I’ve been waiting for a Personal.com (and the similar Singly.com) to come along. What’s more American than having the marketplace harness the havoc that the marketplace hath wrought? So Personal comes along with the bold-but-simple notion that we should individually decide who should see our own data, own preferences, and our own clickstreams — and be paid for the privilege of granting access (with Personal taking 10 percent of whatever bounty we take in from licensing our stuff).
It’s a big, and sensible, idea in and of itself. Skeptics believe the horse has left the barn, saying that so much data about us is already freely available out there to ad marketers as to make such personal databanks obsolete before they are born. They may be forgetting the power of politics. While the FCC, FTC, and others have flailed at the supposed excesses of digital behemoths, they’ve never figured out how to rein in those excesses. Granting consumers some rights over their own data — a Consumer Data Bill of Rights — would be a populist political issue, for either Republicans or Democrats or both. But, I digress.
I think there’s a way for us to reclaim our reading choices, and I’ll call it the News Dial-o-Matic, achievable with today’s technology.
While Personal.com gives us 121 “gem” lockers — from “Address” to “Women’s Shoes”, with data lockers for golf scores, beer lists, books, house sitters, and lock combinations along the way, we want to focus on news. News, after all, is the currency of democracy. What we read, what she reads, what they read, what I read all matter. We know we have more choice than any generation in history. In this age of plenty, how do we harness it for our own good?
Let’s make it easy, and let’s use technology to solve the problem technology has created. Let’s think of three simple news reading controls that could right the balance of choice, the social whirl and technology. We can even imagine them as three dials, nicely circular ones, that we can adjust with a flick of the finger or of the mouse, changing them at our whim, or time of day.
The three dials control the three converging factors that we’d like to to determine our news diet.

Dial #1: My Sources

This is the traditional title-by-title source list, deciding which titles from global news media to local blogs I want in my news flow.

Dial #2: My Networks

Social curation is one of the coolest ideas to come along. Why should I have to rely only on myself to find what I like (within or in addition to My Sources) when lots of people like me are seeking similar content? My Facebook friends, though, will give me a very different take than those I follow on Twitter. My Gmail contact list would provide another view entirely. In fact, as Google Circles has philosophized, “You share different things with different people. But sharing the right stuff with the right people shouldn’t be a hassle.” The My Networks dial lets me tune my reading of different topics by different social groups. In addition, today’s announced NewsRight — the AP News Registry spin-off intended to market actionable intelligence about news reading in the U.S. — could even play a role here. (More on NewsRight here and here.)

Dial #3: The Borg

The all-knowing, ever-smarter algorithm isn’t going away — and we don’t want it to. We just want to control it — dial it down sometimes. I like thinking of it in sci-fi terms, and The Borg from “Star Trek” well illustrates its potential maniacal drive. (I love the Wikipedia Borg definition: “The Borg manifest as cybernetically-enhanced humanoid drones of multiple species, organized as an interconnected collective, the decisions of which are made by a hive mind, linked by subspace radio frequencies. The Borg inhabit a vast region of space in the Delta Quadrant of the galaxy, possessing millions of vessels and having conquered thousands of systems. They operate solely toward the fulfilling of one purpose: to “add the biological and technological distinctiveness of other species to [their] own” in pursuit of their view of perfection“.) The Borg knows more about our habits than we’d like and we can use it well, but let’s have us be the ones doing the dialing up and down.
Three simple round dials. They could harness the power of our minds, our relationships, and our technologies. They could utilize the smarts of human gatekeepers and of algorithmic ones. And they would return power to where it belongs, to us.
Where are the dials? Who powers them? Facebook, the new home page of our time, would love to, but so would Google, Amazon, and Apple, among a legion of others. Personal.com would love to be that center, as it would any major news site (The New York Times, Zite-powered CNN, Yahoo News). We’ll leave that question to the marketplace.
Lastly, what are the newsonomics of the News Dial-o-Matic? As we perfect what we want to read, the data capturing it becomes even more valuable to anyone wanting to sell us stuff. Whether that gets monetized by us directly (through the emerging Personals of the world), or a mix of publishers, aggregators, or ad networks would be a next battleground. And then: What about the fourth wheel, as we dial up and down what we’re in the marketplace to buy right now? Wouldn’t that be worth a tidy sum?
di
Label: ,