Analyzing reputation systems, or a new look at claims

Randy Farmer and Bryce Glass are drafting a book on reputation systems, and a draft chapter is now online. What strikes me about the chapter is that in some ways it strongly resembles the Toulmin analysis that we sometimes assign to rhetoric and composition students in first-year courses. (I'm thinking particularly of my colleague John Ruszkiewicz's textbook Everything's an Argument.)

In a Toulman analysis, the student identifies a claim, supported by reasons, which in turn are supported by evidence. Here's an example:

CLAIM: Harvard is not the best law school for my needs.
REASON: Harvard is expensive.
EVIDENCE: Bill says that Harvard is expensive.

In this case, the evidence is ethos-based: we take it on Bill's authority that Harvard is expensive. His authority constitutes the evidence that underpins the reason. Obviously we could look at other kinds of evidence - for instance, we could undertake our own comparison - but at some point ethos underpins evidence, since we can't investigate everything in the world, even if we had the expertise and inclination to do so.

In reputation systems, of course, it's ethos all the way down. All evidence is explicitly grounded in ethos. So Farmer and Glass say that

All of these reputation statements—and many more besides—can be generalized as:

There's the claim, just as in the Toulmin analysis. But the claim is always an evaluative claim about a target, sometimes explicit, sometimes implicit. And the claim is always underpinned by explicitly ethos-based evidence, which means that we must identify a source. So, in my reading, Farmer and Glass' basic analysis of reputation systems is a special case or subset of Toulmin analysis.

An implicit reputation statement, the authors continue, might look less like an argument although it still is:

And, for completeness, here's another. This is actually an action, and is an implicit reputation statement about the quality of Harvard.

Yes - since reputation statements are built on ethos, a statement like this one - which short-circuits the reason and instead supports the claim with the simple evidence of someone's action - is a legitimate implicit statement. In isolation, such statements stand or fall on the authority of the source (Wendy). But aggregation means that the statement rests on the authority of many, many individuals (Wendy+Suji+Elton+Trudy+ ...). The individuals may have very different evaluative criteria - which is why I always look at Amazon comments, not just ratings, for instance - and arguably each endorsement can be analyzed as a separate argument. But the aggregation becomes its own argument; at some point, we start treating the aggregated endorsements like the index of a set of diversified stocks.

It would be interesting to apply this approach in a rhetoric class, I think, especially since such arguments are increasingly influential. For that reason among others, the book holds a lot of promise, especially if the authors can deliver on this promise:

But this book will attempt to propose a system that accomplishes this very thing for the social web: for the multitude of applications, communities, sites and social games that might benefit from a reputation-enriched approach, we'll take you—the site designer, developer or architect—through a process for: defining the targets (or the best reputable entities) in your system; identifying likely sources of opinion; and codifying the various claims that those sources may make.

Looking forward to seeing more of this.