TPR11: Reputation Systems in Web Communities

Steve Heady 
Designer, Purdue University


The audio for this podcast can be downloaded at


Steve Heady: That was going to be my other title for this. 'Have You Googled Your Name Recently?' Because if you haven't, you probably should, because there's reputation at play in everything that we do and just about every Web application that we use.

We're going to be talking a little bit about that and probably wandering around a little bit. So if you get dizzy, it's probably from the Prezi and not from me, hopefully. Is everyone familiar with Prezi? Ever used Prezi before? Have there been any Prezis today? One Prezi? Yeah? OK, that's good to hear. OK, I'm not like a stranger. Because last year it was like, "Oh, let's bust out the Prezi," and go along some trail. I like Prezi because it reminds me of how I think. It's very scattered, but at the same time it allows you to really drill down into a concept and create a pretty cool outline.

Today we're going to be talking about Reputation Systems in Web Communities. This is a really exciting topic for me because there's not really a whole lot being said about it. It's something that's out there and everything. Your boss might come to you and say, "Oh, I want to put a leaderboard on something," and you're like, "Well... why?"

 01:06

So I'm going to be trying to help you guys answer some of those questions and give you an idea of the types of reputation patterns that are available, the models that you can use to apply those patterns, and then go into a few examples without getting too technical on the TPR track.

So, yeah, my name is Steve Heady. I am off. And my handle is 'heady', if you want to hit me up on Twitter with any questions if we don't get to them, because I may use the full time. And then Confetti, that's what I spam with.

So yeah, let's dive in. OK. So what are we going to be talking about today? Really, what is reputation, why we're using it, what are the building blocks of it, like the core components, and then if you want to create a reputation system, what are some of the steps that you'll want to take in order to do so.

So, really, what is reputation? It's essentially like a contextual value judgment that you make about an action or a history of past actions. And that doesn't make a whole lot of sense, so like any designer, I want to give you something visual, and we'll just dive right in through our guy Ted up here.

 02:09

So Ted just joined my community. Good guy. He got a star for that. He's like a Level 1 user. We'll just think of him in that way. I'm looking at the 10 points on my end. He's not seeing that. He just signed up for my site, and what do you know, he's actually doing something. So good for me.

He posed a photo from the pub crawl, answers a question that was posted, maybe writes a review about this venue, comments on an article, uploads a late-night video from his iPhone. He does all this stuff in my one community because I just make him like that. And there you go. I've increased the points on my end, and he's now a Level 2 user. Way to go, Ted. You've bumped up in rankings.

But it doesn't end there because that would be very Web 1.0. We need some community interaction. We want people to interact with Ted and then say, "Yeah, Ted, that was a great review you wrote last night and I love your comment, but that obscene bullriding was flagged inappropriate." Minus 10 on that. So he bumped up a point, he's still Level 2. No worries, Ted. You're still a good guy. We appreciate your contribution to our community so we're going to keep you around.

 03:12

So what am I saying here? What is reputation? I mean, it gets tossed around as being like, 'Well, what is my reputation as a designer?' 'What's my reputation as an intramural football player?' It's very contextual when you talk about it.

But when it's used online, we're normally referring to it in the way that people make claims about things. So I'm saying something about something. I'll skip ahead; I'll come back to that. But if I favorited a photo or I Digg an article, I'm making reputation statements about those entities. I'm really just using my own stature to say something about some thing.

But the back side of that is that people also make statements about my statements. So that's really where this whole identity comes in. Because if it was just me anonymously rating five-star YouTube video and liking something on Facebook, that would be one thing, but it's what happens when people react to what I'm doing and how do I use that karma, which is essentially people reputation, to affect my Web application.

 04:20

At the end of the day, this is really hard to read. And it says... I mean, really what we're looking to do is reward high-quality content... I can't even read it... motivate quality contributors, and increase visitor value to our site.

So I want people that are active in my site to want to come back, to feel engaged, and I want people that have never even been to my site before to show up, and so we get some value from that because of the reputation miles that I have in place.

So, really, I think you've already got this. Yeah, Google your name. Yeah, there is a reputation statement. Well, truly it is. Google uses Page Rank, which is another form of making a claim about a site, because they need to decide what's going to be at the top of their index. How do they do that? Well, they have a number of factors. There's a bunch of sessions that talk about it. And it's everywhere.

 05:08

We really use it to make better decisions at this moment. And reputation systems help us bring structure to chaos. Our attention will never scale, so really we need to provide more value for the time that we do have visitors on our site.

Here is an application that Purdue made and uses a little bit of reputation. Has anyone heard of NeedForFeed? No one's heard of NeedForFeed? I got a couple of head nods. I don't need hands, that's fine.

But really, NeedForFeed is great because everyone in here that uses Twitter is on NeedForFeed, but they don't realize it. Because NeedForFeed is constantly scanning the HEWeb10 hashtag, and it's pulling in all the tweets and it's assigning every single tweet a reputation score, and from that score, it's calculating how many times the tweet's been retweeted, replied to, favorited, how long it's been up there, and we're putting all these metrics together in order to create a popularity metric.

 06:03

So you can go on NeedForFeed.com right now and see what the most popular tweets are taking place within HEWeb right now, or you can see overall since the conference has began. And we created this tool because we thought it would provide good value for people that don't have time to go on Tweetdeck and scan the hashtag all day.

So NeedForFeed is like leaderboards. It's like reputation experiment per se. It shows all the pictures. It really pulls in a lot of these different elements into it and much of it has to do with the reputation of individual tweets.

OK, so that's like a high-level, just to give you a primer on what the vocabulary is around this. I'm going to go into actually 'why to use it'. I was going to go into the building blocks, but I think it's important to really talk about why you even consider using reputation in the first place. Or not to, for that matter.

There's a few things that you need to take into account, the first one being, what are your goals for interaction? Are you looking to promote a specific feature of your site? Is there content that you want to have being refreshed more often? Are you looking to retain and engage users? I think these are pretty common goals. That first wheel that I showed, they're very similar to these. And then the last thing is, how old is your community?

 07:19

Now I would encourage you to, as you're listening to this, try not to think of this in the frame of your school's website, because while SEO and content and all that play major roles, there's probably something out there that's going to be done in education within the next few years that uses a really solid reputation metric that just hasn't been developed yet.

So I would encourage you to try and think outside of your normal school website and think about a community website and what are the options that... imagine all of the students in your school are tied into one community website, what are the types of things that you can provide for them.

So what's your goals? You need to think about what's going to motivate users. Are they there for a self-interest? For an interest in other people? And maybe they're interested in the content or the object at hand?

 08:06

If it's a self-interest, it's like an egotistical thing... not in a bad sense or anything, but if you're fulfilling a need to be there. So maybe you have an objective that you need to complete, like for all you people that deactivated your World of Warcraft accounts, you want to go out and kill 10 wild boars. And that's a fulfillment objective. That's a self-interest reputation model.

Recognition, being recognized by a community. Leaderboards are good for this. 'Hey, I'm Number 2 on Digg's users', blah blah blah. Or a quest for mastery. 'I'm trying to get everything that I can get out of this service. What can I do to really absorb all that?'

Interest in others, it's like altruism. If you think about it, like 'pay it forward', someone's helped me out in the past so I'm going to help someone else out now, if it's a friend, someone that you want to help out in that way. Or a crusade, or I have information that I think no one else has and I want to share that with them.

 09:05

Combining those goals with the incentives, it's an important thing you start small. You go from there, and that you're trying to rate the things, not the people that are putting those things up. So even though my guy Ted uploaded an obscene video, I'm still talking about the video, not about Ted. He might have the ramifications of that, but it's directed not at him but to the content that he's put up.

I'm going to dive into some examples, because I think examples are really important, and I'm not going to just throw up a bunch of slides because that would be a waste of your time. But some of them are going to be educational, other ones are going to be just social sites that I sure hope some of you are familiar with. One of them is Yelp.

Yelp is one of the most popular community-based restaurant review sites. And they do a really good job of recognizing their community for all the contributions that they make.

Here I have Allen's profile. I can see where the rating distribution is. He's received compliments from other users based on his reviews that he's written. I see here he's also an 'elite' user which means he's probably submitted more reviews than other members in this site.

 10:07

Yelp has been known for actually throwing parties for their elite users across the entire U.S. So if you're an elite Yelp user you can go to these free parties and just get wasted.

[Laughter]

Steve Heady: Who wouldn't want to?

He's got some friends on here. Yelp is really trying to build out like a social network around food reviews. Pretty cool.

But what if you try and copy Yelp's model? What if you have a site like Chowhound?

Chowhound was a site that was founded in the San Francisco Bay Area. Awesome site. People loved it because they can go on to Chowhound and say, "I really like Saddle Peak Lodge for their pastrami. Everything else sucked. Their service was terrible, their appetizers were horrible, it took me 20 minutes to get seated, but they have the best pastramis." It was like a 'diamond in the rough' type of site.

And if you notice here, they've added stars to all of their restaurants, like Yelp. It looks a lot like Yelp. This site has completely died. It's changed ownership a couple of times since they've rolled out this model. And part of the reason why the community has left Chowhound has been because they've gone away from their original intention, which was to spur these discussions about restaurants that are the things that you wouldn't normally expect to get out of them.

 11:20

If you love their pastrami sandwich but you hate everything else, how do you put that into a 5-star rating? Maybe if they came up with a model that's like, 'nominate your favorite food for this restaurant and give that a rating', well, that might have been a more applicable model to use as opposed to just rating the entire restaurant.

So this community is an example of one that's really taken a hit because they haven't applied the proper models for their community.

Another site that we all know and formerly love, Digg. Digg recently went through an entire change of the way that they submit content to their site. Formerly on Digg, you can promote any content, or I should say any user could submit a story, and then it got promoted to the front page from that.

 12:03

Today on Digg, all stories are submitted by publishers, so it's like an RSS reader, and they expect users to vote on stories that have been put in by outside sources. And you notice here, the top story on Digg's got 62 Diggs. I just took this last night. And here back a couple of months ago, in the top Diggs you've got 3,000 Diggs just for something that's being mentioned here. The top one's 395.

It's just incredible how changing a simple pattern can completely remove your community, and it all has to do with reputation. So where have they all gone? They've all gone to Reddit.

OK, so we have an idea of what reputation is and maybe why or why not to use it, taking our goals, what the incentives are, and then aligning that with our community. But what if we actually want to make something and put some of these patterns to use?

So let's go into building some blocks. And there's probably WordPress plugged in that will do this for you, so I can't guarantee anything original here.

 13:03

Reputation patterns. What are some of the ways that reputation is shown to users?

Well, on my first example, I had a user that had a reputation being increased through levels. So this is just a way of me taking a score on my end and converting it to something that's easy to understand for them.

Identifying labels is another common one where you get like a badge for something, or you'll receive something obscure but it gives you an indication of where you stand in the community. And points and totals really go along those same lines.

Continuing that, we have achievements and awards. Here is an example from Yelp. You get different types of compliments if someone thinks something you wrote was funny or something you wrote was insightful. Slashdot is really popular for that as well. Yahoo Sports is another good example of a site that uses reputation to show different awards.

 14:01

And then lastly is the leaderboard. The leaderboard is one of the common things that's thrown out there, but it's also the most commonly misused pattern in reputation by far because it's very competitive.

That we say a reputation board is only as good as n+1 users, meaning the n people that are on the board care about it and the one guy that's not on the board that wants to get on cares about it, but everyone else feels somewhat alienated because they're not a part of it or they don't understand the meaning of it. They just want to come to the site and contribute. They don't think it's about competing with one another to do something.

If you're going to use a leaderboard, you really have to think about what you're going to measure in the first place, like why are you using a leaderboard? What percentage of your community is going to be on the leaderboard? If your community is 50 people and you have a 10% leaderboard, it might make sense. If you want to really motivate that top 20%, good for you. If your community is huge and you have a 10% leaderboard, well, you might run into some problems. Who's going to benefit? That really falls into it. And then the type of community, I'll talk about that here in a moment.

 15:00

Foursquare, great example of a site... loves the leaderboards, loves the accumulation. You might have been going to Starbucks for a year, so you're like the 'Mayor of Starbucks'. If I just came out of Foursquare today and I wanted to catch up to you, it's basically impossible because it's just using accumulation totals. So I'm kind of screwed.

That's an older one. Oh, yeah, I saw this. People were talking about WooRank. They're using an aggregate score. And WooRank is very similar to consumer reports. If you go on to their site, they're set up almost identically once you scroll down.

OK, so we'll get into the actual types of communities. We talked a little bit about the patterns, the labels, the levels, the leaderboards, but the actual types of communities that are available out there, or that are possible, I should say...

This is called the competitive spectrum. And the reason why that is because the level of competitiveness goes up as you go across. So in a caring community like this DailyStrength community for moral support and advice, probably not going to want to have a leaderboard for most topics replied to.

 16:04

Collaborative community. Yelp. I already talked about Yelp. They have shared goals. They're all working to create a great database of popular restaurant reviews. But we want to identify members that are dedicated to this site and allow them to be featured... the trusted ones, not necessarily the ones that are having the biggest impact.

Of course the community, eBay. eBay had a long-standing issue. Everyone knows eBay for their star ratings for sellers, but that's actually one of the biggest problems because if you get one red star on eBay, that means a lot. If you get three red stars on eBay, good luck selling anything because people are going to look at that and be like, "I want to find out what the negative things are as opposed to the positive." They're not equal-weighted. If they have three negative stars, how many positives does it take to weigh that out? A hundred? A thousand? It varies.

So if you actually go to their site, you can't see any negative reviews on their seller page unless you click on that little number right there which will show you, 'OK, this person's had one negative review in the past 12 months, 245 positive, a couple neutral. I think I could trust him.'

 17:12

I'll zip through the competitive. Fantasy Sports are really common for this, and in combative community you're trying basically to knock other people off, like a ladder. So if you're on Warcraft III ladder, you want to beat your boy at RTS strategy. A combative community is great for a leaderboard because you're just trying to work your way up and increase your skill as you go.

So here's a quick run-through on some of the patterns. Here you can see some of the patterns that I've talked through. Top X is similar to a leaderboard where you're showing, say, the Top 10 or so of a given category. Here we have the number levels ranking in leaderboards. As you see, as the spectrum goes up, the use of these more competitive metrics also increases.

Application that we developed, rolled out about a year ago for classroom discussion. I think some of you have heard of it. It's called Hotseat.

 18:06

And it uses a few different metrics to help sort through thoughts. On this particular topic, there's 133 thoughts. If I was just looking at those chronologically, it would take me a long time to get through all them. And it's hard to extract a value from that, but here I'm sorting by deep thought, so it's showing me the most commented thoughts.

And I could probably make some assumptions about this. Well, this one has 20 votes, but the one after that's got zero votes and it's got six comments. So maybe that's telling me that the person said something that they didn't agree with or that there was some issue with it. I'm more interested in this one right here than I am in any of the others. And this metric that we're using is helping us sort through all of that information.

OK, so we talked about what reputation is, why do you use it. Building the blocks. You actually want to make your reputation system. So what can we do to put these pieces together and actually create something that has some tact to it?

 19:10

Content is king, but your community builds the castle. The first thing you want to think about with reputation: what are the inputs? Are the users doing explicit actions or implicit actions?

Explicit actions are claims that you make such as giving something a star, a thumbs-up, a thumbs-down. Maybe you're writing a review for it. It's something that the rest of the community is able to see. Those are all explicit claims.

Implicit claims are things that you do for yourself but allows the application to detect the value of that content. So I might favorite something that only I can see, but whoever is serving me that item that got favorited now has a little bit more information about the value of it. The same thing goes with 'send to a friend', 'add to collection' overall. These are all ways that we can implicitly gauge actions on a site.

 20:01

I wanted to show this graph because I think it's important with 5-star ratings. We always see 5... I mean, we put a lot of value into them. It's one of the first things we look at. Whenever I go to a new town, I'll pull up my Yelp application, 'restaurants nearby', choose a food category, give me the highest rating, that's where I'm going to.

And what we find in 5-star ratings is that they usually resemble a J curve, which is that there is a slightly larger amount of 1 stars and then it dips down 2, 3, 4 and then 5 is like through the roof. In fact, across Yahoo's properties, their average score was a 4.6 out of 5 on a 5-star rating.

There's only one on here that actually has something a little bit out of the norm, and that's their autos page. And the reason why there's so many 1s is from fake posts, like people posting custom cars that are Google search images. Like, 'Look at my Mazda,' and it's page 2.

The J curve is actually pretty common in ratings. And some sites have actually started backtracking from their original models and going to ones that are a little bit more linear, like Facebook has become so popular with their liking yet it's such a simple model.

 21:15

The old YouTube, I've already hinted at it. They had star ratings just as of a month or two ago. And if you go to YouTube today, they're gone, because YouTube was worst of all. YouTube's average rating was a 4.8 out of 5 on their videos.

So what's the point of having a 5-star rating if everything's a 5-star? How do you really gauge disinterest? What do they do? They leave the page. They don't leave a rating. If you're not interested in something, why do you want to give it 1 star? Maybe it's that bad, but what they found was that people were just leaving the page. That was telling them more about the videos than anything.

And they can track that. They could see, 'This is a five-minute video and the average time on this page is 45 seconds. So maybe there's something about this that we can use to affect our popularity scores.'

 22:06

YouTube now has this 'like' and 'dislike' button, and if you look at almost any video, that's still largely skewed. But at least you're getting a little better indication of how many people dislike something versus liking it.

So how are you going to output reputation? We've already talked about some of the patterns, some of the inputs that are used for rep. But, really, if you're going to put out something simplistic, then don't expect to get much from it.

You can look at Twitter and be like, 'Well, that's a reputation system'. It's only showing me how many people I'm following back, how many lists I'm on. Those are all quantitative items. They're not really meant to give me any value beyond just a quantity. So don't expect to get much from that aggregate. We can think back to the NeedForFeed thing. Try and think of ways that you can combine the data, but do it intelligently.

Choosing and testing the right reputation pattern. Don't be afraid to be wrong. It's OK. You might instill a change and have people stop using your site. It's OK to backtrack. I wish Digg would do it because literally that's going to kill the site, and it's unfortunate because they changed their pattern.

 23:15

Don't place outputs out of context. If part of your site is deeply nested and you want to promote the videos that are the content that's on there, don't take that and put that on a completely separate part of your site and not have those two things be related. Find a way to keep the context intact.

And don't ask too much for too little. Don't write reviews for reviews. There's liking of comments now on Facebook. That was a big leap for them, but really, it's like you only want to get as much value as you're putting in.

OK, so a couple of examples of reputation systems. The first one is Netflix. Netflix users. Wow! Do I need to get on? Is Comcast that bad? Yeah, it really is.

 24:03

Netflix is awesome because, in addition to giving you videos extremely fast at really good prices and putting Blockbuster out of business, Netflix has a really slick recommendation engine. And if you're interested in the actual reviews that go onto Netflix, the typical Netflix user reviews about 200 movies. The top 1%, which is a ton of people, is 5,000. So it's 5,000 videos on average, the top 1%.

You get into their elite users, there are about a hundred users that have put in 50,000 video reviews. That's like 136 years of watching movies that these people have apparently experienced. But part of the reason why we're so driven to rate and review movies on Netflix regardless of whether we've seen them has to do with their reputation system. It's their recommendation engine that they have.

 25:00

Here's something actually technical is when we get into personalized recommendations on Netflix. 

[Pause]

[Laughter]

Steve Heady: What am I going to do, throw it at them?

We start looking at their preference for categories, in addition to the actors that are in the movies and how they've rated movies that have the same actors, as well as the directors that were in the movies. Netflix is taking all these things into consideration.

Like when you give "Star Wars" a 5-star rating, it's not just "Star Wars". It's George Lucas, it's everyone else that went into that movie. They take all of that into account to build out their recommendation engine in addition to the community average.

Another one, Yahoo Answers. Yahoo Answers, if you guys are interested in this subject, I'd recommend you go to buildingreputation.com. There's a book written by Bryce Glass and Randy Farmer called "Building Reputation Systems", and a lot of my material, because seriously there's still so much that needs to be uncovered here, has come from these guys.

 26:07

They're both former employees of Yahoo where they did just about every form of testing with their users. And they've always been very social sites, but they've got a lot of solid research on this. And they actually patented their design for how they go about reporting offensive content.

So it sounds simple enough. You hit a flag, it gets removed from the site, the administrator gets a notification and they say, "Is this abusive or not?" and then they say yes or no, and then it goes back out of the site. Well, that sounds fine and all, except that Yahoo is paying $1 million a year and the average response time was 18 hours to get a reported post actually removed or approved from the site.

Say they come up with a new model for this. Score. And they're like, how about we make a threshold for abuse scores. So we'll set our threshold at 3, and then after our post gets flagged we'll increase the score, and then we'll see if it's above or below the threshold. And if it's below, then we'll mark that as a tick, and until three people flag it, because there's so many people on Yahoo Answers as it is, then we can get that post removed. So just set a three-strike rule.

 27:19

Well, this was all fine and all, except that, well, if I'm a spammer or a troll, I could set up three fake accounts in no time. Plus, it has nothing to do with the person that's reporting it. There might be some spammer, troll that's reporting my content, it has nothing to do with the reputation of the person posting the content, and it has no past history of their actions within the site.

So here's a little bit bigger view of what they actually ended up doing. And I won't get too far into detail, but essentially what they did was when someone flagged a post on Yahoo Answers, it would go through this system that would check and see the past history of that user within the site to see whether or not they had any posts flagged in the past, what their overall reputation was within the community, the person that was flagging them. 

 28:14

If, in fact, it passed all of those credentials, the user receives something. It would say, "You've been flagged for removal from the site for posting this one post. Do you wish to appeal this or do you wish to accept it?" If you accept it, you'll only lose 10 points from your reputation. If you appeal it, you'll risk losing up to 50 points.

By instilling this model, what they found was that from going to $1 million a year and 18 hours on average turnaround time, it went down to $10,000 a year. This is in Customer Support Help. And it had a 10% margin of error before, it went down to less than a tenth of a percent afterwards. So it went from 18 hours to less than an hour.

And what they found was that actually the most abusive content was removed so quickly that you couldn't even refresh the page and still see the post on it. That's how many people were trafficking the site.

 29:09

OK, so tying it together. Choose the reputation patterns that match your community. Use inputs that promote your goals. Be careful with karma. I've talked a lot about rating content today, but when you talk about actually rating people outside of a highly competitive spectrum, it's very, very risky.

Don't let reputation stagnate, decay. Like NeedForFeed, it's great for chronological content for things that are ordered over a series of time. Normally, you want reputation to decay so that if someone doesn't come into your site for a year, they're not still the top user. And lastly, being wrong is better than being stubborn.

This last one is Mixable. It's a social application that we developed. It's kind of similar to Facebook except it ties in all of your courses. So think of 'Blackboard meets Facebook', maybe. Maybe that's a better way to think about it.

 30:03

Because it knows all the classes that you're in, and you can immediately connect to all these classes, and then any of your other classmates can also connect to you. So instead of having to friend people and create a whole new social network, you're immediately friends with all of these people on Mixable. And it's for every Purdue student, so anyone with a Purdue alias can log in and check it out.

But we have a couple of different metrics on here. We have liking on our posts. We also have the commenting. And we have a way for us to sort out posts by being 'most recent', 'most popular'... I can't even recall some of the other ones. We have like five different ways of filtering out content.

It really dives into some of the ways... I think down the line, when we get into actual user contributions, we can start looking at what users have done in the past and then seeing how to weigh their content based upon that. So you can start marking the people that are most influential in a given class and giving them some precedence so they can be awarded for their contributions.

 31:03

All right. So I've gone over the reputation, why and what it is, why use it, the building blocks, as well as creating a reputation system. So I think it's a pretty open discussion. There is really no bounds to it. And I'd be happy to talk about or answer any questions, any comments that you guys have.

Now this is where the Web is going. The more interaction that we can provide, the more engagement that we can provide to our users, the better off they're going to be.

Purdue had a project that was developed by our research group called HUBzero, and it was used exactly for that for researchers related to, say, nanotechnology to have a community that they can come into and share their knowledge together. And they developed a point system for that so that anytime a researcher answered a question or posted a new topic, they would get points.

And then they used those points to cash them in for, I think, T-shirts and other swag that they were putting out. But it's really taken off. In fact, there is so much demand for these hub sites, which are very similar to what you're referring to, that they can barely keep up with them.

 32:07

And part of the reason for that is they're a very active community. They have one of the most active nanotechnology communities in the United States. And it's built on a Joomla platform but it uses a lot of these gaming mechanics to help increase engagement and interaction on the site.

And that's really a struggle with any new site that ties social components into it is that they're all great in a blue-sky scenario, but what if no one comes? And what if the people that do come don't interact with the site? And that's a very tricky problem to tackle.

The way I see it is we want to lower the barriers to entry as much as possible, to make it as easy to get involved as we can. So with the NeedForFeed tool, we have reputation models, but no one has to use it. We can build them out from pre-existing data.

OK, well, that's a simple example because it doesn't require anything, but with our tools Hotseat and Mixable, what we did was we made it available both on Facebook as a Facebook application as well as allowing you to insert answers or receive answers through Twitter, so that way they can connect to those services and log into them and get the convenience of having it outside of its native application, but also get more exposure for it, too, for maybe people that aren't familiar with it.

 33:26

Someone has something posted to their Facebook wall, "Hey, I'm on Mixable. Come check it out." Well, if they're a Purdue student, hey, that might be the thing that gets them in there.

Because it's not just about retaining your audience. It's about getting the audience in the first place, getting them to come in the first place. And if you don't have a good means of doing that, then you might be in trouble.

It's one of those things where, if you go to any site that's popular now or what's important right now, there's always decay going on. But decay's one of those things that you almost never see.

 34:00

You never see a score that's ticking down by the minute. You don't want to see something that's constantly decreasing. You always want to see something that's either as current level or increasing, as a user. So when we talk about decay, we're usually talking about something that's present in the current moment.

But I can't think of a social site that doesn't take decay into account. I mean, only recently did Facebook even implement it so that the most popular showed posts from more than two days ago. For a while there, it was everything that's happening today. And that was like their decay method was, 'Hey, if someone posted this yesterday, and you weren't on yesterday, then you might not see it.' Then again, 60% of their audience is signing on everyday, so they didn't really have too much of that problem.

But Digg is the same way. Their stories, if you look at a Digg story that's at the top of the homepage and it's got a bunch of Diggs, and you come back an hour later it's not there, well, why isn't it there anymore? Is it because the other stories are doing that much better? Or is it because that story has had its spot in the limelight and now it's quickly decaying and moving off the page?

 35:03

And you can actually do decay based off of user activity. So if someone comes into your site and they see everything that's hot right now and you can just say that they've been on there for a couple of minutes, when they come back, you might use that as a decay method to bump all those posts down and show other content for them.

I think decay is one of the most interesting parts of reputation because it allows the freshness of the content to really come to life for the user. They don't have to do anything except revisit your site and you're able to tell, 'Hey, they've seen this, they've seen that. Bump this off, bump that off, and bump other things.' They'll go up because of that.

Audience 1: [35:43 Unintelligible]

Steve Heady: Yeah, the turnover of content is very important. If you're promoting blog posts that you're writing and you want to recognize maybe the most popular posts, well, depending on how many blog writers you have, you might do that for the past month before you start dropping those off.

 36:04

Or maybe you just set the model up to be that way in the first place as you're showing it by the week, by the month, by all-time. That's like user-implied decay. That's like something where they can specify the time range and not have to deal with the system doing it for them.

Decay's one of those hidden factors, like karma, which isn't showing a lot but it's in every site that you're using.

Any others? Go ahead.

Audience 2: Are you basing your design decisions on these... together on your own research or existing research? Any tests or designs to see if they work?

Steve Heady: Well, as far as the research area goes, this is a very new topic. These models have been around for a while, but not a lot of people are simulating it all and putting all this together. That's why I was really excited to talk about this today.

And the site I mentioned earlier, Building Reputation, those guys are at the forefront right now of this area. The book that they wrote, I think it just got published in May of this year. So it's just getting started. There's nothing else out there for it.

 37:12

But if you go to that website, they have a list of about 50 different research articles and links that they have put together, so I have essentially gone through everything that's out there and looked through all the examples and everything that's available.

In terms of our applications, it's one of those things. It's not being too stubborn, being willing to test things out and see if they work, and adjusting it accordingly.

I think Hotseat was a great example of an app that, from the get-go, it was really important for us to make the 'vote' button the most prominent item on the page... that it's really raised up, it's got this geometric shape to it... because we wanted users to interact with that. That's the most important action on Hotseat is hitting the 'vote' button. So part of it is influenced by the design decision in addition to making the right application choice.

 38:04

But it definitely is a balance that you can strike, but there is a design aspect to it that can also play into it as well. Good question.

All right. Thank you all for coming. I appreciate your time.

[Applause]