Steve Heady: That was
going to be my other title for this. 'Have You Googled Your Name
Recently?' Because if you haven't, you probably should, because there's
reputation at play in everything that we do and just about every Web
application that we use. Today we're going to be talking about Reputation Systems in
Web Communities. This is a really exciting topic for me because there's
not really a whole lot being said about it. It's something that's out
there and everything. Your boss might come to you and say, "Oh, I want
to put a leaderboard on something," and you're like, "Well... why?" |
|
01:06 |
So I'm going to be trying to help you guys answer some of
those questions and give you an idea of the types of reputation
patterns that are available, the models that you can use to apply those
patterns, and then go into a few examples without getting too technical
on the TPR track. So, yeah, my name is Steve Heady. I am off. And my handle is
'heady', if you want to hit me up on Twitter with any questions if we
don't get to them, because I may use the full time. And then Confetti,
that's what I spam with. So yeah, let's dive in. OK. So what are we going to be talking
about today? Really, what is reputation, why we're using it, what are
the building blocks of it, like the core components, and then if you
want to create a reputation system, what are some of the steps that
you'll want to take in order to do so. So, really, what is reputation? It's essentially like a
contextual value judgment that you make about an action or a history
of past actions. And that doesn't make a whole lot of sense, so like
any designer, I want to give you something visual, and we'll just dive
right in through our guy Ted up here. |
02:09 |
So Ted just joined my community. Good guy. He got a star for
that. He's like a Level 1 user. We'll just think of him in that way.
I'm looking at the 10 points on my end. He's not seeing that. He just
signed up for my site, and what do you know, he's actually doing
something. So good for me. He posed a photo from the pub crawl, answers a question that
was posted, maybe writes a review about this venue, comments on an
article, uploads a late-night video from his iPhone. He does all this
stuff in my one community because I just make him like that. And there
you go. I've increased the points on my end, and he's now a Level 2
user. Way to go, Ted. You've bumped up in rankings. But it doesn't end there because that would be very Web 1.0.
We need some community interaction. We want people to interact with Ted
and then say, "Yeah, Ted, that was a great review you wrote last night
and I love your comment, but that obscene bullriding was flagged
inappropriate." Minus 10 on that. So he bumped up a point, he's still
Level 2. No worries, Ted. You're still a good guy. We appreciate your
contribution to our community so we're going to keep you around. |
03:12 |
So what am I saying here? What is reputation? I mean, it gets
tossed around as being like, 'Well, what is my reputation as a
designer?' 'What's my reputation as an intramural football player?'
It's very contextual when you talk about it. But when it's used online, we're normally referring to it in
the way that people make claims about things. So I'm saying something
about something. I'll skip ahead; I'll come back to that. But if I
favorited a photo or I Digg an article, I'm making reputation
statements about those entities. I'm really just using my own stature
to say something about some thing. But the back side of that is that people also make statements
about my statements. So that's really where this whole identity comes
in. Because if it was just me anonymously rating five-star YouTube
video and liking something on Facebook, that would be one thing, but
it's what happens when people react to what I'm doing and how do I use
that karma, which is essentially people reputation, to affect my Web
application. |
04:20 |
At the end of the day, this is really hard to read. And it
says... I mean, really what we're looking to do is reward high-quality
content... I can't even read it... motivate quality contributors, and
increase visitor value to our site. So I want people that are active in my site to want to come
back, to feel engaged, and I want people that have never even been to
my site before to show up, and so we get some value from that because
of the reputation miles that I have in place. So, really, I think you've already got this. Yeah, Google your name. Yeah, there is a reputation statement. Well, truly it is. Google uses Page Rank, which is another form of making a claim about a site, because they need to decide what's going to be at the top of their index. How do they do that? Well, they have a number of factors. There's a bunch of sessions that talk about it. And it's everywhere. |
05:08 |
We really use it to make better decisions at this moment. And
reputation systems help us bring structure to chaos. Our attention will
never scale, so really we need to provide more value for the time that
we do have visitors on our site. Here is an application that Purdue made and uses a little bit
of reputation. Has anyone heard of NeedForFeed? No one's heard of
NeedForFeed? I got a couple of head nods. I don't need hands, that's
fine. But really, NeedForFeed is great because everyone in here that
uses Twitter is on NeedForFeed, but they don't realize it. Because
NeedForFeed is constantly scanning the HEWeb10 hashtag, and it's
pulling in all the tweets and it's assigning every single tweet a
reputation score, and from that score, it's calculating how many times
the tweet's been retweeted, replied to, favorited, how long it's been
up there, and we're putting all these metrics together in order to
create a popularity metric. |
06:03 |
So you can go on NeedForFeed.com right now and see what the
most popular tweets are taking place within HEWeb right now, or you can
see
overall since the conference has began. And we created this tool
because we thought it would provide
good value for people that don't have time to go on Tweetdeck
and scan the hashtag all day. So NeedForFeed is like leaderboards. It's like reputation
experiment per se. It shows all the pictures. It
really pulls in a lot of these different elements into it and much of
it has to do with the reputation of individual tweets. OK, so that's like a high-level, just to give you a primer on
what
the vocabulary is around this. I'm going to go into actually 'why to
use it'. I was going to go into the building blocks, but I think it's
important to really talk about why you even consider using reputation
in the first place. Or not to, for that matter. There's a few things that you need to take into account, the first one being, what are your goals for interaction? Are you looking to promote a specific feature of your site? Is there content that you want to have being refreshed more often? Are you looking to retain and engage users? I think these are pretty common goals. That first wheel that I showed, they're very similar to these. And then the last thing is, how old is your community? |
07:19 |
Now I would encourage you to, as you're listening to this, try
not to think of this in the frame of your school's website, because
while SEO and content and all that play major roles, there's probably
something out there that's going to be done in education within the
next few years that uses a really solid reputation metric that just
hasn't been developed yet. So I would encourage you to try and think
outside of your normal school website and think about a community
website and what are the options that... imagine all of the students in
your school are tied into one community website, what are the types of
things that you can provide for them. So what's your goals? You need to think about what's going to
motivate users. Are they there for a self-interest? For an interest in
other people? And maybe they're interested in the content or the object
at
hand? |
08:06 |
If it's a self-interest, it's like an egotistical thing... not
in a bad sense or anything, but if you're fulfilling a need to be
there. So maybe you have an objective that you need to complete, like
for all you people that deactivated your World of Warcraft accounts,
you
want to go out and kill 10 wild boars. And that's a fulfillment
objective. That's a self-interest reputation model. Recognition, being recognized by a community. Leaderboards are
good for this. 'Hey, I'm Number 2 on Digg's users', blah blah blah. Or
a
quest for mastery. 'I'm trying to get everything that I can get out of
this service. What can I do to really absorb all that?' Interest in others, it's like altruism. If you think
about it, like 'pay it forward', someone's helped me out in the past so
I'm
going to help someone else out now, if it's a friend, someone that you
want
to help out in that way. Or a crusade, or I have information that I
think no one else has and I want to share that with them. |
09:05 |
Combining those goals with the incentives, it's an
important thing you start small. You go from there, and that you're
trying to rate the things, not the people that are putting those things
up. So even though my guy Ted uploaded an obscene video, I'm still
talking about the video, not about Ted. He might have the ramifications
of that, but it's directed not at him but to the content that he's put
up. I'm going to dive into some examples, because I think
examples are really important, and I'm not going to just throw up a
bunch of slides because that would be a waste of your time. But some of
them are going to be educational, other ones are going to be just
social sites that I sure hope some of you are familiar with. One of
them is Yelp. Yelp is one of the most popular
community-based restaurant review sites. And they do a really good job
of recognizing their community for all the contributions that they
make. Here I have Allen's profile. I can see where the rating
distribution is. He's received compliments from other users based on
his reviews that he's written. I see here he's also an 'elite' user
which means he's probably submitted more reviews than other members in
this site. |
10:07 |
Yelp has been known for actually throwing parties for their
elite users across the entire U.S. So if you're an elite Yelp user you
can go to these free parties and just get wasted. [Laughter] Steve Heady: Who
wouldn't want to? He's got some friends on here. Yelp is really trying to build
out like a social network around food reviews. Pretty cool. But what if you try and copy Yelp's model? What if you have a
site like Chowhound? Chowhound was a site that was founded in the San
Francisco Bay Area. Awesome site. People loved it because they can go
on to Chowhound and say, "I really like Saddle Peak Lodge for their
pastrami. Everything else sucked. Their service was terrible, their
appetizers were horrible, it took me 20 minutes to get seated, but they
have the best pastramis." It was like a 'diamond in the rough' type of
site. And if you notice here, they've added stars to all of their
restaurants, like Yelp. It looks a lot like Yelp. This site has
completely died. It's changed ownership a
couple of times since they've rolled out this model. And part of the
reason why the community has left Chowhound has been because they've
gone away from their original intention, which was to spur these
discussions about restaurants that are the things that you wouldn't
normally expect to get out of them. |
11:20 |
If you love their pastrami sandwich but you hate everything
else, how do you put that into a 5-star rating? Maybe if they came
up with a model that's like, 'nominate your favorite food for this
restaurant and give that a rating', well, that might have been a more
applicable model to use as opposed to just rating the entire restaurant. So this community is an example of one that's really taken a
hit because they haven't applied the proper models for their community. Another site that we all know and formerly love, Digg. Digg
recently went through an entire change of the way that they submit
content to their site. Formerly on Digg, you can promote any content,
or I should say any user could submit a story, and then it got promoted
to the front page from that. |
12:03 |
Today on Digg, all stories are submitted by publishers, so
it's like an RSS reader, and they expect users to vote on stories
that have been put in by outside sources. And you notice here, the top
story on Digg's got 62 Diggs. I just took this last night. And
here back a couple of months ago, in the top Diggs you've got 3,000
Diggs just for something that's being mentioned here. The top one's
395. It's just incredible how changing a simple pattern can
completely remove your community, and it all has to do with reputation.
So where have they all gone? They've all gone to Reddit. OK, so we have an idea of what reputation is and maybe why or
why
not to use it, taking our goals, what the incentives are, and then
aligning that with our community. But what if we actually want to make
something and put some of these patterns to use? |
13:03 |
Reputation patterns. What are some of the ways that
reputation is shown to users? Well, on my first example, I had a user that had a reputation
being increased through levels. So this is just a way of me taking a
score on my end and converting it to something that's easy to
understand for them. Identifying labels is another common one where you get like a
badge for something, or you'll receive something obscure but it
gives you an indication of where you stand in the community. And points
and totals really go along those same lines. Continuing that, we have achievements and awards.
Here is an example from Yelp. You get different types of compliments
if someone thinks something you wrote was funny or something you wrote
was insightful. Slashdot is really popular for that as well. Yahoo
Sports is
another good example of a site that uses reputation to show different
awards. |
14:01 |
And then lastly is the leaderboard. The leaderboard is one of
the common things that's thrown out there, but it's also the most
commonly misused pattern in reputation by far because it's very
competitive. That we say a reputation board is only as good as n+1
users, meaning the n people that are on the board care about it and
the one guy that's not on the board that wants to get on cares about
it, but everyone else feels somewhat alienated because they're not a
part of it or they don't understand the meaning of it. They just want
to come to the site and contribute. They don't think it's about
competing
with one another to do something. If you're going to use a leaderboard, you really have to think
about what you're going to measure in the first place, like why are you
using a leaderboard? What percentage of your community is going to be
on the leaderboard? If your community is 50 people and you have a 10%
leaderboard, it might make sense. If you want to really motivate that
top 20%, good for you. If your community is huge and you have a 10%
leaderboard, well, you might run into some problems. Who's going to
benefit? That really falls into it. And then
the type of community, I'll talk about that here in a moment. |
15:00 |
Foursquare, great example of a site... loves the leaderboards,
loves the accumulation. You might have been going to Starbucks for a
year, so you're like the 'Mayor of Starbucks'. If I just came out of
Foursquare today and I wanted to catch up to you, it's basically
impossible because it's just using accumulation totals. So I'm kind of
screwed. That's an older one. Oh, yeah, I saw this. People were talking
about WooRank. They're using an aggregate score. And WooRank is very
similar to consumer reports. If you go on to their site, they're set up
almost identically once you scroll down. OK, so we'll get into the actual types of communities. We
talked a little bit about the patterns, the labels, the levels,
the leaderboards, but the actual types of communities that are
available out there, or that are possible, I should say... This is called the competitive spectrum. And the reason why
that is because the level of competitiveness goes up as you go across.
So in a caring community like this DailyStrength community for moral support and advice, probably not going to want to have a
leaderboard
for most topics replied to. |
16:04 |
Collaborative community. Yelp. I already talked about Yelp.
They have shared goals. They're all working to create a great database
of popular restaurant reviews. But we want to identify members that are
dedicated to this site and allow them to be featured... the trusted
ones,
not necessarily the ones that are having the biggest impact. Of course the community, eBay. eBay had a long-standing issue.
Everyone knows eBay for their star ratings for sellers, but that's
actually one of the biggest problems because if you get one red star
on eBay, that means a lot. If you get three red stars on eBay, good
luck selling anything because people are going to look at that and be
like, "I want to find out what the negative things are as opposed to
the positive." They're not equal-weighted. If they have three negative
stars, how many positives does it take to weigh that out? A hundred? A
thousand? It varies. So if you actually go to their site, you can't see any
negative reviews on their seller page unless you click on that little
number right there which will show you, 'OK, this person's had one
negative review in the past 12 months, 245 positive, a couple neutral.
I think I could trust him.' |
17:12 |
I'll zip through the competitive. Fantasy Sports are really
common for this, and in combative community you're trying basically to
knock other people off, like a ladder. So if you're on Warcraft III
ladder, you want to beat your boy at RTS strategy. A combative
community is great for a leaderboard because you're just trying to work
your way up and increase your skill as you go. So here's a quick run-through on some of the patterns. Here
you can
see some of the patterns that I've talked through. Top X is similar to
a leaderboard where you're showing, say, the Top 10 or so of a given
category. Here we have the number levels ranking in leaderboards. As
you see, as the spectrum goes up, the use of these more competitive
metrics also increases. Application that we developed, rolled out about a year ago for
classroom discussion. I think some of you have heard of it. It's called
Hotseat. |
18:06 |
And it uses a few different metrics to help sort through
thoughts. On this particular topic, there's 133 thoughts. If I was
just looking at those chronologically, it would take me a long time to
get through all them. And it's hard to extract a value from that, but
here I'm sorting by deep thought, so it's showing me the most commented
thoughts. And I could probably make some assumptions about this. Well,
this one has 20 votes, but the one after that's got zero votes and it's
got six comments. So maybe that's telling me that the person said
something that they didn't agree with or that there was some issue with
it. I'm more interested in this one right here than I am in any of the
others. And
this metric that we're using is helping us sort through all of that
information. OK, so we talked about what reputation is, why do you use it.
Building the blocks. You actually want to make your reputation system.
So what can we do to put these pieces together and actually create
something that has some tact to it? |
19:10 |
Content is king, but your community builds the castle. The
first thing you want to think about with reputation: what
are the inputs? Are the users doing explicit actions or implicit
actions? Explicit actions are claims that you make such as giving
something a star, a thumbs-up, a thumbs-down. Maybe you're writing a
review for it. It's something that the rest of the community is able to
see. Those are all explicit claims. Implicit claims are things that you do for yourself but allows
the application to detect the value of that content. So I might
favorite something that only I can see, but whoever is serving me that
item that got favorited now has a little bit more information about the
value of it. The same thing goes with 'send to a friend', 'add to
collection' overall. These are all ways that we can implicitly gauge
actions on a site. |
20:01 |
I wanted to show this graph because I think it's important
with 5-star ratings. We always see 5... I mean, we put a lot of
value into them. It's one of the first things we look at. Whenever
I go to a new town, I'll pull up my Yelp application, 'restaurants
nearby', choose a food category, give me the highest rating, that's
where I'm going to. And what we find in 5-star ratings is that they usually
resemble a J curve, which is that there is a slightly larger amount of
1 stars and then it dips down 2, 3, 4 and then 5 is like
through the roof. In fact, across Yahoo's properties, their average
score was a 4.6 out of 5 on a 5-star rating. There's only one on here that actually has something a little
bit out of the norm, and that's their autos page. And the reason why
there's so many 1s is from fake posts, like people posting custom cars
that are Google search images. Like, 'Look at my Mazda,' and it's page
2. The J curve is actually pretty common in ratings. And some
sites have actually started backtracking from their original models and
going to ones that are a little bit more linear, like Facebook
has become so popular with their liking yet it's such a simple model. |
21:15 |
The old YouTube, I've already hinted at it. They had
star ratings just as of a month or two ago. And if you go to YouTube
today, they're gone, because YouTube was worst of all. YouTube's
average rating was a 4.8 out of 5 on their videos. So what's the point of having a 5-star rating if everything's
a 5-star? How do you really gauge disinterest? What do they do? They
leave the page. They don't leave a rating. If you're not interested in
something, why do you want to give it 1 star? Maybe it's that bad,
but what they found was that people were just leaving the page. That
was telling them more about the videos than anything. And they can track that. They could see, 'This is a
five-minute video and the average time on this page is 45 seconds. So
maybe there's something about this that we can use to affect our
popularity scores.' |
22:06 |
YouTube now has this 'like' and 'dislike' button, and if you
look at almost any video, that's still largely skewed. But at
least you're getting a little better indication of how many people
dislike something versus liking it. So how are you going to output reputation? We've
already talked about some of the patterns, some of the inputs that are
used for rep. But, really, if you're going to put out something
simplistic, then don't expect to get much from it. You can look at Twitter and be like, 'Well, that's a
reputation system'. It's only showing me how many people I'm following
back, how many lists I'm on. Those are all quantitative items. They're
not really meant to give me any value beyond just a quantity. So don't
expect to get much from that aggregate. We can think back
to the NeedForFeed thing. Try and think of ways that you can combine
the data, but do it intelligently. Choosing and testing the right reputation pattern. Don't be afraid to be wrong. It's OK. You might instill a change and have people stop using your site. It's OK to backtrack. I wish Digg would do it because literally that's going to kill the site, and it's unfortunate because they changed their pattern. |
23:15 |
Don't place outputs out of context. If part of your
site is deeply nested and you want to promote the videos that are the
content that's on there, don't take that and put that on a completely
separate part of your site and not have those two things be related.
Find a way to keep the context intact. And don't ask too much for too little. Don't write reviews for
reviews. There's liking of comments now on Facebook. That was a big
leap for them, but really, it's like you only want to get as much value
as you're putting in. OK, so a couple of examples of reputation systems. The first
one is Netflix. Netflix users. Wow! Do I need to get on? Is Comcast
that bad? Yeah, it really is. |
24:03 |
Netflix is awesome because, in addition to giving you videos
extremely fast at really good prices and putting Blockbuster out of
business, Netflix has a really slick recommendation engine. And if
you're interested in the actual reviews that go onto Netflix, the
typical Netflix user reviews about 200 movies. The top 1%, which is a
ton of people, is 5,000. So it's 5,000 videos
on average, the top 1%. You get into their elite users, there are about a hundred
users
that have put in 50,000 video reviews. That's like 136 years of
watching movies that these people have apparently experienced. But part
of the reason why we're so driven to rate and review movies on Netflix
regardless of whether we've seen them has to do with their reputation
system. It's their recommendation engine that they have. |
25:00 |
Here's something actually technical is when we get into
personalized recommendations on Netflix. [Pause] [Laughter] Steve Heady: What am I
going to do, throw it at them? We start looking at their preference for categories, in
addition to the actors that are in the movies and how they've rated
movies that have the same actors, as well as the directors that were in
the movies. Netflix is taking all these things into consideration. Like when you give "Star Wars" a 5-star rating, it's not just
"Star Wars". It's George Lucas, it's everyone else that went into that
movie. They take all of that into account to build out their
recommendation engine in addition to the community average. Another one, Yahoo Answers. Yahoo Answers, if you guys are
interested in this subject, I'd recommend you go to
buildingreputation.com. There's a book written by Bryce Glass and Randy
Farmer called "Building Reputation Systems", and a lot of my material,
because seriously there's still so much that needs to be
uncovered here, has come from these guys. |
26:07 |
They're both former employees of Yahoo where they did just
about every form of testing with their users. And they've always been
very social sites, but they've got a lot of solid research on this. And
they actually patented their design for how they go about reporting
offensive content. So it sounds simple enough. You hit a flag, it gets removed
from the site, the administrator gets a notification and they say, "Is
this abusive or not?" and then they say yes or no, and then it goes
back
out of the site. Well, that sounds fine and all, except that Yahoo is
paying $1 million a year and the average response time was 18
hours to get a reported post actually removed or approved from the site. Say they come up with a new model for this. Score. And they're
like, how about we make a threshold for abuse scores. So we'll set our
threshold at 3, and then after our post gets flagged we'll increase the
score, and then we'll see if it's above or below the threshold. And if
it's below, then we'll mark that as a tick, and until three people
flag it, because there's so many people on Yahoo Answers as it is, then
we can get that post removed. So just set a three-strike rule. |
27:19 |
Well, this was all fine and all, except that, well, if I'm a
spammer or a troll, I could set up three fake accounts in no time.
Plus,
it has nothing to do with the person that's reporting it. There might
be some spammer, troll that's reporting my content, it has nothing to
do
with the reputation of the person posting the content, and it has no
past history of their actions within the site. So here's a little bit bigger view of what they actually ended up doing. And I won't get too far into detail, but essentially what they did was when someone flagged a post on Yahoo Answers, it would go through this system that would check and see the past history of that user within the site to see whether or not they had any posts flagged in the past, what their overall reputation was within the community, the person that was flagging them. |
28:14 |
If, in fact, it passed all of those credentials, the user
receives something. It would say, "You've been flagged for removal
from the site for posting this one post. Do you wish to appeal this or
do you wish to accept it?" If you accept it, you'll only lose 10 points
from your reputation. If you appeal it, you'll risk losing up to 50
points. By instilling this model, what they found was that from
going to $1 million a year and 18 hours on average turnaround
time, it went down to $10,000 a year. This is in Customer Support Help.
And it had a 10% margin of error before, it went down to less than a
tenth of a percent afterwards. So it went from 18 hours to less than an
hour. And what they found was that actually the most abusive content
was removed so quickly that you couldn't even refresh the page and
still see the post on it. That's how many people were trafficking the
site. |
29:09 |
OK, so tying it together. Choose the reputation patterns that
match your community. Use inputs that promote your goals. Be careful
with karma. I've talked a lot about rating content today, but when you
talk
about actually rating people outside of a highly competitive
spectrum, it's very, very risky. Don't let reputation stagnate, decay. Like
NeedForFeed, it's great for chronological content for things that are
ordered over a series of time. Normally, you want reputation to decay
so that if someone doesn't come into your site for a year, they're not
still the top user. And lastly, being wrong is better than being
stubborn. This last one is Mixable. It's a social application that we
developed. It's kind of similar to Facebook except it ties in all of
your courses. So think of 'Blackboard meets Facebook', maybe. Maybe
that's a better way to think about it. |
30:03 |
Because it knows all the classes that you're in, and you can
immediately connect to all these classes, and then any of your other
classmates can also connect to you. So instead of having to friend
people and create a whole new social network, you're immediately
friends with all of these people on Mixable. And it's for every Purdue
student, so anyone with a Purdue alias can log in and check it out. But we have a couple of different metrics on here. We have
liking on our posts. We also have the commenting. And we have a way for
us to sort out posts by being 'most recent', 'most popular'... I can't
even
recall some of the other ones. We have like five different ways of
filtering out content. It really dives into some of the ways... I think down the
line, when we get into actual user contributions, we can start looking
at what users have done in the past and then seeing how to weigh their
content based upon that. So you can start marking the people that are
most influential in a given class and giving them some
precedence
so they can be awarded for their contributions. |
31:03 |
All right. So I've gone over the reputation, why and what it
is, why use it, the building blocks, as well as creating a reputation
system. So I think it's a pretty open discussion. There is really no
bounds to it. And I'd be happy to talk about or answer any questions,
any comments that you guys have. Now this is where the Web is going. The more interaction that
we can provide, the more engagement that we can provide to our users,
the better off they're going to be. Purdue had a project that was developed by our research group
called HUBzero, and it was used exactly for that for researchers
related to, say, nanotechnology to have a community that they can come
into and share their knowledge together. And they developed a point
system for that so that anytime a researcher answered a question or
posted a new topic, they would get points. And then they used those points to cash them in for, I think,
T-shirts and other swag that they were putting out. But it's really
taken off. In fact, there is so much demand for these hub sites, which
are very similar to what you're referring to, that they can barely keep
up with them. |
32:07 |
And part of the reason for that is they're a very active
community. They have one of the most active nanotechnology communities
in the United States. And it's built on a Joomla platform but it
uses a
lot of these gaming mechanics to help increase engagement and
interaction on the site. And that's really a struggle with any new site that ties
social components into it is that they're all great in a blue-sky
scenario, but what if no one comes? And what if the people that do come
don't interact with the site? And that's a very tricky problem to
tackle. The way I see it is we want to lower the barriers to entry as
much as possible, to make it as easy to get involved as we can. So with
the NeedForFeed tool, we have reputation models, but no one has to use
it. We can build them out from pre-existing data. OK, well, that's a simple example because it doesn't require anything, but with our tools Hotseat and Mixable, what we did was we made it available both on Facebook as a Facebook application as well as allowing you to insert answers or receive answers through Twitter, so that way they can connect to those services and log into them and get the convenience of having it outside of its native application, but also get more exposure for it, too, for maybe people that aren't familiar with it. |
33:26 |
Someone has something posted to their Facebook wall, "Hey, I'm
on Mixable. Come check it out." Well, if they're a Purdue student, hey,
that might be the thing that gets them in there. Because it's not just
about retaining your audience. It's about getting the audience in the
first place, getting them to come in the first place. And if you don't
have a good means of doing that, then you might be in trouble. It's one of those things where, if you go to any site that's
popular now or what's important right now, there's always decay going
on. But decay's one of those things that you almost never see. |
34:00 |
You never see a score that's ticking down by the minute. You
don't want to see something that's constantly decreasing. You always
want to see something that's either as current level or increasing, as
a user.
So when we talk about decay, we're usually talking about something
that's present in the current moment. But I can't think of a social site that doesn't take decay
into account. I mean, only recently did Facebook even implement it so
that the most popular showed posts from more than two days ago. For a
while there, it was everything that's happening today. And that
was like their decay method was, 'Hey, if someone posted this
yesterday, and you weren't on yesterday, then you might not see it.'
Then again, 60% of their audience is signing on everyday, so they
didn't really have too much of that problem. But Digg is the same way. Their stories, if you look at a Digg
story that's at the top of the homepage and it's got a bunch of Diggs,
and you come back an hour later it's not there, well, why isn't it
there anymore? Is it because the other stories are doing that much
better? Or is it because that story has had its spot in the limelight
and now it's quickly decaying and moving off the page? |
35:03 |
And you can actually do decay based
off of user activity. So if someone comes into your site and
they see everything that's hot right now and you can just say that
they've been on there for a couple of minutes, when they come back, you
might use that as a decay method to bump all those posts down and show
other content for them. I think decay is one of the most interesting parts of
reputation because it allows the freshness of the content to really
come to life for the user. They don't have to do anything except
revisit your site and you're able to tell, 'Hey, they've seen this,
they've seen that. Bump this off, bump that off, and bump other
things.' They'll go up because of that. Audience 1: [35:43 Unintelligible] Steve Heady: Yeah, the
turnover of content is very important. If you're promoting blog posts
that you're writing and you want to recognize maybe the most popular
posts, well, depending on how many blog writers you have, you might do
that for the past month before you start dropping those off. |
36:04 |
Or maybe you just set the model up to be that way in the first
place as you're showing it by the week, by the month, by all-time.
That's like user-implied decay. That's like something where they can
specify the time range and not have to deal with the system doing it
for them. Decay's one of those hidden factors, like karma, which isn't
showing a lot but it's in every site that you're using. Any others? Go ahead. Audience 2: Are you
basing your design decisions on these... together on your own
research or existing research? Any tests or designs to see if they work? Steve Heady: Well, as
far as the research area goes, this is a very new topic. These
models have been around for a while, but not a lot of people are
simulating it all and putting all this together. That's why I was
really
excited to talk about this today. And the site I mentioned earlier, Building Reputation, those
guys are at the forefront right now of this area. The book that they
wrote, I think it just got published in May of this year. So it's just
getting started. There's nothing else out there for it. |
37:12 |
But if you go to that website, they have a list of about 50
different research articles and links that they have put together, so I
have essentially gone through everything that's out there and looked
through all the examples and everything that's available. In terms of our applications, it's one of those
things. It's not being too stubborn, being willing to test things
out and see if they work, and adjusting it accordingly. I think Hotseat was a great example of an app that, from the
get-go, it was really important for us to make the 'vote' button the
most
prominent item on the page... that it's really raised up, it's got this
geometric shape to it... because we wanted users to interact with that.
That's the most important action on Hotseat is hitting the 'vote'
button.
So part of it is influenced by the design decision in addition to
making the right application choice. |
38:04 |
But it definitely is a balance that you can strike, but there
is a design aspect to it that can also play into it as well. Good
question. All right. Thank you all for coming. I appreciate your time. [Applause] |