One of the new psychological research says that “Online Ratings Affect Your Judgment”. Are
you influenced by the opinions of other people -- say, in the comments sections
of websites? If your answer is no, here's another question: Are you sure?
A new study co-authored by an MIT professor suggests that
many people are, in fact, heavily influenced by the positive opinions other
people express online -- but are much less swayed by negative opinions posted
in the same venues. Certain topics, including politics, see much more of this
"herding" effect than others.
The results, published today in the journal Science, detail
a five-month experiment conducted on a major news-aggregation web site. The
research group systematically altered the favorability ratings given to certain
comments on the site, to see how perceptions of favorability affected people's
judgment about those comments. They found that comments whose ratings were
manipulated in a favorable direction saw their popularity snowball, receiving a
25 percent higher average rating from other site users.
"This
herding behavior happens systematically on positive signals of quality and
ratings," says Sinan Aral, an associate professor at the
MIT Sloan School of Management, and one of three authors of the study. At the
same time, Aral notes, the results "were asymmetric between positive and
negative herding." Comments given negative ratings attracted more negative
judgments, but that increase was drowned out by what the researchers call a
"correction effect" of additional positive responses.
"People
are more skeptical of negative social influence," Aral
says. "They're more likely to 'correct' a negative vote and give it a
positive vote."
While this phenomenon of social positivity sounds pleasant
enough on the surface, Aral warns that there are pitfalls to it, such as the
manipulation of online ratings by some political operatives, marketers or
anyone who stands to profit by creating an exaggerated appearance of
popularity.
"These positive ratings also represent bias and
inflation," Aral says. "The housing bubble was a spread of
positivity, but when it burst, some people lost their savings and their houses
went underwater. Stock bubbles represent a positive herding, and they can be
dramatically bad in the wrong context."
Still, the experiment also revealed topical limitations in
herding: Stories under the rubrics of "politics," "culture and
society" and "business" generated positive herding, but stories
posted under the topics of "economics," "IT,"
"fun" and "general news" did not.
More
wisdom about crowds
In turn, Aral suggests, we should be as analytical as
possible when it comes to harnessing collective judgments.
"We have to be careful about the design and analysis of
systems that try to aggregate the wisdom of crowds," Aral says.
The research was conducted by Lev Muchnik of the Hebrew
University of Jerusalem; Sean Taylor of New York University; and Aral, who
joined MIT this summer.
The experiment was conducted on a news-aggregation site
whose identity the researchers cannot disclose for legal reasons, although Aral
allows that it operates along the lines of popular sites such as Reddit. Over
the five-month period, the researchers randomly manipulated the ratings given
to 101,281 comments to the site. In this way, they could see how readers
evaluated the same comments when those comments were given different ratings.
This approach was necessary, Aral points out, because in
most circumstances, "It's hard to distinguish the effect of high quality
from the effect of social influence bias. It could be that past positive
ratings have snowballed to create a high score, or it could just be that those
items likely to get high scores are just of high quality."
The researchers also found that comments manipulated to have
positive ratings were 32 percent more likely than untreated comments to receive
a favorable rating from the very next viewer of those comments, and 30 percent
more likely than untreated comments to obtain a very high favorable rating.
Positive
ratings for the research -- but more is needed
Other researchers who are familiar with the experiment say
it is valuable, and helps suggest additional studies that could be conducted.
Duncan Watts of Microsoft Research, who has conducted
pioneering work on social influence, believes the study is "well
done," and says it is an "interesting finding that positive
manipulations persist whereas negative ones tend to be counteracted by
'friends' stepping in to correct them."
That asymmetry would be more explicable, Watts suggests, if
the site in question had a general positive bias in its comment threads
overall. "One question is whether the positive bias is specific to this
site, or is a general feature of social feedback," he adds.
Both Watts and Matthew O. Jackson, an economist at Stanford
University, say that more research is also needed to explore why opinion
herding might be more prevalent in certain areas of life but not others.
"What
accounts for the fact that some topics exhibit herding while others
don't?" Jackson asks. "Is it related to the
subjectivity of the issue, or to emotional content of an issue? This provides a
rich agenda for both theory and empirical studies of herding going forward."
For his part, Aral agrees that the experiment "opens up
as many questions as it answers." He suggests that it would also be
valuable to have more work "explaining the psychology of the correction
effect on the negative side," as a way of understanding how collective
judgments are formed.
"Our
message is not that we should do away with crowd-based opinion
aggregation," Aral says. "Our point is that you need
solid science under the hood trying to understand exactly how these mechanisms
work in a broad population, what that means for the diffusion of opinion, and
how can we design the systems to be fair, to have less incentives for
manipulation and fraud, and be safe in aggregating opinions."
Funding for the research was provided in part by a National
Science Foundation Career Award and by a Microsoft Research faculty fellowship.
Post a Comment