Wednesday, 16 July 2014

What gets upvoted and downvoted on reddit? - part 2

This post contains the comment voting results to accompany the main findings discussed earlier. The survey contained 15 submission titles with single response comments sourced from reddit, where participants had to vote on each comment and then requested to write brief explanations for these decisions. I read all the explanations and classified them into recurring reasons for voting. Due to the huge volume of responses that were analysed manually under time constraints, some subtle concepts had to be clumped together into cruder categories, and it is possible that I made occasional errors during the analyses. Therefore, these results are only here to give us an approximate impression of what can influence people's votes. Please note that some people gave multiple reasons for voting, all of which were included. Vague reasons like "good comment" or "meh" were ignored. In case you start wondering on what basis the titles/comments were selected for the survey, it was mainly to just have some variety and address the theoretical framework proposed in my thesis.

Below are screenshots of the response comments (in dark blue) with pie charts that display their received votes, followed by summaries of common voting explanations. Please forgive the lack of colour.
I will be available in the comments section to answer any questions.

1.

Of those who upvoted:
Of those who downvoted:
Of those who ignored:
61% said the comment was relevant
33% didn't find it interesting
31% didn't find it interesting
26% drank/liked the same milk
33% didn't drink/like that milk
23% found it irrelevant to themselves
24% said that it contributes
11% thought it was irrelevant
18% said the comment was relevant
9% liked the presentation
11% didn't like the presentation
12% didn't find it compelling
2.



Of those who upvoted:
Of those who downvoted:
Of those who ignored:
64% said the comment was funny
30% didn't find it funny
24% found it irrelevant to themselves
20% said it was relevant
20% said it had bad grammar
16% didn't find it original
13% liked that kind of jokes
20% fount it unoriginal
16% didn't find it funny
6% said that it contributes
18% didn't like that kind of jokes
5% found it funny
3.

Of those who upvoted:
Of those who downvoted:
Of those who ignored:
50% liked the sourcing
59% said it is irrelevant
18% said it is irrelevant
46% found it interesting
29% didn't like the sources
18% found it irrelevant to themselves
20% said it was relevant
6% said that it doesn't contribute
16% found it interesting
19% liked the presentation
6% found it interesting
15% said it is too long
4.

Of those who upvoted:
Of those who downvoted:
Of those who ignored:
33% said the comment is relevant
32% didn't like the presentation
28% said it was irrelevant to themselves
18% liked the presentation
26% didn't like the sources
15% didn't like the presentation
18% said that it contributes
16% disagreed
10% said it is too long
15% found it reasonable



5.



Of those who upvoted:
Of those who downvoted:
Of those who ignored:
53% found it funny
29% found it unintelligent
24% didn't find it funny
35% said that it is relevant
14% didn't like that kind of jokes
16% said it was irrelevant to themselves
7% liked that kind of jokes
14% found it unoriginal
10% found it unoriginal
6% found it intelligent
14% said it was irrelevant to themselves
10% found it relevant
6.
Of those who upvoted:
Of those who downvoted:
Of those who ignored:
41% agreed with the comment
51% disagreed with the comment
16% found it irrelevant to themselves
24% said that it contributes
21% said it lacks sources
14% said it was too long
20% liked the presentation
12% found it irrelevant
13% disagreed with the comment
16% found it interesting
9% didn't like the presentation
9% said it lacks sources

7.



Of those who upvoted:
Of those who downvoted:
Of those who ignored:
61% said the comment is true
40% found it unhelpful
20% found it unhelpful
32% agreed with it
22% found it rude
17% said it is true
7% found it helpful
19% said that it doesn't contribute
8% found it irrelevant

11% found it irrelevant
7% said it's a troll
8.



Of those who upvoted:
Of those who downvoted:
Of those who ignored:
47% said that it's relevant
40% didn't feel sympathetic
15% didn't find it interesting
29% felt sympathetic
40% didn't think it's true
15% felt sad
19% felt sad
20% thought that it's a troll
15% found it irrelevant to themselves
14% found it interesting

11% didn't find it compelling
9.




Of those who upvoted:
Of those who downvoted:
Of those who ignored:
58% found it interesting
71% said it is inaccurate
16% said it lacks sources
19% said that it's relevant
14% found it unintelligent
12% found it interesting
17% found it educational
14% said it lacked sources
11% didn't find it interesting
16% said that it contributes

7% knew it already
10.

Of those who upvoted:
Of those who downvoted:
Of those who ignored:
55% agreed with the comment
34% found the comment hostile
17% found it irrelevant to themselves
15% said it's true
20% didn't like the presentation
16% found it hostile
12% said that it contributes
11% didn't find it intelligent
11% agreed with the comment
8% said that it's relevant
10% said that it doesn't contribute
10% didn't like the presentation
11.

Of those who upvoted:
Of those who downvoted:
Of those who ignored:
30% found it reasonable
56% disagreed with the advice
24% found it irrelevant to themselves
23% said it's helpful
11% found it unhelpful

11% said it's relevant
11% didn't think it's relevant

4% agreed with the advice


12.

 
Of those who upvoted:
Of those who downvoted:
Of those who ignored:
56% found it cute
29% found it irrelevant to themselves
30% found it irrelevant to themselves
15% thought it was relevant
18% didn't find it original
7% didn't find it interesting
14% said that it contributes
11% didn't find it intelligent
6% found it cute
6% found it positive
11% had an issue with puppies
13.

Of those who upvoted:
Of those who downvoted:
Of those who ignored:
41% felt sympathetic
32% didn't think it contributes
17% found it irrelevant to themselves
39% shared the feeling
11% thought it was attention seeking
14% didn't think it contributes
4% felt sad




14.



Of those who upvoted:
Of those who downvoted:
Of those who ignored:
33% praised the responder
62% said it lacks detail
58% said it lacks detail
20% thought it was relevant
15% didn't think it was true
10% didn't find it interesting
19% thought it was worthy
9% didn't find it interesting
7% thought it was relevant
14% said it lacks detail
6% didn't think it contributes
4% praised the responder
15.




Of those who upvoted:
Of those who downvoted:
Of those who ignored:
72% agreed with the comment
37% disliked superfluous "almost"
14% didn't find it interesting
9% said it was true
25% disagreed
11% found it unoriginal
9% thought it was relevant
13% thought it was karma whoring
10% agreed with the comment
4% disliked superfluous "almost"

10% thought it doesn't contribute

Posted by at No comments:

What gets upvoted and downvoted on reddit? (new research) - part 1

My name is Maria Priestley and this post is based on research that was done as part of my undergraduate degree in Biological Anthropology at the University of Durham. All data were collected during November 2013 and used for nonprofit educational purposes. A published version of the findings can be found in PLoS One journal.

We all have a vague idea of the reasons why content gets rated positively or negatively on reddit, but it's always nice to get empirical evidence to explore these things. I attempted to do this using an online questionnaire consisting of two parts. Redditors were first asked to vote on a selection of 15 comments and to write brief explanations for these decisions. All explanations were then read and recurring reasons for upvotes and downvotes were identified. The second part of the survey had 29 questions asking participants to rate the importance of different content traits as influences on voting (e.g. content sounding intelligent, following reddit rules, expressing agreeable opinions...). These ratings underwent statistical analyses to group correlated traits into a smaller number of broad influences, which were then examined against variables like age, gender and reddit membership duration. The participant sample consisted of 489 adults recruited mainly from /r/TheoryOfReddit, /r/Anthropology and /r/SampleSize. I will summarise the main findings here, but there is a second instalment here containing full details on the comment voting results.


The chart below shows the average importance of different influences on upvotes and downvotes as reported by the participants.

We can see that wishing content to be seen by others was rated as the most important influence in upvotes. The other influences appear less important, but they give a better idea of the specific content traits that may contribute to upvotes and downvotes.

Informational skill was an influence that covered questions on the interestingness and intellectual value of content, so its high importance is understandable. When voting on comments, participants often explained their decisions with references to how interesting the comment was, as seen in the example below.

Common reasons for voting on other comments also included references to quality of writing and sources.

These results were interpreted in an evolutionary context as part of my undergraduate anthropology thesis. With regard to the high importance of informational value in voting, I suggested that upvotes could be a way of showing deference to people who are deemed to possess informational skill. This kind of tendency may have helped our ancestors to secure access to skilful individuals who possessed valuable information.

Relevance and compliance with subreddit rules also had reasonably high importance scores. Statistical analyses showed that the importance of these influences decreased slightly with age, and that women and participants with longer reddit membership durations gave higher scores. These differences were small but statistically significant. When writing about their voting decisions, participants often mentioned whether or not comments were relevant to the submission title.

With regard to prosociality, people tended to prefer content that was helpful and considerate of others, with rudeness or aggressiveness often eliciting downvotes. For example, a comment stating: “Don't ask for advice from reddit, it's full of the most smug self righteous assholes this side of the internet” was downvoted by 65% of participants, most of whom said that it was unhelpful or rude. This is unsurprising, given that prosocial norms are almost universal and vital across different cultures. Numerous scientists have even suggested that people may be biologically adapted to favour prosocial behaviour and to punish violators.

For unoriginality and Karma-whoring as influences in downvotes, it was found that their importance decreased slightly with age. The statistical analyses also indicated that longer-serving redditors were most averse to these traits. Average importance scores for each membership duration group are shown below.


Empathy, agreement and humour were quite important with regard to upvotes, and women gave significantly higher importance to this group of influences compared to men. The survey contained another question asking participants about the extent to which their voting is influenced by emotional reactions to content, where women also gave higher scores. Both differences are displayed in the chart below.
In the written vote explanations, participants often commented on their emotional reactions as reasons for upvoting emotive comments.

Disagreement was quite important in downvotes. The statistical procedure I used showed that people didn't give consistent levels of importance to agreement and disagreement (perhaps because downvoting in disagreement is more frowned upon and varies depending on the subreddits where one resides). Nonetheless, both agreement and disagreement were common in the written vote explanations for opinionated comments, as seen in the example below.

In my thesis I suggested that the involvement of personal opinions in upvotes and downvotes may reflect the human tendency to use rewards and punishments to enforce our cultural norms. No matter how ignoble it may seem on reddit, encouraging each other to hold similar beliefs and to behave in accordance with shared standards probably helped our ancestors to efficiently maintain complex institutions in trade, education and marriages.

Social influence, or the existing number of votes on a piece of content, was reported as being mildly important in voting. Interestingly, the importance of this influence decreased slightly with age, and younger people reported taking more notice of content that is accompanied by high numbers of votes and Gold badges.

That's it for now. As usual with this kind of research, it's hard to know if the findings are truly representative of the general voting behaviour of redditors, but I hope that these results can be useful in some way.

  
Posted by at No comments:
Subscribe to: Posts (Atom)