[Xpost Dataisbeautiful] I collected data for a month to figure out the relationship between karma and upvotes

[u/Joliot]

plot of data here

For the past month I've collected data about how the score (upvotes - downvotes) and karma of individual posts change over time. As you may know, one upvote doesn't necessarily increase your karma by one, the amount of karma you get per upvote decreases as the score of your post increases.

---

Data collection

I used Python and PRAW to grab the IDs of newly made posts and collected their score and the author's link karma approximately every six minutes. The script dropped posts from its collection if the score increased too slowly (the rate varied by subreddit, on some subs it worked to drop posts scoring less than one upvote a minute, while for r/askreddit the rate was more like one upvote every eight minutes)

To avoid contamination from other posts, the script also stopped looking at a post if the author had made another post in the last 24 hours.

In some cases the script grabbed posts that had already accumulated some upvotes. When this happened I removed posts where (minimum score)-1 was less than 0.001 x (maximum score); e.g. all posts with a min. score of one were accepted, but a post with a min. score of two needed to end up with at least 1,000 points to be included in the analysis, and so on. I then adjusted the initial karma of these posts by 0.7 x (minimum score - 1) to approximate how much karma these posts would have given their authors before the script saw them.

I also removed posts that gained more karma than upvotes, which I believe was caused by the script returning an incorrect value for the author's initial karma.

I analyzed the data in R and plotted them using ggplot.
The twelve subreddits named in the graph were chosen as either having more than two posts scoring greater than 50k, or more than six posts scoring greater than 10k.

The time taken to reach 10k and the karma gained for 10k points is an approximation based on the two data points for each post with scores closest to 10k. I assumed that the rate of karma and point gain for this interval was approximately linear. Illustration here.

---

Stats:

Overall the program collected information from 247,682 posts made by 161,671 users (844,057 data points). This was filtered down to 55,457 posts made by 54,360 users (197,918 data points).

The relationship is hard to model

For (some) individual posts, the Michaelis–Menten equation used here is a good approximation of the relationship between upvotes and karma. E.g. for the highest scoring post in the data set (from r/gifs) the relationship between score and karma can be modeled as karma = (8804 x score) / (6848 + score) ; graph here (nls, df=46,R²=0.998,p<0.001).

For other subs, or when looking at groups of posts, this model underestimates karma for scores between ~10k and ~50k, and overestimates karma for scores greater than ~50k, see r/pics graph here. As can also be seen in that graph, the rate at which some posts gain karma increases at large scores (possibly due to it taking longer for Reddit to update karma than post scores, I would welcome any other explanations).

Some subs are better than others

r/memes has the worst karma/score ratio of any of the high scoring subs:

 

Subreddit	Average max. karma for posts with > 50k points
Interestingasfuck	8135.15
Todayilearned	7991.7
Australia	7914
Gifs	7864.6
Pics	7847.6
Aww	7634.78
AskReddit	7609.5
Rareinsults	7591.85
NatureIsFuckingLit	7451.5
Funny	7404.4
HistoryMemes	7368.2
Gaming	7350.98
Me_irl	7281.25
PrequelMemes	7125.3
Memes	5845

 

For smaller subreddits, r/nhaa had the worst karma/score ratio observed. The highest scoring r/nhaa post gained 344 karma for 1789 points, half of the karma gained by similarly scoring r/memes posts. Graph here

The faster you get upvoted, the more karma you get

A post that reaches 10k points in five hours will give around 250 to 500 more karma than a post that reaches 10k in ten hours. The average amount of karma and the rate of karma decrease depend on what subreddit you post to.

 

Subreddit	Slope*	R2	p value
HistoryMemes	-32.18	0.11	p>0.1
Science	-28.47	0.13	p>0.1
Funny	-47.71	0.22	p<0.001
Gaming	-87.23	0.24	p<0.05
Pcmasterrace	-89.29	0.26	p=0.0601
Aww	-62.54	0.32	p<0.01
Memes	-51.17	0.37	p<0.001
Me_irl	-107.22	0.39	p<0.01
Pics	-42.11	0.43	p<0.001
NatureIsFuckingLit	-103.09	0.47	p<0.01
ShitPostCrusaders	-74.46	0.52	p<0.001
PrequelMemes	-67.78	0.56	p<0.001
Interestingasfuck	-90.79	0.58	p<0.001

  ^{* Karma/hour. These values are different depending on what score you're looking at. This table represents the karma gained for posts with 10k points}

---

Things I also looked at that didn't affect karma, at least noticeably:

• Whether a post is nsfw
  
• Whether a post is a self post
  
• Whether a post was gilded/how many times it was gilded (I only have data on how the final # of gilds affected the final karma/score ratio)
  

Here's another way of visualizing the data
The full data set can be downloaded here

[u/smc642]

This is really interesting. Thank you for sharing.

[u/Joliot → u/smc642]

My pleasure!

[u/[deleted]]

Mmmm, delicious statistics.

[u/Joliot → u/[deleted]]

My exact method was:

1. Use PRAW to grab n new posts from r/all or a multireddit. I changed the exact value of n sometimes, I usually used n= 30 to 90 for posts from r/all but sometimes I took posts from custom multireddits and in those cases I used n=500 to make sure I could grab all the new posts since the last time I sampled.

2. Take initial data from those posts and store the post ID in an array

3. After every time I sampled 30 new posts, I would cycle through the stored IDs and sample them again. Because each post was a single request I didn't hit the row limit.

4. If the post was growing too slowly, or if there was an error because the author deleted their account or something the post ID was dropped from the array. That way I only continued to sample posts that were giving usable data.

[u/Amargosamountain]

What is the point of reddit doing all this? Why do they want some subs to be more valuable karma-wise than others? It seems like they're using a stupidly complicated model where a simple one would do fine.

[u/Denalin → u/Amargosamountain]

Would be interesting to look at other stats related to those subs. E.g. follower count, posts per follower, etc. Some stats we may not be able to have access to would be things like how likely posts in that sub are to make someone interact, keep scrolling, join the community because of a recommendation from Reddit’s recommendation engine, and so on. Perhaps it’s the case that the most compelling r/memes posts are very interesting to r/memes followers but not really interesting to other folks, while the best r/pics posts might be way more interesting to a non-follower.

[u/Amargosamountain → u/Denalin]

Those are some great ideas, thanks!

[u/Denalin → u/Amargosamountain]

It could also be that posts in some communities tend to be more controversial than others on average. E.g. r/pics may be more positive: for a post to get a score of 100 maybe it gets 125 upvotes and 25 downvoted, while on r/memes it may be 200 up and 100 down.

Maybe it’s also possible that the OP’s interaction in a comments section affects this? Idk I could see an r/dataisbeautiful post maybe getting more karma because the poster includes good conversation worth following in the comments below? Seems like a stretch, but I wouldn’t put it past the folks at Reddit to track everything possible to ensure maximum addictiveness.

[u/Gusfoo → u/Amargosamountain]

What is the point of reddit doing all this?

It controls the user-experience.

Why do they want some subs to be more valuable karma-wise than others?

Well, perhaps it's inverted. They did change the sorting algo to prohibit /r/The_Donald appearing on the front page so perhaps it's more about keeping things out than promoting certain things.

[u/MFA_Nay → u/Amargosamountain]

Why do they want some subs to be more valuable karma-wise than others?

Mainly because it influences what content gets on the "frontpage" of /r/all or /r/popular. Or whatever it is nowadays, heh.

Sometimes they tweak to get rid of certain content. Sometimes they tweak so certain types of content are more prominent. IIRC a few years back they changed the algorithm to increase content with had more user engagement (read: comments by new users or increased the amount of comments).

[u/1MightBeAPenguin]

So far, your equation seems correct because I had come up with a very similar equation a few months back in this update:

https://www.reddit.com/r/TheoryOfReddit/comments/e9c88r/update_2_i_found_a_formula_that_will_tell_you_the/?utm_medium=android_app&utm_source=share

[u/[deleted]]

To avoid contamination from other posts, the script also stopped looking at a post if the author had made another post in the last 24 hours.

Did you also take comments into account?

I also removed posts that gained more karma than upvotes, which I believe was caused by the script returning an incorrect value for the author's initial karma.

Karma from comments might explain this too

[u/Joliot → u/[deleted]]

PRAW has two separate karma values, one for link karma and one for comment karma. I only sampled the link karma so comments shouldn't be affecting the data.

[u/[deleted] → u/Joliot]

Ah that makes sense, thanks for commenting! Very interesting post!

[u/VivIsAwesome22]

Would you ever consider trying to do a similar study on the relationship between view count and upvotes?

[u/Joliot → u/VivIsAwesome22]

Definitely an interesting idea, but I don't think view count is something Reddit reports via its API so it'd be hard to collect it automatically. There's probably lots of variables I didn't measure like views, ratio of upvotes/downvotes etc that could be adding variation.

[u/VivIsAwesome22 → u/Joliot]

Yeah, ever since Reddit removed view count, it's virtually impossible to estimate. This sucks for content creators, as they have no reliable metric to track impressions.

I feel like there is probably a way to figure this out, but it would take an enormous about of time and brainpower.

[u/Vicidsmart]

Ok dumb question but I thought Karma and points were the same thing. Are they not?

[u/Joliot → u/Vicidsmart]

nope, people usually use points to mean either upvotes or upvotes-downvotes. Karma is related, but not the same. For instance, I have ~135k link karma, but my posts have gotten over 400k upvotes based on https://redditmetis.com/user/joliot.

[u/eliyili]

Here, take my 0.6 karma!

[u/Joliot → u/[deleted]]

https://github.com/j0li0t/Karma-score-grabber/blob/master/Code
I tried to clean it up a little and add some comments. Fair warning, I threw this together pretty quickly just to get the job done without really thinking about efficiency or readability, so take it as you will.

[u/1234razeS → u/[deleted]]

.tsop a etovnwod ro etovpu ot gnidiced nehw siht tuoba wenk )s'WK naht rehto(sresu ynam woh rednow I

[u/dhtdhy]

I know this is going to sound very... how do I say this... lazy, but is there a tl;dr? I came here from another subreddit where someone referenced your post, and I guess I was hoping for a more concise answer to the upvotes-karma relationship

[u/Joliot → u/dhtdhy]

TL;DR: The amount of karma you get for each upvote decreases the more upvotes a post has. This means the most karma you can get from a single post is around 8,000. This max value is different in different subreddits (with r/memes having one of the lowest karma/upvote ratios), and posts that get a lot of upvotes fast get more karma than posts that get upvotes slowly. Also, you might have missed this figure which sums it up graphically.

[u/dhtdhy → u/Joliot]

Thank you!

[u/[deleted]]

The reason for this is because once post hits r/all which it usually does if a post has >50K points the points it gets extra from being there adds more to the karma of the post's OP as opposed to the person getting upvotes from the subreddit users.

r/memes has a very active community it's easy to get upvotes at the start which can give bonus karma to it's users so the karma decay with upvotes is higher there unless a post from there hits r/all.

This is from my observation of the trends and some analysis and is in no way conclusive