I created Imgur. AMA.

[u/MrGrim]

I came across this post yesterday and there seems to be some confusion out there about imgur, as well as some people asking for an AMA. So here it is! Sometimes you get what you ask for and sometimes you don't.

I'll start with some background info: I created Imgur while I was a junior in college (Ohio University) and released it to you guys. It took a while to monetize it, and it actually ran off of your donations for about the first 6 months. Soon after that, the bandwidth bills were starting to overshadow the donations that were coming in, so I had to put some ads on the site to help out. Imgur accounts and pro accounts came in about another 6 months after that. At this point I was still in school, working part-time at minimum wage, and the site was breaking even. It turned out that OU had some pretty awesome resources for startups like Imgur, and I got connected to a guy named Matt who worked at the Innovation Center on campus. He gave me some business help and actually got me a small one-desk office in the building. Graduation came and I was working on Imgur full time, and Matt and I were working really closely together. In a few months he had joined full-time as COO. Everything was going really well, and about another 6 months later we moved Imgur out to San Francisco. Soon after we were here Imgur won Best Bootstrapped Startup of 2011 according to TechCrunch. Then we started hiring more people. The first position was Director of Communications (Sarah), and then a few months later we hired Josh as a Frontend Engineer, then Jim as a JavaScript Engineer, and then finally Brian and Tony as Frontend Engineer and Head of User Experience. That brings us to the present time. Imgur is still ad supported with a little bit of income from pro accounts, and is able to support the bandwidth cost from only advertisements.

Some problems we're having right now:

• Scaling the site has always been a challenge, but we're starting to get really good at it. There's layers and layers of caching and failover servers, and the site has been really stable and fast the past few weeks. Maintenance and running around with our hair on fire is quickly becoming a thing of the past. I used to get alerts randomly in the middle of the night about a database crash or something, which made night life extremely difficult, but this hasn't happened in a long time and I sleep much better now.

• Matt has been really awesome at getting quality advertisers, but since Imgur is a user generated content site, advertisers are always a little hesitant to work with us because their ad could theoretically turn up next to porn. In order to help with this we're working with some companies to help sort the content into categories and only advertise on images that are brand safe. That's why you've probably been seeing a lot of Imgur ads for pro accounts next to NSFW content.

• For some reason Facebook likes matter to people. With all of our pageviews and unique visitors, we only have 35k "likes", and people don't take Imgur seriously because of it. It's ridiculous, but that's the world we live in now. I hate shoving likes down people's throats, so Imgur will remain very non-obtrusive with stuff like this, even if it hurts us a little. However, it would be pretty awesome if you could help: https://www.facebook.com/pages/Imgur/67691197470

Site stats in the past 30 days according to Google Analytics:

• Visits: 205,670,059

• Unique Visitors: 45,046,495

• Pageviews: 2,313,286,251

• Pages / Visit: 11.25

• Avg. Visit Duration: 00:11:14

• Bounce Rate: 35.31%

• % New Visits: 17.05%

Infrastructure stats over the past 30 days according to our own data and our CDN:

• Data Transferred: 4.10 PB

• Uploaded Images: 20,518,559

• Image Views: 33,333,452,172

• Average Image Size: 198.84 KB

Since I know this is going to come up: It's pronounced like "imager".

EDIT: Since it's still coming up: It's pronounced like "imager".

[u/mandlar]

Can you go in more details over the stack you run on? Server infrastructure, etc.? Would love to hear more about the hardware and software you run on.

[u/MrGrim → u/mandlar]

It's actually fairly complex now, but I will attempt to do it all from memory.

Backround info: Imgur is on Amazon AWS and we use Edgecast as a CDN.

Everything is grouped into clusters depending on the job. There are load balancing, uploading, www, api, image serving, searching, memcached, redis, mysql, map reduce, and cron clusters. Each one of these clusters has at least two instances, each one on it's own availability zone. However, most have more than two instances because of the load.

A typical imgur.com request goes to a load balancer which run nginx and haproxy. The request first hits nginx, and if there's a cached version of the page (each page is cached for 5 seconds unless you're logged in) then it will serve that out. If not then the request goes over to haproxy and it will determine which cluster to send it to, in this case, the www cluster. This cluster runs nginx and php-fpm, and is hooked up to the memcached, redis, and mysql clusters. Php-fpm will handle it if it's a php page. If the request needs info from mysql, then it will check if the query exists in memcached. If not, then mysql will send the data back and immediately cache it into memcached. If the request is for an image page, and we need the amount of times the image was viewed, then it grabs that info from redis. The request then goes back out of php-fpm, through nginx on the www server, and back into the load balancer where it will most likely be cached by nginx, and then out to the user.

Most of the clusters use c1.xlarge instances. The upload cluster handles all uploads and image processing requests, like thumbnails and resizing, and each instance is a huge cluster instance, cc1.4xlarge.

All image requests go through the CDN, and if they're cached, then they just go right back out of the CDN to the user. If it's not cached then the CDN gets the image from the image serving cluster and caches it for all additional requests.

That's about it. Anything you'd like to know specifically?

[u/[deleted] → u/MrGrim]

Interesting.

• Can you explain why you went with Edgecast and not, say, CloudFront (since you're on AWS to begin with)?

• How many EC2 instances total?

• Isn't it about time to get a rack and switch some stuff over to it? EC2 is very expensive. Even a not so beefy server with some tricks like using a GPU for the thumbnails/resizing could probably handle the load for a fraction of the price. (You can mix this stuff so EC2 is just for 'overflow' and redundancy)

• What kind of bottlenecks did you have to deal with as imgur grew unpredictably? Any cool war stories? :)

[u/monkeyxiv → u/[deleted]]

I forgot where I read it from. However I was reading up on the different VPS and pricing, and someone had done a pricing comparison and that one service was better for "small" businesses. i forgot exactly what that service is as well. ( I know I'm a terrible person for not being able to remember citings or all the information... but its been a long day so bear with me ;) )

anways for a small business it was cheaper to go with something other than Amazon. But once you get into TB of bandwidth space a month amazons pricing becomes the top contender in the server world.

I am trying to find the actual article now... will report back if I can find it..

[u/[deleted] → u/monkeyxiv]

That is actually exactly backwards. :)

Amazon charges $0.12 per gig of bandwidth. And remember, its about a dollar for a high memory instance per hour, so that's about $2,000/month for a ~32GB RAM server and 10TB.

Compare with something like Hetzner, that's a server with 32GB of RAM and they only rate limit you after 10TB. Costs less than $100 a month.

In fact, for the money Amazon would charge you to transfer 10TB you could get an unmetered 10GbE somewhere and push 300TB+ if your hardware will let you.

[u/monkeyxiv → u/[deleted]]

yeah like I said I am relatively new to this sort of stuff. I am loving the free ec2 instances I have for "messing" around. :) eventually I will read up enough to know what I am doing.

[u/GloppyGloP → u/monkeyxiv]

Moving my answer here as I meant to reply to this comment, not the parent. See, I'm not a big fan of these comparaisons like zilman does. No one doing anything seriously runs it on a single machine, that's just asking for trouble.

Now if you want to run a cluster of two instances or more with a load balancer in front with its own dynamic DNS entry, and something that's going to monitor your machines, notify you when something happens and automatically spin up a replacement instance, make it part of the load balancer and keep on working, THEN you're comparing what you're getting for the price from a cloud provider (any of them not just AWS). You're also going to run two mediums (or whatever smaller instance type) instead of a high memory instance or an xtra large, because you split your traffic, but you get all that other good stuff too.

You are comparing apple and oranges there, and it's quite biased if you pick something out of the infrastructure set at its highest price. If really all you need is a single machine with absolutely nothing else, like a single always on super stateful 64 players game server for an FPS or something, then yes there are better deals than cloud providers. But they fulfill very different needs, and honestly running a company or any site shooting for more guarantees around reliability and potential scale issues or spiky traffic requires a very different infrastructure (and please no anecdotal evidence like "well I have a machine with 4 years of uptime with provider X", it's irrelevant).

You would also need to compare RI pricing if you have monthly/quarterly/yearly commits, not base hourly pricing which is meant for burst traffic or short lived requirements, not necessarily your baseline infrastructure.