I hope this post helps someone else out. On Monday, I had page on another website go viral via a Reddit post, and it briefly hit the front page (whooo #19 on r/all at around noon). It sounds great right? Well, I noticed this as I was sitting in an airport ready to fly back home after Christmas. My website is hosted on a Digital Ocean droplet, and I had kept it at the 2GB plan which was generous as it usually only has about 700-2000 users a day. At 9AM all was well, and my thinking was that I could resize to a larger droplet if I needed to.
At 10am, I noticed that my position on Reddit kept climbing and the number of active users was flat. I thought that was weird. So, I tried to access my website. It. was. soo. slow. But, this should be fine, all I had to do was resize the droplet. I shutdown my droplet and resized to 8GB / 4 CPU’s. I waited a few minutes and fired up Google Analytics again, I was still at ~900 users and my website was still borderline non-responsive.
I was in a panic. My flight was getting close to boarding. In a rash decision, I decided to try resizing again. Bigger is better, right? A couple clicks later, the droplet was racing around at the 64GB / 20 CPU plan. It did not help. The post kept rising on Reddit, and my little droplet couldn’t keep up. The page in question was the savings percentile calculator.
I enlisted a tech friend to help, while I was on the plane. We tried increasing the memory limits on MySQL, PHP, and some obscure WordPress memory limit. Nothing worked. Hours passed by, I knew there was a bottleneck somewhere, but I couldn’t figure it out. My droplet basically had unlimited CPU and RAM for all intents and purposes, yet it was barely chugging a long.
The Solution
Finally after some hundreds of google searches, I came across this post on increasing the number of concurrent users in Apache. My server is running Ubuntu 14.0.4 LTS and the setup was a little bit different, but these changes finally helped. I basically doubled the max number of Request Workers that my Apache was allowed to generate:
/etc/apache2/mods-available/
mpm_prefork.conf
Added: ServerLimit 300
Change to: MaxRequestWorkers 300
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
ServerLimit 500
MaxRequestWorkers 500
MaxConnectionsPerChild 0
</IfModule>
mpm_event.conf
Change to: MaxRequestWorkers 300
mpm_worker.conf
Change to: MaxRequestWorkers 300
Then restart your Apache server for the changes to take place: service apache2 restart
If you need to roll back your settings, you may need to clear your cache by running:
sync; sudo echo 3 > /proc/sys/vm/drop_caches
Finally I saw the number of users climb past ~900 well over 1000, and my site was as responsive as normal. By this time though, my site was dropping off of r/all and r/rising’s frontpages, so I can only imagine how many people I missed during the 5 hours that my server was throttled. I hope that if you’re reading this, and you are facing a similar problem that this will help you. I know I tried googling and people mentioned CDN’s and mirroring databases, but no quick fixes like this.