Scaling Up WordPress Website with A Simple DNS Load Balance

One of my established website runs on WordPress but was hosted on Windows IIS. Have to admit that it’s not a perfect match. It works but not perfectly. I eventually had to move it to an Apache powered web host after I got a big bad hit that took down the site for over 2 days.

Migration ran very smoothly and had the site up and running in no time. And it all looked working beautifully until a few days later it was got suspended because the high usage caused by the amount of incoming traffic trigged the switch they had it set up on the web server that suspended my website and my whole account. No complains here but I did call them back asked what the options I have after I requested to reactivate my account. The suggestion was that I ought to move up to higher level of hosting plan to avoid future suspension.

Fair enough, so I headed out shopping for a better deal on a plan that can scale up to handle my website. I looked up WordPress.com’s option and compared a fairly long list of VPS (Virtual Private Server) options, and finally landed on a plan on Linode. Paid fee and had the server ready within a few seconds, really got myself excited. But unfortunately, the excitement went away pretty quick. The site built on VPS didn’t seem to handle the amount of traffic well. It’s constantly hang and deadly slow when it’s live. I knew that there must be some settings in Apache I didn’t do right but felt have no time left for me to fiddling around how to make it right.

And that’s when I decided to take different route to scale my site up, and do it in a way that not only works but also doesn’t cost a whole lot of money.

The method

The idea is simple, called Round-robin DNS.

How it works theoretically?

Basically, I set up two identical mirrored websites with each on different web hosting company, and made 2 "A" DNS records that point to each site respectively. Because over 80% of the visitors to my site is new, they all need to query the DNS before locating to it. And that’s how the traffic gets split. The first new visitor gets one A record that points to one site and next new visitor gets one that points to the second site.

The result

Two words: it works. A second hosting plan at $3.95 per month did the trick and takes half of the traffic off my site. Just awesome.

Here are the screenshots from each hosting’s Awstats traffic log. As you can see, both sites took nearly half of the total traffic.

How to set up WordPress using this method?

There are a few things that need to be taken care of when running a setup like this adjacent with WordPress. Mostly, the questions will be like how I know which site I am actually in and how to sync the content on both sites. There are for sure a number of ways of doing this but here is what I did and I think at least it works for me.

1. Set up two additional DNS A records, for example, www1 points to site A, and www2 points to site B.

2. Set up the second site identical with the first one.

3. Set up each site’s WordPress Address in General Settings to www1.domain.com and www2.domain.com respectively. That way, you can access each site’s control panel easily by going to www1.domain.com/wp-admin or www2.domain.com/wp-admin.

4. You also need to change the settings on how your sitemap.xml generates. It’s equally important that you make sure the sitemap.xml on each site is generated with the identical content. If you are using Google XML Sitemaps plugin, you need to change the Location of your sitemap file setting from Custom Location to something like below:

5. From this point on, post content to both sites respectively to keep the content on both site identical. And you need to do this consistently every time when you post or update the post. If you are using Windows Live Writer, you need to set up two different accounts, one of each site. And when posting, post it in a future time in schedule mode so that the content on both sites has the same posting time showing.

6. Adopt a 3rd party comment management system, i.e. Disqus. Or, you will have trouble managing the comments on either of the sites.

Catches?

It works almost flawlessly, except for a few incidents where the auto-tweet service constantly sends tweets for the same post repeatedly.

It does require a bit maintenance to keep both site identical. Since I couldn’t find any good tool or service that can automate some of the tasks for me I am going to just take it manually. But I can’t really complain too much. I get more than what I paid for. A monthly total of $8 to run a site that can easily cost me 10 times more.

Scaling up a website certainly is a much bigger topic and this is just one simple way that happens to be working with the size of my website. I am sure I will learn more along the way as this site grows bigger and bigger.