May 9, 2014

NGINX - Performance Tuning Tips

WWW Sub-domain Redirect

Typically, one can use this code to redirect to the WWW domain:

server {
    ...
    if ( $host !~* ^www\. )
    {
        rewrite ^(.*)$ http://www.domain.com$1;
    }
}

This has performance issues since NGINX has to evaluate the "If" block for every request. Instead, use server blocks to do a 301 redirect:

server {
    listen 123.123.123.123;
    server_name domain.com;
    return 301 http://www.domain.com$request_uri;
}

server {
    listen 123.123.123.123;
    server_name www.domain.com;
    ...
}

May 3, 2014

Best Root Servers

A Root Server or, most commonly, a root virtual private server (VPS) is typically a Linux or Windows based server where you have full control over the operating system. This post covers VPS providers, some of which feature also dedicated server (Hetzner) besides the cheaper offers.

Having been an user of virtual servers for the last 5 years, I've been a faithful user of three different companies for some period of time. I've tried a whole bunch of VPS providers but arrived at these as some of the best. The winners were, by chronological order:

TL;DR

Use Hetzner if you are in Europe or don't mind some latency due to working somewhere further away. All their plans are well priced and the machines perform great. You won't get any support 24/7 in case their hardware breaks, so don't use for something that requires 99.9% uptime. Software problems you can fix by using the offline console, which many other VPS providers don't have.

General Tips

In most quality providers, you have access to an offline console and a rescue system to install your own distros/kernels or whatever you may need. Installing BSD is another issue for most of them, as you would need a way to provide the boot ISO as an alternative to their rescue image - I have not found a provider that allows you to do this. I would not put servers without access to an out of band console or a way to backup and restore a limited range of Linux distributions on-the-fly, as even mainstream distro upgrades can sometimes expose bugs that leave the system unable to boot.

The use of good virtualization software is the first and most important feature of a quality root server, as OpenVZ/Virtuozzo (operating system level virtualization) allows oversubscribing that delivers both lower prices and terrible performance (most of the time). You also can't use swap with OpenVZ, so that doesn't leave a lot of margin for error in the server's configuration for periods of high load. I recommend you to stay away from OpenVZ virtual servers.

KVM and Xen solutions, on the other hand, do not allow oversubscribing and deliver much more close to metal virtual servers with higher performance and a broader software compatibility, as you are not stuck running only one kernel. The performance of these solutions is theoretically lower than OpenVZ solutions but the fact that the machines can't be oversubscribed by the provider means that what you lose in performance due to the usage of the hypervisor allows you to have more stable performance 99% of the time, even if slightly lower.

Slicehost

Slicehost was one of the big VPS providers for cheap. They used to provide high quality Xen based root virtual servers, loaded with RAID 10 and a bunch of other extras were available, like cheap full automatic backups of the VPS. Eventually, Rackspace purchased them and turned them into the current Rackspace Cloud solutions. Since the new offer was not compatible with what I needed for servers, I looked for another provider.

Linode

Still a big provider nowadays, and one of the oldest ones, Linode had my business for a good while after the Slicehost takeover. Linode still provides all the kind of functionality you expect like offline console and good hardware at great prices and I never experienced any issues with them - slow downs, downtime or any kind of issue with customer support or billing. It also had free DNS servers for your domain names, which is quite good and something I  hope others replicate soon.

If I recall correctly, I paid $20 a month for 512 MiB of RAM and 20 GB of HDD space, all backed by a Xen hypervisor. Currently these are the specs for the cheapest plan, of $10 a month:
  • 1 GiB RAM
  • 1 CPU Cores
  • 24 GB SSD Storage
  • 2 TB Transfer
  • 40 Gbit network for download
  • 125 Mbit network for upload
Quite the deal, though I don't think you need more than 1GiB of RAM for most stuff, even multiple sites, and as long as you stay away from Apache, you may very well do all you need with even as low as 256 MiB and swap (which Xen allows).
I personally wouldn't go lower than 512 MiB and a properly tuned NGINX + PHP-FPM configuration is almost a requirement though, in either case.

So if it was great, why did I stop using Linode services? A while back there was a high profile hack that involved a big amount of crypto currency getting stolen and Linode did not provide any specifics as to why the hack was possible and what did they do to mitigate future problems. More hacks have happened in the recent years, one as recently as in 2014, so I'll keep using other providers which better handle issues like this.

After looking quite a while for a new provider, I found Hetzner:

Hetzner

This German based provider offers not only root servers of the VPS variety but also dedicated servers, web hosting and collocation services. I only use the VX6 VPS service, which has the following characteristics:
  • 1 Virtual CPU
  • 1 GiB of RAM
  • 25 GB of HDD space
  • 2 TB of traffic
  • KVM Virtualization
  • Full root access
A good virtualization solution is one of the most important features of a good VPS (I go into more detail about this in Top Tips) and was one that brought me to Hetzner's offers. What I looked for next was a good price (6.90eur), enough resources for a decent web application server and an offline console that was also present on the other providers I had worked with. 
Hetzner not only provides the offline console but also a Linux rescue system, which then lets you install any distribution or lets you recover the machine from any state - not just from a network bad configuration or a kernel crash - which is something I can't live without.  Offline consoles are especially important if the providers don't have 24/7 support or any kind of management of the instances, as is the case with Hetzner. If you need 24/7 support, you need to contract one of their more expensive root server offerings, which start at €50/$69. This is still an offer that is considerably cheaper than competing offers at more renowned companies.

As for uptime, customer support or other issues that I've run into with Hetzner, I have little to report thus far (2+ years). I have only had a few hours of downtime on one of my VPS due to some network hardware failure which was promptly resolved during work hours, since as I've mentioned only dedicated root servers have 24/7 support.

I have tried other cheaper providers, from time to time, but even the ones that offer good Xen based services don't have enough of a supporting infrastructure to offer a Linux rescue system or offline consoles. Given the price range and the service available, Hetzner has been the best I've been able to find.

Do leave me other companies you like in the comments and I'll have a good look at them!

Apr 28, 2014

Linux Kernel - Forcing a Crash for Testing

In software development, it is sometimes important to have resiliency to crashes in the software. It is less impractical though, to induce a crash in a remote machine, which may not be easily power cycled. For this purpose, the Magic SysRq Linux kernel functionality can be tapped into:

echo "b" > /proc/sysrq-trigger # Emulates a crash

The most relevant options, from the Linux kernel manual:
  • b - Reboots the kernel without first unmounting file systems or syncing disks attached to the system.
  • c - Crashes the system without first unmounting file systems or syncing disks attached to the system.
  • o - Shuts off the system.

Sep 8, 2011

Bash - String Tokenizing

Bash has some functionality to enable building a string tokenizer, by means of four modifiers:
  • #
  • ##
  • %
  • %%
We can then use to to get the first or last elements, from the end or the start of the string:

#!/bin/bash
TEST="i/am/a/very/big/dir"
echo ${TEST#*/}
echo ${TEST##*/}
echo ${TEST%/*}
echo ${TEST%%/*}

# am/a/very/big/dir
# dir
# i/am/a/very/big
# i

echo ${TEST#*/*/}
echo ${TEST%/*/*}

# a/very/big/dir
# i/am/a/very

I have omitted a leading slash in this string, which was a directory, so that it would print "i" instead of nothing. So you can use echo ${TEST%%/*}  to get the first element and saving echo ${TEST#*/} on every iteration of a cycle that tokenizes the string into an array.

The last two expressions serve to show that if you're dealing with some hardcoded delimiters, you can further expand it so that it matches what you want precisely.

Jul 7, 2011

VCS - Converting Mercurial Repository to Git

First, clone the original Mercurial repo with hg:

hg clone ssh://hg@bitbucket.org/owner/repo

Now we'll need Git-Hg (massive kudos to offbytwo for making it possible):

git clone git://github.com/offbytwo/git-hg.git
cd git-hg/
git submodule update --init fast-export
cd ../repo
../git-hg/bin/git-hg clone ssh://hg@bitbucket.org/owner/repo
touch .gitignore
git add .gitignore
git pull git+ssh://git@github.com/mdvcs/md.git

Here I had some problems with the git repository already having a README file, so I just removed it and proceeded with the push.

git rm README
git push git+ssh://git@github.com/mdvcs/md.git

And you're done.

Mar 2, 2011

Apache - Selectively Disabling mod_rewrite

Say you have redirect a domain from .net to .com and have a rule such as:

# Rewrite to the new domain and add "www"
RewriteCond %{HTTP_HOST} !^www\.domain\.com$ [NC]
RewriteRule ^/(.*)$ http://www.domain.com/$1 [R=301,L]

These are catch all rules, which check if the host is domain.net, if not it just redirects it to it. (See the motives why you should use "www" or base domains and never both on this page)

In this scenario, if you want to disable rewriting for a certain folder for the .net domain, you can add the following rule at the top of the code:

RewriteRule ^/oldfolder(.*)? - [L]
# Rewrite to the new domain and add "www"
RewriteCond %{HTTP_HOST} !^www\.domain\.com$ [NC]
RewriteRule ^/(.*)$ http://www.domain.com/$1 [R=301,L]

Here we will match the folder and stop there, thereby serving the page from the alternative domain name (a .net domain) for the specified folder.

http://www.domain.net/oldfolder/somethingelse

If you need more information on www or non-www URL rewriting, please see the www-redirection tutorial page.

Apache - URL Rewriting To WWW or non-WWW Domains Names

I'm a very strong proponent of having website redirection turned on for most websites. I have even written a post about this, with examples of failure to do this and it's consequences - go here if you want to take a look. I assume that if you've arrived here, you already know why you need to do it.

To do the rewriting I'll be focusing on Apache's mod_rewrite, in a Linux installation, which is what is available to people typically using a LAMP install for their content management system of choice - like Wordpress or phpBB. It's all quite simple, so let's get down to it.

Two ways to do it

This rewrite is the rewrite if you need to use a .htaccess file on your server:

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www\.chosendomain\.com$ [NC]
RewriteRule ^(.*)$ http://www.chosendomain.com/$1 [R=301,L]

There's another option when you want to keep it stored in a safer location, which is including it in Apache's httpd.conf file. There is a detail that must be taken care of, in this case: if you don't include the slash at the start of the rewrite rule, you'll find that some pages get a double slash at the end.

RewriteEngine On
RewriteCond %{HTTP_HOST} ^chosendomain\.com$ [NC]
RewriteRule ^/(.*)$ http://www.chosendomain.com/$1 [R=301,L]

These two blocks of code have slight differences. Notice the bold slash at the end, replacing the RewriteBase / directive. While they do the same thing, they are not interchangeable depending of where you are placing the mod_rewrite rules.

Note: Slashes are important on the RewriteCond rule as a safety precaution so that it matches "." and not some other random character - even though it is highly unlikely you have a domain "www2domain.com", where 2 is the random character, pointing to your server's IP.

If after you've done this, you want to exclude some folders(or paths) from the URL redirection you've just inserted, see the instructions on this page.