APC, memcache and varnish

I've recently set up these little puppies as some web developer friends of mine explained their usefulness in speeding up certain things like Drupal.

It's kinda not surprising when you realise that people are developing things on Drupal, which is a layer on top of PHP, itself an interpreted language (i.e. S.L.O.W.), which then has to go and talk to a MySQL database (useful, ubiquitous, relatively easy to use, but not necessarily the best designed or speediest database out there, so I've been told), which then has to go and grab the content, perhaps off a S.L.O.W. spinning hard drive, and serve it to the end user. When you consider that according to Drupal's web site, some of the biggest sites on the web are running Drupal, you can see that there may be room for some optimisation.

Enter a couple of enhancements to PHP itself, which implement and improve caching, and a reverse-proxy, which itself is another form of caching.

Bear in mind that this post is written from the POV of a sysadmin, I possibly don't have the insights into how these things actually do their job at a deeper level, but I know how to get them installed and working.

APC

APC, or the Alternative PHP Cache is fairly self explanatory, according to Wikipedia, it "optimises..and caches data and compiled bytecode", so takes stuff that has been interpreted and passed along and sticks it in memory so it's probably several orders of magnitude easier to access.

Getting this into PHP is as easy as finding the relevant package for your distribution's package manager of choice. I'm going with Ubuntu here, as that will possibly find a wider audience, but I did it first on my Gentoo box at home, and there wasn't really much difference.

$ sudo apt-get install php-apc 
..................... 
Setting up php-apc (3.1.7-1) ...

 Easy! And in Ubuntu, installing this package will insert the necessary line into the relevant PHP config files:

apache2/conf.d/apc.ini:extension=apc.so
cli/conf.d/apc.ini:extension=apc.so
conf.d/apc.ini:extension=apc.so

 To check that this is enabled, we make a small webpage available which uses the phpinfo function, like this:

<?php
 phpinfo();
 ?>

We should now be able to see that apc is enabled by going to http://<your webserver/test.php

Memcache

OK, memcache next. Cribbing from Wikipedia again, it seems that memcache is a more general purpose application, which many other apps can use to stuff data into memory, so it's faster to access.

This is installed in much the same way as apc:

$ sudo apt-get install memcached
...........................
Starting memcached: memcached_1

$ sudo apt-get install php5-memcached
............................
Creating config file /etc/php5/conf.d/memcached.ini with new version

and the latter line sticks similar lines in PHP's configs:

apache2/conf.d/memcached.ini:extension=memcached.so
cli/conf.d/memcached.ini:extension=memcached.so
conf.d/memcached.ini:extension=memcached.so

 phpinfo shows memcache is enabled in PHP:

A word on security

It seems that on older machines, memcache may be started in an insecure manner, allowing anyone on the internet to connect to it, if firewall rules allow:. The quick answer to this is to only let memcache listen on its local interface. You can check this by doing the following:

$ ps aux |grep memcache
memcache 24752  0.0  0.0 316888  1108 ?        Sl   12:18   0:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1

You're looking for the bit that says "-l 127.0.0.1" , if that isn't there, find the config file that it should be in (according to which distro you're running), put it in, and restart memcache. On Ubuntu, this file is /etc/memcached.conf, and it's there as of 12.04. On Redhat/Fedora/CentOS, it appears to be /etc/sysconfig/memcached, on Gentoo, it's /etc/conf.d/memcached, etc. etc.

This will ensure that memcache only communicates with things that are running on your server, and no digital Tom, Dick or Harry can come along and connect to your particular memcache instance. Phew, you're a little bit safer from the denizens of the internet for the time being.

Varnish

Varnish is slightly different in that it isn't linked to the LAMP stack as such, but sits in front of the webserver, basically caching stuff that goes out to the internet, making said stuff much quicker to get to than, for instance:

Web client (browser or app) -> Apache -> PHP -> MySQL -> PHP -> Apache -> Web client

Instead, with Varnish installed, you're potentially looking at:

Web client -> Varnish (cached data) - > Web client

Assuming the content has been accessed once, and of course assuming Varnish has been set up correctly, it's easy to see how this could potentially be much faster, especially considering that once a request hits the PHP and MySQL parts of the process, it could be slowed down even further if APC and memcache are not installed.

So, installation is essentially the same as we've seen above - via the package manager

$ sudo apt-get install varnish
.............................
 * Starting HTTP accelerator varnishd                                           [ OK ]

To configure varnish as it would be used in the wild we're going to have it listening on TCP port 80, the standard destination for HTTP requests. Of course, this being the case, we'll need to tell the webserver to listen on a port other than 80, in this example, we'll use port 8080. We'll then tell varnish where the webserver is. The webserver I'm using in this example is apache. The distro is Ubuntu, YCFMV (your config files may vary)

In /etc/apache2/ports.conf, change:

NameVirtualHost *:80
Listen 80

to

NameVirtualHost *:8080
Listen 8080

in /etc/apache2/sites-enabled/000-default, change:

<VirtualHost *:80>

to

<VirtualHost *:8080>

Then, restart your webserver.

Now, we point varnish to the webserver (the "backend" in varnish parlance). We're using localhost because varnish is running on the same machine as the webserver.

Editing /etc/varnish/default.vcl:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

To get varnish to listen on port 80, edit /etc/default/varnish:

DAEMON_OPTS="-a :6081 \

becomes

DAEMON_OPTS="-a :80 \

And then restart varnish via /etc/init.d/varnish restart .

You can check this is working by running varnishlog from the command line, and accessing some pages on your server - you should see some ouput scrolling up your console window.

There is some optimisation to be done on the configuration of varnish - but this can vary according to the needs of your site. I'll cover these in a subsequent post.

Enjoy!