How to run Drupal on Amazon EC2 using a new micro instance

Amazon Web Services

A while ago, I wrote an article about the steps required to set up a web server that runs the Drupal content management system (like this site).  Enough has changed in the interim that I figured I'd revisit this article with more up-to-date information.  In particular, Amazon just introduced their micro instances, which offer you the ability to host a website for as little as $5 a month.

About EC2 and micro instances

I covered Amazon's Elastic Compute Cloud (EC2) in my previous article, along with the reasons why I think it is a fascinating option for web hosting, so I won't go into detail on that here.

In short, EC2 lets you create virtual servers that run within Amazon's data centers. Amazon bills you by the hour that these servers are operating, as well as by the disk capacity and bandwidth they consume. These rates are very low, and they let you easily scale your site to fit immediate needs.

I've been running both this site and SonoPlot's from an EC2 small instance for the last two years. The performance has been awesome, and the sites have not once slowed down due to spikes in traffic. I've been able to experiment with new configurations without disrupting the main site by creating virtual clones and playing with them. The only reliability problems I've had have been due to my own stupidity when I've messed up backup scripts or other system configuration settings.

The new micro instances that Amazon has unveiled are pretty interesting. In my testing, I've not needed anywhere as near as much processing power or memory as the entry level small instances offer. A micro instance provides nominal computing power all of the time, with the ability to jump up to 2 EC2 Compute Units on demand, and 613 MB of memory. A normal small instance gives 1 EC2 Compute Unit all the time, along with 1.73 GB of memory.

This comes with a significant cost decrease for the micro instances. Current EC2 pricing has small instances priced at $0.085 per hour, or about $63 per month. Micro instances are only $0.02 per hour, or about $15 per month. If you pay a fee of $82 to reserve a micro instance for three years, your pricing drops to $0.007 per hour or around $5 per month ($7.50 per month with the reservation fee amortized across the three years).

$5 per month (plus bandwidth and disk space charges, of course) for a website that sits behind Amazon's pipes seems like a pretty good deal to me. Note that this is a site you can configure however you like, because it's a pure Unix virtual machine.

The sudden scaling capability of a micro instance fits the needs of a traditional website, which won't have high computational needs most of the time. This site is now running on a micro instance, and I've seen no dropoff in responsiveness in moving from the small instance to a micro one.

However, micro instances are a little different in structure than the original EC2 instances that I described two years ago, so my old setup instructions no longer apply. Micro instances do not have any local (ephemeral) storage, with all disk access being through Amazon's elastic block store (EBS). These instances use the relatively new capability for an image to boot from an EBS volume. In this article, I'll describe how to create an image that is compatible with the new micro instances.

Setting up an EC2 account and configuring security group

Not much about this part of the process has changed since the last time I wrote. I refer you to that article for how to create an account, get a key pair, and configure the security group you will be using.

Starting and configuring your micro image

When I did this last, I used a Rightscale CentOS Amazon Machine Image (AMI) as my basis and built from there. This time around, I decided to start with an Ubuntu image because I'd heard good things about their ease of use. I grabbed one of the images linked to by Alestic, ami-1234de7b, which is a 32-bit Ubuntu 10.04 Lucid image that boots from an EBS volume. There is one slight glitch with this image that I will discuss later.

One new addition to the control software that I did not mention before is Amazon's new AWS Management Console. We will be using this to start up the micro instance because Elasticfox currently does not support creating new instances of this type.

Sign in to the AWS Console and go to the Amazon EC2 tab. Click on the AMIs link on the left side of the screen and do a search for ami-1234de7b among all images. Right-click on that image and select the Launch Instance option. In the wizard that pops up, change the instance type to micro (t1.micro, 613 MB) and choose the availability zone, if you'd like. Continue twice, and select the appropriate key pair that you would like to use with this instance. Continue again and choose the security group you set up before. Continue once more and launch the instance.

Your instance will start and take a short time to get into a ready state. Click on the Instances link in the left side of the console to switch to your list of running instances. Right-click on the instance and choose Connect to bring up a help window. Copy the command that it lists at the bottom of the screen and paste that into a terminal window.

Edit that command so that it looks something like the following:

 ssh -i [path to key file]/[key file name].pem ubuntu@ec2-123-45-678-90.compute-1.amazonaws.com

where you provide the path to the .pem key pair file associated with this instance and keep the public DNS address that you were given in the window from the AWS console. Note that I'm using ubuntu instead of root as the login user, due to the way that the Ubuntu image is configured to prevent root logins.

This will connect you to the instance as the administrative ubuntu user. While you aren't provided the root or ubuntu user password, the ubuntu user can use the sudo command without a password.

If you'd like, at this point you can use sudo to add a user account for yourself and set the ubuntu and root passwords:

 sudo passwd sudo passwd ubuntu
 sudo adduser newuser

If you wish to enable password-free sudo access for another user, edit /etc/sudoers and add a line like the following:

 newuser ALL=(ALL) NOPASSWD:ALL 

As I mentioned earlier, there is a slight flaw in this Ubuntu image when running on an EC2 micro instance.  It was built to expect the presence of ephemeral storage (the temporary disk space that comes with a normal EC2 small, medium, etc. instance).  This storage is not present on a micro instance, so if you reboot this instance it will freeze on startup and you won't be able to access it.

The simplest fix for this that I've found is to edit /etc/fstab and comment out the line

 /dev/sda2       /mnt    auto    defaults,comment=cloudconfig    0       0

I also like to enable passwords for SSH logins so that I can use tools like Transmit for managing my site.  To do this, change the appropriate line in /etc/ssh/sshd_config to read

 PasswordAuthentication yes

and restart the SSH daemon using the command

 sudo service ssh restart

Installing web server and Drupal

For this, I followed the Ubuntu Community Documentation on installing Drupal.

The first step is installing the Apache2 / MySQL / PHP configuration (commonly referred to as LAMP) necessary for the web server.  This can be done using a single command:

 sudo tasksel install lamp-server

Drupal needs Apache's rewrite functionality to be able to do clean URLs, so run the following command to enable this:

 sudo a2enmod rewrite 

We then can install Drupal 6:

 sudo apt-get install drupal6

If it isn't obvious, you should take note of the drupal6 user password you've set so that you can use it later on.

If you want to have Drupal use the root path for your website, you'll need to edit /etc/apache2/sites-enabled/000-default and change the appropriate line to read

 DocumentRoot /usr/share/drupal6

Within that file, also comment out the block that starts with

 <Directory />

Additionally, edit /etc/apache2/conf.d/drupal6.conf and comment out the line

 Alias /drupal6 /usr/share/drupal6

Restart Apache to have these settings take effect:

 sudo service apache2 restart

Once you've done this, you're ready to set up and start using your Drupal site.  If you are just installing a site, you can simply go to your site's Drupal install page at

 http://ec2-123-45-678-90.compute-1.amazonaws.com/install.php

where you have replaced the base address there with the public DNS name you used above for your SSH connection.

If you're migrating a Drupal installation, you'll first want to dump your old Drupal site's database using

 mysqldump -Q -u [drupal user] -p [drupal database name] &gt; drupaldatabase.sql

on the new site, drop any database that exists with the same name as your migrated Drupal database and create a new one by logging in to mysql as the root user and using the commands

drop database drupal6;
create database [newdatabasename];
GRANT ALL PRIVILEGES ON [newdatabasename].* TO [drupal MySQL user]@localhost IDENTIFIED BY '[password]';

You can then import the old database using a command like the following:

 mysql -u [drupal user] -p [drupal database name] < drupaldatabase.sql

Creating a new image from this configured instance

Unlike how we previously had to package up the non-persistent portion of the instance or lose it upon termination of the instance, now everything in our instance's file system is persistent.  We can easily snapshot this filesystem, clone it, and attach it to another AMI running the same kernel as this one to replicate the site as it is.

Still, it's pretty easy to generate an image to allow for quick booting of clones of your server.  To do this, go to the AWS Management Console, go to your running instances, right-click on the instance, and choose the menu item Create Image (EBS AMI).  Give it a name and description and a snapshot of your instance will be taken to be used as a basis for the new image.  Note that this process will lead to a reboot of your current instance, but that should only take it down briefly while this is being performed.

After that is finished, you can clone as many copies of your instance as you'd like from this base image.  You can even take this image and use it to spin up an m1.small instance, if you need a little more processing power for your server.

Making the site publicly visible

The last step in making the site visible to everyone is pointing one of Amazon's elastic IP addresses at it.  I describe this process in my previous article, so I won't repeat that here.

Also, you can use the instructions I have there for setting up regular snapshotting of your EBS store, only now the store is your entire root volume and it's not using the XFS filesystem.

With this, you should have a Drupal website running on an EC2 micro instance, like I do here.  As I said, I think this provides a great price / performance ratio for a small website.

Comments

Finally! Thank you so much for this. I have been wanting to set up a micro-instance since it was launched 2 weeks ago, but I could not figure out how to do it properly. I think this is the first step by step set of instructions. I'm going to give it a shot and hopefully it will work for me.

How does the smtp aspect of Amazon work? I send out a lot of reports to our regional leaders and this amounts to about 1,000 - 10,000 emails a day. (depending on a combination of what roles need reports)

Though I am strongly considering trying to using someone like mail chimp, it's precisely that I never know exactly what combination of emails I am sending out until that morning that I've been reluctantly.

It has been my experience that emails sent from an EC2 instance tend to be blacklisted as spam. This appears to be because a few bad actors seized on EC2 as a spamming platform early, due to its scalability.

I'd highly recommend a solution like MailChimp for mass-mailings over an EC2 SMTP server.

Great tuto. Will do the same with Wordpress ;-)
To send email from EC2 this was a problem. That is why we are using either Gmail as SMTP gateway with a limit of 1000 email per day or external email gateway like authsmtp.com.

But I found really reliable to use an elastic IP address to my instance to let it send email to webmail like Hotmail and Yahoo for example.

Now with the new functionnality of Reverse DNS Proxy of Amazon this can open new way not to be considered as spam when email are sent from Amazon.

http://aws.typepad.com/aws/2010/03/reverse-dns-for-ec2s-elastic-ip-addre...

It's not a good idea to enable password logins. Don't spread bad practices, please!

You cannot revoke shared access granted via passwords, unless you change the password and notify everyone who knows it.
But this process is tedious and requires the password being transmitted via various channels.
It's not safe for a secret to travel, i hope we can agree on that ;)

With SSH public keys you can grant and revoke access to an account for any number of people and you never have to handle any secret and they can change their corresponding private key passphrase without your knowledge any time and it won't affect their granted access...

I think Transmit can utilize SSH agents.
Just add your key to your agent with the "ssh-add" command, provide your passphrase and Transmit shouldn't ask for anything when you are connecting.
If I'm correct, by default the keys generated by Amazon don't have a passphrase.
It's not very secure, so encrypt your downloaded key with the "ssh-keygen -p" command.

Otherwise thanks for the article. It can help my friend to move his server without too much help from me.

Great tutorial, just what I needed!

One thing I couldn't get though... Say you reserve a micro instance which works just fine, but during the workday the load increases, so you'd like to increase the resources a bit.

Can you bring another micro instance into the game for a few hours and have the database use the combined instances' cpu/ram ?

Is that possible? Or you'll need to upgrade to a higher instance type, like Small or use some complex load balancing?

Thanks once again for the useful tut!

Each instance is its own virtual machine, so they won't automatically pool resources. You could use some of the load-balancing options to spread this out, but that will require some work on your end.

Generally, the easiest approach would be to clone a new small instance from your existing micro one, then simply switch over the elastic IP to point at this new small instance. You'll get the faster performance of the small instance (which can take quite a bit of load in my testing) while minimizing downtime.

Thanks! great instructions. One note thought, I needed to do a
sudo apt-get update
before doing
sudo tasksel install lamp-server

What do you mean by "comment out the line" in ( edit /etc/fstab and comment out the line)? Please.

I think this is a great tutorial but some more explanation about editing stuff would be really helpful please. I am learning this and i am finding ec2 a great environment to try stuff out and learn while not destroying a well developed website or db.

Place a pound sign (#) before that line. Amazon EC2 might not be the best place to just get started with server administration, because it is fairly technical. I'd recommend starting to work with local Linux / Unix setups first before jumping into EC2.

Excellent document, this is exactly what i wanted to do, it saves my 2 days time, and i could able to setup fully loaded EC2 drupal site in one hour.

regards
Brett vincent.

i had one EC2 micro instance on amazon and i created another image and trying to install the load Balancer ,but failed ,is load balance is possible for two micro instance ,in my case its not added to inserive

Syndicate content