Klaas Bals' Blog

I've successfully created a public Amazon EC2 AMI, that allows you to start a Fedora Linux system with Drupal 5.1. When the image is started, you can simply connect to the assigned public DNS Name, and you can configure and play with Drupal. Actually, creating this image wasn't that hard, but I figured some people would want to play with Drupal on EC2, so I decided to make this AMI public.

I haven't solved any of the problems with persistence, meaning that this AMI is only usable for playing around with Drupal, as none of the changes you make or content you add will survive a restart of the virtual machine instance. But you can use this AMI to take a look at the Drupal installation procedure, or get a quick tour of a scratch Drupal installation without needing to install all the prerequisites and configure the database.

How to get started?

To be able to start your own instance of this AMI, perform the following steps:

• Subscribe for the Amazon Web Services. More specifically, subscribe to Amazon S3 and Amazon EC2.
• Install the Firefox Extension for Amazon EC2, launch it, and fill in your account details you received when creating your account.
• Find the AMI with the AMI ID or manifest shown on the AMI's info page, and create a new instance.
• Wait a minute or to, click the 'Your instances -> Refresh' button, and copy the public DNS Name.
• Paste the name in your favorite browser, and you will see the Drupal installation screen.

For a more in depth guide on how to start with Amazon EC2, check out the Getting Started Guide.

Apparently, the Drupal Group in Vancouver has had a session on Amazon Web Services. It was given by a guy named Alex and his slides contain a nice overview of the Amazon Web Services (but doesn't give details about Drupal on AWS).

No real solution for the persistent storage problem though, although he has a nice name for this problem. He call this the "Ephemeral Nature of EC2".

On a regular computer, the operating system and applications are loaded from a local hard disk into memory. Every application you launch runs in memory. When you want to keep data, you need to store it to the local hard disk. The advantage of memory is that it is fast, but it is not persistent. The local hard disk is slow, but it is persistent. Memory is also smaller than the hard disk. For example, the memory is 2GB and the hard disk is 160GB.
An application such as a database runs in memory (for speed) but writes all data to the hard disk to not lose it when the computer crashes.

Now with Amazon EC2, this problems is complicated even further. An EC2 virtual machine has 1.75GB memory and a hard disk of 160GB. The virtual machine is launched from an Amazon Machine Image (AMI), and once the machine is running it can write to the hard disk. However, when the machine fails, the newly written information is not persisted. Restarting the machine will reload the original AMI, and any information you saved to the hard disk in a previous instance is lost. If you want to persist data, you need to store it to Amazon S3, which is slower than the hard disk. So storing data on Amazon S3 is similar to the regular computer, but needs an extra level of storage before you know the data is persisted.

This means regular applications that thing they are persisting data to hard disk, lose all their information when the instance crashes. And instances can crash!

I think that is the major problem when working with Amazon S3. The storage paradigm changes, and it takes time before applications get adapted.

The problem can be solved generally by performing regular backups of the data to Amazon S3, and automatically loading that backup when the EC2 instance starts. But a backup every hour means you can still lose up to an hour of information. It can also be solved by running multiple EC2 instances, and performing replication to another machine, and then backing up that second machine.
For me, the ideal solution would be that applications become aware of this problem and solve it themselves, by storing their information to S3. some people have started this with a number of applications. For example Mark Atwood built a A MySQL Storage Engine for AWS S3MySQL. Other people are working on (or thinking about?) an Amazon S3 File-Store for Alfresco.

I haven't tried any of these solutions yet, so I can't tell you how good they work at this time. When I want to run Drupal on EC2, I will need to find some solution to this problem.

I've ran my first Amazon EC2 virtual machine instances. I started of with some of the default Fedora AMIs (Amazon Machine Images) provided by Amazon themselves.

But I don't feel at home on a Fedora Linux as much as on a Gentoo Linux, so I tried the Gentoo 2007.0 Base System AMI provided by Eminent, and that worked perfectly. (I started of with another one created by Geert Bevin, but that virtual machine had some network problems, so I switched to the AMI by Eminent.)

The first thing I had to do was modify a text file, so I installed the only good text editor ;-) and created my own AMI.

So, I was able to complete an entire EC2 cycle: Create a virtual machine for a provided AMI, connect to it, play around with it, install some extra software and dump it into my own AMI. Nothing complicated, but still nice for a starter.

Great. I just got an email: This is a brief note to confirm that we've enabled your access to the Amazon Elastic Compute Cloud (Amazon EC2) limited beta.

Yay! Now I 'just need to make some time' to get started...

Almost a month ago I subscribed for some Amazon Web Services (AWS). For Amazon EC2 I wasn't that successful as I got a message that EC2 was full. But I have been using Amazon S3 to do some backups of stuff I certainly don't want to lose.

Today I finally received some email regarding EC2 saying that they "have begun accepting additional developers into the beta, and wanted to let [me] know that [they]'re aiming to provide [me] with access soon." Let's see what soon means.

For a while now, I want to test Amazon Web Services (AWS). Now that I have some free time (May 1st is a National Holiday here in Belgium, I decided to give it a try. I would like to run an installation of Drupal on Amazon Elastic Compute Cloud (EC2) just to see what it does.

I followed the Getting Started Guide, so I subscribed for Amazon Simple Storage Service (S3). I entered my credit card number and subscribed.