Last week I did a presentation on Amazon Web Services meet up in Greece. During the presentation I showed how you can start an AMI of fedora 8 base, install apache, php and mySQL and make them all store their data on an EBS volume. This attracted a lot of attention and participants asked me to post this as an online tutorial. This tutorial requires basic knowledge of AWS like running an EC2 instance, creating an EBS volume and assigning an elastic IP to the instance. During the presentation I used ElasticFox which is a firefox plugin that allows you to perform several operations on Amazon through a graphical user interface.
Now before starting the tutorial, I would like to write a few things about the purpose of it. The idea behind it, is to be able to run a server with data stored on EBS volumes. This gives you a lot of flexibility. For example, let’s say that you have two EBS volumes: One 10GB volume for you mySQL database and another 150GB volume for serving files through apache. If your website grows and you need to have a bigger storage just for your apache server you can simply take a snapshot of that volume, create a new volume using that snapshot but with a bigger storage (up to 1000 GB) and attach it back to the server. This is a matter of just a few minutes. Moreover this can happen for CPU problems as well. Once you need more CPU, you can create a new, bigger instance and attach your EBS volumes to that. Amazon offers great options for that, which practically can cover most of internet websites. Should you need more, scaling is much easier since they are many tools out there like RightScale or scalr which can add / remove instances on the fly using very advanced triggers for that.
Now let’s get our hands dirty:)
To get started you need to do the following things:
1. Launch an instance of the following AMI: “ec2-public-images/fedora-8-i386-base-v1.07.manifest.xml“.
2. Create an EBS volume and attach it to the instance (attach it to /dev/sdh).
3. Request a new elastic IP and assign it to the instance.
4. Connect to your new server using SSH.
All the above steps can be done using ElasticFox (or any other tool, even through command line).
Once you have logged in as root to your server the first thing to do (only do this once) is to format your newly attached hard drive (EBS volume actually:)). To do this, run the following command:
Then you need to create a directory on your server where the new volume will be mounted.
To mount the volume to that directory run the following command:
mount -t ext3 /dev/sdh /ebs1
Note for the steps above: If you ever terminate your instance and create a new one, you will only need to create the directory and mount the volume there. The volume would be formatted already and your data will be in there!
Now you can install apache, php and mySQL using yum:
yum -y install httpd php mysql mysql-server php-mysql
To make sure that these services are started each time your instance boots, run the following commands:
/sbin/chkconfig httpd on
/sbin/chkconfig –add mysqld
/sbin/chkconfig mysqld on
You can start apache and mySQL using:
/sbin/service httpd start
/sbin/service mysqld start
Once you have started mySQL it is advised that you do some stuff for security reasons. The first thing to do is to change the root password:
mysqladmin -u root password ‘new-password’ (where new-password is the password you want for root).
Then connect to mySQL:
mysql -u root -pnew-password (replace new-password with the password you specified before).
Remove the test database:
DROP DATABASE test;
Disable anonymous access:
DELETE FROM mysql.user WHERE user = ”;
Then exit mySQL using quit.
Now it’s time to move apache to EBS:
First we need to stop the service:
/sbin/service httpd stop
Then we move httpd to EBS:
mv /etc/httpd /ebs1/httpd
And create a symbolic link to it:
ln -s /ebs1/httpd /etc
Also do this for the document root:
mv /var/www /ebs1/www
ln -s /ebs1/www /var/
And restart apache:
/sbin/service httpd start
Note: If you terminate and start a new instance simply remove the /etc/httpd and /var/www folders and just run the commands to create the symbolic links.
Then we do the same for mySQL:
/sbin/service mysqld stop
mv /var/lib/mysql /ebs1/mysql
ln -s /ebs1/mysql /var/lib
/sbin/service mysqld start
This only moves the mySQL data, not the logs. You can edit /etc/my.cnf to configure where logs are stored and also move the log directory to EBS if you want to.
So now you have a fully functioning server on EC2 that stores and retrieves data to and from and EBS volume. The EBS volume and be backed up almost instantly using a snapshot. If you don’t use innodb for your tables it is advised that you lock your tables while you take the snapshot.
Next steps / thoughts:
Some might want to create a hosting enviroment for many websites. What you can do for this is to edit your httpd.conf file (located on EBS /ebs1/httpd/conf/httpd.conf) and enable Virtual Hosts. What I did is to create a directory /ebs1/sites and then create users on fedora using the following command:
useradd -d /ebs1/sites/mysite1.com user-for-mysite1
Also don’t forget to specify a password for the user using:
Once you have created the user you can point the document root for that specific site to his home directory.
Also to enable the user to upload files to his/her website using SCP you need to allow password access through SSH (be careful on security issues though). To do this, edit /etc/ssh/sshd_config and change the line “PasswordAuthentication no” to “PasswordAuthentication yes”. Also restart sshd by running /etc/init.d/sshd restart.
The final step is to link real domains to your new server. We assume that you have already assigned one or more IPs to your server. To point a domain to a specific IP you need to edit its DNS records and add an “A record” that points to that IP. Once the domain resolves on your server you will be able to use it for your website. If you need to host multiple websites, create one VirtualHost record for each domain (using the appropriate servername and documentroot values) and off you go!
Special thanks to http://agiletesting.blogspot.com/ for the useful information that I found there for this tutorial.