Amazon EC2 as a Reseller Web Hosting Platform – Part 3 – EBS Volumes and Elastic IPs
EBS Volumes
EBS (Elastic Block Storage) is a way to attach disk volumes to your running EC2 instances. The main difference between EBS volumes and the default ephemeral storage that is attached to EC2 instances by default is that EBS volumes persist beyond reboots and can also be moved between instances without losing the data contained within.
EBS Volumes are billed out in 1GB increments at $.10/GB/month. This is not per usage, this is per allocated GB. For example, if you have a 10GB volume allocated and only 1GB of it is in use, you’re still paying $1/month.
For the purposes of this example, we’ll start with 2 1GB volumes ($.20/month for allocation) which will be mirrored for redundancy at the OS level. This will ensure that a failure of one of the volumes will not affect the operation.
Allocating an EBS Volume
NOTE: For the purposes of this example, the architecture of the running instance is irrelevant. Therefore, to save myself some cost while putting together this part of the demo, I’ll be using a 32-bit ’small’ instance. This will not affect the operation, and will work identically to the 64-bit ‘large’ instance used in the previous examples.
Before you create a volume, you need to determine which availability zone your instance is running within. Think of an availability zone as a server room. To check this, click on the row in your Instances section of the AWS console, and look for the ‘Zone’ entry in the panel below. It should read something like ‘us-east-1d’. Take note of this zone, as it will be required for assigning your EBS volumes (Volumes can only be attached to instances within the same availability zone).
To start the process, click the ‘Volumes’ shortcut under the ‘Elastic Block Store’ header in your AWS console. The default view will show no volumes.
To create a volume, click on the ‘Create Volume’ button in the upper left. You will be presented with a menu asking for the size and availability zone. The size can be anywhere from 1GiB to a theoretical unlimited TiB. In this case, choose 1GiB, and pick the availability zone that matches your running instance. Do this twice, so you end up with 2 1GiB volumes.
NOTE: You may not be familiar with the GiB and TiB nomenclature. For our purposes they are the same as GB and TB, but they do have a significant difference. A TB refers to 1,000 GB, which is 1,000MB, which is 1,000KB, which is 1,000 bytes. A TiB is the binary-counting equivalent, meaning: a TiB is 1,024GiB, which is 1,024MiB, which is 1,024KiB, which is 1,024 bytes. So while a TB is 1,000,000,000,000 bytes, a TiB is 1,099,511,627,776, an eventual difference of 99.5 Billion bytes.
Once the volumes have been created, it’s time to attach them to the system. If you haven’t already, launch an ssh session to your running instance and issue the command ‘tail -f /var/log/messages’. This will allow you to see when and how the system detects that the volumes have been attached.
To perform the attachment, select the checkbox next to one of the volumes and click the ‘Attach Volume’ button that becomes available. You will be provided with the option of which instance to attach the volume. There should be only one. Choose the next available device name (/dev/sd* in the list). If you don’t see your running instance, be sure you created the volumes in the proper availability zone. Do this for both volumes, making sure that you have different device names for each.
Configuring the Volumes
When the EBS volumes are created, they are just like brand new disks out of the box, completely empty, and without any information whatsoever. Since we’re going to be using them as PVs (Physical Volumes) within LVM, this is ok, as they’ll be initialized properly when we start to use them. The first of these steps is to tell LVM that we have 2 new PVs we’d like to use. To do so, issue the following commands (assuming /dev/sdf and /dev/sdg for our new EBS volumes):
- sudo pvcreate /dev/sdf
- sudo pvcreate /dev/sdg
Next, we’ll want to create a VG (Volume Group) out of these new PVs, which will allow us to then start creating usable file space. To create a VG, issue the following command
- sudo vgcreate datavg1 /dev/sdf /dev/sdg
Next, we need to create an LV (Logical Volume) within the VG, which will house our filesystem. In this case, we want to create a roughly 1GB LV that is mirrored (2 copies, one on each physical volume). This will take up both of the EBS volumes we attached to the server earlier. To create this mirrored LV, issue the following command
- sudo lvcreate -L 1000M -n datalv1 -m 1 datavg1 –corelog
This creates a 1000MB volume named ‘datalv1′ with 1 mirrored device (2 total copies) within the datavg1 VG. The ‘–corelog’ option tells LVM to manage logging (the system keeps track of changes made to the LV) internally to the LV, otherwise, a third device would need to be used for logging. We can’t use a size of 1GB (-L 1G), because of the ‘–corelog’ option, which requires some space within the LV. This leaves a little bit of unused space within the VG, but it’s negligible.
Using the Volume
Finally, we need to create a filesystem within the LV that will allow us to actually use the space. To do so, issue the following command:
- sudo mke2fs -j /dev/datavg1/datalv1
This will create an ext3 filesystem (standard within the Linux world, but you may update the command to use whichever filesystem type you desire) that fills up the entire LV called ‘datalv1′. As a test, you can create a mount point, which is simply an empty directory, and mount the file system, this will show you your available space in the volume.
If you’re not planning on utilizing these volumes right away, you can essentially reverse the process to unmount the filesystem, delete the LV, remove the VG, detach the EBS Volumes, and finally remove the EBS volumes. I recommend you do this until you have the process down and can get things running in more-or-less one sitting.
Elastic IPs
Fortunately, this section is rather straightforward, and will just give an overview of how Elastic IPs work, and how to attach them to a running instance. Later, I will show you how to create a nameserver, and then begin running resources through your Elastic IP into your running instance.
To allocate an Elastic IP for your use, click the ‘Elastic IPs’ option under the ‘Networking & Security’ heading in your AWS console.
Since you won’t already have any Elastic IPs allocated, you will be asked to create one right away. Click the button you’re presented with, and you will see an IP appear almost instantly.
NOTE: Elastic IP usage is billed differently than other AWS resources. Since IPs themselves are relatively inexpensive, Amazon doesn’t charge you for using them. But, to keep people from hoarding IPs and not using them, Amazon charges $.01/hour for any Elastic IPs allocated that are not being actively used. While this isn’t much in the short-term, keep aware, as this can add up if you’re not watching.
To attach this IP to your running instance, simply click the checkbox next to the new IP, then click the ‘Associate’ button that becomes available. Again, since you are only running one instance at this time, you should be presented with only one option. Click ‘Associate’ in the menu, and then return to your instances display in your AWS console. After a few seconds of clicking ‘Refresh’ (this could take up to 10 minutes, according to Amazon, but I’ve never seen it take more than 2 minutes), you should see the ‘Public DNS’ item change to match your new IP.
At this point, you should be able to access your instance via SSH to the new IP address, rather than the old one.
That’s it for dealing with Elastic IPs. In the next installment, we’ll work with setting up our hosting control systems, including configuring a domain and DNS.