Structured Thought . ORG

EBS Volumes

EBS (Elastic Block Storage) is a way to attach disk volumes to your running EC2 instances. The main difference between EBS volumes and the default ephemeral storage that is attached to EC2 instances by default is that EBS volumes persist beyond reboots and can also be moved between instances without losing the data contained within.

EBS Volumes are billed out in 1GB increments at $.10/GB/month. This is not per usage, this is per allocated GB. For example, if you have a 10GB volume allocated and only 1GB of it is in use, you’re still paying $1/month.

For the purposes of this example, we’ll start with 2 1GB volumes ($.20/month for allocation) which will be mirrored for redundancy at the OS level. This will ensure that a failure of one of the volumes will not affect the operation.

Allocating an EBS Volume

NOTE: For the purposes of this example, the architecture of the running instance is irrelevant. Therefore, to save myself some cost while putting together this part of the demo, I’ll be using a 32-bit ’small’ instance. This will not affect the operation, and will work identically to the 64-bit ‘large’ instance used in the previous examples.

Before you create a volume, you need to determine which availability zone your instance is running within. Think of an availability zone as a server room. To check this, click on the row in your Instances section of the AWS console, and look for the ‘Zone’ entry in the panel below. It should read something like ‘us-east-1d’. Take note of this zone, as it will be required for assigning your EBS volumes (Volumes can only be attached to instances within the same availability zone).

To start the process, click the ‘Volumes’ shortcut under the ‘Elastic Block Store’ header in your AWS console. The default view will show no volumes.

To create a volume, click on the ‘Create Volume’ button in the upper left. You will be presented with a menu asking for the size and availability zone. The size can be anywhere from 1GiB to a theoretical unlimited TiB. In this case, choose 1GiB, and pick the availability zone that matches your running instance. Do this twice, so you end up with 2 1GiB volumes.

NOTE: You may not be familiar with the GiB and TiB nomenclature. For our purposes they are the same as GB and TB, but they do have a significant difference. A TB refers to 1,000 GB, which is 1,000MB, which is 1,000KB, which is 1,000 bytes. A TiB is the binary-counting equivalent, meaning: a TiB is 1,024GiB, which is 1,024MiB, which is 1,024KiB, which is 1,024 bytes. So while a TB is 1,000,000,000,000 bytes, a TiB is 1,099,511,627,776, an eventual difference of 99.5 Billion bytes.

Once the volumes have been created, it’s time to attach them to the system. If you haven’t already, launch an ssh session to your running instance and issue the command ‘tail -f /var/log/messages’. This will allow you to see when and how the system detects that the volumes have been attached.

To perform the attachment, select the checkbox next to one of the volumes and click the ‘Attach Volume’ button that becomes available. You will be provided with the option of which instance to attach the volume. There should be only one. Choose the next available device name (/dev/sd* in the list). If you don’t see your running instance, be sure you created the volumes in the proper availability zone. Do this for both volumes, making sure that you have different device names for each.

Configuring the Volumes

When the EBS volumes are created, they are just like brand new disks out of the box, completely empty, and without any information whatsoever. Since we’re going to be using them as PVs (Physical Volumes) within LVM, this is ok, as they’ll be initialized properly when we start to use them. The first of these steps is to tell LVM that we have 2 new PVs we’d like to use. To do so, issue the following commands (assuming /dev/sdf and /dev/sdg for our new EBS volumes):

• sudo pvcreate /dev/sdf
• sudo pvcreate /dev/sdg

Next, we’ll want to create a VG (Volume Group) out of these new PVs, which will allow us to then start creating usable file space. To create a VG, issue the following command

• sudo vgcreate datavg1 /dev/sdf /dev/sdg

Next, we need to create an LV (Logical Volume) within the VG, which will house our filesystem. In this case, we want to create a roughly 1GB LV that is mirrored (2 copies, one on each physical volume). This will take up both of the EBS volumes we attached to the server earlier. To create this mirrored LV, issue the following command

• sudo lvcreate -L 1000M -n datalv1 -m 1 datavg1 –corelog

This creates a 1000MB volume named ‘datalv1′ with 1 mirrored device (2 total copies) within the datavg1 VG. The ‘–corelog’ option tells LVM to manage logging (the system keeps track of changes made to the LV) internally to the LV, otherwise, a third device would need to be used for logging. We can’t use a size of 1GB (-L 1G), because of the ‘–corelog’ option, which requires some space within the LV. This leaves a little bit of unused space within the VG, but it’s negligible.

Using the Volume

Finally, we need to create a filesystem within the LV that will allow us to actually use the space. To do so, issue the following command:

• sudo mke2fs -j /dev/datavg1/datalv1

This will create an ext3 filesystem (standard within the Linux world, but you may update the command to use whichever filesystem type you desire) that fills up the entire LV called ‘datalv1′. As a test, you can create a mount point, which is simply an empty directory, and mount the file system, this will show you your available space in the volume.

If you’re not planning on utilizing these volumes right away, you can essentially reverse the process to unmount the filesystem, delete the LV, remove the VG, detach the EBS Volumes, and finally remove the EBS volumes. I recommend you do this until you have the process down and can get things running in more-or-less one sitting.

Elastic IPs

Fortunately, this section is rather straightforward, and will just give an overview of how Elastic IPs work, and how to attach them to a running instance. Later, I will show you how to create a nameserver, and then begin running resources through your Elastic IP into your running instance.

To allocate an Elastic IP for your use, click the ‘Elastic IPs’ option under the ‘Networking & Security’ heading in your AWS console.

Since you won’t already have any Elastic IPs allocated, you will be asked to create one right away. Click the button you’re presented with, and you will see an IP appear almost instantly.

NOTE: Elastic IP usage is billed differently than other AWS resources. Since IPs themselves are relatively inexpensive, Amazon doesn’t charge you for using them. But, to keep people from hoarding IPs and not using them, Amazon charges $.01/hour for any Elastic IPs allocated that are not being actively used. While this isn’t much in the short-term, keep aware, as this can add up if you’re not watching.

To attach this IP to your running instance, simply click the checkbox next to the new IP, then click the ‘Associate’ button that becomes available. Again, since you are only running one instance at this time, you should be presented with only one option. Click ‘Associate’ in the menu, and then return to your instances display in your AWS console. After a few seconds of clicking ‘Refresh’ (this could take up to 10 minutes, according to Amazon, but I’ve never seen it take more than 2 minutes), you should see the ‘Public DNS’ item change to match your new IP.

At this point, you should be able to access your instance via SSH to the new IP address, rather than the old one.

That’s it for dealing with Elastic IPs. In the next installment, we’ll work with setting up our hosting control systems, including configuring a domain and DNS.

The last two years, I have participated in a student competition sponsored by IBM that has been designed to keep interest in Mainframe operations in the younger generations.

Mainframe systems have been around for decades, but over the last 10 years or so, they have fallen out of the public eye, and interest in learning how these systems work has waned. Consequently, the population of Mainframe experts is aging quickly, and some companies are ending up in crisis as these people retire with nobody to take their place.

IBM began this contest with the idea of rekindling interest in Mainframes within student populations.

The competition consists of three parts:

1. General How-To and Getting Around – A simple set of tasks that are designed to give even the absolute beginner an idea of how the system is laid out, how to work with data, and basic operations.
2. Practical Experience – Part 2 gets more involved and gives the student competitor a chance to see what kind of operations real Mainframe admins perform  on a daily basis. This also includes some debugging and job control operations, and actually gets fairly in-depth in the operations of the system.
3. Real-World Challenges – This is a much longer part that pits the student against problems and challenges that have been seen in the real world. There are job control, debugging, coding, administration, and many other challenges.

Last year, I managed to make it through Part 2, but didn’t get a chance to start working in Part 3. This year, I completed Part 2 on the first day of competition, and have started working on Part 3. I plan to work as much as I can on Part 3 until the deadline of December 29.

So, what does all of this have to do with blogging? Well, as part of attempting to get the word out among the younger generations, IBM has offered to work with bloggers and other social media consumers. By feeding us information and soliciting us to share our input and opinions, we can help IBM show the world that Mainframes are still a big thing and worth learning.

Once in a while, you will see a post here with information about the Mainframe world from IBM. I do this because I think it’s a worthy endeavor. Mainframe systems are still the supreme workhorse of the processing world. (These are my own opinions and experience, BTW) While many of the systems with which we interact with Mainframes feel antiquated and counter-intuitive in our point-and-click computing world, they are purpose-built and work perfectly for the system. Getting over the hump of transitioning from a graphical interface to a textual one is one of the biggest challenges for people attempting to get into Mainframe systems anew.

But I think it’s worth it.

So, the LHC fired up this morning in a (completely expected) hail of non-event-ness. Early this morning, the first laps of protons around the 27km ring were made, ensuring that the systems in place are all functioning, and defining what, if anything, needs further tuning before real experimentation begins.

If you’re wondering what the heck the LHC is, in short, the largest particle accelerator built to date. It consists of a 27km ring that runs under the border between Switzerland and France. There are two paths, within which will be circulated bunches of small (to us) particles that will eventually be accelerated to 99.9999999999% of the speed of light (299,792,458 meters/second, or 670,616,629.4 Mph, in other words REALLY fast). The hope of the LHC (among other things) is to find the Higgs Boson, which is a particle that is suspected to be involved in giving things mass. I would get into the science of this, but I myself BARELY understand the surface of all of this, and to try to explain it would probably make my brain become teh brokun.

But anyway… Some of the other things that might be found/created (though not dangerous in any way, please shut up, doomsayers): Nano-black holes, these are super small black holes that (through Hawking radiation) evaporate and collapse into themselves over the period of a few femtoseconds (one billionth of one millionth of a second); Dark matter, believed to make up a huge portion of the bulk mass of the universe, dark matter is unique because it doesn’t interact with light, electromagnetism, or any other force that we can measure, except gravity; Conditions much like the creation of the universe, one of the detectors is in place to measure the collision of lead ions (which are REALLY heavy), which is predicted to give us a look at what it was like a few nanoseconds after the universe came into being.

So, there’s some interesting stuff going on over there at CERN. None of it is even remotely likely to do anything that would affect us in our daily basis. Least of all, nothing that will destroy the planet or the universe. I am a little leery, however, of strange looking men in blue suits, and particle physicists with glasses and goatees.

I’ve been checking out (and posting to) Veritocracy over the last couple of days. It works kind of like Digg, in that users choose to push an article up or down the stack based upon their opinion. It’s a bit different than Digg in that you actually have three options: ‘I like this article, push it up’, ‘I have no opinion on this article’, and ‘I don’t like this article, push it down’. Being able to give an actual down vote to an article, rather than just making it disappear from view, is a real benefit that Veritocracy carries.

On the Veritocracy home page, you are presented with a clean, very AJAX-ified interface, with search capability and five primary topic headers (Politics, Technology, Entertainment, Business, and Veri). Each of these headers takes you to a list of sub-topics, under which are organized the articles themselves.

What I consider to be the ultimate feature of Veritocracy is the ability to syndicate your own content and publish your stories to the site, all through the easy interface. All it takes is providing Veritocracy with your feed’s URL and adding a flag to a post to verify that you are indeed the owner of the content. From there, Veritocracy will read your feed, and with two clicks, allow you to publish your article under the topic(s) of your choosing.

Veritocracy is currently in an invitation-only beta, but TechCrunch was able to snag some invites. If there are any left, hop over there for your invite code and check it out.

Personally, Veritocracy’s layout and content presentation fit pretty much perfectly with the way I want to get my information. I only have one tab left on my default Firefox screen (thanks to Google Chrome), and it now goes directly to Veritocracy. A nice, quick way to scan the latest goings on out there on the intertubes.

I’ve been running Google Chrome for a little while now, and it’s already beginning to supplant Firefox (3.0.x) as my primary browser (Internet Explorer hasn’t been a serious player in my book since Firebird 0.5). There are still a few things that will make me keep Firefox around for a while, yet.

1. Extensions - Firefox is the king of extensibility. I just can’t live without adblock, TwitterFox, and FoxMarks. Once reasonable equivalents begin to make thier way in, though. It’s going to be harder for Firefox to hang on to my system.
2. Supportability – A lot of sites don’t yet support Chrome, and give the good old “You’re using an unsupported browser, please use one of these…” message, even though the rendering engine is the same as Safari (Woopra’s monitoring script detects Chrome as Safari, even). This is a minor thing that will be fixed in due time.
3. Scrolling Issues – Scrolling using the edge of a touch pad, or by middle-clicking on the page are currently not working. This is pretty minor, as most of the time, I’m using a mouse, but it can be problematic when I’m expecting it to work and nothing happens.
4. Security – There are a couple of vulnerabilities in the version of WebKit that’s being used in current builds. These have been addressed in later builds, and it will be nice to have them implemented as things progress.

Other than these things, which are really minor for a .2-level product that has been out for 3 days, Chrome is a truly fantastic piece of engineering. I have heard from more than a couple of Googlers that they have been using earlier builds internally for quite a while. It’s nice to know that they eat the proverbial dog food before serving it to the rest of us. It really shows that they are committed to creating something with real quality, usability, and future usability.

One more thing, I think I kind of touched on this in the last post: Chrome (under the Chromium project) is OpenSource, meaning anyone can download all of the code written so far (minus the Google Trademarked stuff), and create their own browser project. It seems that Google is hoping for this to happen, counting on the Open Source community to make improvements and push them back into Google’s development stream.

The next few months are going to be interesting. I can’t wait.

I’ve been using Google’s Chrome browser for about 30 minutes now, and I just wanted to give people a few points of interest that I’ve found so far.

Rendering: This thing is unbelievably fast. I was impressed with the recode of Gecko that was released with FireFox 3, but the Webkit backend of Chrome is nothing short of insane. Gecko managed to expose what I call download lag (the time it takes for the first renderable chunk of HTML to come down from a server), but Chrome has managed to hang it out for all to see. Even historically fast pages now show some download lag because the rendering engine in Chrome literally snaps pages into place when they’re loaded.

Features: The ability to pop a browser window out and use it as a discreet application on your computer is phenomenal. Using GMail as an example, it really makes the web application feel as though you’re using something you actually installed. Couple that with the increased rendering speed of Chrome, and you have something that approaches or, in some cases, exceeds the performance of desktop applications.

Compatibility: This is something ongoing, and will improve over time, but even now, I’m having trouble finding a site that won’t render properly. I have been hearing scattered reports about plugins not working, but that’s about it so far.

Extensibility: It’s OpenSource. Don’t like it? Fix it. Want a new feature? Create it.

Security and Stability: Every tab has its own discreet process on the system. This keeps web pages from bringing down the entire browser when something goes wrong. It also appears to make memory management easier, as freed memory is flushed better than when the browser is re-using memory segments. This should show great benefits as power users that open tabs continuously don’t lose performance as quickly (or at all, as is the possibility)

I plan on playing with this a lot over the next few days, so keep checking back here for more details.

Google scores huge with this one.

Yep… That’s 2.0 Firmware (Notice the App Store link) with Cydia Installer and a *gasp* T-Mobile connection you see there.

Well, yes. That is also a literal statement, in that a huge battery does contain a great amount of potential electrical energy. But that’s not the potential I mean. I mean the potential to transform industries, particularly the Auto industry.

A company called Altair Nanotechnologies has developed a Lithium-Titanate battery system that can be scaled to 250kWh, and deliver a 1.0MW draw for up to 15 minutes. The system is also able to be charged at roughly the same rate, which far exceeds any quick-charging technology we have available elsewhere.

Down to a reasonable scale, this technology could transform the way electric and plug-in hybrid cars are built. With these batteries, a charging station could bring a car back up to full power in a matter of a few minutes, rather than several hours, which would be required even for such advanced current technology as Lithium Ion. The batteries can also be charged via ‘trickle charge’ which sends a much smaller current, but charges the battery much more slowly. In fact, to create a practical full-power charging system for a home, serious power reworking would need to be done, as the batteries would require a 3-phase industrial power source to utilize the system’s full charging potential.

That’s where the real cleverness of ideas comes in. Rather than attempt to rework the wiring in a consumer’s home, why not create a dual installation: One in the car, one in the home.

Home Installation: Continuously trickle-charges off of the home’s existing grid connected power, holding enough potential to charge the in-car system several times over. In particularly sunkissed areas of the world, these systems could even be exclusively trickle-charged from a solar-power installation on the roof, requiring no grid access at all.

Car Installation: Standard electric vehicle installation, with two-charging modes available. Full power and trickle.

Configuration: So we have the large in-home system charging continuously for 23.9hours every day. When the car arrives at home, the consumer plugs into the home system, and selects a trickle charge for overnight. If, by the time the consumer needs to leave the house again, the car is not fully charged, flip into full-capacity charging mode, which is powered by the in-home system, and have the car ready within a few minutes. The in-home system still has enough remaining capacity to charge the car twice over, and is still being trickle-charged in the background.

I think this would be a fantastic way to utilize this new technology and really put a dent into the U.S. (and world) requirement for petroleum-based energy, which is beginning to outstrip the budgets of even middle-class individuals.

Since I’m a sucker for free hits, and @UStreamTV asked, here are some thoughts on live video, specifically, live video on the web.

I think the idea of being able to connect with people face to face (in a manner) instantly, from almost anywhere, is achieving a level of world-shrinking that hasn’t been seen since the early days of the telegraph and telephones. Before the telephone, realtime communication with someone more than a few feet away from you was pretty much unheard of. There was the telegraph, but morse code is slow, and sending messages was generally pretty expensive. When the telephone came along, suddenly people could communicate over astoundingly large distances, several thousand miles in some cases (New York to Los Angeles is almost 2,500 miles), instantly. Suddenly, your favorite cousin that moved to Texas a few years ago, was merely a few turns of the rotary dial (or a few plug changes by the switchboard operator) away.

The Internet itself has done much to accelerate this world-shrinking, but (in my own opinion) has not participated directly in the process, as it was originally built upon the infrastructure of the phone system in the first place.

Enter Internet video: Now, with about $30 of equipment, and a reasonably fast internet connection, anyone can create a video that is able to be distributed to the entire world in an instant. Video sharing sites like YouTube allow anyone to create a short video that is viewable by anyone in the world.

The next step, live video, has truly shrunk the world to the point that the only thing separating many of us is a time zone. Now, with that same $30 of equipment and internet connection, people can have live, realtime, face-to-face conversations, anytime, regardless of physical location. This new experience is the closest we have been able to come to actually being in the same room with someone so far away.

After thinking about it, I honestly think that there will not be any real world-shrinking technologies developed that will do what the telephone and live video have been able to do until we’re able to transport ourselves instantly from place to place.

So, iPhone developers have put out a fantastic little application called Twinkle, which is a native iPhone application that interfaces with Twitter. I’ve decided to work on developing a Twitter application for Windows Mobile. Partly because I think it would do well, but mostly because I have a Windows Mobile phone, and using the Google Talk gateway to Twitter just isn’t that practical for me.

I put up a site for the development that is slowly taking shape. Check it out, subscribe, and if you have a WinMo phone, let me know. I’m going to need testers, so I’ll probably be sending out early releases to lucky individuals.