A proxy server is a server that sits between your computer and the internet and services your requests to the internet. Many companies use proxy servers to filter employee internet requests since a proxy server can also modify or block your request.

My goal in setting up a proxy server was to give myself a means of encrypting my web traffic when I am in insecure locations such as airports and restaurants.

Setup
To set up a proxy server, you first need a server. You can use a machine on your home network, get a dedicated server, or get a virtual server. I chose to get a virtual server through slicehost.com. I am using their base $20 256 slice with Ubuntu 8.04 LTS. In addition to proxy services I also use it as a virtual desktop and, since I can leave it on all the time, run long term jobs.

I set up the proxy server to only allow connections from the local host. I can then connect to the local machine via SSH and surf the web through the SSH tunnel. This way, the connection to my server is encrypted and no one can use the proxy server unless they have an account on my machine.

The setup is simple but I had to go to several sources to figure it all out, hence this article.

First, install Apache:

apt-get install apache2 apache2.2-common apache2-mpm-prefork apache2-utils libexpat1 ssl-cert

Next, link to the mod_proxy modules:

cd /etc/apache2/mods-enabled
ln -s /etc/apache2/mods-available/proxy_http.load
ln -s /etc/apache2/mods-available/proxy_ftp.load
ln -s /etc/apache2/mods-available/proxy_connect.load
ln -s /etc/apache2/mods-available/proxy.load

Then, in your httpd.conf file located at /etc/apache2 make
entries to configure the server:

ProxyRequests On
ProxyVia Block
<Proxy *>

   Order deny,allow
   Deny from all
   Allow from 127.0.0.1
</Proxy>

Here, ProxyRequests configures Apache as a forward proxy. <Proxy *> and </Proxy> delineate the proxy configuration block. The block tells Apache to deny all connections except from the local machine (127.0.0.1).

Now, restart Apache:

apache2ctl graceful

Finally, in your browser configure your host to use the proxy server you just set
up:

Sources:
slicehost.com: Ubuntu Hardy Apache and PHP install guide
serverwatch.com: Configuring Apache 2.0 as a Forward Proxy Server

While researching the NiteCore SmartPD EX10 R2 I was also looking for a 2nd flashlight that took AA batteries.  The AA battery form factor is bulkier than the CR123 batteries that the NiteCore uses but the advantage is that AA batteries are far cheaper and easier to find.  The low cost and easy availability of AA batteries makes a AA light an excellent companion for traveling.  After some research I found a light I felt was perfect, the Fenix LD10, and decided to purchase it from Outfitter Country on Amazon.com.

Background:
The Fenix LD10 is a follow-on to the popular Fenix L1D, the LD10 features a redesigned exterior and improved reflector.  I don’t have a L1D so I cannot compare the change in the reflector. However, looking at pictures of the L1D I can see that the exterior has been updated to put a multi-sided surface around the head of the flashlight.  The surface makes the flashlight easier to hold and it prevents the light from rolling around when you put it down on a flat surface.

The LD10 still uses the Cree Q5 LED that the L1D used and doesn’t come with the R2 version used in the NiteCore EX10.  I can see a definite but not significant difference in brightness between my NiteCore EX10 and the LD10.

A really cool feature of the Fenix L series  is that the heads and bodies are swappable with other L series lights and most of the P series lights.  So, I can use the head of my LD10 with the body of a 2xCR123 Fenix P3D or PD30.  This will give me longer runtime and up to 220 lumens out of my LD10 head because the CR123 battery can source more current than a single AA.  So I can pack different bodies and carry what I need based on my plans.

Features of the LD10 include:

Conventional push button switch.
Durable aluminum body.
Access to 6 light levels – min, med, max, turbo, S.O.S, and strobe.
Waterproof.

The strobe mode only works at the turbo light level. This video shows off the different modes available on the Fenix LD and L1D series of lights.

Personal Impressions:
I’ve have the LD10 for for a few weeks now, almost as long as I’ve had my NiteCore EX10, and I’m happy with it.  Just like the EX10, the LD10 is solid, well built, and gives me loads of light on a single battery.  However, unlike the EX10, the LD10’s controls are easier to use and are much more reliable.  I always know where the light is going to start and I can easily switch between modes by soft-clicking the tailcap switch.

The fact that AA batteries don’t store as much power as CR123 batteries is readily apparent in this light.  I can get barely 20 minutes of light out at turbo mode before turbo mode is dims down to be just as bright as the high light mode.  The battery type doesn’t seem to matter, I’ve seen the same result using both lithum and eneloop brand rechargable batteries.  I don’t think we’ll see an improvements in the duration of light output until large gains are made in LED efficiency.

The low mode is great for getting around the house and the high/turbo mode is good for night walks and general outdoor night tasks. In addition, The strobe mode is great for biking or walking at night – it really grabs attention which ensures drivers will notice you. For hands free operation, Fenix sells both a bike mount and headband that you can use to attach the light to your bike or body.  Strobe mode is also great for alerting other drivers when you have to pull off the road in your car – the anti-roll design means that you can just turn the light on and put it on your car trunk or roof to alert fellow drivers.

I really like the fact that Fenix offers a variety of acceseories to modify their lights to fit your needs. Besides the interchangable bodies, headband, and bike mount I mentioned earlier; Fenix also has diffuser tips for general lighting, filters for night vision, and a pressure switch for remote activation.  It would be nice if they had a magnetic mount I could use to attach the light to my car or other metallic surfaces for when I need both hands free and don’t have a flat surface for the light.

Conclusion:
While a little bulky to use as a everyday carry light, the Fenix LD10 is great for travel and general use around the house and outdoors.  The AA form factor makes it cheap to operate but it also means that the maximum light output is unsustainable for long. I really like the strobe mode and available acesseories – I plan on at least getting the bike mount or headband for night biking during the winter.

Pros:

• Variety of accessories – headband, bike mount, bodies, and diffusers.
• Uses easily available AA batteries.
• Amazing light output for a 1xAA battery light.
• Reverse polarity protection.

Cons:

• Expensive!
• Very short lived turbo mode.

I’ve been using Komodo ever since version 3.5 came out several years ago. I first found out about it while looking for a Windows Perl distribution and happened upon the Activestate website. So, what is Komodo? Komodo is a IDE geared towards dynamic languages like Python/Perl/Tcl/Ruby that runs on all 3 major OS platforms (Windows, Linux, and OSX). It is put out by Activestate, the same company that puts out the most widely used Windows Perl distribution - Activestate Perl.  I bought Komodo because I liked the built in Perl debugger and syntax highlighting. I purchased the upgrade to versionMain Window showing debugger and variable viewers 4.0 because it added VI keybindings. Version 5.0 didn’t have any features I thought were compelling, but I purchased the upgrade anyway because I wanted to support Activestate.

Features:
Komodo is a feature rich editor. Besides the required syntax highlighting, it has editing of remote files over FTP and SFTP, version control integration, VI emulation mode, a very nifty graphical debugger, http request and response inspector and editor (HTTP inspector), source control integration, a regex constructor (RX toolkit), and an interactive shell that lets you try out commands on the fly. These are just the features I use on a regular basis! Komodo is also extensible via scripts, supports macros, customizable keyboard shortcuts, and much more.

Usability:
Komodo is very user friendly.  Besides the VI keybindings, you can also customize the menu by assigning your own key combinations to the commands you use the most.  The interface stays out of your way until you need it, the main typing window takes up most of the screen with the quick link buttons  listed along the top.  You can bring up the debugger or your source control windows at the bottom and there are side tabs to let you quickly access files in your project, variables, and functions.  One area that is lacking is the help – it is sparse in some areas, especially on how to access the API and so sometimes I have to resort to trial and error.

The syntax highlighting is great, it makes the code readable and there are little red squiggles show you where you have errors in your code.  A little drop down pops up when you access member variables, however it is not as comprehensive as Microsoft’s intellisense in that it doesn’t perform the drop-down for every variable.

A great feature I use regularly but is not found in most IDEs is the ability to edit remote files over FTP and SFTP.  This allows me to get syntax highlighting, code folding, code completion and all the other nifty features of the IDE on files that can only be accessed via SSH or telnet.  This is great because a lot of my work is done on UNIX machines that don’t come with fancy editors but allow access via SSH and telnet.

Komodo Regular Expression Editor

I mainly use Komodo for Perl and C/C++ development but I have dabbled with Ruby on Rails using Komodo and it also shines in this area.  It has shortcuts to automatically generate scaffolding items and you can watch your site execute using the built in debugger.

I’ve discussed the features I use the most and like the best but Komodo has many more features that will interest others who work with different languages or with different needs.  Check out their features page for a full list of everything Komodo can do.  Activestate also offers a trial version and has a lite version of their IDE in Komodo Edit.

Conclusion:
Overall, Komodo is great software and I would highly recommend it to anyone who works with dynamic languages.  I don’t regret buying it since it has paid me back many times in saving me time and reducing the frustration of debugging code.

Pros:

• Versatile and feature rich
• Easy to modify to suit your tastes
• Advanced debugging support
• Great code editing features
• Works on all major platforms – Windows, Mac, Linux

Cons:

• Expensive!
• Only supports dynamic languages

I have been looking for a small flashlight I can carry with me at night and use to light up dark areas.  After doing some research, I decided to purchase a NiteCore SmartPD flashlight from http://www.4sevens.com about two weeks ago.

Background:
The PD in SmartPD stands for Piston Drive.  Piston drive is a new switching technology that uses a piston to drive a switch on the flashlight head to communicate with the flashlight microcontroller.  The Piston Drive technology is supposed outlast the push button switches used on  conventional flashlights.  However, I don’t see why it would last longer since the Piston Drive also uses a switch that is built into the head to drive the flashlight.

NiteCore makes two flashlights with the PD technology, the D10 which uses 1 AA-battery and the EX10 uses a CR123 battery.  I chose the EX10 over the D10 because it is smaller and has a longer run time.

The exact model I got was the SmartPD EX10 R2.  The EX10 indicates that it takes CR123 batteries and R2 means that it uses a CREE R2 type LED.  The CREE R2 type LED in the EX10 is rated at 145 lumens which makes this one of the brightest small flashlights I could find.  This light has several advanced features including:

• Instantly jump to max or min brightness.
• Ramp to 100 light levels between min and max.
• Light turns on at the last light level.
• Waterproof.

The light can also be used in momentary or intermittent mode where the light is only lit when you push the piston down.  This video shows off the interface.

Personal Impressions:
The flashlight is is well built and feels solid in my hands.  I feel that I could drop it and it would still keep working.  The flashlight is unbelievably small, I can carry it around in the change pocket of my jeans and it will fit entirely in my palm.  The piston drive is stiff and has very little travel so it takes a while to get used to it. The max mode is very bright and will illuminate pretty much any room.  Does it outshine a 3-D LED maglight? No.  But is it bright enough to replace a 3-D maglight in most situations? Yes.  Likewise, the min mode is really dim and will really squeeze out every erg of energy out of a battery.  4sevens claims that the light is good for 90 minutes of illumination at max light mode – however, the 90 minutes counts till it is at 50% brightness.  In reality, the light is good for about 20-30 minutes of bright light on a fresh battery, after which it starts to noticeably dim.  This means that you have to use the bright mode sparingly and instead rely on a dimmer setting for most of your work unless you plan to replace your battery often.  Or, use the bright mode and then end up working with just a dim light until your battery runs out.

Occasionally, the light refuses to go into high/low mode using the shortcuts.  I think this is because the processor gets confused and needs to be reset.  This seems to be the case because it can be fixed by unscrewing till the light turns off to cut off power to the CPU and then screwing it back in.   Another annoyance is that the light is prone to roll around if placed on a flat surface which makes it unsuitable for any kind of work where you can’t hold it in your hand.  In addition, the black anodizing looks like it will easily wear off – I’ve already noticed a couple worn off specks just from carrying the light around in my jean’s change pocket.

Another caveat is that the microprocessor is always on so it draws current from the battery even when the light is turned off.  I’ve read that a new battery will power the microcontroller for ~400 days while the light is in standby.  I couldn’t find a statement on how much light I would get from the light after ~400 days or even ~200 days, I imagine it wouldn’t be much.  This makes the light unsuitable to use as an emergency light. Some say that you should store the battery separately, but do you want to be fumbling around for a battery in the dark when the light goes out or if you are in an emergency situation?  They may also tell me to unscrew the head a couple turns so that the head doesn’t make contact with the battery, but I think that should be unnecessary in a $60+ flashlight.  Just look at Fenix (a NiteCore competitor) which makes great flashlights that don’t drain batteries when turned off.

Summary:
The NiteCore EX10 is a nice small flashlight that is useful if you plan to use and carry it daily and don’t mind keeping it at low/medium most of the time in order to be able to access max mode when needed.  I wouldn’t recommend it for non-handheld uses since it has a tendency to roll around, or for long term storage in an emergency kit.  I like it because it is solid, tough, and has min/max brightness modes.  I don’t like it because the PD switch isn’t as great as it is made out to be – it is stiff and relies on a mechanical switch despite NiteCore’s hype about PD being superior to mechanical switches – and it will drain your battery if you are not careful.  Overall, I feel that this is an overly hyped light and I would recommend against purchasing it.

Pros:

• Small, tough, and waterproof.
• Extra bright when you need it.

Cons:

• Expensive!
• anodizing is prone  to wear off.
• Short lived max mode.
• Poor candidate for long term storage.
• Unknown long term reliability of piston drive.
• No reverse polarity protection.

Sources:
There is an extensive review of the D10 (AA-battery version of the EX10) and EX10 over at CandlePowerForums.  I consulted this review before choosing the EX10 over the D10.

I ordered my Toxic Green Deck 82 key keyboard in April of 2005 to go with my Razr Diamondback and the new PC I had built from scratch. I still use the PC and Deck keyboard daily, the Razr Diamondback sadly turned out to be an over-hyped piece of trash. The Deck keyboard is billed as tough and a easy to mod keyboard. The keyboard is also unique in that each key is back lit by a super-bright LED. I use the keyboard for 1-2 hours almost everyday at home. While this is not the most comfortable keyboard I’ve ever used, I have been impressed by its durability and construction.

Background:
I bought my Green Deck keyboard more than 4 years ago on April 10th, 2005 from ThinkGeek.com for $119.99. The Toxic Green Deck keyboard had just been released and it was in high demand. ThinkGeek no longer carries the keyboard but it is still available through Deck Keyboards for almost the same price I paid years ago – $119.

Description:
Unlike some manufacturers that try to cut costs by painting letters onto the keys, each key in a Deck keyboard has its letter imprinted using a ’sublimated negative printing process’, this means that the letter goes all the way through the key and will only become unreadable if the key wears through. Each key uses a ‘Cherry MX1A-11NN linear switch’ which is rated for ~50 million key presses or 20 years of use (according to the website). Each key is backlit by a super-bright LED (green in my case), the spacebar has two LEDs, and the caps lock key has an additional blue LED to indicate when it is active. The LEDs are readily visible during the day and especially in the dark which really makes the keyboard stand out. The brightness of the LEDs can be adjusted to one of 7 levels, 8 if you count off, by depressing the ‘FN’ key along with one of the 0-7 number keys. The brightness controls come in handy for when you want to turn the light down in order to watch a movie or when you are trying to avoid distraction. They keyboard’s edge housing is made of a very tough polycarbonate plastic and can be interchanged with other colored face plates available on the Deck website. The bottom of the keyboard consists of an approximately 1/8th inch thick metal plate that is screwed into the polycarbonate casing.

Mods:
The Deck website really plays up the modifications (mods) possible on the keyboard. The Deck website sells two main mods for the Deck keyboard. One is different colored polycarbonate casings and the other is key caps with different logos on them. Other mods include opening up the keyboard and changing the LEDs to different colors or performing different kinds of paint jobs. Deck even covers modded keyboards under its warranty – one caveat is that the modifications have to of good quality and performed with competence. In the 4 years I have owned the keyboard, I have not made any changes to it – it is flashy enough the way it is and I don’t feel the urge to spend extra money on new keys or casings.

Keys:
The cherry switches used on the keyboard are quiet, they don’t click like the switches on a Model M or Kinesis Contoured Keyboard. You only hear a click when the key hits the the base of the switch. The activation threshold on the switches is very low, so it is possible to type by lightly depressing the keys. This requires more concentration since you don’t get much tactile or audio feedback, but it allows one to type silently. The total key travel is deeper than on other keyboards I’ve used, this can lead to fatigue especially when you are are engrossed and start to pound at the keyboard.
As expected, the ’sublimated’ lettering on the keyboard is very durable. Unlike most keyboards I’ve had, none of the letters have faded despite daily use. The keys are made of a hard, smooth plastic and they feel like they will last.

Usage:
The Deck Keyboard is a very solid keyboard. I use it regularly and its solid nature and silent keys make you want to pound at it in order to get feedback. It has stood up well over the last three years, all of the LEDs still work and it feels the same as the day I bought it.  You have to watch out for the screws on the bottom of the keyboard or they will scratch your desk if you drag the keyboard.

Ergonomics:
The Deck Keyboard doesn’t have the best ergonomics. It is a straight keyboard which means that typing on it stresses your wrists. The keyboard is also has a 7 degree tilt and is about 3/4 inches high. The Deck website claims that this was an overwhelming choice by their focus group. However, the extra tilt and height puts more strain on my wrists since I have to bend them back further in order to hit the keys.
In addition, the deep key travel quickly leads to fatigue if you start treating it like a regular keyboard and push the keys down until you hear a audible click.

Customer Service:
Early in its life, my keyboard would sometimes double print a character – that is if I typed in ‘a’ it would show up as ‘aa’ on the screen. I decided to contact Deck since the keyboard was still within its 1 year warranty period. They responded promptly and asked me to send the keyboard back. They never found anything wrong with the keyboard, however they said that they cleaned it before sending it back. This cleaning, whatever they did, fixed the problem and my keyboard has been trouble free since. I was impressed by their quick response and communication throughout.

Lighting:
The lighting on the keyboard is great, it is readily visible during the day and really lights up a dark room.  One thing to note is that the lighting is not even at low brightness levels.  For example, on my keyboard the ‘k’ key really stands out at lighting level 1 and doesn’t blend in with the rest of the keys until I get to light level 4 or 5.  Similarly, a few other keys stick out at low light levels but this isn’t as noticeable as on my ‘k’ letter key.  This is understandable since the LEDs used in the keyboard probably vary in their forward bias voltages, this sort of variation is common in semiconductor manufacturing since it is hard to create completely identical units during die fabrication.  I’m mentioning it here because it was a small annoyance when I got my keyboard.  The annoyance has since faded as I’ve come to see it as a uniqueness.

Summary:
Overall the keyboard is very strongly constructed and feels like a quality product. it is not very heavy but you can feel the sturdiness when you are typing on it. However, it has poor ergonomics and so isn’t suited for all day use.

• Pros:
  - Solid keyboard that will add pizazz to your work environment
  - Excellent customer support
  - Easy to modify to suit your tastes
  - 20+ year lifespan (claimed)

• Cons:
  - Expensive!
  - Poor ergonomics
  - Uneven lighting at low brightness levels

I was having a tough time finding how to display database data from a primary key in a drop-down and then have the selected item show up in my business class object when it returned.

I wanted to do this using an Objectdatasource because I had to reference several different tables in order to display my data.  As a newbie, I spent several days struggling on it when I found this set of tutorials on Microsoft’s MSDN site, specifically tutorial 20.  These tutorials nicely cover the basics of data manipulation in ASP.NET using C# and the Objectdatasource.

I hope this helps other like me who are struggling with the same problem.

The Kinesis Contoured Keyboard is billed as the best available design for both comfort and productivity.  Since I spend most of my time at work on a computer, anything that enhances my comfort and productivity is always welcome.  I decided to try one out just to see if it lived up to its billing and to find out if it would help me with the wrist pain I experience after a long day at work.  I’m sitting here typing this on a contoured keyboard I bought more than 6 months ago and have been using for 8+ hours a day and I can say that it definitely lives up to its billing.

Background:

I bought my keyboard, a near mint condition Professional QD, online on Ebay for about $200.  In addition to the fancy split bowl layout, the keyboard comes with high-tech built in electronics: it can be hardware switched between QWERTY and Dvorak layouts, you can re-map any key, create macros, have the keyboard beep on every keypress, and you can even attach a foot switch for your control keys.  The Kinesis website no longer lists the Professional QD, the closest match I can find is the Advantage Pro.  You may think that $200 is a crazy price to pay for a used keyboard, but they cost even more brand new!  Prices on the Kinesis website range from $289 to $359 + shipping.  The Human Solution has the mid-level keyboard for about $269 and you can find good deals on E-bay.  So, it is wise to shop around.  The price is high but you have to consider that these are high quality keyboards that are probably custom built in small batches, they have fancy electronics, and the company also has to make a profit.

Description:

The Kinesis Contoured Keyboard is split down the middle into two scooped out halves.  There is a wide gap between the two halves in order to make you correctly position your shoulders.  Your wrists rest on the bottom half of the keyboard and your fingers rest into the scooped out halves.  According to the Kinesis website, this downward rest is “the best available design for both comfort and productivity“.  The keys most used by your pinky (Enter, Backspace, Del e.t.c) have been moved to the bottom center of the keyboard so they can be operated by your thumbs.   This makes sense since the pinky is the weakest finger and the thumb is the strongest.  The arrow, insert, and other keys have been moved around so that you can operate them with your fingers without moving your hands.  The function keys remain in their normal spot at the top of the keyboard.

The keyboard as well as the keys are made of a hard plastic.  The white keyboard picks up stains easily and requires periodic cleaning if you dislike the grunge look.  The home keys are colored light blue.

Usage:

If you are a proficient touch typist like me, it will take about two weeks to get used to this keyboard.  The hardest part is getting used to hitting space, enter, backspace, and delete with your thumbs.  Another tough hurdle is learning to use the arrow keys with both of your index and middle fingers.  It is slow going for those first two weeks but your brain figures it out and you catch up rapidly after that.

My typing on a regular keyboard is largely unaffected.  I make some mistakes when I first switch over, but my brain warms up and I start typing normally after a few minutes.  I mostly get the space-bar and enter keys mixed up.

Keys:

The keys on the keyboard are made of a hard plastic.  They have a very nice clicky feedback, they are not quiet though – my cubemates have commented that they can hear them when things are quiet.  The force required to activate the key is not large like on the IBM Model M keyboard, they need a gentle but firm tap.  Neither are they hyper-sensitive like those on a Deck keyboard.  You have to push them down till they click before the computer registers a key-press.  The keys and switches feel high quality, however the QWERTY letters on the keys feel like they are glued on and the Dvorak characters are painted onto the keys.  I doubt the lettering will last.  I like the negative printing process on the Deck keyboards that embeds the character into the key and makes it impervious to wear.  I think for $300 Kinesis could have put more durable lettering on their keys.

The function keys are rubber with the same painted-on feeling letters and they have a noticeable wobble in their sockets.  They have a lighter, more rubbery clicky tactile feedback.  The function keys are about 1/2 the size of a function key on a regular keyboard, this is probably done in order to include the rarely used keys like Scroll Lock and Pause/Break along the top of the keyboard.

One oddity on my keyboard was that the backspace key was titled ‘Space’.  I only discovered its function by trial and error.  There are keys named space on both sides of the keyboard and my guess it they named both the space and backspace keys ‘Space’ to allow switching them to either one of your thumbs.

Ergonomics:

As advertised on the Kinesis website, this keyboard has excellent ergonomics.  I used to experience wrist pain when using my regular flat keyboard for long periods of time, the pain has mostly gone away since I switched and only returns if I spend extremely long periods, say 12+ hours, at my computer.  In addition, the bowl like shape makes my shorter fingers travel less than the longer ones which has also reduced fatigue.

There are some drawbacks though.   Any kind of one-handed typing is difficult.  The keyboard is harder to use for programming since the bracket keys are at the bottom which makes them hard to access.  The split arrow keys require two hands to operate which makes using the arrow keys and the mouse at the same time impossible.

Final thoughts:

I’ve used many keyboards including the Microsoft ergonomic split keyboards, the Happy Hacker, the Deck, the Keyovations Goldtouch, as well as the IBM Model M.  Of all the keyboards, the Kinesis contoured keyboard is the most comfortable and usable for long periods of time.  I think it is worth the price after you consider the damage you can do to your hands with the wrong keyboard.  I make my livelihood with my hands and I would gladly pay full price if I had to purchase it all over again.

• Pros:
  • Very comfortable for daily typing.
  • Highly customizable with the built in macro support.  Settings are preserved even when the keyboard is unplugged.
  • Switches between Dvorak and QUERTY layouts.
  • Ergonomic layout allocates most-often used keys to the thumb, your strongest fingers, and separates your arms to about shoulder width.
  • Long USB cord.
  • Expandable with factory upgrades and foot switches.

• Cons:
  • Expensive!
  • Programming guide should be printed on the bottom of the keyboard.
  • I don’t expect the keyboard lettering to last.

Images:

I have a VPS at Slicehost that I use to play around with Linux, it is much easier and convenient to use their remote service than dealing with the security, noise, and heat of running my own machine at home.  However their VPS doesn’t come with much disk space, the base 256MB plan only allows for 10GB of disk space.  So, I needed a way to offload my files so that I can shutdown my server when I don’t need it.  Slicehost pro-rates their charges so I don’t keep my VPS active unless I’m using it which saves me money.  I did a lot of research before settling on s3sync to save my files.  I tried using Jets3t but it requires Java which was slow and it is GUI based which means I had to run VNC to access it.  s3sync runs on Ruby and is on the command line which makes it perfect for running over SSH.

s3sync consists of two utilities, s3sync and s3cmd.  s3sync is a utility that will keep folders synchronized between Amazon S3 and your hard drive.  This utility is perfect for syncing your personal directories with Amazon for backups or to keep your development and production sites in sync.  s3cmd is billed as a counterpart to s3sync for managing your Amazon S3 account, however it is also great as a standalone tools for uploading, downloading, deleting, and managing single files.  I mostly use s3cmd on my VPS and that is what I’ll focus on here.

Assuming you are staring from a fresh installation, first install Ruby.  From Slicehost’s Ubuntu Ruby-on-Rails guide:

apt-get install ruby1.8-dev ruby1.8 ri1.8 rdoc1.8 irb1.8 libreadline-ruby1.8 libruby1.8 libopenssl-ruby

Then, install the symlinks:

sudo ln -s /usr/bin/ruby1.8 /usr/bin/ruby
sudo ln -s /usr/bin/ri1.8 /usr/bin/ri
sudo ln -s /usr/bin/rdoc1.8 /usr/bin/rdoc
sudo ln -s /usr/bin/irb1.8 /usr/bin/irb

Now, download, unzip, and untar s3sync:

wget http://s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz
gunzip s3sync.tar.gz
tar -xvvf s3sync.tar

You should now have a directory called s3sync containing the s3sync files.  Create a directory called .s3conf and copy the s3confg.yml.example file to it as s3config.yml.  This is where s3sync will look for its configuration options.  The README file lists additional locations you can specify the configuration options.

mkdir .s3conf
cp ./s3sync/s3config.yml.example ./.s3conf/s3config.yml

Finally, open .s3conf/s3config.yml and edit the aws_access_key_id and aws_secret_access_key.  Set them to your S3 access key and S3 secret access key respectively.

You are now set use s3cmd and s3sync.  Run the utilities without any arguments to get a listing of the options and commands.  I have found the -v (verbose) and –progress (progress bar) options to be great in giving me an update on what is going on during the file load/download process.  Some examples:

s3cmd.rb listbuckets  –> Lists all your buckets
s3cmd.rb createbucket/deletebucket –> Create and delete buckets
s3cmd.rb list <bucket> –> Lists all tokens in bucket
s3cmd.rb get/put <bucket>:key filename –> Copies filename from/to S3

s3sync can be used to synchronize local folders with buckets:

./s3sync.rb <bucket>:token local_dir –> sync your bucket with the local dir.  The –make-dirs option will create local directories as needed for first time downloads.

You will see that s3sync and s3cmd are very capable tools for interacting with S3.  Besides the command line, they are also scriptable and you can set up regular syncs and uploads/downloads via cron.  They have given me access to virtually unlimited cheap disk space and simplified the complexity of managing my files.

This is the first of several articles I’m going to write about Amazon S3.  I’ve done a lot of research and cheap, secure, reliable, and publicly hostable disk space is hard to come by.  Sure, there are some companies such as dreamhost and mediatemple that offer enormous amounts of disk space at a low price but often they put restrictions on the file size, locations, and types of files you can save.  Other companies even give it away for free, but I wonder how they plan to survive by giving their product away for free.  Ads may pay for some of their costs initially, but I doubt they will pay enough as they scale up their operation.  I have a feeling that many of them will simply disappear along with the data entrusted to them.  I invite you to come back to this post in a year and see how many of these hosts are still around.

My search for a cheap, reliable, and secure place to backup my data ended when I found out about Amazon S3 (Simple Storage Service).  S3 is touted as a data storage service and that is all that it is.  There is no limitation on the types or quantities of files one can store.  They charge you on a pay as you go plan and there are no upload and download limits. You simply pay for what you use.  Amazon’s own website is a testament to their experience in building massively scalable services.  These are their rates at this time:

Storage
$0.15 per GB-Month of storage used

Data Transfer
$0.100 per GB – all data transfer in
$0.170 per GB – first 10 TB / month data transfer out

Requests
$0.01 per 1,000 PUT, POST, or LIST requests
$0.01 per 10,000 GET and all other requests*

Considering how much other hosts charge, their rates are very reasonable.  Their storage and transfer rates have been coming down and I think this trend will continue.

Amazon has built a very simple API with a high level of abstraction for its S3 service.  Data is stored in ‘buckets’ and each bucket can contain an unlimited number of objects up to 5GB in size.  Objects can be made public or private which makes it ideal for hosting large files as well as using it for backing up your data.  In addition, you can choose to store your data in either Europe or the United States which gives your data an additional level of geo-redundancy and allows you to deliver files to users around the globe with minimal latency.

As a results of this simple API, Amazon S3 is very versatile and has a diverse eco-system built up around it.  A simple search for amazon s3 tools turns up several tools that abstract away the S3 API and simplify the user experience.  Of these, s3sync and JetS3t are my favorite.  I use jetS3t to back-up important files on my computer to S3 and s3cmd from the s3sync package to upload files I want to share.

In future columns, I’ll write more about these tools and how I use them to simplify my life.

A while back I wrote an article of how I came to choose RailsPlayground for my next host.  I have been their customer for the past 3 months and I thought I’d write a review for those that are interested.  After looking at several hosts, I chose RailsPlayground because of their rails support, ssh access, excellent support, and no limitation on how I use my disk space.

Signup was painless, I entered my information and credit card info on April 4 at about 9 AM and I received my log-in information at 4PM on the same day.  These are the major details of my hosting plan, The Developer Plan:

• Storage: 5GB
• Bandwidth: 30GB/month
• Shell Accounts: 1
• Cost: $5/month

In addition, I also receive the following:

• Access to PERL, PHP, Ruby, Python, TCL, and Ruby on Rails in your shell
• Unlimited domains, sub-domains, e-mail accounts e.t.c
• Postgres and MySQL databases
• Nightly backups
• 1GB storage/10GB transfer SVN and Trac hosting

With all this, no wonder it is called the Developer Plan! It has everything a developer needs to get started with Rails for web development and work with their favorite language.  The SVN and Trac hosting makes version control and issue tracking for yourself or your team easy.  According to the description page, their “latest server features Dual Opteron processors with 2 GB of RAM and Raptor 10k RAID 5 protected hard drives.” They don’t list how this maps to their other servers.

Hosting:

So far, their hosting has been solid.  I’ve never experienced a slow connection and my server, baldr, has been up longer than I have been a subscriber.  My machine is lightly loaded, I’ve seen a load average of 1 only once and it was due to a run-away process.  You can always check its uptime on my uptime page or visit my main site.

Control Panel:

RailsPlayground uses CPanel as its control panel.  The panel has all the features you need to manage your account without ever logging into the shell account.  You can add/delete e-mails addresses, databases, backups, domains, sub-domains, and everything else that involves web hosting.  The panel includes Fantastico, a one-click installer for many free popular programs such as blogs, wikis, and CRM software.

Support:

I’ve had to contact RailsPlayground twice and both times I’ve received excellent support.  The first time was to report a run-away process, they answered my e-mail 10 minutes after I sent it and they killed the process right away.  The second time, I e-mailed them to set up the SVN account that is included with my hosting package, I sent the email at 9:42PM and they had it ready less than 9 minutes later – I received the confirmation e-mail at 9:51PM!  I am very happy with their support.

Documentation:

I haven’t used the documentation at RailsPlayground much.  In addition to contacting support, you can also have a wiki for articles and a forum for questions.  I once had trouble setting up an e-mail address for automatic notifications, they have a very nice tutorial for it in their wiki that I was easily able to follow that fixed my problems.

Shell Access:

One of my deciding factors for choosing RailsPlayground was their shell access.  The developer account gives you 1 shell account and this one account is sufficient for me.  The shell is accessible via SSH and they put you in a Jail-shell to limit potential damage, that has never gotten in my way.  I haven’t seen a time limit, I’ve been logged into my shell days at a time.  Likewise, I’ve seen no process limitations, I have several cron jobs that run from once daily to every half-hour and I’ve never seen them delayed or stopped.  From the shell you have access to everything you would in a standard UNIX environment – vim, emacs, cron, lynx e.t.c.  One thing I noticed is that they don’t have screen installed so you have to log into multiple sessions if you plan on doing any serious editing.

Final Thoughts:

I’m very pleased with the features, hosting, and support at RailsPlayground.  Rather than overselling and trying to hook customers on quantity, they deliver quality and strive to please.  They have features that fit all users, shell access makes it easy for experts and CPanel makes it easy for newbies.  The multiple language support make it easy for users to develop applications in a familiar environment.  If you don’t believe it, feel free to visit my site or my uptime page to see how they are doing .