Data Storage and Backup Headaches

If you have been shooting digital for a few years, you will face the storage and backup problem sooner or later. If you shoot RAW, storage becomes more serious. If you shoot RAW + jpg like me, the situation is even worse.

I did a rough calculation last night, when I was copying files to my new Drobo, my G1 pictures took up about 175 GB of space, in the span of 4 months. At this rate, a terabyte hard drive will be good for a couple of year, if I don't do any edits.

In the past, I bought drives as needed without much regard to data safety. Even though I have been messing around on and off with SCSI RAID setup since the early 90s, and later IDE RAIDs, mostly for speed (striped), data safety didn't register in my head. At least I didn't take action on it, until few years ago. Two of my primary hard drives blew up due to a faulty power supply. More than 200GB of pictures, which were not backed up, were lost. Still have the 250GB dead drive lying around and I am hoping one day I can afford to recover the data off of it.

Since then, I took some action, like getting a workstation class of computers (Dual Xeon Dell) with good reliability, uninterpretable power supply (UPS) with power conditioning, and small steps in burning data on DVD/Bluray. Later on I tried the NetGear NAS (SC-101) and Western Digital World Edition, but both are disappointing.

Netgear NAS SC-101 -- One of the very first affordable NAS and Took IDE drives with data mirroring feature. It never worked for me. It was SLOW, unreliable network connections, has no local connection, and has no internal fan for cooling and consequently overheats easily. Average speed was about 3 MB/s. Barely usable as a backup solution, but for primary storage, forget about it.

Western Digital World Editing -- With 1 Tera Byte capacity (two 500GB) that also has mirroring feature. Unfortunately, this one suffers the same limitations as the Netgear -- Slow. It's a lot more reliable, has internal cooling fans and relatively quiet. Average speed is about 6 MB/s. With a gigabit network, this is just unbelievably slow. So, again, it's used only as a backup solution.

Enter the Drobo storage robot. This thing is a godsend. First, it does RAID 5 configuration to provide data redundancy without needing same size drives, and second, you can add larger drives to increase storage without reconfiguring the RAID setup. Just plug in the drive, and the Drobo does the rest.

I have the second generation of the Drobo which has both a firewire 800 and USB 2.0 connection. If you want to turn this into a Network Attached Storage (NAS), you can buy a DroboShare. I am currently connecting it to my computer through the USB port. It is more than fast enough for me, averaging about 20MB/s, about twice that speed with Firewire 800.

I started with two Western Digital 1 Terabyte Drives. This gives me roughly 1 Terabyte of storage, since Drobo uses the space equivalent to the space of the largest drive you have in Drobo to mirror the data. But, if I add another 1 Terabyte drive, I will have 2 TB of space, the space needed to mirror the data does not increase.

For those of you who have a conventional RAID setup will know, you need exact same size drives to mirror the data in a RAID. This is not a real problem for large companies, which typically have a few spare drives on hand at any time, in case one or more drives die. But for a home user, having a couple of drive lying around not used is expensive and unproductive. So, most home users do not have spare drives for their RAID setup. This is OK until a few years later, when one of your drives die, and you need replacement. Now comes the headache. Where to find a 3 year old drive that's the same as the one you are using. You can not buy a new one on the market, because technology advances so fast. Hard drive models usually have 1 to 1.5 years of shelf life. What to do?

The Drobo, on the other hand, can mirror data without the same size drives. If one of your drives die in a few years, just yank out the dead one, and pop in a larger capacity drive. Voila! Your data will be rebuilt automatically, and you can even do this while Drobo is still actively serving data. That's the beauty of Drobo, and is what sold me.

There is only one drawback. High initial cost. The Drobo Firewire 800/USB 2.0 edition costs about $500CAD, without drives, when not on sale. A couple of 1 TB drives costs another $200. This adds up to a lot of money for a home user, especially when you factor in the taxes. So, you really have to consider the balance between costs and data protection as well as future expandability.

Personally, I look at it this way -- Buying the Drobo is like buying a lens. Hard drives are like camera bodies. The initial cost is high, but the storage cost is incremental. You add more space as needed, for a maximum of 16 TB. I think that will last quite a few years, until holographic storage becomes a reality.