Making RAID easier

Submitted by brad on Tue, 2008-05-27 21:49
Topic: 
Technology

Hard disks fail. If you prepared properly, you have a backup, or you swap out disks when they first start reporting problems. If you prepare really well you have offsite backup (which is getting easier and easier to do over the internet.)

One way to protect yourself from disk failures is RAID, especially RAID-5. With RAID, several disks act together as one. The simplest protecting RAID, RAID-1, just has 2 disks which work in parallel, known as mirroring. Everything you write is copied to both. If one fails, you still have the other, with all your data. It's good, but twice as expensive.

RAID-5 is cleverer. It uses 3 or more disks, and uses error correction techniques so that you can store, for example, 2 disks worth of data on 3 disks. So it's only 50% more expensive. RAID-5 can be done with many more disks -- for example with 5 disks you get 4 disks worth of data, and it's only 25% more expensive. However, having 5 disks is beyond most systems and has its own secret risk -- if 2 of the 5 disks fail at once -- and this does happen -- you lose all 4 disks worth of data, not just 2 disks worth. (RAID-6 for really large arrays of disks, survives 2 failures but not 3.)

Now most people who put in RAID do it for more than data protection. After all, good sysadmins are doing regular backups. They do it because with RAID, the computer doesn't even stop when a disk fails. You connect up a new disk live to the computer (which you can do with some systems) and it is recreated from the working disks, and you never miss a beat. This is pretty important with a major server.

But RAID has value to those who are not in the 99.99% uptime community. Those who are not good at doing manual backups, but who want to be protected from the inevitable disk failures. Today it is hard to set up, or expensive, or both. There are some external boxes like the "readynas" that make it reasonably easy for external disks, but they don't have the bandwidth to be your full time disks.

RAID-5 on old IDE systems was hard, they usually could truly talk to only 2 disks at a time. The new SATA bus is much better, as many motherboards have 4 connectors, though soon one will be required by blu-ray drives. What I propose is readymade RAID modules for PCs, with slightly different goals than found in the commercial RAID market. In particular, it is possible to build a module that fits in a 5.25" DVD drive bay which holds 2, and possibly 3 drive mechanisms from more standard 3.5" drives. It is necessary to use standard mechanisms to make it cheap. Holding 3 is difficult as we would need to use slightly slimmer drives, which are not so readily available, at least for one of the drives. An alternate and useful product would have 2 3.5" drives (mounted sideways) and a slimline DVD drive on top. Such slimline DVD drives are commonly made for laptops.

Inside the module would be one piece board of drive electronics controlling the 2 drives, and possibly performing RAID duties. That's cheaper, but considered a no-no in the raid world because the failure of the drive electronics would cause the failure of the RAID. Real RAID tries to avoid any single points of failure. However, dead drive electronics rarely cause loss of data, they just cause loss of ability to get at the data. If you can buy and swap in replacement electronics to get back up, it's reasonably good.

The basic unit would offer RAID-1 (or raid-0 which is a double fast disk with no protection from failure, if desired.) However, it would also have two SATA connectors on it, which could then connect to one or two regular disks in other bays, to do 3 or 4 disk RAID-5. Or they could connect to another dual-drive array similar to the first one, for a 4-disk RAID-5 in two CD/DVD bays. (Admittedly many cases don't have two CD/DVD bays any more so the former may become a more common choice.)

To be more reliable, the two drive mechanisms in any 2-pack would come from two different drive manufacturers. This is important because sometimes drives from the same batch from the same maker will have the same flaw, and could fail together. The extra drives would of course come from different sources.

Unlike most RAID designs, it would not necessarily be easy to replace one drive in the 2-pack if one failed. It might make sense to require users to replace the entire 2-pack, since this would make it cheaper to manufacture. Why?

  • Usually by the time a drive fails in a RAID, drive technology has improved, and current drives are bigger/cheaper/faster/greener.
  • Drives are cheap. If one has failed after a few years of service, might as well replace the stack.
  • Most users will not be up to doing such a replacement, though service centers could.

The system would be designed to facilitate the double replacement by connecting the old pack (with one bad drive) and any other drives to the new 2-pack.

Ideally, the system would also allow you to connect 2 smaller drives to the 3-pack, for a special RAID-5 that consists of 2 large disks and 2-smaller disks striped to be the 3rd component. No matter what disks you have, no matter what their sizes, it should figure out the best thing to do and do it, and of course handle drives going bad and being replaced. It must use the OS to warn you and guide you through the replacement.

For many users, there would be an "upgrade" process: Take a standard system running on a single drive, then insert the 2-pack and plug the old drive into the 2-pack. It would then build a RAID-5 from the 3 disks, plus create a second partitions that are normal or RAID-1 if the disks are different in size.

Disks are cheap enough and data valuable enough that there's really no excuse for building all our systems on vulnerable single drives. This should be standard. Of course it can be done with regular drives and software RAID (which is how I've done it) but this is beyond most users.

In fact, over time, the drive industry should move to a new form factor that makes a cheap and simple 3-pack drive which fits in existing bays, and make this be the standard sale. They would be tempted to build the 3-packs all from their own parts. As noted this runs the risk of a batch being bad, though it's still a lot better than what we have today.

Some further notes:

  • People must not forget that regular backups and offsite backups are important. We still have more lost file incidents due to software bugs and accidental deletes than we do to hard drive failures, though the latter cause many more losses per incident.
  • Yes, this is not as green, since 3 drives consumes more power than one. Efforts to make the 2 or 3-packs more energy efficient should be part of this.
  • As an alternate, the 2-pack might work with software RAID, providing hardware to help the software RAID. The extra drive or drives outside the 2-pack would still talk directly to the system via their original connection. Right now 3 gigabit SATA is more than up to this challenge.
Share on:
  • Reddit logo
  • Facebook logo
  • Twitter logo
  • LinkedIn logo

Comments

Moz

Wed, 2008-05-28 04:31

I've seen units that stack 4 3.5" disks into 3 5" bays, but I think your maths is dodgy if you expect to get them into two. With 2.5" disks that's not a problem size wise, just cost wise. But then, for general use most users can get by with ~.5TB from a few of the current 200GB-ish 2.5" drives.

There are increasingly many external NAS boxes that do RAID5, some of them quite well. For most users these make more sense, especially then genuine NAS ones that allow multiple users to connect rather than the cheaper USB-only ones (why anyone would buy a disk where filling it takes a week I do not know... but people do the same with USB keys).

I think the better solution is those two combined - run a pair of small-ish disks mirrored by your motherboard plus an external big disk. That way a single disk (or mobo) failure does not kill you, and even if the PC dies you will probably still have the external media disk in usable condition. With NAS, I would probably go for some sort of near-live backup, probably daily replication to a second copy of the live array. Buying two NAS boxes sounds expensive until you look at the cost of reacquiring 2TB or more of data (even just scrounging recorded TV shows off friends will take a while).

There are also increasingly affordable PCIe RAID cards that don't completely suck, and even a few motherboards that do it too. When I bought my current PC and extra $50 or so got me a "super deluxe" motherboard with RAID1 (and claytons RAID) built in, and while it chokes at about 50MB/s I still use it to mirror a pair of drives for booting. These days a mobo with 8 SATA ports is not uncommon, and I believe some of them even have dedicated RAID processors (although with quad core that may not be so necessary any more).

At the higher end, I like quiet PCs so I bought a big case with slow fans and lots of bays (P180), plus an expensive RAID card (Areca 4 port). That let me start with 4x250GB disks then upgrade some time later to 4x750GB (tripling disk capacity means I can copy the old array onto one disk, build a new array with the other three and copy onto it, then add the fourth disk, so I always have two copies of the data). Then when 1TB disks came out I bought a couple of 2TB external backup units (two 1TB disks in claytons RAID) and a FireWire 800 card so they ran a little faster.

My lessons from that experience are mostly to do with buying excess capacity in non-upgradeable components - I should have spent 50% more and got an 8 port card so that I can just keep adding disks. Areca offer RAID6 even on the four port cards, but with 8 ports it makes sense to bulk up a 4 disk RAID5 to 6-8 disk RAID6 once drive prices drop (if you have the bays for it).

Oh, and TrueCrypt currently sucks when combined with RAID because it serialises access. But I do not want 2TB of every file I've got falling into the hands of some random burglar or sneak thief. 2TB via USB2 is more than two days, with FireWire800 it's under a day. That $50 card was a bargain :) When I bought the first 2TB disk the first thing I did was copy my current system across to it... it took a very long time. The second one was much faster with the better connection. And much cheaper, thanks to an online pricing error... $450 instead of the $800 it should have been or the $990 I paid 3 months before that.

brad

Wed, 2008-05-28 11:40

Even bought one of those multi-bay units but it didn't fit well in my case.

There are 3.5" disks that are less than 5.25 inches long, and thus will fit sideways in a 5.25" bay. They are just slightly longer this way than many CD-rom drives so should fit in most systems. The electronics add very slightly to the length, and the connectors add a bunch, but in this case I am proposing different electronics and thus no connector issue. And there is room for a low-profile laptop style DVD drive on top.

This is useful because you get one unit you can slot into a PC's DVD bay, and get Raid-1 and DVD all in one box. I will contend that every PC needs this. Add another drive for RAID-5, though probably with some software assistance for the conversion of the raid-1 to a raid-5.

The goal is to make it cheap and easy. It's only cheap of course if this does become standard and everybody is buying it.

With today's multi-core processors, I like software raid but as yet it's not nearly simple enough for most computer users. Hell, it's too much work for most advanced admins. An ideal would be hardware raid with full software instrumentation so the software can control it, configure it etc. but doesn't have to do so except for complex operations (like convert Raid-1 to Raid-5).

For servers on all the time, power is a factor. Here it's $8 per watt for a device on for 3 years. An 8w drive uses $64 of power, a raid controller even more. That does make RAID power-expensive, since the drives cost only a bit more than that. (750gb for $99 last weekend at Fry's) This makes RAID-6 or frankly even 4-disk Raid-5 seem less practical for personal machines.

Moz

Wed, 2008-05-28 04:37

My approach to greening the system is to turn it off.

A great deal of what I use the computer for can be done using a second hand ex-corporate laptop, so I do that in the mornings and after work, then once I've had dinner I crank up the big machine. Although most of the power actually goes into dual 20" monitors rather than the PC. According to the UPS the monitors run about 50W each on ~80% brightness (to the nearest 3W) while the PC uses about 60W when I'm browsing and 80W when the RAID disks are all spun up (more when defragging, about 230W when defragging all disks and running the Prime CPU torture test).

The laptop uses about 20W. It has just enough disk to run Windows and uTorrent, enough RAM that when I have to I can run PhotoShop, but mostly it just idles along.

So 20W to browse... 200W to play C-Evo :)

jess

Wed, 2008-05-28 11:44

it's nice that you worry so much about using your laptop instead of a regular PC and monitor, but let's be honest, the energy you use to power your pc is probably pretty low in comparison to other devices (lights, AC, and so) on actually contribute.

as to brad's post. i definitely think it sucks big time you cannot buy and isntall new components for laptops as easily as for regular pc's. ia m sure that somehow this would be technically possible, but then the companies would sell fewer new laptops...

Moz

Wed, 2008-05-28 15:18

Jess, we don't have air conditioning and we have a total of 10 light bulbs in the apartment, all compact fluoros and rarely do we run them all at the same time. Our major electricity use is the fridge, and after that it's computers. Since we mostly cook with gas the electric stove doesn't use much power either. While we're on that topic, our transport emissions mostly come from the food we buy, since we all ride bikes and occasionally use public transport (yes, we carefully selected an apartment where that is possible. It didn't even cost more, because the average person doesn't yet consider it important).

Jay Cuthrell

Wed, 2008-05-28 17:14

I still like the concept of the Drobo. Folks that I know with them seem very content to use them. These are the same folks that deal with storage for a living too so it isn't that they don't want to roll their own... they just choose a simpler path.

brad

Wed, 2008-05-28 19:26

USB (and certainly 100 mbit ether) won't cut it for your main disk. They are fine for remote servers for files that do not need to be accessed quickly, such as video. I would not want it for photo storage. USB seems to only pull about 30 megabytes/second. A 3 disk RAID-5 should be able to give you about 140 megabytes/second sustained read, a 4-disk even more.

And that is too much even for gigabit ether, and would saturate the gigE while doing so.

An external box using eSata would make sense. The 3 gigabits there should keep you happy. Such a box could then also have gigE for access by other computers.

However, most people don't want to have an external box for their disks.

Now, one reasonable architecture is a regular disk in the PC, and regular mirroring to a NAS box. In that case you are fine with the ethernet, even at 100mbps. However, you should also be looking into offsite.

One product that might make sense is a NAS box which not only provides reliable storage and mirror of the fast disk in each PC, but also which can be "paired" with a friend's similar NAS box in a remote location. The two boxes would then do offsite backup for one another, automatically. Using an efficient incremental algorithm. Of course, your data would be encrypted on the friend's box with a key stored in a 3rd place, as well as on your home box.

In such a system, there would be a special disk allocation system, mounted on two drives. New files on the server would be stored twice, once on each disk. Files from local PCs would be stored only once (there's already a copy on the PC.) Over time, new files would be mirrored to the offsite box. Once mirrored, they need only be stored once, freeing up storage.

Thus the only thing stored twice would be files not yet sent offsite. If, due to internet connection limits, this grew too large, users would have the option of connecting a USB drive and writing the delta to it, and physically moving it to the offsite location, which need not be too far away -- just not so far that a major catastrophe would not get both.

In this circumstance you could survive any disk failure, though physical travel would be needed for failures on the server for fast response. And you could survive your house burning down, too.

brad

Fri, 2008-06-06 00:22

Disks don't fail very often, but it's still too often. I've had 4 or 5 fail -- eventually -- in the last 20 years or so. Doesn't sound like a lot, but when there's important data on the disk, even one is too many. Failed drives have cost me more in disk recovery fees than I have spent on 10 drives, and more than that in lost time. And more than that in lost data which was not recovered.

So of course, you should make backups. But there would be no drive recovery companies if people always did that. Tools to automate backup, especially offsite, would be better than RAID, but until this becomes the norm, there's a need for more reliability. My proposal is to make something that's simple, in fact in raid-1 mode it's just as simple as a regular drive until one of the units fails, at which point it beeps at you and you go get a replacement to put in. (No need for hot swap.)

Power is a concern, though as I said, I don't mind having one piece of drive electronics. Turns out a two-platter drive does take a fair bit more power than a 1-platter drive for the motor, so it's not twice the power, but it is more. 3-disk raid certainly is more power.

Also good would be better SMART support, so the OS is very vocal with you about failing drives and getting you to replace them -- and making it easy to do so.

Add new comment