This is an article which discusses the increase in storage capacity while performance and hard error rates have not improved significantly in years, and what this means for protecting data in large storage systems. "The concept of parity-based RAID (levels 3, 5 and 6) is now pretty old in technological terms, and the technology's limitations will become pretty clear in the not-too-distant future â€" and are probably obvious to some users already. In my opinion, RAID-6 is a reliability Band Aid for RAID-5, and going from one parity drive to two is simply delaying the inevitable."
To read all comments associated with this story, please click here.
Member since:2005-07-08
That only works if you have relatively small amounts of data that is not modified frequently. If you have a large operation with frequently changing data the potential for failure increases significantly.
I am not a fan of cheap storage because what most people think they are getting in savings they end up paying for in terms of management, performance and reliability. I don't care for SATA storage because I am not convinced that it will work reliably in the long run as opposed to SAS or Fibre Channel.
While the possibility of a massive failure for most cases is slight, it depends entirely on how the solution is deployed and what mechanisms are in place to protect the data. Having redundant storage to prevent data loss can get real expensive.
Edited 2009-09-18 15:33 UTC
Member since:2007-07-27
"It is not the end of the world. The mainframe solved this one decades ago."
You seem to have missed the point. If a drive fails in a raid, it takes hours to rebuild the raid. The larger the drives, the longer it takes to rebuild the raid. If you have 1TB, it takes maybe 10 hours. 2TB may take 24h. 4TB maybe 2 days. 8TB drives may take one week? Because drives doesnt get much faster, only larger.
At some point, it will take long time to rebuild the raid. Scaringly long time. Say it takes one week. When you rebuild a raid, it stresses the other discs very much, to the point it is common another disc breaks! This happens more often than you think. Then you are screwed, if another disc fails.
Therefore you use raid-6, which allows two discs to fail. But there is likelihood that both drives fail during rebuild. At some point in the future, the discs get so large, another disc will fail as fast as you rebuild a broken drive. This is due to larger and larger drives.
A decade ago, the mainframes didnt have this large drives. To rebuild a raid was no problem, it went very quick. Today, it takes a very long time. Therefore, you are wrong, mainframes have not solved this problem. This is the reason people say that raid-5 is soon obsolete. This is what the article is about.
Also, enter Silent Corruption. Discs will read/write bits errorneously without even noticing it! You will not get a notification: there was an error. This is a BAD thing. 20% of a discs surface is dedicated to error correcting codes, and the codes can not fix every error, nor even detect every error. There are lots and lots of errors in every read and write, that gets corrected on the fly. But sometimes there will be errors that can not get error corrected by the disc. Nor even detected. It is like the lamp on the oven says it is turned off, but the lamp shows wrong, the oven is in fact turned on - the HW doesnt detect this, so it lies to you. Look at a spec of a new drive, it says "unrecoverable error: 1 in 10^14". There are errors that even doesnt get detected. But ZFS detects, and also recovers the data. The "1 error in 10^14" doesnt apply with ZFS. Because ZFS detects and corrects them.
SUN knows about these problems and ZFS does fix this problem. ZFS also allows other more safe, configurations than raid-5 and raid-6, which makes ZFS less susceptible to taking a long time to rebuild a raid. For instance, three discs are allowed to fail in raidz3 configs. Or you can mirror lots of discs, and combine them into flexible raid configs. The best thing is that ZFS does NOT like HW raid and they only disturb ZFS. Therefore, ditch HW raid while you can get a fair price, and use ZFS to get a cheaper and safer solution. 48 SATA 7200 rpm discs, reads 2-3GB/sec and writes 1GB/sec. And the data is safe too.
CERN did a study on silent corruption, and the moral was that one error in 10^14 is not correct. This article is not correct, according to studies at CERN. The errors occur more frequently, in practice:
http://storagemojo.com/2007/09/19/cerns-data-corruption-research/
Your data is at risc. Silent Corruption and bit rot eats your data. Silently. Without the HW telling you. The HW doesnt even notice.
Clearly, something has to be done to cope with these errors that large drives and future filesystems will face. The main architect of ZFS explains some of the future problems that will be more and more common.
http://queue.acm.org/detail.cfm?id=1317400
Edited 2009-09-19 09:54 UTC
Member since:2007-01-13
At an event earlier today, Microsoft unveiled the next Xbox - the third model, but confusingly named Xbox One. The big focus was TV, integrated Kinect, and all the other stuff we all expected to be forced down our throats. I think it took them 25 minutes to actually come to what should be the core of the story: gaming. Nothing groundbreaking in the gaming department, except for how Microsoft intends to handle the used games market and borrowing games from friends: pay up, buddy!
The day has finally come! Jolla has finally announced the launch device for its Sailfish operating system - and by god this is a looker. It's decidedly different from other phones out there, but it has good specifications and carries a relatively reasonably price tag - EUR 399, and it's up for pre-order today, shipping in the fourth quarter of this year.
Why does Google get so much credit in the technology industry? Why, despite the company's many obvious failings, do many geeks and enthusiasts still hold a somewhat positive view on the all-knowing technology giant? A specific talk at Google I/O this week provides the answer.
OSNews is a sponsor of this year's O'Reilly OSCON in Portland, Oregon, USA. A lucky OSNews reader will win a free three-day pass, including two tutorials days. To win the pass, post a comment on this story saying something about Open Source Software or OSCON. We'll pick a winner at random next week. If you don't have an OSNews account, you may email us your entry. Part of the conference is the 9th annual Open Source Awards, and today the 16th is the deadline for nominations. If you'd like to nominate an outstanding open source contributor, do it here. Read on for more information about OSCON. Update: The 20% discount code for OSNews readers is "OSN."
"Windows is indeed slower than other operating systems in many scenarios, and the gap is worsening." That's one way to start an insider explanation of why Windows' performance isn't up to snuff. Written by someone who actually contributes code to the Windows NT kernel, the comment on Hacker News, later deleted but reposted with permission on Marc Bevand's blog, paints a very dreary picture of the state of Windows development. The root issue? Think of how Linux is developed, and you'll know the answer.
This is one of those news items that's fun to write, fun to read, fun to comment on, and where no one will be able to say anything unkind. It's all just one big ball of awesome fluffiness. TuneTracker, the BeOS radio automation software, has just released something very special: TuneTracker System 5, the first version designed entirely and specifically for Haiku. In fact, it actually includes Haiku in the software package. Better yet, TuneTracker also unveiled several system-in-a-box products - which have Haiku and TuneTracker pre-installed.
Exactly twenty years ago, a document was published that played a huge role in establishing the web as we know it today. Twenty years later, and this simple and straightforward document is proof of an irrefutable fact: while closed technologies can change markets, open technologies can change the world.Oh multitasking. That staple of computing that got thrown out the window with many modern smartphones. We got some rudimentary thing in its place - but even as multitasking on phone and tablets improves, its user-visible side remains cumbersome. Windows 8 has a neat implementation, and now it's time Android follows in it footsteps.They're here! Whether that excites you or not remains to be seen, but the Galaxy S4, which will most likely become the best selling Android smartphone of the year by a huge margin, has been reviewed by all the major sites, and there's lots of interesting conclusions in there - although I think most of you will get the gist.Nokia has posted its quarterly results for the first quarter of 2013, and just like the quarters that came before, there's not a whole lot of good news in there. The rise in Lumia sales still can't even dream of making up for the sales drop in Symbian phones, and when broken down in versions, the sales figures for Windows Phone 8 Lumias in particular are very disappointing. In North America, Nokia is getting slaughtered.
Member since:
2006-01-04
What the article states might be true, but the cost of storage goes down far faster than the problems rise.
So you can just throw more disks and a permanently running replication daemon at the problem will have a working solution.
It is not the end of the world. The mainframe solved this one decades ago.