This raid issue makes me want to invest in IDE raid heh.

You’re correct. I was working from E, the expected number of bit errors — with 7 TB, the expected number of bit errors is 0.56. This is obviously not the same as probability of at least one failure, which is 0.44 as you say. I’ve updated the post to correct this error. 44% is still pretty scary!

Regards,
–Jered

I think that should be 44%.

Let’s work it out together.

1 in every 10^14 bits has an error. Let’s refer to 1/10^14 as alpha for the
sake of ease of notation.

The probabilty that a bit has no error is thus 1-alpha.

The probability that there will be no error while reading 7TB (as disk
manufacturers count terabytes) of data is

(1-alpha)^(10^(1024*1000*1000*1000*8*7)), or, more practically,

10^(log(1024*1000*1000*1000*8*7*log(1-alpha))).

calc from the wonderful apcalc package can work that our for us:

% calc ‘10^(7*1000*1000*1000*1024*8*log(1-(1/10^14)))’
0.56358337345776143653

There you go, 56%.

The chance that it cannot be done is then 44%.

That’s an interesting theory. I’m doubtful, as I’ve never seen a storage vendor knowingly undersell a product on the spec sheet, but it’s possible. There was a recent study using data from NetApp that suggested that the published BER rates were pretty accurate — I think it may have been “An Analysis of Data Corruption in the Storage Stack” (http://www.cs.wisc.edu/adsl/Publications/corruption-fast08.pdf).

I can see two reasons to do that: (a) promising high reliability makes for higher costs in warranty replacement, so it carries a premium; (b) the vendors don’t want enterprises buying consumer drives, so they make weaker statements about the consumer drives.

This wouldn’t be illegal, or even particularly deceitful; they’re promising that the BER is no *greater* than such-and-so, and so exaggerating the BER is still true.

Lev,

Yeah, the Seagate data sheets use different terminology — they call the error rate out as “1 sector” instead of “1 bit”, which would be 512 times more reliable than any other vendor! I think they might just be referring to the fact that a single bit error causes the read of the entire sector to fail as you don’t get partial data, you get a read error on the sector.

Clarification from Seagate would be great here. If they really are 512X more reliable than every other drive vendor, you’d think they’d be shouting that from the treetops!

–Jered

These same vendors offer “enterprise” SATA drives which, presumably, go through this more intensive testing. They go for around a 20% price premium over the “consumer” SATA drives. Why not just stick with one interface, then? I stand by my claim that enterprise : SAS :: consumer : SATA is a false market segmentation.

Steven,

You’re mixing “enterprise” and “SATA” here, but I get the point you’re making, but I don’t think it’s correct. Serial-ATA has CRC checks on bus protocol frames. With both SATA and SAS you have protection of data from the drive to the host controller.

I’ll agree that SAS controllers may be more likely to use ECC memory, or that SAS drives may maintain better ECC data internally to protect the data until it’s converted to a SAS protocol frame, but these are again manufacturing decisions on the part of the vendors involved. SATA can be just as reliable as SAS in these areas.

Of course, both are vulnerable to data corruption between the application and the host controller. This is why application vendors like Oracle add additional ECC data at the application layer — they know they can’t fully trust any component firther down the chain.

–Jered