OSNews

Indeed an excellent article!

I think it would happen in all 4K sector drives regardless of size. If you started at LBA 63, the drive would be forced to update 2 4K sectors because the virtual sectors would lay across 2 physical sectors.

P=======P=======P=======
=V=======V=======V======

Edited 2010-02-14 12:20 UTC

Reading the Aandtech article, it appears intended for 1TB+ drives, but as you say, it could apply to drives of any size if WD decided to adopt it across the board.

Hm, how is LBA addresses specified exactly?

I thought it was specified as multiples of drive sector size, in which case addressing via LBA should always be aligned - but is it in reality always a multiple of 512?

Is there a difference between how LBAs in the MBR are interpreted, and how ATAPI interprets LBAs?

...but perhapse the 4k-sector WD drives are currently running in 512b-sector emulation mode? That would at least explain the "LBA weirdness"

My drives are 1GB, but I'm sure it'll affect any 4K-sector drive the same.

As if gparted (which is just a pretty gui + parted) can't screw things up too. fdisk and the likes are for people that know what they are doing.

In case you don't know there are distros that have text mode installs and use utilities like fdisk/cfdisk. Just because you can use gparted it doesn't mean you have to.

I use it all the time. Furthermore I actually use gdisk so that I can create hybrid MBR-GPT partitions. Parted (and gparted) totally trashes hybrid MRB-GPT setups.

I've never used anything but fdisk, and never had any bugs or problems with it. But then I run Gentoo and don't use the installer.

The fdisk man page (on my machine at least) contains a lengthy warning about how fdisk will quite happily create some pretty dodgy partition layouts and it recommends parted for doing anything even remotely unusual. I guess this falls into that category.

All the major noob-friendly distros use gparted for doing the partition editing, don't they? Will that protect users from these kinds of problem, then?

All the major noob-friendly distros use gparted for doing the partition editing, don't they? Will that protect users from these kinds of problem, then?

I do consider Mandriva to be pretty newbie-friendly all in all; it's clear, consistent, and provides an extensive selection of documentation and loads of online help if needed. Also, it's really stable and has excellent control center utility.

But alas, Mandriva doesn't actually use gparted. They use some sort of a tool of their own which apparently uses libparted as its backend. As far as I know quite a few distros actually do it that way. But as the article states, you seemingly have to use "--align optimal" option which does the right thing. It doesn't automatically align the partitions properly without that. And I have no idea if those custom partitioning tools employed by various distros pass such an option to libparted. If they don't then that'll be a very important issue to fix immediately.

I'd actually prefer if distros rolled out an update of some sort which will check the currently installed system and its partitioning scheme and warn if they are misaligned and would provide a way of fixing it; not everyone re-installs their system all the time and as such could be using misaligned partitions for years before next re-install.

I think it will be mostly ok if they use it on new installs from now on, these WDs are not widely available on the market yet (and when you do buy them: their is a BIG warning on the front).

gparted get it's right:

"When enabled, Round to cylinders aligns partition boundaries on the cylinder boundaries for the disk device. Enabled is the default setting."

The Ubuntu installer uses Partman from Debian, which uses parted in the background.

So that's a start.

But if parted will do it right when run from partman, I don't know yet.

Edited 2010-02-14 13:19 UTC

I just had a look at what partman does and what parted does. Parted does align by default as well, just like gparted. And partman just passes the MB's to parted, if I looked in the right places. That means it will do the right thing by default I think.

Aligning partition to "cylinder" boundaries is BAD. The cylinders are fake and they're in units of 63.

You could be right.

I guess my brain is off because it's weekend.

Yeah, Mandriva would certainly fall within the category of distros which should sort all of this stuff out automatically without the user having to worry. Distros like Arch, Gentoo & Slackware all generally expect their users to be aware of the technical issues. But for the mainstream distros this does need to be fixed.

Maybe we could take a look at our respective distros and file a bug report if there could be an issue.

Mandriva was the first distro to make partitioning the hard drive mortal friendly.

Mandrake Linux 7.0 was when they first released it and I recall thinking "Holy crap, this thing needs to be sold separately"

I went so far as to use the install disk up to the partition step to repartition my hard drives for a while.

So, since you're using Linux, wouldn't it behoove you to use the GPT (GUID Parition Table) scheme which handles, by design, the new block size?

ref: http://en.wikipedia.org/wiki/GUID_Partition_Table

Well apart from anything, fdisk doesn't support GPT. Parted does, but as he said, you can get parted to automatically solve the problem anyway.

The problem as TFA describes it is that a bunch of the tools & tutorials out there today will give you bad results if you use them with the new block size. They will probably give you bad results if you use them with GPT, too.

Edited 2010-02-14 17:57 UTC

Hi,

No.

Very old disk drives used "CHS" (Cylinders, Heads, Sectors) instead of LBA. Due to limitations this didn't work for drives larger than about 500 MiB, so the industry shifted to LBA; and created a CHS->LBA translation scheme.

Due to BIOS limits, this CHS->LBA translation scheme usually uses "63 sectors per track", which is the highest number of sectors per track that the old BIOS disk interface can handle.

For performance reasons OSs make partitions that start/end on track boundaries (having a few sectors at the start or end of a partition that are on a track by themselves causes more disk head movement).

Basically what I'm saying is that the problem wasn't caused by *any* OS. The problem is caused by 30 years of backward compatibility (and the lack of foresight, from BIOS, disk and OS designers).

The ironic part is that the original IBM design supported floppy disks and hard drives with different sector sizes. It's unfortunate that this aspect of the original design was lost, and unfortunate that these new hard drives need to emulate 512-byte sectors to begin with.

-Brendan

Reporting disk topology requires the hardware to also communicate that topology. AFAIK many of these drives do not. Christoph Zimmermann summarized the same observation in the thread you point to:



It looks like the discussion in that thread is about aligning partitions by default on a sufficiently large granularity to "get by" (as Vista and above do.) Note that other technologies (eg. SSDs, RAID) benefit from large alignment (larger than 512 bytes or 4Kb.)

I was using a 2.6.32 kernel. It clearly did not "do the right thing."

I think I might have the same problem. What model are you talking about?
mine is WD15EADS.

Perhaps not only linux is not ready. They are not ready even at WD support.

Currently the only "Advanced Format" disks (as WD calls it) are the Green models ending in "EARS". WD15EADS is of the previous generation and therefore not affected.

The Advanced Format disks are called WD10EARS, WD15EARS and WD20EARS.



I read that every major HDD manufacturer has agreed to using a 512-byte-emulation mode until the end of 2014.

I also read that there is a way for software to ask the disk about its real physical layout but that Western Digital hasn't implemented such feature in its current line of 4k disks. Therefore no software can detect them and take care to stay aligned.

Edited 2010-02-15 14:11 UTC

I know it is not supposed to be an advanced format drive, but running the code at the and of the article indeed shows performance differences with alignment other than 0 and 8. (I ran it on a partition starting at sector 64)

Also the disk freezes a lot when doing io.

I am sending it back.

Where did you get this info? If you read WD's site:

http://www.wdc.com/en/products/products.asp?driveid=773

Formatted Capacity 2,000,398 MB
Capacity 2 TB
Interface SATA 3 Gb/s
User Sectors Per Drive 3,907,029,168

That's 512 byte sectors, model # WD20EARS

I made a mistake by leaving "be" out of my sentence. I think you're smart enough to make the appropriate inference. This was not intended to be a professional publication, so I didn't bother editing it 20 times and having it proof-read by lots of other people.

Aside from forgetting a word ("should /be/ unaffected"), I see no problems with what he wrote.

wow, he forgot one word, and therefore he's incapable of communicating in English?

Please, drink a fifth of scotch and calm the hell down.

That seemed pretty understandable to me, perhaps you need to bone up on your reading comprehension.

I would like to apologize to you all.

I was angry for other reasons.

I just lost my job.

I love OSNews, and I hate grammar mistakes on the front page. A little more care by the authors, that's all I'm asking for.

Cheers...

Just make sure to do that on a disk whose data you don't care about. I only say this in case some naive user decides to test their primary hard-drive in this fashion and ends up destroying the first X megabytes of the drive (including bootloader and partition tables).

it's the duty of the blockdevice-driver and/or the filesystem to collect several such manipulations before writing them to the disk

Well, no. cp doesn't do 1 char at a time, it tries to minimize io. It might not be clear from the code though. Even stdio keeps an internal io buffer you can't normally see.

A quick truss/strace on recent FreeBSD/CentOS/Solaris shows 64k buffers/4k buffers/mmap the whole darned file approaches.

This drive does not in any way report its physical sector size. It says its sectors are 512 and that's it.

Shouldn't WD correct that then? How is a disk tool supposed to partition a drive properly if the drive itself is reporting incorrect data? If the physical sectors are 4096, the drive should report 4096 shouldn't it?

I believe the whole point is to emulate the 512b sectors so that legacy OS's will work. The drives can't query the OS and then modify how they report themselves depending on what is supported. They do provide a jumper so you can manually turn the legacy emulation on or off, but most people aren't going to mess with that.

We have knowledge of the problem. We can deal with it, regardless of how the drive lies.

Yes it can be dealt with, yet at the same time it seems as though these drives should be able to report the correct geometry when queried properly. That would mean the partitioning tools would be aware of it from the start rather than having to manually deal with the problem. Most people I know, even ones with good technical knowledge, wouldn't have known how to handle this one as they don't delve that deep into drive partitioning. For the sake of avoiding trouble whenever possible the drive should report the geometry properly when queried by an os that knows how to ask for the *real* geometry and not that ridiculous LBA compatibility hack we've had to live with for so long thanks to bios and Windows.

First of all it's important to distinguish between logical block size which is used when sending commands to a device and the physical block size which is used by the device internally.

Linux has supported (SCSI) drives that present 4KB logical block sizes for a long time. For compatiblity with legacy OS'es, however, consumer grade ATA drives with 4KB physical blocks continue to present a 512-byte logical block interface. The knob indicating that the drive has 4KB physical blocks is orthogonal to the logical block size reporting, allowing the information to be communicated without interfering with legacy OS'es like XP that only know about 512-byte sectors.

We have worked closely with disk manufacturers for a long time to make sure we were ready. Western Digital have been instrumental in the ATA specification in terms of the alignment and physical block size parameters. The engineering sample drives I have received from WDC have all implemented the physical block size knobs correctly. Which makes it even more baffling that they end up shipping an advanced format drive that gets it wrong. I have no idea why they did that. The location of the block size information in IDENTIFY DEVICE is unlikely to be inspected by legacy systems, so I highly doubt it's a compatibility thing. Brown paper bag time for Western Digital...

It is true that the effects of this particular drive reporting incorrect information could have been mitigated by a 1MB default alignment. However, that would still have caused misalignment for other drives that come wired with 1-alignment to compensate for the legacy DOS sector 63 offset. So blindly aligning to 1MB won't cut it. Windows Vista/7 don't do that either. Like Linux, they compensate based upon what the drive reports.

Linux 2.6.31 and beyond will report device alignment and physical block size for all block devices. It is then up to the userland partitioning utilities etc. to adjust start offsets accordingly. You'll find that both parted and util-linux-ng have been updated to do this. And that modern fdisk will in fact align on a 1MB (+/- drive alignment) boundary by default.

Caveat being that Fedora is the only community distribution I know of that's using the updated bits. I don't think all of them made it into Fedora 12 but I'm sure Fedora 13 will do the right thing.

So I encourage you to work with your distribution vendor to ensure they start shipping recent partition tooling.

Martin K. Petersen
Kernel Developer, Oracle Linux Engineering

4k blocksize is the default value, not something that needs to configured manually.