OSNews

how is HFS+ last century? curious.

Good question. I have little familiarity with ZFS, but HFS+ is certainly more sophisticated than what you see on Windows.

I have little familiarity with ZFS, but HFS+ is certainly more sophisticated than what you see on Windows.

In which ways? I'm not choosing either side, I'm just curious. I don't really give a rat's ass about what filesystem I use, so I know little about it.

God I hate NTFS and its defragging problem. Try a little experiment. Defrag your computer and I mean offline and online defrag. Then reboot, you will see your computer boot fast in less than 20 seconds....well mine does anyway....reboot 3-4 more times and watch your boot times grow longer and longer as the disk gets more fragmented because of all the I/O that is being done. That is the single biggest problem I have....is fragmentation with NTFS.

Are you sure thats not the prefetching optimisaton on XP?

God I hate NTFS and its defragging problem. Try a little experiment. Defrag your computer and I mean offline and online defrag. Then reboot, you will see your computer boot fast in less than 20 seconds....well mine does anyway....reboot 3-4 more times and watch your boot times grow longer and longer as the disk gets more fragmented because of all the I/O that is being done. That is the single biggest problem I have....is fragmentation with NTFS.

I'm going totally OT here, but from my past experience with 2K/XP, wiping the swap and then setting it to a fixed size after a defrag helps a lot with this problem. Windows will create the swap in a contiguous space so it doesn't become fragmented, helps with a quicker bootup and slightly better performance overall.

But you're right, defragging ntfs is a PITA.

Regarding a fixed swap to prevent disk slowing due to fragmentation and/or resizing, for those with 2GB of RAM fix the swap size at 0.

Completely disabling swap is not recommended in Windows. I've never personally tried it on my box with 2 GB of RAM, but I've heard that some programs cease to function when no page file is available.

A disk won't get fragmented just because of I/O going on. It will get fragmented if you're appending to files after you've consolidated all of your free space (and packed the files in close to each other).

I call bullshit on this test, as I deal with 3-4 PCs getting benchmarked every day, and fragmentation only goes up minimally (ie. 3%) every few days.

Not really. NTFS is actually fairly sophisticated. For example, it was built with journaling in mind, while HFS+ had journaling bolted-on as an afterthought.

All the merits go out the window when one has to waste time defragmenting.

No one said you have to defragment. Out of the 35 Windows users that I know better, only 2 regularly defragment. The other 33 don't have any problems. :-)

Hmm...true defragging is not required....but it is for having decent I/O performance isnt it?

Indeed, but NTFS fragmentation isn't even close to being as bad as the parent made it out to be.

Not really. NTFS is actually fairly sophisticated.

Absolutely. NTFS is one of the best things MS has ever done. Compression, encryption, journaling -- it's an advanced FS by most accounts. Completely closing off outside support for it has somewhat invalidated that for multisystem users like me, though. Also, I think it still stores entries by filename, so it can't have hard links and just doesn't have symlinks, but the Windows line was never particularly about hands-on control, so few will miss those.

Anyway, the complaint was about fragmentation. I remember how happy everyone was back in the day about how much better the Win2K defragmenter was than the 98 one. NTFS itself didn't help any. MS didn't design around that little problem. They just improved the tool to manage it.

For example, it was built with journaling in mind, while HFS+ had journaling bolted-on as an afterthought.

Successfully, without breaking backwards compatibility. OS9 can still boot from a journaled HFS (but won't access the journal). Let's see 98 even access an NTFS volume. The first version of NTFS is older than FAT32. There's no excuse for the complete lack of included support in the 9x line. But I digress.

Apple has been very conservative with filesystems. Their own HFS is getting up there, and UFS (which they support with OSX) is about 30 years old now. I know Apple doesn't have a reputation for backwards compatibility, but it's very important to them in this case. They'd rather see the user buy all new software and hardware in order to upgrade than lose one byte of data.

With the new Intel Macs, they jumped over to the default filesystem for EFI so they could have nice, shiny software-firmware integration. Apart from that, I know nothing about it. If they're considering ZFS, their comfort with it must come from the fact that Sun is a big name with lots of experience and a reputation of reliability behind it. But I don't think it would ever be used for the system partition (i.e., replacing HFS/EFI in any way.) ZFS was designed primarily for storage. There aren't even any Solares that can boot from it yet. If the post is for real, it sounds like Apple might be looking to sweeten their oft overlooked Xserve / SAN product lines.

I really hope ZFS makes it into Leopard. I'd love to turn my G4 into a low-power (as in, electricity) fileserver once I get a new system. I'd put Solaris on my Athlon server right now if I could trust that I'd be able to add and upgrade software.

This is certainly a throw away comment. NTFS can store double the amount of information on volumes and file size. Up to 16 exebytes. HFS+ can store 8 exebytes. NTFS is actually a very advanced file system with refinements happening with every release of Windows.

Of course, that can't be the only differences, maybe you should checkout this and see what other factors come into play:

http://en.wikipedia.org/wiki/Comparison_of_file_systems

P.S I do own a powerbook and absolutely love it. Modern filesystems don't mean much on smaller laptops but can play an important role on larger systems.

NTFS is also NOT OPEN.

http://www.sun.com/2004-0914/feature/ ZFS is ENDIAN NEUTRAL, this is a reason why Apple is intrested in it. They can jump architectures easier in the future.

What license is this, Sun's own tight license?

Edited 2006-04-29 15:05

ZFS is under Sun's CDDL license, which is probably one of the reason's Apple is interested in it. After all it doesn't contain any viral clause (like the GPL), though any changes Apple make to files licensed under CDDL need to be released under that license.

Basically it's the best of BSD and GPL licenses combined. The original coder/community gets any improvement to the code (ala GPL), and commercial companies can include it in the OS/whatever and not have ro relicense their entire codebase (ala BSD)

Which is the same you can do with LGPL

That's all fine and dandy. No one mentioned LGPL though. GPL != LGPL.

That's hardly a point.

"After all it doesn't contain any viral clause (like the GPL), though any changes Apple make to files licensed under CDDL need to be released under that license. "

I don't think you know what you're talking about.

"though any changes Apple make to files licensed under CDDL need to be released under that license"

is exactly the "viral problem" people describe in the GPL.

The GPL applies the fullest definition of "derivied work" reasonable within copyright law. The CDDL applies a much narrower definition of "derived work", based on files. Under the GPL, if you use a file from a GPL'ed program in your program, your whole program must be GPL'ed. Under the CDDL, you can use a file from a CDDL'ed program in yours, but you can keep the other files under whatever license you want, as long as you publish any changes you make to the CDDL'ed file.

To use an analogy, the GPL says: "if you use any pages from this book in yours, your book is a derived work." The CDDL says: "you can use pages from this book in yours, but the only pages that are derived works are modifications of the page you use."

Well then. It'd appear I'm the one who doesn't know what he's talking about ;-)

is exactly the "viral problem" people describe in the GPL.

No, what makes the GPL viral is that GPL + any other file == GPL. It's like being "infected" by a virus. If I have a virus and infect you, then you are infected as well.

Whereas using CDDL items is like being near a non-contagious infected person. Unless you do certain things you don't become infected

> NTFS is also NOT OPEN.

Of course it isn't. Who claimed otherwise? And how does it make it less superior?

It means that the other operating systems on your multi boot system cannot access files on your Windows partition. That does make it less useful in my book, and thus less superior.

Imagine if all operating systems used open file systems. Not necessarily open source, just properly documented. Wouldn't that take away most of the data sharing pain we all take as part of life today?

If Apple switched to ZFS, dual booting OS X + Linux would be far more convenient than Windows + Linux.

I doubt it.

I still havent found a very stable ext3/reiser drivers *for* Windows.

Just because its open source doesn't mean much in some things.

Also, you can access (including write) to NTFS filesystems from Linux, I currently do it on Ubuntu and its fine.

http://www.linux-ntfs.org/

That's because the IFS (Installable File System) Kit for Windows (which includes the required headers to make the development process feasible) costs around 150$ and few lone developers are going to buy it. I think the last time I checked the kit costed several times the current price.

I doubt it.

I still havent found a very stable ext3/reiser drivers *for* Windows.

Just because its open source doesn't mean much in some things.

Linux is a little guy created by programmers all over the world of varying tastes and interests, so they work to be able to access and manipulate all forms of data.

Apple is kind of a little guy and wants its users to be able to have access to their data, and so includes support for major filesystems where it sees it as practical (FAT, NTFS, UFS).

Microsoft is a big guy that wants to stay the big guy, and thus doesn't want any revolving doors on their platform. Windows XP supports exactly two filesystems (three variants each), both of them Microsoft native, and MS is a ton less helpful in allowing amateur developers to write drivers than they are with apps. A closed filesystem or just a closed system in general will allow for vendor lock-in. It's not that having the filesystems open doesn't help. It's that MS found a way to hook you in anyway.

Also, you can access (including write) to NTFS filesystems from Linux, I currently do it on Ubuntu and its fine.

http://www.linux-ntfs.org/

This is helpful. I've never heard of it before. I was still trying to get Captive to work right. But a very belated solution doesn't excuse the problem. NTFS's propriety has caused a lot of headaches.

Shades of RMS... and it had to be defragmented on a scheduled basis too or performance would begin to lag.

Limits
Max file size 16 EiB
Max number of files Unlimited
Max filename size 255 characters
Max volume size 16 EiB


from wikipedia

Aqua is just for one system, Apple systems.

Mind you, there's no reason other than Steve Jobs' ego why Sun couldn't license the Aqua "look."

Well, that doesn't need to change. AFAIK it's widely considered as a fact, that Solaris (the core OS, without graphical stuff) is much superior to Darwin.
Since OpenSolaris was announced, I am hoping that Apple will use Solaris as the core for Mac OS X.

All higher level features (Quartz, QuickTime, CoreVideo,...) would still be Apple exclusive, but with a more powerfull core.

At an official capacity, this will never happen. Given Solaris 10 is based on Gnome with the free licenses that are associated with the GTK toolkit, I highly doubt Sun will consider Aqua. This falls in line with they decision not to back QT for their next toolkit even though its GPL licensed as well, the downfall is seen by its commerical history and backing from Trolltech...

To me this doesn't seem to suit most of Apple's consumer market.

Leopard+1 Server.

Server is a consumer product now?!?

I just think that porting it probably isn't worth the time and expense for how a small a market it would fit.

Server is a consumer product now?!?

Of course it is. It is just a different consumer product than the iMac or MacBook consumer products.

Edited 2006-04-29 13:53

While the speed and disk pooling features of ZFS might be more appealing to the workstation or server market, ZFS also offers some great improvements at the desktop and consumer level, not the least of which is file-system level snapshots and rollbacks. ZFS also offers self-healing and excellent reliability.

If ZFS becomes an install-time option, the benefits to OS X users will be immediate and vast.

"which is file-system level snapshots and rollbacks. ZFS also offers self-healing and excellent reliability."

I think an explaination of what those are, what they does and how they do it would help.

ZFS is "endian-neutral" which might be making it attractive to Apple; right now Intel Mac's are taking a bit of a disk access performance hit because they're little-endian and HFS+ is big-endian.

The bits of ZFS that protect against corruption might be even more attractive to smaller businesses and home users, who in my experience aren't always very good at sticking to a backup schedule.

Depending on your computer, compressing the filesystem can actually speed things up because it reduces the amount of data you have to read from the hard disk. Granted, this will only be the case if your CPU isn't already working hard.

Alongside ext2/3, reiser and XFS there is no real benefit from a home user perspective. I use ext3 everywhere for servers & workstations at home, but have a couple of macs too.

If you have a couple of machines I recommend scheduling a job to backup, even to each other (systems). This is what I do with Windows and Linux systems over Samba.

You should ask yourself the question, do I need compression at all? With harddrive prices the way they are, its almost inconceivable for needing to compress.

Well said.

Also, look at how many people want to do RAID-5 at home, but don't want to spend $300+ on a decent RAID card.
With ZFS, you can use RAID-Z, which at least in theory performs better than any software-based RAID-5.

The higher reliability (the key feature of ZFS) alone is worth the consideration for any home RAID users.

"Simply put, it's very easy to append disparate volumes together and make larger single volumes. So, for example, now you can easily just keep chaining those new 750G Seagate drives together as you continue to rip your DVDs or home video projects."

But if one of the disk dies, does the whole filesystem die or can one "repair" parts of it from the other parts? (not talking raidz here).. Say disk 1,2,4,5 lives, 3 died. Can i remake the filesystem with disk 1,2,4,5 and have the files which wasn't on disk 3 still available?

I believe so. Generally that problem arises because the volume manager and filesystem are separate, so the FS doesn't actually know it's being spread out over multiple physical devices. When a big chunk of itself goes missing, it doesn't know what to do, especially if that chunk has the only superblock. ZFS manages multiple physical volumes by itself, which gives it the equally important advantage of being able to avoid getting cornered like that in the first place. If one physical disk is giving you trouble, you can command ZFS to start bailing out the files on that one onto other disks, and then remove it from the pool. Its featureset is really amazing.

Or maybe someone at Apple sent a private email to someone at Sun, who then told the whole world. I guess Apple should have negotiated an NDA first... :-)

Something like this WOULD be great, but in what future release would we SEE it? I remember when Mac OS X came out you could set it up under HFS or UFS... but Mac OS X NEVER worked 100% under UFS... they eventually removed the option, did they not?

Anyway, it seems like the OS uses the HFS in ways that may have to be carefully considered when switching file systems, which to me means it will not ship with Leopard, maybe not even ocelot, or whatever the next version of OS X is called...

...This is just one of numerous points that make Mac OS X somewhat seem like OS/2 in its days...

You often had to install applications on HPFS (High Performance File System) in order for them to work, mainly because of the support for long filenames, and possibly a few other technicalities...

Apart from that, just to mention, is the way everything seems to be drag'n'drop oriented, the nature of the WinOS/2 vs Classic Environment is also a lot alike

...But then again, I've always loved OS/2.... -It's always been way more stable than my female relationships

Since BootCamp many of us believe that Apple will implement some kind of virtualization technology on 10.5. Can ZFS ease the implementation of virtualization

If you're going to virtualize your system, you'll also want to virtualize your storage, and ZFS provides fine-grained, efficient, easy-to-use storage virtualization (aka volume management). Although for the kind of virtualization that Mac users are generally interested in (Parallels), you really just need good sparse file support so that those disk image files take up less space if they're not full. ZFS would be really useful if OS X implements something like jails, but I doubt that will happen.

How does ZFS relates to other file systems? Can it "see" ext, fat or NTFS?

ZFS is a different file system. Different file systems don't interoperate, so you're either using ZFS or you're not.

how does ZFS manages both very small and very large files, in terms of disk space and speed?

Efficiently, using multiple block sizes to reduce wastage on small files and increase speed on large files.

raid on my home linux box is the dogs dangly bits (thats good) i have 2 ide drives one on each cable and used software raid built in to linux to make raid 1, it goes rapid... useful for all kinds of things (openoffice is quick now

Edited 2006-04-29 20:05

apple doesnt have enough money to buy sun. and they have different interests.

"AAPL" market cap is 60 Billion; "SUNW" is 17 Billion, I think they can buy sun. And for the difference it would be the power to add to Apple throne in the server arena.

Market Cap alone won't determine one company's ability to purchase another. Although I'd be all for some sort of collaborative merger between Sun's server-oriented expertise and Apple's desktop prowess.

Exactly. No need to buy-out to collaborate.
Apple didn't spend billions on a BSD company to build OSX on BSD.

Also, from my point of view, Sun is a much bigger and more influential _technology_ company than Apple, while Apple is a giant in the consumer market, it doesn't make sense for one to buy another, neither has the capability.

However, it makes lot of sense for them to collabroate on the OS side, it would be a win-win situation.
Sun does not "sell" Solaris, they sell enterprise services for systems. Even if Apple makes a "better Solaris than Solaris" by migrating to Solaris kernel but keeping the UI technology, it doesn't have the capabilty to service the kind of enterprise environment that Sun has been supporting.
In the end, more people will use OSX on the desktop/workstation and more people will use Solaris on enterprise servers. That's what I call win-win.

can a ZFS Solaris formated volume read and write to FAT, NTFS, ext2/3 partitions? In OS X/HFS+ I can read and write to FAT and read NTFS. Sorry for the noob question but is this a function of the file system or is implemented at a different level on the operating system?

It's done at the OS level with drivers that tell the OS how to handle a given file system.

You can read this for more information on file systems.

http://en.wikipedia.org/wiki/Filesystem

One filesystem (or "volume") does not read or write another filesystem. That's what Wes meant by "different filesystems dont' interoperate".

For each filesystem, you have a separate module which understands its own filesystem on-disk and kerenl structures. The operating requests (cd/open/read/write etc.) are passed to different module for different filesystem by the OS.

You can read/write FAT and read NTFS on OSX suggests OSX provides the module that read/wrte FAT, and another module which only supports NTFS read.

It has nothing to do with HFS+.
If you replace HFS+ with ZFS, you can still read/write FAT and read NTFS, because those modules are still there.

Here is hopefully a more full explanation.

Most modern OSs have what is called a "virtual file system". The actual directory structure you see is implemented inside the OS, and plugins handle mapping on-disk filesystems (like ZFS or FAT), to the logical VFS hierarchy. When you "mount" a filesystem, you're basically tell the VFS: "use such and such filesystem plugin to expose such and such partition at such and such directory". The plugin then takes care of the logical mapping between the VFS semantics and the on-disk filesystem semantics. It'll handle, for example, working with a case-insensitive filesystem like VFAT in a case-sensitive system like Linux.

FYI: At the OS level, filesystems generally don't interact. In OS X, a CD filesystem might appear in /Volumes, which is a directory on an HFS+ disk. However, at the kernel level, the HFS+ plugin really doesn't do anything when you read/write to the CD. The part of the kernel that handles filename lookups will see a path like "Volumes/My_CD/foo", and realize that 'foo' is actually a file on the CD drive, not a file inside a directory inside an HFS+ volume, and redirect the request accordingly.