A French user has discovered that ZFS is now ‘officially’ supported in Leopard [French], while testing the latest build of Apple’s upcoming operating system. Indeed it is now possible to create disk images or partitions formated in ZFS (screen capture included). However, it seems that it is not possible yet to install the OS on a ZFS partition.
This is very lovely. I only hope that they use it for timemachine, or even the OS itself. I’m not sure though that will happen (probably Leopard+1/XI). I would also really like it if they added live resizing of partitions to Disk Utility.
Congratulations, Mac fans. I’m a Linux user, but recognize that ZFS is a filesystem second to none.
If Sun goes GPL on Solaris, which seems likely to happen, we might get a shot at this, too. (GPL is nice and all, but sometimes its unintended consequences are a damned pain.)
Whether ZFS would ever make it into the mainline kernel is another matter. Andrew Moton has described it as a “rampant layering violation”, though I should hasten to add that his statement was actually praising ZFS’s feature set while noting this fundamental clash with Linux kernel policies.
I suspect that we would see an out of tree port first, followed by individual ZFS features making there way into the appropriate layers of the Linux kernel.
At any rate, looks like you guys are getting there ahead of us. Kudos!
-Steve Bergman
I would say that ZFS is second to Reiser4. Reiser4 is a far more innovative filesystem that really addresses the point that byte-streams are just byte-streams, and that any distinction between metadata and data is just arbitrary. Furthermore, once the distinction between data and metadata is eliminated, and small files are stored efficiently, it becomes a completely usable design for apps to stop storing small record in custom databases, and use the filesystem for such tasks.
That said, its nice that OS X is getting a decent filesystem. The longer I use OS X, the more it becomes painfully apparent how badly the kernel sucks, and any step forward is a much needed one.
“The longer I use OS X, the more it becomes painfully apparent how badly the kernel sucks….”
_______________________________
It would really be worth an article, if you can find the time, to explain this at reasonable length. Agree or disagree, it will certainly be worth careful reading. Hope you can find the time.
Speaking of OS X and filesystems, Slashdot recently had a thread going about Mac automated backup solutions. pesc brought up this gem: http://blog.plasticsfuture.org/2006/03/05/the-state-of-backup-and-c… While the writeup is generally about backup software, it’s goes into a lot of detail about how OS X handles metadata.
Brief synopsis: be glad you aren’t in the business of making automatic backup software for Macs!
If you decide to go ahead, drop me an e-mail. I’ve seriously thought about writing something like that myself.
Being a Linux guy myself, last September I bought a MacBookPro. I was on a hiatus using OSX for nearly two months non-stop. I thoroughly understand what you say about its kernel. The interface is very flashy and all, but seriously, its kernel is far worse than an unoptimized… I don’t know what to compare it to.
Once I an on Linux (2.6 + CK) full time again, I feel I have unleashed the real power this machine has.
Tell me about leaving a lengthy CPU bound process like encoding an HDTV video with x264. You get a barely usable system on OSX, even nicing it +19. On Linux, I can keep doing that not only way faster than it was on OSX, but interactivity doesn’t suffer at all.
I’d need to compile myself mplayer on OSX, tweaking it the way I do on Gentoo, to do a fair comparison. But still, the difference is awesome.
My 2¢.
Encoding HDTV is a CPU-intensive process with a limited number of potential bottlenecks: simply pointing the finger at ‘the kernel’ isn’t in the slightest bit useful, all told.
Realistically, there are only three real places that something like that could suffer: an I/O bottleneck (Linux could be more efficient at buffering than OS X), a memory-management issue (was XNU swapping when Linux wasn’t, for example?), or a scheduling problem coupled with the two previous: for example, was there a resource over-allocation applied to the encoding process which makes task switches slower than they should be (i.e., causing a slow-down in both overall encoding performance and interactivity)?
What you really should be doing is profiling the encoder on both platforms to find out what’s slowing it down: with a process that actually uses the kernel as infrequently is this, it wouldn’t be particularly difficult to do and perform a real comparison.
I’m afraid an article like that would just invite a flame-fest. But I’d be happy to summarize some of the most glaring problems that I’ve encountered.
1) The toolchain is just weird. If Apple was going to use the GNU toolchain, they should have done it properly, by adding support for Mach-O to BFD, and fixing the GNU tools so they could target Mach-O, just as they can target PEF for Windows. However, Apple chose to instead ship custom hacked-up versions of the GNU tools. While that is a solution, for people writing software, Darwin’s “almost GNU but not really” toolchain is a complication that could’ve been avoided. In an ideal world, they would’ve used ELF like everyone else and avoided the problem entirely, but that’s probably too much to ask.
2) Darwin’s shared library infrastructure is… strange. Mach-O just isn’t as clean a format as ELF. There are fairly arbitrary distinctions between loadable modules and shared libraries, restrictions on section types in dynamic libraries, etc. The linker is also slow, and startup of programs with extensive shared library dependencies is pokey.
3) The Mach/BSD API schism is not entertaining, to say the least. Their tracking of the FreeBSD API has stagnated since 5.x. Darwin is BSD… almost.
4) The system does not handle huge VM loads gracefully. Giant compiles won’t even cause Linux to blink, while they’ll cause Darwin to have very noticable hiccups.
5) I/O performance, and general system performance under high I/O loads is not steller.
6) Many primitive UNIX operations (process/thread creation, page-fault handling, etc) are much slower on Darwin than on Linux. This has been documented repeatedly, and extensively.
In the grand scheme of things, Darwin just isn’t very good, at anything, except maybe low-latency audio. It just doesn’t deliver what I’d expect from a modern, high-performance UNIX kernel. It’s not as scalable as Solaris, its not as fast as Linux, its not as stable as FreeBSD, its not as clean and elegant as NetBSD, etc, etc, etc. It’s merely “good enough”, what Apple inherited when it bought NeXT, and what they surely realize is behind all of the major competition, but what they (rightly) think is not enough of a handicap in their target market to be worth the trouble of replacing.
points 4 and 5 simply give clarity that Vista is better than OSX.
I am no expert but if Vista is based on Server 2003, then that means I/O performance and VM loads should be handled much better in Vista.
But saying this how different is Apple’s server OS?
No desktop version of Windows has ever had adequate interactive performance under high I/O and VM loads. OS X isn’t as good as Linux in this regard, but its still better than Windows (which will hiccup even without heavy load).
I’ve never used a 2k3-based Windows, so it could very well be better. However, good VM and I/O performance still won’t fix the brain damage of the UI…
Edited 2006-12-16 23:29
I’m afraid an article like that would just invite a flame-fest.
You’re probably right there. Maybe ArsTechnica would be a better place to submit such an article – the readership there seems to be a bit more technically-knowledgeable and mature.
Innovation’s nice, but last I checked Reiser4’s performance was somewhat lacking.
It’s also been seriously neutered since the original design… all of the innovative metadata features have been removed (or never implemented). The implementation (a hidden directory filled with files containing the metadata) was a bit wacky, but it was a great idea exposing a file’s metadata “directory” so it could be manipulated by standard tools.
– chrish
Those innovative features are still there, they are just not exposed to userspace or fleshed out as they could be. The hidden directory with metadata files isn’t the underlying implementation of metadata, but rather how the system is exposed at the semantic layer. At the level of the implementation, Reiser4 names byte-streams in the storage layer using an abstract key, and directory plug-ins map names using a particular namespace scheme to keys. You could, right now, write a directory plug-in that exposed metadata in a different way.
he longer I use OS X, the more it becomes painfully apparent how badly the kernel sucks, and any step forward is a much needed one.
Yeah, you are telling us that you are able to talk directly with the kernel.
How many people can feel how bad is a kernel while using an application …
Well, I might say that if you’ve been customizing a kernel for some time, seeing the progress along yourself, noticing the impact different changes have had, without even having to program it directly, you’d definitely be able to realize it’s the kernel what really sucks on OSX.
Try to put the machine to its limits, and it’ll become readily apparent how many of the progresses Linux has made over the years (specially since 2.6) are not still there on OSX.
I swear I can’t understand how OSX on servers is doing anything out there.
Could someone with experience on it comment whether there are important changes between the desktop and server version of OSX, WRT its kernel?
Well regarding your comments on ZFS, I will just quote a Sun engineer.
ReiserFS v4 may be a significant step forward for Linux file systems, but looking through the feature list on its Web site, I don’t see anything like the ability to add storage space dynamically or integrated checksums to protect against data corruption. ReiserFS v4 is also not 128-bit, so its ceiling is much lower than that of ZFS.
http://blogs.sun.com/dilpreet/entry/nice_s10_article
I’m not a big Reiser4 fan. While it is certainly interesting, one must be careful to sort the hype from the reality. And Namesys is very good at hype. I lost some faith in it when it was exposed that benchmarks on the Namesys site had the phases of the benchmark where ext3 beats Reiser4 (and badly) systematically removed, a situation that still has not been corrected years after Hans admitted to it on lkml. (I can dig up a link if requested.)
Plus, looking over the threads on lkml, I get the impression that the most interesting stuff, like file as directory, is not necessarily safe.
Furthermore, the most interesting features are not ones that developers could really use. For example, its database-like speed with large numbers of objects, and the file as directory semantics might make it easier to to write certain kinds of programs, even eliminating the need for a database… as long as you are writing for Linux only and don’t care about portability. If you do care about portability, you have to write it to run on other filesystems anyway, and that same code would work on Linux. So why bother writing it 2 ways?
ZFS has features that would be useful right now, and without the drawbacks, which is why I give it the nod over Reiser4.
Reiser4 is pretty well-hyped, but the underlying concepts really are sound, and are very innovative. The major benefits in Reiser4 are the following:
1) Seperation of the storage layer and the semantic layer. Hierarchical filesystems need to go, for the simple reason that years of experience has shown that most people really don’t understand them. All the unexperienced computers I know have “My Documents” directories with hundreds of files in it, because it doesn’t occur to them to create subdirectories, and the only reason its even in “My Documents” to begin with is that Word defaults to there. Seperating the storage layer from the semantic layer is a prerequisite to moving the semantic layer towards something people find usable.
2) High-performance storage of small files. Storing data in small files is a good thing. BeOS did it, and OS X is doing it (Spotlight’s architecture is really set up for it). Search-based filesystems, in one way or the other, are going to be a UI paradigm that’ll weigh heavily in the future, and Reiser4’s design is better positioned to take advantage of it than any other filesystem.
The politics surrounding Reiser4 are regretable, and from a quality of implementation point of view, I have no doubt ZFS is much more solid and trusthworthy. But ZFS’s design is a baby-step while Reiser4’s is a giant leap, and I think that deserves an enormous amount of credit.
Hmmm. I’m not sure I agree about heirarchical filessytems having to go. Surely most people understand the idea of folders within folders. Some people just don’t use them. Are you proposing that they search by metadata rather than organizing in folders? If they don’t want to organize into folders, I don’t really see them tagging every file with relevant metadata.
I’m not sure how Reiser4 is doing in this respect, but it is common for people to say that reiser3 is fast for lots of small files. And guess what? In the testing I’ve done, ext3 with dir_index blows the doors off of reiser3. (Yeah, I was shocked.) That’s with or without “notail”. Of course, with “notail” resier3 is still far more space efficient when storing tiny files.
“””I have no doubt ZFS is much more solid and trusthworthy.”””
Well, reliability is far and above my number one requirement in a FS. However, WRT Resier4 I’m not really thinking about the actual implementation details, but concern about more fundamental things. I believe it was Sep, 2004 that Reiser4 was submitted to mainline and there was a lot of discussion. I was struck by how many times kernel devs would say something like: “Hey Hans, look at the nastiness that occurs when you mix files-as-directories and symlinks” and the number of times that Hans basically had to say: “Uuuuhhh… we hadn’t thought of that. But we’ll fix it!”. Also all the times that when presented with problem X, he would quickly say “Oh, we’ll just do this.” And his own employees would explain to him why that won’t work.
Namesys tries to hard to “sell” or “market” it. I’m used to filesystems “earning” inclusion in mainline.
As a user, it just doesn’t give me a good feeling about that FS.
Yes, the politics are truly unfortunate because they cloud the issue so badly.
On the one hand, the kernel reviewers may be excessively nit-picky WRT resier4. The flip side is that Reiser4 proponents dismiss the valid criticms from kernel devs as just “politics”.
It’s like trying to evaluate an art piece in a smoke filled building. It’s too fuzzy to tell anything.
Edited 2006-12-16 22:11
1) Seperation of the storage layer and the semantic layer. Hierarchical filesystems need to go, for the simple reason that years of experience has shown that most people really don’t understand them. All the unexperienced computers I know have “My Documents” directories with hundreds of files in it, because it doesn’t occur to them to create subdirectories, and the only reason its even in “My Documents” to begin with is that Word defaults to there. Seperating the storage layer from the semantic layer is a prerequisite to moving the semantic layer towards something people find usable.
ZFS has that separation. The Posix semantics are handled by the ZPL (ZFS posix layer), which interacts with the DMU ( Data Management unit) for storage operations on object. Like the UNIX directory plugin works in Reiser4.
From the OpenSolaris website.
“The DMU is responsible for presenting a transactional object model, built atop the flat address space presented by the SPA. Consumers interact with the DMU via objsets, objects, and transactions. An objset is a collection of objects, where each object is an arbitrary piece of storage from the SPA. Each transaction is a series of operations that must be committed to disk as a group; it is central to the on-disk consistency for ZFS.”
Clients of the DMU furthermore include the ZVOL volume interface and /dev/zfs. ZFS already has a clear separation of the storage layer and semantic interface layer.
http://www.opensolaris.org/os/community/zfs/source/
You might find this useful to read.
If Applications require the kind of separation, it wouldn’t be hard to create another interface on top of the DMU to accommodate them.
I would say that ZFS is second to Reiser4. Reiser4 is a far more innovative filesystem that really addresses the point that byte-streams are just byte-streams, and that any distinction between metadata and data is just arbitrary.
I disagree. If meta-data and data distinction is the only feature that marks innovation in filesystems then maybe Reiser4 is more innovative.
ZFS completely redefines storage on a computer by making it more like main memory. What do you do on a box if you need more memory? Add a DIMM and presto more memory instantly no extra work required. With ZFS you need more storage space for your filesystems just add a disk or too and presto instantly more space, no extra work required.
Snapshots, clones and block checksums make it far more innovative than any other filesystem I am aware of. More users can use those features right away than one innovative feature in Reiser4 that only developers can use that too if it becomes popular enough.
The snapshot and cloning features for ZFS makes it perfect for Timemachine in leopard. Snapshots and clones don’t take up extra space because of ZFS’ COW semantics and are pretty instantaneous. ZFS can create volumes on which you can overlay other filesystem types. Apple could make a ZFS volume and overlay HFS+ on it and have all the ZFS features like cloning and snapshots and maintain backward compatibility.
I suspect that we would see an out of tree port first, followed by individual ZFS features making there way into the appropriate layers of the Linux kernel.
You might want to check this out:
http://zfs-on-fuse.blogspot.com/
As I see it it really doesn’t matter if Linux gets ZFS or not, most of the features are allready available from lvm, even though I must give Sun that ZFS is easier to manage. Anyway, most of us can’t afford the huge amount of disk that ZFS supports.
To Mac users on the other hand, ZFS will be a large step forward, as far as I know current MacOS filesystem can’t do things like snapshots. Something I really would like to have to create verifyable backups.
Yeah, LVM + RAID5 gets you a lot of it. But management is a bit of a pain. And it does not get you raidz or the the self-checking/self-healing features.
Personally, I feel very comfortable with ext3 and feel that it is a much maligned and underrated filesystem. (Current ext3 is actually quite fast, but nobody knows it.) 😉
But it’s not ZFS. There is no reason, however, that features could not be added to the Linux RAID and VFS layers + ext4 to close the gap.
Edited 2006-12-16 20:43
When one takes a close look at zfs – beyong SUN’s marketing machine – it’s easy to see that it’s not totally new. It takes certain features, combines them (that’s the actual achievement of zfs), and sells them as zfs. It’s not vital for Linux to have this since lvm and xfs achieve roughly the same.
“””
It’s not vital for Linux to have this since lvm and xfs achieve roughly the same.
“””
RAID-Z? Error detection? Self-healing?
And arguments that LVM/RAID/XFS is comparable would be more convincing if we had decent all-in-one management interfaces. And if XFS had ZFS’s (or ext3’s) resilience in the face of abnormal shutdowns.
Linux is my OS of choice. But I’m not blind to where Sun’s ZFS shines. If we had it in Linux we’d be trumpeting the fact from here to Alpha Centauri.
So let’s be good sports and not downplay Sun’s accomplishments here.
Edited 2006-12-17 21:51
We don’t have an all-in-one management interface. True. But that’s not the point here. I said it’s roughly comparable to zfs – which is certainly true. One has to look beyond SUN marketing and realize what zfs really does and is.
Let’s not downplay Linux’ accomplishments …
We don’t have an all-in-one management interface. True. But that’s not the point here. I said it’s roughly comparable to zfs – which is certainly true. One has to look beyond SUN marketing and realize what zfs really does and is.
Let’s not downplay Linux’ accomplishments
Yes, let’s downplay Sun’s (and the Solaris community’s) accomplishments instead!
But seriously, I could argue that a Win95 PC is roughly comparable to a Linux box. The differences lie in the details.
Over LVM, ZFS offers (among other things):
1 – End to end checksumming. Every time you read or write a block, it is checksummed. If you have RAID redundancy and the checksum shows the block to be corrupt, it is regenerated from parity/mirror. If not, at least you know something is wrong.
2 – NetApp style snapshots as opposed to VxFS style snapshots. People hear snapshot, they think ‘LVM can do that!’. LVM snapshots are geared more to backup, that is, a single point in time consistant snap of an active filesystem which is destroyed after the backup runs. Their changed blocks are stored on a separate logical ‘device’. ZFS snapshots are geared toward creating one every hour, keeping it indefinitely, and navigating to it as a UNIX directory when you realize, oh, I should not have deleted that. ZFS snapshots are stored in the same pool as their filesystem.
3 – No RAID5 write hole using pure software / no NVRAM RAID, and without killing performance. Your parity and data blocks are going to match! Even if they did not, upon reconstruct, ZFS checksums would tell you instead of blindly regenerating garbage data.
4 – (in OpenSolaris) Transparent iSCSI sharing. A volume (the ZFS volume manager is not limited to backing a ZFS filesystem) can be shared out as an iSCSI lun with a single command line switch.
On this board, Linux folks seem to be satisfied with saying, oh, ZFS is not that great, we could almost do all that stuff, if someone would just hack together a scripted implementation. What happened to technical leadership? The first step is admitting you have a problem
ZFS is not perfect. It is new software and is still maturing and adding features. Software like VxVM still has advantages over it in a few scenarios. But I am searching for an attribute where Linux/LVM leads it, and I just can’t find one.
“””We don’t have an all-in-one management interface.”””
Or the other features I mentioned… plus others I may have left out.
I would not downplay the accomplishments of the Linux community. Linux is my OS of choice for a reason.
I’m just tired of factionism.
I was a Unix guy a *long* time before I was a Linux guy. I love Linux because it is a great Unix that has momentum and is going places.
Linux, with its grass roots origins, addressed basic things, that you and I care about, that the big Unix vendors couldn’t be bothered to because they cared only about selling big expensive servers at a high profit margin.
Who needs a usable desktop? The desktop is not that important. Right?
So the quickest way in the world to draw my ire is to say, “Hey, just cobble together this, that, and the other, and it’s as good as what they’ve got!” … when it’s not true.
Our greatest strength is in recognizing (not denying) all the beneficial ideas that others produce and, where possible, making them available to a larger audience. (This also applies to our *BSD brethren; Don’t think I’m leaving you guys out!)
If we are behind in an area, we are behind. That’s OK.
Rest assured that useless face-saving tactics are seen by the rest of the world as useless face-saving tactics.
More important than “where we are” is “where we are going”.
When Sun, *BSD, Haiku, or MS puts forth a good idea or implementation, it benefits *us*.
And there is no shame in their getting there first. They “copy” from us, too.
-Steve
“When Sun, *BSD, Haiku, or MS puts forth a good idea or implementation, it benefits *us*.”
It depends. In the case of SUN one can rest assured that they chose an incompatible license on purpose. Don’t get me wrong: zfs is great altho due to license restrictions not interesting for me. And zfs-fuse will suffer a great deal from running outsiude the kernel.
In the case of *BSD I see little where Linux nowadays can benefit from. Same applies to Haiku.
I don’t use MS, and I am not interested in it – so I cannot comment on that.
As a mac illiterate, what file system does it use now? What are the advantages of using ZFS? I realize its not the base filesystem but mac now supports it.
Browser: LGE-VX9900/1.0 UP.Browser/6.2.3.2 (GUI) MMP/2.0
A demo of ZFS capabilities can be seen here:
http://www.opensolaris.org/os/community/zfs/demos/basics/
ZFS is incredibly powerful, but one notable feature is to be able to span a file system seamlessly across several disks and allow you to add more disks dynamically to that file system. No more need for partitions.
Quite impressive.
But did I get this right:
Striping is turned on by default?
This would really suck!
I know it can make write operations nearly twice as fast but it rougly doubles the probability of data loss.
———quick calculation———
If the disk fails with a probability of “p” then the chance of both disks not failing is (1-p)^2=p^2-2p+1 so the chance that one of them fails is 1-(1-p)^2=2p-p^2
However, since p should be pretty small we can neglect p^2
Therefore, for small numbers of disks, the probability of data loss grows linearly with the number of disks.
Now lets see what raid1 would give us:
For the array to fail, every single hdd has to fail, yielding p^n where n is the number of disks
———-end of calculation—————
So I think they should present the user with a question like:
Would you like to have
a) default storage space, speed and security
b) full storage, double speed, double risk (striping)
c) half storage, double reading speed, usual writing speed, greatly increased security
Or does it have some cool ways of preventing the higher risk of data loss that comes with striping?
Edited 2006-12-16 19:56
Or does it have some cool ways of preventing the higher risk of data loss that comes with striping?
Checksum on write.
Does it really stripe by default (like, split actual files), or just not-mirror? I have a testing zpool going with two disks I added separately, so I don’t know how data would actually be written in. I should try dumping a single, large file on and see what it reports for per-disk activity.
Checksumming is the major reason I wanted ZFS, along with the flexibility of throwing whatever odd hard drives I have in the pool. Unfortunately, it turns out ZFS doesn’t have volume evacuation implemented yet, so if you change your mind about having a certain drive in the pool, you can’t take it out without either destroying the pool or providing a replacement volume of the same size or larger, and it won’t grow into the space if it’s larger. In short, it turns out LVM is a little more flexible at this point. I hope ZFSv4, or whatever they’ll call it, is fast in the works.
“Does it really stripe by default (like, split actual files), or just not-mirror?”
Well, from what I understood, yes!
I went to the link http://www.opensolaris.org/os/community/zfs/demos/basics/
and after the first half of the presentation he says:
“And once again we can take a look at the statistics.
And now we’re seeing that IO is flowing through both of the mirrors.
This is called dynamic striping in zfs and it’s on by default and automatically and it means that you always get the best write and read bandwidth for your storage.”
Well, with Solaris, you get less hand-holding than in e.g. Linux. You have to know, what you are doing.
You can create striped-volumes (like RAID-0), mirrored volumes (like RAID-1), RAIDZ volumes. Mirrored volumes can be mirrored more than once. Since Solaris 10 11/06, there’s support for RAIDZ2, which is RAIDZ with 2 parities (i.e. -2 disks down in capacity).
It all depends on what HW you have and what you want to achieve. You can even mix-and-match, I think, all of the above. Right now, I have a zpool where some parts are mirrored and some are not. (I am planning to add devices to mirror everything today
.
That sounds like a nice feature, but it also sounds incredibly dangerous from a data loss point of view. What happens to the filesystem if one of the physical disks fail?
They have a special RAID called RAID-Z so it is not dangerous at all. RAID-Z is very advanced and does not use expensive hardware RAID. ZFS likes lots of cheap SATA drives.
Details on RAID-Z http://blogs.sun.com/bonwick/entry/raid_z
Also this PDF may help, http://www.sun.com/software/solaris/zfs_lc_preso.pdf
But the most fun is watching Bill Moore tell you what makes ZFS so good.
High quality – http://www.sun.com/software/media/real/zfs_learningcenter/high_band…
Low quality – http://www.sun.com/software/media/real/zfs_learningcenter/low_bandw…
All from the ZFS learning center – http://www.sun.com/software/solaris/zfs_learning_center.jsp
Mac OS currently use the HFS+ filesystem. More about it here http://en.wikipedia.org/wiki/HFS_Plus
As a mac illiterate, what file system does it use now? What are the advantages of using ZFS? I realize its not the base filesystem but mac now supports it.
There are two defaults. The first, inherited from the “old” MacOS, is called HFS+ and supports Unix semantics (mostly) along with Mac features like case-insensitive filenames and (I believe) attributes such as creator codes (which are the MacOS way of saying “this is a ‘whatsit’ file, and it’s created by the application ‘thingummy'”).
The second is UFS, which is from the BSD underpinnings and is a “normal” Unix filesystem.
ZFS improves on UFS by adding the ability to dynamically resize filesystems (the layout of data) and partitions, and snapshotting the contents of the FS. There may be other features I haven’t mentioned.
Oh, yeah, and it allows colossal filesystems.
HFS+, also with optional user space journalling now as well, since 10.3 IIRC. (preceded by HFS & MFS(flat file system with “fake” folders))
UFS is/was also supported, but was problematic as UFS did NOT support resource forks, which broke some mac programs. IIRC I tried UFS with the 10.0 release and even some Apple programs were broken to such a point that it really wasn’t a usable option at the time for a desktop environment.
HFS+ is a great filesystem, but its is reaching its EOL rapidly I believe and Apple would be smart to replace it. Going with something like ZFS would be a great choice, especially for its OSX Server line; XServes would definitely be a contender in the Data Centers then. Also having that kind of FS horsepower would be a great match with new features like TimeMachine. I would also like to see this happen because it could make one day Linux will probably get full support and if Solaris is GPL’ed then there is a good chance that an IFS will even be written for Windows. Wouldn’t that be nice, being able to read the same disc on MS, Linux, Mac OS, and Solaris
Yes sure Rayiner, the kernel sucks, of course you are sureley a kernel architect to say that!
You don’t have any single piece of competence to say wether or not a kernel sucks. So please don’t come here to say people that everything related to Linux works and everything else sucks. That’s the comments that you do which suck actually! Terribly!!!!!!!
I can not believe that there is someone in Earth who believe that Reiser4 will ever go somewhere. The Reiser file system is even not supported by many linux distributions (look at opensuse 10.2 recently), just because its development is simply chaotic.
<alert> zealot
MacOS X is becoming one of the best UNIX(tm) operating systems for workstations/daily use day by day, it’s got EVERYTHING! Now, imagine using a large ZFS pool thru iSCSI on your local network!
Well done! a superb filesystem for a superb OS.
a joke i hope?
very few electric, chimestry, physic… software available for mac os X
for linux, you have a lot of software for this kind of domain
you can convert it to macos X but few does it
How is the lack of such software the fault of the OS? If the software developers see enough of a demand for OS X support, they’ll port it over. For me, Matlab and to a lesser extent R support all my numerical needs.
Mac OS X is POSIX compliant and has an X server, so most Linux software can be compiled easily once the correct libs are installed. Check out http://fink.sourceforge.net/ if there’s a Linux package you want with no OS X binaries.
that cool to be everytime 3 late release
>very few electric, chimestry, physic… software
>available for mac os X
>for linux, you have a lot of software for this kind of
>domain
You can try Fink or even Gentoo’s emerge/portage project straight on MacOS X.
One thing you WON’T GET is ZFS on GNU/Linux.
That’s why IMHO MacOS X has reached an incredible outstandig level regarding other “UNIX players”.
Sun FAQ about ZFS, a Linux port “is currently being investigated.”
anyway, linux have a lot of system file for different use….
for a home use, ZFS utility is very debatable
don’t think zfs on macos X will give it more popularity…..
zfs it’s nice for server
mac os x server represent why 0.0001%?
linux server arount 25%
The site is down.
Here’s a fake-looking ZFS screen shot; maybe it’s the same one: http://forums.macrumors.com/showpost.php?p=3151944&postcount=131
I am getting me a mac.
You could always just install Solaris.
Solaris is extremly user unfriendly, it would be impossible to switch for average mac user. Don’t know about hardware support, but I guess it is a lot worse than Linux. For today only zfs and dtrace (IMO) are worth looking at in Solaris.
I could respond with “Solaris is extremely choosy about who it makes friends with” but that would be churlish as well as childish.
User-friendliness is all about perception. My perception of Solaris is distinctly different to yours. That’s fine. What matters is whether you can run the applications that you need with a minimum of fuss on your OS of choice. I think you are seriously underestimating the intelligence of the “average Mac user,” too.
Regarding HW support .. perhaps you should investigate the Solaris HCL rather than making assumptions.
If you can only see ZFS and DTrace as worth looking at, then perhaps the other features which Solaris provides are not relevant for you. However, you shouldn’ be fooled into thinking that your opinion is the be-all and end-all of “what matters in Solaris.”
For today only zfs and dtrace (IMO) are worth looking at in Solaris.
Apparently you’ve never used Solaris.
print “$subject”;
I think this will be one of the “top secret” features
Checksums are great, but won’t help at all when you lose a disk.
ZFS always uses striping; there is no “concatenate” mode.
The “default” configuration of ZFS depends on the user interface of the tool that you use to create the pool. If you use the “zpool create foo disk1 disk2 disk3” command, then you’ll get striping (like RAID-0); if you type “zpool create foo mirror disk1 disk2 mirror disk3 disk4” then you’ll get mirroring and striping (like RAID-10). If you use OS X Disk Utility, I don’t know which configuration you get.
Yes, checksums won’t really help you when you lose a disk. That’s why you should be mirroring your disks. But you knew that already 🙂
And re striping-by-default …. good thing ZFS has variable block sizes to help with making efficient use of the space on your disks.
does this man HFS+ will posibly be replaces with ZFS? if so that would be awsome. not that i have anyhitng against HFS+ i relaly think ZFS is the future…
Is there something like fuse to implement userspace filesystems for the mac? What’s the nearest I could do – is it possible to ‘fake’ disk image mounting for example, and do something like write/read to gmail?
Well, OS X supports mounting FTP and WebDAV shares, and that’s done with the help of a user-space daemon for the actual I/O, I don’t know if there’s a publicly-documented interface for it, though (the source is part of Darwin, so you’d be able to find out yourself).
It may be that it just uses NFS loopback mount tricks to do it (i.e., private NFS server running in user-space proxies to GMail/WebDAV/whatever)
Thanks – fun excuse to poke around the source
the politics behinf reiserfs are unfortunately something to take into account as Hans Reiser was convicted for killing his wife.
Also Suse dropped reiserfs for other reasons:
http://www.linux.com/article.pl?sid=06/10/12/2120204
Actually, he has not been found guilty, as of today (18-Dec-2006). He has been charged, but that is not the same thing as being tried and found guilty.
The Oakland Tribune has more information. http://www.insidebayarea.com/search?sortBy=mngi&similarTo=&similarT…
Hans was arrested not convicted. You should check your facts before you make such extreme allegations.
That being said, reiserfs has always felt too experimental for me to trust it with any important data. For that, I use ext3 or XFS; although, the features shown in the ZFS video make me wish it were available on Linux.
I mean, I’d be just extra bouncy and giddy if I could get ZFS on my Mac, particularly bootable. If I could boot to a 4 drive RAID 0+1 500G “usable” space on my machine, with ZFS reilability, that would be slick.
My only nit regarding this screenshot is simply that it says “Zettabye”. I may be mistaken, but I’d think it was Zeta, with one T.
And, frankly, adding something as fundamental as ZFS to Mac OS I think would be on Mac OS Server where it could be used to spectacular effect on Apples Raid product vs the version for the Desktops and Laptops.
But maybe in the next version we’ll get it.
This is from their FAQ reguarding the Zetta with one T or two. It is indeed spelt with two T’s.
What does ZFS stand for?
Originally, ZFS was an acronym for ‘Zettabyte File System.” The largest SI prefix we liked (‘yotta’ was out of the question) was ‘zetta’. Since ZFS is a 128-bit file system, it was a reference to the fact that ZFS can store 256 quadrillion zettabytes (where each ZB is 270 bytes). Over time, ZFS gained a lot more features besides 128-bit capacity, such as rock-solid data integrity, easy administration, and a simplified model for managing your data.
Developers of “other UNIX players” like DragonFly BSD and FreeBSD are working on ZFS integration.
I think the performance for MacOS X’s kernel has been blown out of proportion. Would it be great if it was first-class? Sure. But it wouldn’t significantly affect MacOS X’s utility for almost anyone.
The utility of the launchd system and commands like “sudo fs_usage” far outweigh any performance issue. Add to that a full suite of POSIX tools, dev tools, and an X server, all of which come with the OS, and it has available almost any software that linux does.
As for ZFS, Apple’s TimeMachine (one of the most innovative features coming in any OS, IMHO) doesn’t depend on it now, but I could imagine a future version that might. Part of Time Machine’s power, though, is is that it’s not just file based, but depending on how the applications use it will recover individual records, while using the application’s own search and UI. (Things like open your Address Book, search through history for a contact, and restore that 1 contact to the present with a couple mouse clicks… nothing else like it…) This type of thing is more than a file system snapshot hack…
There’s nothing innovative about Time Machine. Voluem Shadow Copy in Vista/Win2k3/XP does the same exact thing but better. Any application can hook into the VSS (Volume Shadow Service) just like in OS X. In fact, all SQL Server/MSDE based apps have a special API in Windows that is made just for them to use the VSS.
VSS does copy-on-write which only writes the differences of the new data not the entire file. You can also choose to have the entire file copied as well.
VSS has an API that allows hardware developers to make disk controllers that interface with the system too.
Here’s something innovative: VSS lets you make shadow copies of volumes that are not NTFS. So you can install an HFS+ or Ext2/3 driver and use Vista to shadow copy that as long as you have at least one NTFS drive mounted on the machine to store the shadow copies on.
You can also access the shadow copies of a Vista machine from an XP machine. Set up a home file server running Vista and while you’re browsing it via an XP machine you’ll see the ability to access and restore “previous versions” right from the XP sidebar.
VSS doesn’t need to run at intervals like Time Machine does. It runs as soon as a file is changed. With Time Machine, any time you make multiple changes to a file between the intervals (which by default are once a day) you lose those intermediary changes. Not so on Vista.
Vista doesn’t require a separate non-bootable volume like Time Machine either.
Comparably, Time Machine is a nasty hack next to VSS… just like File Vault is next to NTFS Encryption/Bitlocker.
Oh did I mention you can store you’re incremental VSS backups on CD/DVD/HD-DVD/Blu-ray as well? Try that with Leopard.
Edited 2006-12-17 06:38
blalbala
Vista is better.
blalba
Vista is better.
blabla
I had enough Vista here and there articles/comments over the past 5 years. This is about a rumor that ZFS might be part of Leopard which is totally unrelated to Vista.
If you like to praise Vista, move on to an appropriate fangirl site ..
Wow, someone is a little bitter. lol
My comment is related to samkass saying that Time Machine is the most innovative thing coming in any OS and that there’s nothing else like it. He’s just plain wrong on both points.
Don’t hate, celebrate. If you can’t deal with comparisons to other operating systems or any positive talk about Vista ( I’m sure if I would have bashed Vista I would have been modded up) then go surf distrowatch or badvista.org or whatever.
Edited 2006-12-17 06:54
*Hehe*
Obviously you seem to be a little bit bitter that there are people ingoring Vista and its features.
I never said I like or dislike Vista. I said your comment is totally out of place in this discussion and that there were and will be enough of those discussions about Vista and OS camparisons where you can post your praises.
One thing you seem to forget totally (and which you obviously don’t get):
As a user of operation system A, feature X can be the best thing in the world regardless of the fact that maybe operation system B has a similar feature. Why? Because as a user of operation system A OS B is irrelevant. Point.
Or does “Hey, my C64 boots in less than 2 seconds and it’s security is far better than Vista!” sounds interesting to you? Should I start posting that to Vista discussions?
I guess MS has enough money to make their own marketing – the don’t really need fangirls who put their “but Vista is better” or “but Vista has that feature, too” marketing in every discussion about every other OS or feature thereof.
Personally, I think that comparing a feature of Vista with a feature of an OS which is not released yet and which is only speculative makes you IMHO look like an .. – well – never mind.
OS News really needs an “ignore list” feature.
“I never said I like or dislike Vista. I said your comment is totally out of place in this discussion and that there were and will be enough of those discussions about Vista and OS camparisons where you can post your praises. ”
And I told you it wasn’t out of place because the original poster referenced other operating systems (presumably Vista too).
“As a user of operation system A, feature X can be the best thing in the world regardless of the fact that maybe operation system B has a similar feature. Why? Because as a user of operation system A OS B is irrelevant. Point.
Or does “Hey, my C64 boots in less than 2 seconds and it’s security is far better than Vista!” sounds interesting to you? Should I start posting that to Vista discussions? ”
You’re a little off with that comparison. If someone was going around touting Vista as the fastest booting and more secure OS ever then you would be on topic by making that commment, just as I was when I explained the Time Machine wasn’t unique in the feature he mentioned and the VSS does it much better.
“Personally, I think that comparing a feature of Vista with a feature of an OS which is not released yet and which is only speculative makes you IMHO look like an .. – well – never mind. ”
Clearly you have reading problems if you think I was comparing Vista to a speculative feature. Time Machine is public knowledge and many people, including myself, are actually running Leopard. There’s nothing “speculative” about it. Don’t try to make it seem that way just because YOU don’t know anything concrete about Leopard or Tiime Machine.
The point to remember is that “Time Machine” is not a shadow volume mechanism. It’s a user interface for backup. Shadow volumes would be an implementation detail (and not even the one it uses). As a user-interface for backup (for regular users), Time Machine is very innovative.
Ok so lets take the problem with another way. Microsoft do Time Machine better than Apple (that’s your point of view), but where are the equivalent of Dtrace and ZFS on Vista ? Hummm
The answer is: none. NTFS just suck and there is no equivalent of Dtrace.
The answer is: none. NTFS just suck and there is no equivalent of Dtrace.
The analogue to DTrace on Windows is Windows Eventing (aka Event Tracing for Windows). It first debuted with Windows 2000, has received updates in subsequent versions of Windows, and a new architecture in Windows Vista. You can publish, subscribe to, forward, and query live or logged events.
Windows Events
http://msdn2.microsoft.com/en-us/library/aa964766.aspx
Flight Data Recorder: Monitoring Persistent-State
Interactions to Improve Systems Management
http://research.microsoft.com/~emrek/pubs/FDR-OSDI-CR-FINAL.pdf
n4cer: First, thanks for the info on Windows Event Tracing which I wasn’t aware of before.
But from the looks of it, both FDR and Windows Events is still quite different and uses static instrumentation (the driver writer using WPP for example, needs to define events/messages), with FDR geared towards monitoring file I/O. Dtrace allows dynamic and arbitrary probing. There’s no overhead at all in areas where there is no probing because Dtrace does not “monitor” for certain calls or events like other static tracing methods, nor does code need to be modified to supply tracing points.
Rather, it modifies the running code to branch to and execute your probe (compiled from a D script) at the points you tell it to, which at its most fundamental level is any function call in any unmodified kernel and user land code, plus other extra tracing points within a function for Dtrace-aware code. Even without the modified, Dtrace-aware code, global variables, function arguments, return value, and type information (including user defined data structures) are all available to you.
Linux now aims to do the same with Systemtap. (competition is good, eh?
)
Edited 2006-12-19 04:22
(Edit: this message is regarding someone’s claim that Time Machine isn’t innovative because Vista includes file backup and implements it well.)
Okay, perhaps I’m missing something, please enlighten me. You seem to be discussing “innovation” from the point of view of low-level technology implementation. I’m talking about innovation from the value a feature offers to a user.
Let’s take a use-case. In Leopard, you can open the Address Book, and if you’ve found you no longer have a contact in it you need, you can click on Time Machine. Now you’re STILL in the Address Book app, but when you enter a user’s name in the search field, you can go back in time to the first time that person appears. When you find the record, you can click on it, click “Restore”, and it’s back. 3 mouse clicks and typing someone’s name.
Maybe I’m wrong. Maybe that is possible in Vista. I don’t really care how much better a *file* backup system is implemented, though, because *file* backup is a tiny part of what makes Time Machine so useful. It’s all above *record* *retrieval*. The additional value of the extra features you mention don’t seem so important to me compared to tight tie-ins with iPhoto, Address Book, and the other common apps that let me zip back in time using the application’s own search feature and snag individual records, photos, comments, etc., from the backup.
Edited 2006-12-18 01:53
The “record retrieval” and search features you mention are the easy part, especially when the backups are clones and not copy-on-write. It’s relatively trival for any developer to have their VSS-enabled app search through its own backups.
The other features I mentioned like copy-on-write and compatibility with all file systems must be implemented on the system level to be used by any developer.
“There’s nothing innovative about Time Machine. Voluem Shadow Copy in Vista/Win2k3/XP does the same exact thing but better”
Yeah, but can you fly through a visual representation of your files as if you were warping through space and time?
“Yeah, but can you fly through a visual representation of your files as if you were warping through space and time?
”
You act like that’s a good thing.
It’s a good thing, because it means mom and dad non-tech users will be able to rescue lost data on their own using a nice interface.
I hope you’ll be able to switch that off … 🙁
There will be three shareware apps and a dashboard widget to do that less than a week after the launch.
well if zfs will be start fs for leopard i consider buy it.
otherwise i will not waste my money for apple.
ZFS is very cool.
I hope it’ll be the default OS in Leopard, but don’t know if it can handle HFS+ metadata and resource forks. Anyone have an idea?
I haven’t RTFM’d on ZFS yet, but I can’t imagine anyone designing a filesystem without support for arbitrary metadata these days. Even the original NTFS supported it (although almost nothing could take advantage of it).
It’s entirely possible that the UNIX arrogance (ugo rwx bits, owner user/group, etc. is enough metadata for everybody!) might have hobbled it, but I doubt it, ZFS seems to support all kinds of other advanced filesystem features.
– chrish
Trust me when I say that the default OS is on Leopard is… uh… Leopard!
I think you mean the default file system, and I’d bet that it’s pretty late in the game to change the DEFAULT. I’m not even certain that you can boot off of ZFS yet.
I suspect it won’t be until the next release of OS X that ZFS is even entertained for a default.
“I hope you’ll be able to switch that off … :-(” Ha ha! Unlikely!
Although I can’t afford a Mac at the moment, I will strongly consider it in the future.
For those suggesting that ZFS is only for servers, I think you’re mistaken.
And, in truth, it will be Apple (assuming they bundle it) that will drag ZFS kicking and screaming in to the consumer space.
The two biggies are snapshots and volume management.
Specifically, snapshots, if exposed in a reasonable manner, give users the ability to put parts of their system in to a “known” state, and recover easily. Not as safe as an offline backup, but safe enough for many uses. It’s also instant and could be done with a keystroke or a mouseclick.
Volume management is important because it lets users simply slap in a drive and be done with it. Volumes, per se, can “go away”. Is you 100G drive filling up? Add another and its 200G. It’s not “Old Drive” with 2G free and “New Drive” with 100G free, it’s “The Drive” with 102G free.
Very handy, very simple.
So, yes, major win for ZFS is large arrays of drives on backend servers, but don’t discount it’s properties for simple consumers, particularly if the skilled UI folks at Apple can expose those features in a intuitive and easy to use manner.