Linked by diegocg on Thu 7th Nov 2013 22:19 UTC
Linux Linux kernel 3.12 has been released. This release includes support for offline deduplication in Btrfs, automatic GPU switching in laptops with dual GPUs, a performance boost for AMD Radeon graphics, better RAID-5 multicore performance, improved handling of out-of-memory situations, improvements to the timerless multitasking mode, separate modesetting and rendering device nodes in the graphics DRM layer, improved locking performance for virtualized guests, XFS directory recursion scalability improvements, new drivers and many small improvements. Here's the full list of changes.
Thread beginning with comment 576477
To read all comments associated with this story, please click here.
Btrfs dedup
by WereCatf on Fri 8th Nov 2013 14:28 UTC
WereCatf
Member since:
2006-02-15

I'm interested in Btrfs's newly-gained support for data deduplication, but after taking a look at this app called "bedup," which is supposed to do the actual heavy lifting, I'm left confused. The application only looks for identical files, it doesn't look for identical blocks of data, so does this mean Btrfs's dedup is also limited to file-level deduplication or is it just a limitation of the app in question?

Reply Score: 3

RE: Btrfs dedup
by Bill Shooter of Bul on Fri 8th Nov 2013 16:42 in reply to "Btrfs dedup"
Bill Shooter of Bul Member since:
2006-07-14

I think its just the bedup app, but don't quote me on that. I think its just a case of crawling before walking. Files are easy to examine for dupes, and its fairly useful for a lot of people like myself. Although, I may wait a few kernel revisions for any bugs to shake out before using it on anything vital.

Reply Parent Score: 3

RE: Btrfs dedup
by Laurence on Fri 8th Nov 2013 17:50 in reply to "Btrfs dedup"
Laurence Member since:
2007-03-26

If you want deduping then you're better off with ZFS - not only does it work against fragments, but it works live.

In fact on the whole I've been unimpressed with Btrfs during my recent testing.

Reply Parent Score: 2

RE[2]: Btrfs dedup
by Alfman on Fri 8th Nov 2013 18:57 in reply to "RE: Btrfs dedup"
Alfman Member since:
2011-01-28

Laurence,
"In fact on the whole I've been unimpressed with Btrfs during my recent testing."

I've been meaning to do this myself, I'm very interested in what you tested and the results of your tests, if you don't mind elaborating.

I've maintained nonstandard kernels in the past, but I'm trying to get away from that and stick to mainline as much as possible, so ZFS is less appealing for that reason even though it seems to be robust and mature.

I have a very strong desire to install a copy-on-write FS on production servers to replace an rsync --link-dest solution I have deployed for generational backups. It's pretty clever and quite efficient, however one major problem with this approach is that files cannot be moved, otherwise they fail to link. It's possible to relink dups after the fact using 'hardlink' or 'fdupes', but it's not ideal and raises other concerns about unintentionally hardlinked files getting restored.


One question I do have is whether it's possible to copy one Btrfs FS in deduped form from one host to another without having to rededup it anywhere in the process? I'll research it eventually, but maybe someone here knows...?

Reply Parent Score: 2

RE[2]: Btrfs dedup
by diegocg on Fri 8th Nov 2013 20:14 in reply to "RE: Btrfs dedup"
diegocg Member since:
2005-07-08

Live dedup is definitively not a perfect solution, it hurts performance and for many people it's a bad choice. Btrfs will add live dedup in next releases, so users can choose which method suits better to their needs.

Edited 2013-11-08 20:14 UTC

Reply Parent Score: 4