Linked by Thom Holwerda on Thu 6th Nov 2008 15:33 UTC, submitted by Gregory
Hardware, Embedded Systems It's no secret that SSDs suffer from performance penalties when it comes to small random writes. Even though more modern SSD try to solve some of these issues hardware-wise, software can also play a major role. Instead of resorting to things like delaying all writes until shutdown and storing them in RAM, SanDisk claims it has a better option. At WinHEC yesterday, the company introduced its Extreme FFS, which it claims will improve write performance on SSDs by a factor of 100.
Order by: Score:
so...
by hobgoblin on Thu 6th Nov 2008 16:02 UTC
hobgoblin
Member since:
2005-07-06

either users have to install a driver everywhere they go, or these devices pretends to be fat formated.

as this is not developed my microsoft, it will not be natively supported by microsoft (see things like rw optical media formated to act as "floppies", no native support by the big gorilla of the market).

like it or not, fat has become the lingua franka of removable storage media. but as its showing its age (even the fat32 version is closing on retirement in computing lifetime), microsoft have rolled out exfat in a hope to corner the market. and they probably will, sadly...

Reply Score: 2

RE: so...
by poundsmack on Thu 6th Nov 2008 17:23 UTC in reply to "so..."
poundsmack Member since:
2005-07-13

sooo when you want to access your files on your linux computer from a windows box you have to install the driver to read ext3 (or whatever file system it may be)?

thats the deal with new technology, why people complain about it i dont know. you have to install a PDF reader when u have windows, same with java. people are just used to them being there these days. back in my day, if you wanted ot do something, you installed something to do it with. kids these days are spoiled. haha

Reply Score: 5

RE[2]: so...
by hobgoblin on Thu 6th Nov 2008 19:40 UTC in reply to "RE: so..."
hobgoblin Member since:
2005-07-06

yes there are drivers for ext3 on linux.

but what about the proverbial "aunt tillie"?

the ones that gets scared silly by a simple printer install?

Reply Score: 0

RE[2]: so...
by Laurence on Fri 7th Nov 2008 09:45 UTC in reply to "RE: so..."
Laurence Member since:
2007-03-26

sooo when you want to access your files on your linux computer from a windows box you have to install the driver to read ext3 (or whatever file system it may be)? thats the deal with new technology, why people complain about it i dont know. you have to install a PDF reader when u have windows, same with java. people are just used to them being there these days. back in my day, if you wanted ot do something, you installed something to do it with. kids these days are spoiled. haha


The point is it shouldn't have to be like this any more.

It's all very good and well saying "people used to cope in my day", but the fact of the matter is people aren't using 286s and Windows 3.x any more. People expect their modern, bulky, multi-functional OS be all inclusive. They expect part of the hefty 1GB install (or whatever size Windows demands these days) to contain all the tools required to read all the mediums they use from day to day.

To take your argument further: in my day people coped without GUIs, HDDs, CDs and the internet - however I wouldn't expect anyone go back to the desktop BASIC days, nor would I say anyone complaining about a lack of internet or frustrated with their slow / uninturative GUI was "spoilt".

Oh, and your PDF analagy isn't wholely accurate either as you're effectively comparing a floppy disk (removable storage) to a word document (document format).

Edited 2008-11-07 09:47 UTC

Reply Score: 3

RE[3]: so...
by poundsmack on Fri 7th Nov 2008 19:56 UTC in reply to "RE[2]: so..."
poundsmack Member since:
2005-07-13

who is to say you would have to instal something everywhere you go. its perfectly plausible that MS could release something via automatic update and BAM, supported. its just that simple. the fact is no truely new technology (that is technology that differs enough from current standards) is going to work on systems that were developed before it was. the OEM's and OS developers have to issue patches or have the driver downloadable or inclueded with the new storage media.

Reply Score: 3

RE[4]: so...
by Laurence on Sat 8th Nov 2008 10:45 UTC in reply to "RE[3]: so..."
Laurence Member since:
2007-03-26

I didn't say you would have to - i'm only responding to the previous poster saying you shouldn't have after he called users spoilt.

Reply Score: 2

RE[2]: so...
by StephenBeDoper on Fri 7th Nov 2008 18:07 UTC in reply to "RE: so..."
StephenBeDoper Member since:
2005-07-06

thats the deal with new technology, why people complain about it i dont know.


It's largely because many people have unrealistic expectations of technology, due to a lack of understanding. The "I don't understand it, so I will assume it works by magic" effect.

Reply Score: 2

RE: so...
by judgen on Fri 7th Nov 2008 08:52 UTC in reply to "so..."
judgen Member since:
2006-07-12

I dont know the exact limit of fat32. (ive only used a 400gb disk with fat so far) But ther filesize limit of 4gb is a showstopper non the less, so they need something new and as NTFS isnt as free as fat is so i guess its up to the hardware guys to make something new, probably with a fat32 comp-layer or so. Im sure those guys just make it wirk, just like they have come through for us before.

Reply Score: 2

RE[2]: so...
by hobgoblin on Fri 7th Nov 2008 14:11 UTC in reply to "RE: so..."
hobgoblin Member since:
2005-07-06

iirc, the issue is that ones one go beyond a specific size, the cluster size has to grow.

that means that if a file is below the size of the cluster, it will still take up the whole cluster, even if most of it is empty space.

still, it may be that im jumping the gun, as reading the right hand table here:
http://en.wikipedia.org/wiki/File_Allocation_Table

fat32 should be able to address drives all the way to 2TB.

Edited 2008-11-07 14:13 UTC

Reply Score: 2

RE: so...
by christian on Fri 7th Nov 2008 14:27 UTC in reply to "so..."
christian Member since:
2005-07-06

either users have to install a driver everywhere they go, or these devices pretends to be fat formated.


No. The device will just look like a hard disk. Just an array of sectors, on which you write whatever file system you like.

The device will map your logical sector numbers to physical flash pages, using what is essentially a dynamic map.

Having not read the details of ExtremeFFS (it is probably patented) I theorize it operates by collecting together temporally close sector writes in a cache, and writing them all in one go to a fresh page. The garbage collection looks for stale or partially stale pages, queues any live data for writing in the next write batch, then cleans the (now) stale page ready for use in the free list.

Of course, this may all be wide of the mark, in which case sorry for the noise, but a SSD that required drivers over and above the link layer (SATA) would have a very limited market and just wouldn't make sense.

Reply Score: 2

RE[2]: so...
by hobgoblin on Fri 7th Nov 2008 14:33 UTC in reply to "RE: so..."
hobgoblin Member since:
2005-07-06

i read the FS part as File System. and unless one file system can pretend to be something else, its at the very least exposed to the os. and if so, it will need drivers, either built into the os or installed afterwards.

Reply Score: 2

RE[3]: so...
by christian on Sun 9th Nov 2008 15:12 UTC in reply to "RE[2]: so..."
christian Member since:
2005-07-06

i read the FS part as File System. and unless one file system can pretend to be something else, its at the very least exposed to the os. and if so, it will need drivers, either built into the os or installed afterwards.


Well, TrueFFS, of which this is an evolution, provides a block mapping, plus some FAT specific optimisation. It certainly didn't provide a filesystem in the classic operating system sense.

Reply Score: 1

Greate news.
by SReilly on Thu 6th Nov 2008 16:03 UTC
SReilly
Member since:
2006-12-28

I can't wait to try this out. I'm looking forward to having a laptop that doesn't burn my goolies every time I leave it on my lap for more than 2 minutes ;-).

Reply Score: 3

can anyone clarify?
by Yamin on Thu 6th Nov 2008 16:04 UTC
Yamin
Member since:
2006-01-10

So basically the only overhead of flash comes in when writing to a non-empty location. It needs to erase and then write... 2 operations.

If writing to an empty sector, there is only 1 operation (write).

So basically the way to optimize flash drives is
1. To make sure they write to empty locations
2. To cache writes and then write them at a later time

Is that understanding correct?

Now I'm a little skeptical on this 100x number. I mean the most you could ever really improve is 2x by removing the extra erase cycle.

The caching aspect can basically be done on any kind of drive and improve its performance just based on the fact that RAM is faster than disk.

But I'm assuming they taking the overall system into account
increase = optimal FS with caching / raw write to disk.

Reply Score: 3

RE: can anyone clarify?
by tyrione on Thu 6th Nov 2008 23:01 UTC in reply to "can anyone clarify?"
tyrione Member since:
2005-11-21

Oh come on now. Stop being rational. The theoretical throughputs of PCI to the latest have always fallen drastically short of their theoretical fanfare.

How dare you come in here and poop all over fantasy!

Barkeeper! Give this man a double!

Reply Score: 3

RE: can anyone clarify?
by TQH ! on Fri 7th Nov 2008 15:34 UTC in reply to "can anyone clarify?"
TQH ! Member since:
2006-03-16

TheRegister explained it nicely:
http://www.theregister.co.uk/2008/11/07/sandisk_extremeffs_dram_buf...

Short version:
They keep a cache of free pages to write to, and mark the old one as dirty. Another thread then cleans the dirty page and puts it in the free cache in the background.

Reply Score: 2

Nice but...
by mmu_man on Thu 6th Nov 2008 16:05 UTC
mmu_man
Member since:
2006-09-30

Is it open ?
Will it work in every OS I use? Or can it be ported to them?
Else there is just no use to it.
I don't put my data on something I can't control, period.

Reply Score: 6

v RE: Nice but...
by PJBonoVox on Thu 6th Nov 2008 16:32 UTC in reply to "Nice but..."
v RE: Nice but...
by Chicken Blood on Thu 6th Nov 2008 18:16 UTC in reply to "Nice but..."
RE: Nice but...
by Michael on Thu 6th Nov 2008 20:20 UTC in reply to "Nice but..."
Michael Member since:
2005-07-01

Given that flash drives could one day replace HDDs, I'd say SanDisk have more to gain from growing the market generally than from trying to secure their own position in it.

If that is the case then it would make sense to spread this technology about as much as possible, either through licensing or open source-ing it.

If they try to sit on this, then it's always going to be a minority tech. But they may see things differently.

Reply Score: 6

I'm hoping...
by IanSVT on Thu 6th Nov 2008 16:21 UTC
IanSVT
Member since:
2005-07-06

I'm hoping that this means more craptastic tools like anything that involving U3 throws at you. I love that stuff so much. The only way it could get better would be if it would use up some more drive letters mounting pointless volumes!

Reply Score: 3

RE: I'm hoping...
by helf on Thu 6th Nov 2008 18:09 UTC in reply to "I'm hoping..."
helf Member since:
2005-07-06

That is why you run a handy little tool that wipes that crap off the memory key.

Reply Score: 5

RE[2]: I'm hoping...
by Clinton on Fri 7th Nov 2008 06:21 UTC in reply to "RE: I'm hoping..."
Clinton Member since:
2005-07-05

It shouldn't be on there in the first place. I'm buying some external storage for my personal data. I don't want somebody else's data on it.

If I was interested in their crapware, I'd download it from their website.

Reply Score: 3

About time
by Liquidator on Thu 6th Nov 2008 17:04 UTC
Liquidator
Member since:
2007-03-04

Last week, I purchased a Transcend SSD for my laptop. It's silent but very very slow, even using the legacy FAT32 filesystem. It took an afternoon to install Windows XP. If it weren't for silence, I would be using an old good WD Scorpio HDD ;)

Reply Score: 3

Not the Answer
by MightyPenguin on Thu 6th Nov 2008 18:32 UTC
MightyPenguin
Member since:
2005-11-18

This is just a stopgap solution that probably won't really work for most people. Just like the all the crazy memory compressor software that used to be out there that tried to give you more ram with a bunch of software compression tricks. It kind of worked some of the time, and most of the time your system was slower and more unstable.

No, the real solution as always is newer technology on the hardware end. There are new kinds of flash memory coming out that are faster to read/write and can handle more write cycles. I foresee the day when people might not buy ram for mid/lower range systems because the flash storage they use for their hard drives is fast enough to use a modest 30gb of their 500gb flash drive for ram. This is similar to what's already being done on a lot of mobile devices now.

And I think this would help a lot with suspend/resume support. Yeah the hardware devices would need to be reinitialized, but your ram image could be on your hard drive so if you loose power, no big deal.

Reply Score: 3

What classifies as "empty" then?
by UltraZelda64 on Thu 6th Nov 2008 21:39 UTC
UltraZelda64
Member since:
2006-12-05

For a section of a flash device to be "empty," does it just have to be zeroed? Or when a block's been written to once, does it somehow end up forever-written (ie, it will always have to be written to twice from then on)? In other words, if you performed the following command:

dd if=/dev/zero of=/dev/sdx

...does that automatically clear the "written" flag of the entire device and make it so that all future writes are one write instead of two (ie, make it "new" again)? Well, until you've put it through some decent use, creating and deleting files, of course, which is when you'd end up with random data again as before.

Reply Score: 2

Its the erase
by transputer_guy on Thu 6th Nov 2008 23:32 UTC
transputer_guy
Member since:
2005-07-08

I hadn't thought about why Flash is so slow from a user point since I really can't see through all the complexity involved in both the OS, PC and the Flash technology used and it also varies by OS, and Flash and Controller vendor.

Now I put my chip design hat on and something horrible clicks.

The erase time in almost every non volatime storage chip that I have been familiar with over 3 decades has always been 2 or many many orders of magnitude greater than the write time all the way back to the 1st EEPROM which is really not that different from modern Nand EEPROM Flash.

Back in the 70s, 80s, erasing had to be performed on the entire chip in one go with an external UV lamp and it could take several minutes or longer (maybe 20 mins), and all the UV lamp did was to help the trapped electron charge in the floating gates sneak through the silicon oxide interface, it used a see through quartz window.

Jump forward a few years and the erasure was done internally by a high Voltage charge pump that still had to erase the whole chip. These pumps take many clock cycles to pump up a voltage grid to power the erase cycle. Soon afterwards, it became possible to partition the entire chip into smaller blocks and route the high erase voltage to specific blocks. This still took several orders longer than individual writes.

It is almost certainly the case that erasing any block takes possibly microsecond or more based on the physics of the storage mechanism even if the SSD or Flash vendor says writes can occur at upto 100MB/s. I will have to look up that value.

If every write includes erase then performance is going to be terrible for small data writes so it makes sense to group writes into each block.

One solution if for the OS to perform background Erases on all blocks not containing data so that in the future cleared blocks will be available.

my 2c

Reply Score: 4

RE: Its the erase
by dsmogor on Fri 7th Nov 2008 09:52 UTC in reply to "Its the erase "
dsmogor Member since:
2005-09-01

I guess this is exactly what UBIFS (and I guess Extreme FFS too) is supposed to do: treat whole FLASH as a huge log file registering all write operations and run garbage collection in some backroung thread to reclaim outdated entries. The block map resides in ram, is written to flash device on umount time but can always be rebuilt from the scattered logs (which contain enough metadata aside from content) in case of power failure.
This fully embraces FLASH performance specifics:
Slow inplace write + 0 seek time.

Reply Score: 3

RE: Its the erase
by TQH ! on Fri 7th Nov 2008 15:36 UTC in reply to "Its the erase "
TQH ! Member since:
2006-03-16

And that is what they do.

Reply Score: 1

Linux support?
by ruel24 on Fri 7th Nov 2008 02:31 UTC
ruel24
Member since:
2006-03-21

If it doesn't work in Linux, I'll pass on it...

Reply Score: 2

Except the Intel SSDs....
by looncraz on Fri 7th Nov 2008 02:44 UTC
looncraz
Member since:
2005-07-24

From a detailed performance review linked from overclockers.com, the Intel SSD entries do not suffer from the random writes performance bug ( it really is just a bug ( in the SSD core logic ) ).

Given this, the development of a new file system to deal with improperly designed hardware seems a bit... uninformed :-)

However, even on normal disks, it is often better to group and "burst" write ( or writev[ector] ) to disks[/sectors/cylinders], simply because it is more efficient ( though can lead to integrity issues, so journaling is a must ).

In any event, the biggest advantages can be seen when the application side is designed to work with the performance characteristics of the hardware on which it is running. For instance, my LoonCAFE ( still unreleased ) tries to determine best performance for writes to optimize file system accesses - especially for the read-ahead caches.

Oh well, so long as purely generic solutions are employed to address specific problems, I will reign king :-)

--The loon

Reply Score: 2

RE: Except the Intel SSDs....
by dsmogor on Fri 7th Nov 2008 09:36 UTC in reply to "Except the Intel SSDs...."
dsmogor Member since:
2005-09-01

Well the incompatible performance characteristics have to be overcome somewhere. Be it the microcontroller embedded in a SSD (on FAT devices) or a OS driver (custom FS route).
I guess SANDISK tries to make a bold move to establish a standard facilitating the 2nd route on consumer PCs. Moving the logic out of the devices will make them cheaper to produce and result in some additional cash from IP royalties.

Reply Score: 2