Linked by Thom Holwerda on Sun 14th Apr 2013 20:30 UTC
Hardware, Embedded Systems "In the past five years, flash memory has progressed from a promising accelerator, whose place in the data center was still uncertain, to an established enterprise component for storing performance-critical data. It's rise to prominence followed its proliferation in the consumer world and the volume economics that followed. With SSDs, flash arrived in a form optimized for compatibility - just replace a hard drive with an SSD for radically better performance. But the properties of the NAND flash memory used by SSDs differ significantly from those of the magnetic media in the hard drives they often displace. While SSDs have become more pervasive in a variety of uses, the industry has only just started to design storage systems that embrace the nuances of flash memory. As it escapes the confines of compatibility, significant improvements in performance, reliability, and cost are possible."
Thread beginning with comment 558498
To read all comments associated with this story, please click here.
Comment by TempleOS
by TempleOS on Mon 15th Apr 2013 03:12 UTC
TempleOS
Member since:
2013-04-03

Operating Systems with Paging could not do this -- my operating system is single-address-map. In all the cases where I read files and place them into pointer-linked structures... I would never have to serialize them unless exporting to another machine. My crazy heap pointer-linked structures could be sphegetti with one file mixed in other files, freely, just like heap memory naturally gets all mixed-up.

Reply Score: 1

RE: Comment by TempleOS
by ssokolow on Mon 15th Apr 2013 06:42 in reply to "Comment by TempleOS"
ssokolow Member since:
2010-01-21

Operating Systems with Paging could not do this -- my operating system is single-address-map. In all the cases where I read files and place them into pointer-linked structures... I would never have to serialize them unless exporting to another machine. My crazy heap pointer-linked structures could be sphegetti with one file mixed in other files, freely, just like heap memory naturally gets all mixed-up.


Filesystems are already spaghetti. That's what fragmentation means and different filesystems have different approaches to keeping track of which file a fragment belongs to and their ordering.

You just don't see that because the OS does such a good job of providing an abstraction layer between you and the on-disk format.

Honestly, the most promising use I can think of for what you're envisioning is what WinFS and GNOME Storage were trying for... A filesystem that, rather than being hierarchical, is an SQL database.

Both attempts failed for lack of performance but, if they can bring non-volatile storage up to the same performance level as RAM, that problem might go away.

Reply Parent Score: 4

RE[2]: Comment by TempleOS
by Laurence on Mon 15th Apr 2013 11:36 in reply to "RE: Comment by TempleOS"
Laurence Member since:
2007-03-26

You're missing his point there. I think he's talking about the future of storage working a bit like this:

Say you have a character array of the alphabet, you can just request the 5th element in that array and the value returned would be "e". Say if that array was actually a file in RAM, then we'd only ever need to pass pointers rather than having to duplicate the the file (ie persistent version in storage and and a volatile version in RAM).

There's a few issues with that mindset though:

1/ You'd need to know all the memory addresses of each bit of data you're wanting to retrieve before hand. That would require an epic centralised database that would fragment in no time at all. This isn't a show stopper though, there will be ways around it, but it's certainly a major issue that would need to be resolved. (it's probably worth noting that we can already jump to specific blocks of data within a file without having to preload the file beforehand. But it's beyond impractical to do so, which is why we use the much simpler approach we have currently)

2/ The contents of files are not static. Take XML parsing, the tags will land in different locations within the file depending on the data held (bad example, but the 5th line of a dictionary will differ depending on the language of the dictionary). So most forms of structured data will still need to be stepped through before they can be parsed - which defeats the point of "spaghettifying" pointers.

What wouldn't be an issue is "in memory" changes no affecting the saved versions (eg opening a Word document, making edits and the version I'm editing not automatically editing the version I have saved unless I explicitly click "save") as we already have technology developed that works around that limitation (CoW file systems).

While TempleOS's vision is interesting, I think it adds waaay too much complexity for comparatively little gain. Particularly when most instances of structured data will still need to be stepped through.

Reply Parent Score: 3

RE[2]: Comment by TempleOS
by Flatland_Spider on Mon 15th Apr 2013 19:35 in reply to "RE: Comment by TempleOS"
Flatland_Spider Member since:
2006-09-01

Honestly, the most promising use I can think of for what you're envisioning is what WinFS and GNOME Storage were trying for... A filesystem that, rather than being hierarchical, is an SQL database.

Both attempts failed for lack of performance but, if they can bring non-volatile storage up to the same performance level as RAM, that problem might go away.


BeFS did the filesystem as a database thing well before those two, and it worked really well.

Reply Parent Score: 1