Linked by Thom Holwerda on Mon 30th Jan 2012 20:39 UTC
General Unix Finally something really interesting to talk about. If you've used UNIX or any of its derivatives, you've probably wondered why there's /bin, /sbin, /usr/bin, /usr/sbin in the file system. You may even have a rationalisation for the existence of each and every one of these directories. The thing is, though - all these rationalisations were thought up after these directories were created. As it turns out, the real reasoning is pretty damn straightforward.
Thread beginning with comment 505784
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[6]: We are stuck in the past.
by axilmar on Fri 3rd Feb 2012 12:27 UTC in reply to "RE[5]: We are stuck in the past."
axilmar
Member since:
2006-03-20

As I said before, metadata. Or maybe media information from something like 'mediainfo'.


It's not metadata. It's data. And it's not only media that have useful data.

Nor should it be. Wrong level to have it.


Yes it should. There should be a standard for it, to allow applications to interoperate at data level.

Not quite, but it's not really hierarchical either, not with links, it's more of a network. Still easy to argue is a database of a form.


You can't run queries on a filesystem regarding the data inside the files, and therefore it is not a database.

The case in question is meta data


Nope. The discussion is about information management, not metadata only.

the tool 'extract' sticks the meta data in a set structure out to stdout.


The schema of the information output cannot be queried at runtime.

With that, find and grep, you could write a "query" that find all jpegs that where taken with a certain camera.


But not all jpegs taken on a certain afternoon, or within a specific time period, or with multiple cameras or users, or many other things.

Plenty of programs that build custom indexs/database of meta data for this kind of purpose


If this functionality was supported out of the box, these programs would be redundant.

Example please. Windows WMI certainly hasn't convinced me.


Almost every application has a layer of data input/output from/to files. This layer would be redundant if databases were supported out of the box.

What parts exactly? Not all fileformats are like that, many are just blobs


Nope. All file formats have an internal structure, otherwise they could be read after they were written.

What about a video, once you dismiss the metadata stuff, how are you going to store the stream?


As an array of frames, and each frame being an array of pixels.

On their own one is likely meaningless, and won't have anything in it you can search.


Nope. One can search for a particular scene, employing pattern matching with another picture, or even a hand-drawn one, for example.

Many filesystem do offer transactions


Many, but none of the major ones, as far as I know.

The SQLite site has some a doc on how they do it generically


SqlLite achieves transactions through various mechanisms, as described in the paper. These mechanisms are based on the functionality provided by filesystems, but the filesystems themselves do not have the concept of transaction.

In order to enable transactions in another application, one has to rewrite the same SqlLite mechanism.

Pretty much every OS has a notification system for file or folder changes.


But these notification mechanisms are not compatible with each other, many O/Ses don't even have such notification mechanisms, these notification mechanisms don't work over the internet, and applications cannot be notified about what exactly changed inside a file.

Found this, sounds like what you want:
http://en.wikipedia.org/wiki/Pick_operating_system


Not bad. It's a step in the right direction.

Reply Parent Score: 2

jabjoe Member since:
2009-05-06

I don't think you have understand much of what I just said. I give up. Please try and program your crazy idea.

Just please ponder why Pick is in the dust bin of OS history and Unix not only survived but was endlessly copied.

Reply Parent Score: 2

Alfman Member since:
2011-01-28

jabjoe,

"I don't think you have understand much of what I just said. I give up. Please try and program your crazy idea."

Thanks for the support, haha. I offer you two pieces of advice for talking to people like me in the future. 1) try to talk to us as intelligent peers, nobody likes to be talked down to. 2) Focus less on discouragement and more on constructive criticism which might yield better ideas.


"Just please ponder why Pick is in the dust bin of OS history and Unix not only survived but was endlessly copied."

Never used pick. On a side note unix was backed by one of the biggest monopolies in it's time, which might have a lot to do with it.

Reply Parent Score: 2

axilmar Member since:
2006-03-20

I don't think you have understand much of what I just said.


I've understood everything you said.

I give up.


It's ok, you don't have any arguments, I understand.

Please try and program your crazy idea.


Apparently, it's not that crazy, since there are various implementations around.

Just please ponder why Pick is in the dust bin of OS history


For the same reason BETA, a superior video format, died and VHS, an inferior format, prevailed.

For the same reason the Amiga, a superior computer, died and the PC, an inferior computer, prevailed.

For the same reason the MC68000, a superior CPU, died, while the 80x86, an inferior CPU, prevailed.

For the same reason LISP machines, a superior computer architecture, died, and other, inferior ones, prevailed.

For the same reason BeOS, a superior IS, failed, and other inferior OSes prevailed.

See? it's not always the better product that succeeds.

Unix not only survived but was endlessly copied.


And that's why he have so many problems in IT.

Reply Parent Score: 2

Alfman Member since:
2011-01-28

Edit: oops, I got conversations mixed up and replied earlier to wrong post.

Anyways, it is interesting how two people can look at the same thing and come up with different opinions. But as long as we recognize that we all have different needs, then we should be able to get along. ;)

Edited 2012-02-03 14:55 UTC

Reply Parent Score: 2

JPollard Member since:
2011-12-31

"As an array of frames, and each frame being an array of pixels."

Um.. video isn't like that. You CAN tread video that way, but it is too incomplete. For instance, a video frame is actually a collection of blocks, each block is compressed. Subsequent blocks use tables generated from the first block for further decoding...

And then there is audio... each audio track associated with a frame is blocked.. and there can be many separate audio tracks.. And for any video track, there can be multiple alternate video tracks.

Now we get to the additional packaging... The video may be encrypted as well... Some tracks encrypted, some not.

NOT GOOD FOR A DATABASE.

And trying to coerce hundreds of different formats into a database would make the database useless. Especially when most data won't need the complexity.

I have worked with wether simulation data. A single run is 100GB or more (one week at low resolution). Some simulations are in 3D for just a few hours (also 100GB). Searches are made for intersections...(weather patterns)... yet the pattern is not something that can be described in SQL, which is really really bad at it.

These searches are closer to what is used in gaming - A 3D mathematical intersection from different points of view....

Database queries are just really stupid at that. And really really slow.

Reply Parent Score: 1

Alfman Member since:
2011-01-28

JPollard,

"Um.. video isn't like that. You CAN tread video that way, but it is too incomplete. For instance, a video frame is actually a collection of blocks, each block is compressed. Subsequent blocks use tables generated from the first block for further decoding..."

I agree that a typical database would not do a good job with pixel level data. In theory it could be done, but the overhead would be insane. Some day a database might be bold enough to include video codecs, but not today's databases. Even with codec support, I wouldn't be very keen to use "SQL" to manipulate frame buffers... although it might for an interesting challenge.


"Now we get to the additional packaging... The video may be encrypted as well... Some tracks encrypted, some not."

Ether the decryption key is available or it is not, this would be true whether the streams are stored in a database or not, so I don't really follow what you are saying.

Reply Parent Score: 2