OSNews

I'm pretty much in agreement with that. During Vista development, I was really put off by Microsoft dropping meta data for everything but Office files. Losing WinFS and then any semblance of meta cut the heart out of the new Explorer. Now Apple may have delivered detailed meta searching (and decent saved searches), but there is still no up front way of tagging and editing meta on the filesystem, and this is where NEPOMUK is stepping up the game.

My only real concern is that It's going to be near impossible to breakdown the desktop/folder metaphor that everybody is so used to, and go fully meta.

I think it would be fairly easy to emulate folders using a metadata-based system. For the end user everything could look as usual, while under the hood a folder is implemented as a query for files with some specific attributes.

> It's going to be near impossible to breakdown

these sorts of changes will inevitably be multi-generational. but they need to start at some point, right? =) we shouldn't be discouraged if we don't get to the final destination with the first iteration. we should only be discouraged if we aren't moving towards that destination as quickly as possible.

there are several things in kde4 that are like this, btw.

Application developers can give the notion of semantic organization a kick-start by guiding users toward contributing metadata. For example, a media player could offer to add a common tag to files in a playlist. The "Save As" dialog for any application would be an obvious place to "remind" the user to add tags (maybe this done already).

Also, I think it might be useful to introduce generic key/value pairs to the possible metadata types. MIME types could be associated with default keys, and multiple applications could use well-known keys to communicate through metadata without requiring action from the user.

There may be some potential worries about malicious applications doing bad things with metadata, but I'm not sure there's any way to prevent that even in the current design. Maybe applications should be allowed to create private keys that no other application can modify. This would be a convenient way for applications to save per-file state.

Just some ideas. Keep up the good work!

> useful to introduce generic key/value pairs to the
> possible metadata types

from the pure metadata POV work on standardizing this is being done already (and common sense has already brought us a long ways there) and from the RDF side we have shared ontologies.

The thing is people already use folders and file names for this purpose. You have pictures/family/vacation/2001/franksallybeach22.jpg. Instead, it could be /personal/nice day.jpg tagged with family, vacation, vacation 2001, frank, sally, beach. Using folders for this purpose, you end up with extraordinarily long mazes with your information at the end, with very little flexibility in navigation. Using a meta approach is such a jump forward in every way, i really think people would jump on it, even though it is quite different.

I mean, look at flickr, facebook, gmail, digg, etc. People get tagging, they love tagging, and they use tagging. The technology is here for operating systems, and it boggles my mind that neither Apple or MS is moving forward with this.

Hmm. I personally don't add tags to anything at all. It's just so much easier and faster to give files a proper filename and save them in a logical place. I'd be annoyed a bit if I had to f.ex. specify tags for a file when saving it. Oh well, I guess this is just not my thing.

Manual tagging is just one of the sources envisioned here.

Manual tagging allows you to enhance meta data an relations by information that is closer to your way of thinking, thus allowing you to take this personal information into account when searching.

But semantic information can also be derived automatically, by the software handling your data.

The article offers the example of relating a file to the email it was saved from. Such an information can be added by the email program automatically on save.

Together with the also automatically derivable information about the email's sender (e.g. by indexing the emails), you can then search for the file by just knowing who sent it.

Proper integration into applications will ensure that (less) information is lost during operations.

Manual tagging can be used to add information not available to any of the involved applications, especially subjective values like "uhh, pretty!"

I just started thinking about something..If you f.ex. save a file from an email, and the saved file automatically acquires metadata such as the sender's email address, couldn't this be considered potentially a threat if there's several users on the same computer? If the file's metadata is readable by other users and they have read-access to even some of your files, they could learn email addresses of the people you stay in contact with etc. In that case it'd help if the metadata was accessible only by the owner of the file, but what if sooner or later f.ex. system files are populated with metadata? The only way I can think of how to fix that would be to have two kinds of metadata: private and public.

metadata currently is private to the user. sharing/merging metadata will be a by permission thing only, and we're still a couple years away from that anyways.

Aaron already answered this, but just a bit more detailed: the acquired relation data is stored in an relation database separate from the file. Thus the file itself remains "clean".

For some kind of data it might make sense to also store it in the file or in extended file system attributes. This is currently not implemented AFAIK and as Aaron pointed out, would be subject to policies.

For some kind of data it might make sense to also store it in the file or in extended file system attributes. This is currently not implemented AFAIK and as Aaron pointed out, would be subject to policies.


IMHO all such metadata information should be stored per-file, ie. in extended attributes cos then it could be shared by all users. And I don't really like the idea of a single database with all the metadata cos it could get corrupted, or if it got deleted one way or another you'd lose ALL metadata. But to have public and private metadata saved per-file would most likely require modifications to existing filesystems, or a completely new one. Hopefully this will happen, I'm sure a lot of people would find such a thing useful even if I can't imagine myself gaining much from that.

"Then MS starts talking about all the cool new things they will do with it, and I thought "Man, what a slap in the face for apple, to be beaten to the punch for once by MS"

Actually, MS started talking about integrated search long before spotlight. This was one of the rationals behind the failed "WinFS" DB driven filesystem. Apple came out with spotlight to counter that, and they succeeded. MS couldn't make it work, and had to drop it from Vista.

I wasn't talking about search, I was talking about using relationships between data and metadata for new forms of organization over the traditional file/folder thing we have been using for decades.

"I wasn't talking about search, I was talking about using relationships between data and metadata for new forms of organization over the traditional file/folder thing we have been using for decades."

What do you think is the basis of spotlight and WinFS? Rich metadata on file and directory structures to enable these "new forms of organization" The point of WinFS was to base the filesystem on a SQL store, to allow those relationships to be defined.

The Nepomuk group is involved in the freedesktop.org discussion to create a standard for metadata databases which would allow them to be interchangeable and would allow any compliant front-end to be used to search any database. this is the general direction most projects are looking to move in, they just need to sort out the framework.

> to create a standard for metadata databases

you are referring to xesam? if so, that has very little impact on the ability for apps to tap into the sort of framework nepomuk offers.

it's a step in the right direction as it starts to bring some commonalities into the search space, but it's the first step of many, many more.

of course ... what could end up happening with xesam, is xesam is adopted and the strigi+nepomuk combo gets chosen by more and more users and so full text searches return information stored by nepomuk enabled apps.

however, for proper integration to occur in non-kde apps they need to be instrumented to be able to add to the metadata and traverse the rdf stores... there's a lot of work there, but i hope it starts to happen.

does anybody have links that show that GNOME is discussing the use of nepomuk?

I don't know how this is related to official Gnome and Beagle, but you already have Beagle++ http://beagle.kbs.uni-hannover.de/ an extensions to Beagle using NEPOMUK.

Edited 2007-07-14 23:17

The demostration is here:

http://www.youtube.com/watch?v=Ui4GDkcR7-U

Beagle++: "http://www.youtube.com/watch?v=Ui4GDkcR7-U"

Cool... Maybe I could even consider moving back to "big fat" Beagle from Tracker just to try all those new semantic Beagle++ features.

"tracker has the ability to do almost anything nepomuk does"

Edit: Tracker has similar semantic features like Nepomuk or Beagle++? Well, I better look into that too. The reason why I've used Tracker and not Beagle is simply that it is more lightweight than Beagle.

Edited 2007-07-15 11:26

Well tracker is a semantic web store minus the rdf complexity

it implements the triple store which is unified across indexer and user defined metadata (unlike strigi and nepomuk which have incompatible dbs and have two daemons)

tracker has the ability to do almost anything nepomuk does + everything strigi does but using only one DB and one daemon.

The end result is that tracker is faster, lighter and a lot easier to use (very little complexity)

The second iteration of XESAM will cover metadata storage and semantics so hopefully nepomuk and tracker will be compatible that way.

So you are saying that the talk of KDE 4 is reaching new heights? An OS X killer? This is the epitome of hype.

I am cautiously waiting KDE 4 myself. I am fairly good with GNOME right now, but not seeing them doing anything *interesting*.

I think as Linux and free BSDs grow more reliable and their driver base grows, UIs become easier to use and configure and the tons of (what I consider to be) office-grade software grows and matures, I say yeah, it could be considered that.

Now KDE4 is looking slicker than ever (lets face it, a big part of why people like OS X is because it LOOKS GOOD), it is being simplified with a useful out-of-the-box look and feel, full of drag and dropness (which will improve as all the frameworks/api's are developed to make UI cross development improve with GNOME and KDE) but still has the capability to be tweaked by confident users, etc.

I don't know, what more does OS X offer? It is only now becoming more of a gaming platform...

I am not suggesting the masses would dump their macs just because of KDE 4... I am saying that for ME I think it has a lot of appeal.

I'm not sure, but isn't Quasar something like CoreAnimation? And Lars Knoll said: "CoreAnimation will give us a lot of great ideas to help build our version of this technology in the future."
http://arstechnica.com/journals/apple.ars/2007/06/12/ars-at-wwdc-in...

> Leopard is WAY ahead in terms of UI

and far behind in other ways. and we're catching up with the UI advances; in some areas we're at parity, in others we're approaching parity and we'll just continue until we are undeniably ahead.

as for CoreAnimation, we've been toying with similar things in KDE. Quasar and Phase/Animator are a couple of examples; Quasar, once released and ready to use, will replace the Animator bit of Phase/Animator and we should at that point have something that quite successfully competes with CoreAnimation.

I haven't used KDE since 3.0 - when I think of KDE I envision "clunky" QT components. It appears 4.0 is a major change...and change is good (when done right).

I wish all the KDE developers best of luck.

KDE gained very significant improvements in speed in each major version update from about 3.3 through to 3.5, and some minor speed tweaks in between.

KDE4 promises to be faster again.

If you have tried only KDE 3.0, then you will be in for a very pleasing surprise if you were now to try KDE 3.6 or KDE4.

Windows gets slower and slower ... while KDE is getting faster and faster.

http://kde.org/announcements/announce-3.5.3.php
http://kde.org/announcements/announce-3.5.4.php
http://kde.org/announcements/announce-3.5.5.php
http://kde.org/announcements/announce-3.5.6.php
http://kde.org/announcements/announce-3.5.7.php

GNOME also has had some speed-ups in recent times I am lead to believe, although not as significant as KDE.

Edited 2007-07-16 05:54

actually, windows search is essentially equivalent to strigi. except that you'll find that as time goes on many more applications will use strigi on the platform than will use window search due to how these things disseminate in the open source world (and perhaps KDE even more so)

nepomuk adds a few new angles to it, being oriented towards social metrics, using common ontologies and providing graph-based rdf stores.

it is easy to glance at two systems in the same general space ("search") and go "oh, they are the same", but if you actually educate yourself as to what each does then you may find differences. in this case they certainly exist and windows search is not comparable to nepomuk+strigi.

enjoy your vista, though.