Linked by Nicholas Blachford on Thu 19th Feb 2004 20:06 UTC
Editorial No, I'm not going all "New Age" on you, this time I'm looking at how computers are going to get a 3rd dimension and how this will change the way we interact with them. The previous parts of this series have been based on extrapolations or previous history. This time I'm looking further forward, when technologies currently in long term development become available and open up a whole new realm of possibilities.
Order by: Score:
3D display.
by Jimbo on Thu 19th Feb 2004 20:23 UTC

MS research did a 3d display, the link is here.
http://research.microsoft.com/adapt/TaskGallery/

There is another company that took and sold this idea but I don't have their site.

My opinion does not reflect "average joe XP user" but I think first there is room to improve the task management interface in out existing 2D desktops to better handle multitasking. XP's "grouping" function is too difficult to figure out which window is which. I don't don't like having my operating system ove stuff around on me because I have to go looking for it.

So far multiple desktops seems to be the best existing solution to this problem. I think MS has a powertoy to add multiple desktop support to windows but I have not tried it yet.

Just flat wrong
by Anonymous on Thu 19th Feb 2004 20:33 UTC

"Architectural advances in microprocessors seem to have slowed down in recent years...."

Huh? The tremendous die area has been utilized to implement numerous refinements to all sorts of things--larger and faster caches, better branch predicting, register renaming, longer pipelines, speculation, larger issue width, out of order execution....The Opteron and the Power5 both represent major advances beyond Alpha.

"Graphics Processing Units (GPUs) get more performance at a much higher rate than CPUs but have to date always been limited to producing graphics. These days modern GPUs have programmable vertex and pixel shaders and these are starting to be used for general purpose computations[1] in research and even in some applications. With the next generation shaders they will become not only more powerful but also more general purpose."

Graphics processing is about the most embarrasingly parallel task you could can imagine. They are fast because there are few data dependencies and everything can be done in parallel. For data dependencies, you implement multiple passes. Sure researchers have used them for some embarassingly paralle problems that happen to not be graphics, but it is not like these GPUs will replace a CPU any time soon.


"The human brain has a very large portion devoted to the processing of visual information."

Wrong. The visual cortex is small in comparison to the rest of the brain.

"When I say the 3D GUI is coming I mean a GUI displayed in three dimensions, not a 2D representation of a 3D space such as games deliver. I mean a real 3D display where objects on screen have Height, Width and Depth. Our displays are one way or another going to gain another dimension, 2D displays are going to seem somewhat quaint by comparison."

What's the point of making a hologram? You still only have two eyes. Put on some goggles!

Augmented reality will be the future. Why even have a monitor that sits there on your desk for you to look at, when you can have a lightweight pair of glasses with a wireless connection?

I'm not even going to bother reading this. If he is as far off as his previous parts then Pluto can be seen from Earth by the naked eye.

i agree, Sabon
by Reflekt on Thu 19th Feb 2004 20:49 UTC

This guy keeps writing this HOPEFUL drivel.... quite sad actually

Neural interfaces, replies
by Bascule on Thu 19th Feb 2004 20:49 UTC

Neural interfaces allowing direct interaction with computers at the level of abstract thought will replace any need for an "interface", and are likely to render a 3D environment obsolete before it can really become useful.

Anonymous (IP: ---.CS.UCLA.EDU)
What's the point of making a hologram? You still only have two eyes. Put on some goggles!

Augmented reality will be the future. Why even have a monitor that sits there on your desk for you to look at, when you can have a lightweight pair of glasses with a wireless connection?


We're already in the process of creating 3D displays which are physically the same shape as a standard LCD. Goggles are obtrusive, they either need wires or a rechargable power source, the latter of which means that you'll have to interrupt what you're doing to swap batteries/recharge. It also means that you're all consumed with what's being presented, you can't say... mess around with your guitar while watching something on one or more monitors, etc.

I'd certainly prefer a 3D LCD-like display to goggles any day.

Headsets are the primary resaon why VR failed.

"Wired" for 3D
by BR on Thu 19th Feb 2004 21:19 UTC

"Neural interfaces allowing direct interaction with computers at the level of abstract thought will replace any need for an "interface", and are likely to render a 3D environment obsolete before it can really become useful."

Interesting idea. So what protocals does the brain use? What about language? We'll most likely see 3D (no pun intended) first, although as pointed out above, we need to fix our 2D interfaces first before embarking onto 3D with it's own set of problems. Crawl before walking and all that.

Not goggles, glasses, and non-intrusive
by Anonymous on Thu 19th Feb 2004 21:24 UTC

Augmented reality has a hell of a lot more to it than simply a monitor on your eye glasses' lenses. The system tracks your head movement and orientation and embeds digital images into your view of reality. So for example, instead of an actual LCD monitor sitting on your desk, your glasses will constantly monitor your position and orientation and which direction you are looking and make the same image appear to be the same physical location as a physical monitor would have been.

You could therefore embed the display anywhere: floating in the center of the room, on a wall, on the floor, on the face of a book....if you wanted to get away from the monitor, you can walk away.

If the glasses (I envision something the size and weight of eyeglasses, with a small computer the size of a watch battery behind one ear and a watch battery behind the other ear) are light enough and use little enough power that they can be wireless, then you could have a virtual 50" on your wall, 23" LCD on your desk, etc.

But of course the whole point of augmented reality isn't to make a rectangular display appear to float somewhere, but to make virtual objects appear to be physically located places where they actually don't.

Imagine an art gallery with blank walls, and the glasses let you see virtual paintings on the walls. Imagine a road with no signs, and the goggles show you both waypoints and the names of streets, buildings of your interest.

Imagine a battlefield where every soldier can have the positions of their squadron overlaid with reality, and foreign objects highlighted, categorized, and tracked in real time on their glasses. Waypoints, objectives, statuses, a flood of information embedded in reality (and of course a way to reduce the flood to currently relavant information).

I think we will eventually enter a time when physical reality is far less important than the digital overlays that we will create and share. We will need to build far fewer things, and everything will be more efficient, even daily life.

Re: Just flat wrong
by debian_semi_guru on Thu 19th Feb 2004 21:28 UTC

"The human brain has a very large portion devoted to the processing of visual information."

Wrong. The visual cortex is small in comparison to the rest of the brain.


actually, vision does indeed take up a lot of the brains processing power, the visual cortex is devoted to one sense, which is vision. there is no other part of the brain dedicated to just one function.

3d Gui's
by Paul on Thu 19th Feb 2004 22:00 UTC

I think 3d is totally overrated and just making things 3d isnt going to do a whole major amount different for interacting with things. All hail 2D forever!

I can't see these `new advances` being much more than a passing fad or novelty. Wow, let's use the funky touch gloves and move the knows. Cool, see how they turn. Who cares.

I don't think it matters if computers are 1D, 2D, 3D of infinityD, it's not the dimensions of the display device that make a different, it's what's being displayed, how it is designed and what it allows you to do.. Okay so a selling point is all this new `what you can do that you couldn't do already` stuff, but it's not much.

The closer we get to a true representation of the physical world the less interesting it is. While we linger in a limbo of `not quite there` things are much more interesting. If we had displays that looked real we'd be bored. Who cares, it's reality. People like to live in fantasy. The audience will have to overcome their addiction to illusions and delusions in order to think that this super-real system would be appealing.

Do I get excited and freaked out about seeing a perfect real-looking 3d representation of an orange, apple or banana? No.. it's just there, who cares. All this hype is computer junkie mania.

Paul

RE
by Jean-Louis on Thu 19th Feb 2004 22:07 UTC

hi guys,

I beleive "tangible interfaces" are more likely to take over
in the near future than virtual reality .. check the amazing
work of James Patten of the MIT Media Lab :

http://web.media.mit.edu/~jpatten/

Cheers,

Jean-Louis

RE
by Jean-Louis on Thu 19th Feb 2004 22:11 UTC
3d
by Chris on Thu 19th Feb 2004 22:15 UTC

3d isn't gonna help anybody. We seem to be thinking that as we increase power we should throw all the increase of power (CPU speed) into the input/output systems. And we leave the IMPORTANT step, process, back in the 90's.
Instead of fixing our "broken" interfaces maybe we should fix our broken process code. Humans are quite adaptable, and things are fairly ergonomic as it is, so I don't see much need for great changes in UI. Optical (not gyro, because I only want it to move when my hands are in front of it) mice controlled with my finger would be cool, and useful for some situations (those where you are only on the machine for short periods and are likely standing up or walking).
A better keyboard would be nice. I'd use dvorak but nobody else does, and I just don't wanna have to switch back and forth. I would find dictation annoying, as my voice wears out faster than my hands; but a businessman would love it.

In the end the mind makes the display of the interface good or bad. 2d is still 3d because the mind allows itself to be deceieved. But our physical input systems kinda suck. I have to move my hand from my keyboard to use my mouse, that's pretty annoying (and the reason why many prefer CLI).


There will be new dimensions in computing when computing makes some serious, not small incremental, improvements.

Re: debian_semi_guru
by Bascule on Thu 19th Feb 2004 22:16 UTC

Wrong. The visual cortex is small in comparison to the rest of the brain.

There are at least 27 distinct regions of the brain dedicated to vision, accounting for almost a third of the brain's total mass. See the blurb for this Oxford Press book: http://www.oup.co.uk/isbn/0-19-852479-X or this University of Rochester press release: http://www.rochester.edu/news/show.php?id=1191 or this article from the International Biometrics Group: http://www.biometricgroup.com/in_the_news/03.07.03.html

actually, vision does indeed take up a lot of the brains processing power, the visual cortex is devoted to one sense, which is vision. there is no other part of the brain dedicated to just one function.

How about the hippocampus, which regulates the storage and recall of memory (much like a cache)? Or the medulla oblongata, the body's autonomic control center? There are many parts of the brain which are specialized to perform a single task, although you could certainly quibble about their scope.

Perception
by Jack on Thu 19th Feb 2004 22:21 UTC

I can see the next generation, UI's in fact being 3D or at least pseuddo 3-D. I'm certain that alot of the hardware manufacturers would love for MS push a such an UI, it certainly would drive alot of new units out of their warehouses.

As far as Gogles v. 3D Displays v. Augmented reality go in my opinion the majority of people would not accept any out of the ordinary computer interface. VR is pretty much dead, and as far as cyber punk Augmented realit goes we (techies) might be likely to adopt them but there will be along time before they become common place.

Just some thoughts.

Re: debian_semi_guru
by Bascule on Thu 19th Feb 2004 22:22 UTC

Sorry, the first half of that reply appears to be rightfully directed at Mr. Anonymous (IP: ---.CS.UCLA.EDU)

What about voice?
by Tom on Thu 19th Feb 2004 22:44 UTC

When I can ask my wrist watch/computer link: do I have any messages (and if I do, repeat them to me) what is the price of soAndSoStock, what is 9380.4 X .03125, .... Then we will have something. And, I think that is the next big thing!!!!!!!!

Neural Interfaces, My Behind
by Anonymous on Thu 19th Feb 2004 23:01 UTC

"Neural interfaces allowing direct interaction with computers at the level of abstract thought will replace any need for an "interface", and are likely to render a brain an electrofied puddle of goo."

Yes, neural interfaces are progressing, but the actual ability to decode the numerous electrical impulses of the brain, to re-represent them, and to re-transmit them through the brain which will always simultaneously have natural, non-artificial electrical activity in any meaningful, non-destructive manner is a long way off.

Attempts to re-represent visual information is advanced when the subject is a fruitfly. When it is a human, a neural interface is capable of producing some connect-the-dots phosphene activity that can "suggest" the most basic visual cues; however, anything sophisticated, visually, is difficult to re-represent as electrical impulses that can be understood by the brain.

Re: Anonymous (IP: ---.ded.pacbell.net)
by Bascule on Thu 19th Feb 2004 23:17 UTC

Yes, neural interfaces are progressing, but the actual ability to decode the numerous electrical impulses of the brain, to re-represent them, and to re-transmit them through the brain which will always simultaneously have natural, non-artificial electrical activity in any meaningful, non-destructive manner is a long way off.

Not really, we're testing prototypes of an artificial hippocampus (http://www.newscientist.com/news/news.jsp?id=ns99993488), the center of short term memory. It isn't too much to envision such a device being networked, allowing someone to, say, query the internet in the same way they query their short term memory.

v yadda, yadda, yadda
by paul on Fri 20th Feb 2004 00:05 UTC
Size. It's always been Side.
by Chris on Fri 20th Feb 2004 00:19 UTC

"When I can ask my wrist watch/computer link: do I have any messages (and if I do, repeat them to me) what is the price of soAndSoStock, what is 9380.4 X .03125, .... Then we will have something. And, I think that is the next big thing!!!!!!!!"
You're right, portability is where it is. Smaller and faster, not bigger and more comfortable. The ultimate comfort is it never leaving your side. I think if cell phone providers would pull there heads out of there asses some of these new phone would be wonderful to browse the web, but I just can afford the minutes.

3D will be for the computer not us
by Don on Fri 20th Feb 2004 00:58 UTC

I am not sure that 3D will play that big of a roll in future interfaces. A need must be created before it will become common place, and history has not been good to 3D technologies. For example the technology to make good 3D movies has been round for 50 or 60 years, holograms for around 40 years. But where are these technologies in terms of popular culture?

IMHO The big advances in 3D may not be in terms of what we will see but what the computer will see. If our computers can watch and recognize our gestures within a 3D field they could use those gestures as queues to what we want them to do. Image a computer noticing that I am looking at a particular window’s title bar and switching its focus to that window (some 35mm SLRs do this already). Then when I say, open, bring to the front, or let me see that, the computer brings that window to the front. A key component to this is that the computer would learn about how I work. Ultimately it could occasionally recognize gestures and complete a command before I have fully formed the command in my head. It would be like having a butler that knows what I want (or need) before I do.

Electrons??
by PasserBy on Fri 20th Feb 2004 01:28 UTC

Why does this guy say that a raytracer follows the path of an electron? Silly me, I thought they were called photons.

awesome
by daniel on Fri 20th Feb 2004 01:37 UTC

this is a really smart guy!

he is an iluminated! he should work at corporate strategy at ms, oracle, ibm, apple or something like that

congratulations for the series!

daniel from buenos aires

Not really
by Anonymous on Fri 20th Feb 2004 01:47 UTC

"Not really, we're testing prototypes of an artificial hippocampus"

yeah, IN RATS!!!

We've already produced computer-generated vision systems for flies, and have had baboons power video games...

That's not saying much though really...

Nova just did a special on the future symbiosis of medicine and technology and covered a lot of cyborg stuff.

They have actually performed a human experiment to create "vision" in a blind woman. All she gets is some phosphene flares that denote boundaries... It works, I'm not suggesting it's never going to happen; however, this woman is very disappointed. She has a permanent implant wedged into her visual cortex, a ribbon of wires coming out the back of her head around to a camera mounted on a pair of glasses, and all she sees is sparkles. And even at that, it is suspect-- it seems to work well for the most fundamental modes of visual cognition (borders, intersections of lines, etc...) However, the cognition of a human was so complex that whatever electrical impulses were being generated by the vision system produced ZERO visual effect... no blurrs, no boundaries, SIMPLY NOTHING.

crackpot.
by j on Fri 20th Feb 2004 01:56 UTC

this guy's articles are *always* fun to read. they make me giggle. madly. come back to earth. it's comfy here.

a new sense
by rain on Fri 20th Feb 2004 02:59 UTC

the only interface that IMO can remotely compete to the 2D screen is something that is directly connected to your brain.
Either by emulating regular signals or use completely new ones. I don't know how the brain would react to a "new sense", perhaps there's someone here that knows more about brains? I find the subject interesting though. If the brain could learn to interact with this new sense it would allow us to do a lot of crazy things. But as I understand it we wouldn't be able to share memories but descriptions of our memories (somewhat like drawing a painting or telling a story). Just imagine getting your daily spam directly delivered to your brain ;)
aah, it's an exciting thought though. Scary but exciting.

Anyway, mindcontrolled interfaces allready exists even though they are slow and simple at the moments, but I think they will replace the physical input devices in the long run because they can be a lot more effecient.
Voice controlled interfaces has to be one of the worst for general computing. It's noisy and slow. The fingers are faster than the mouth any day.

re: What about voice?
by rain on Fri 20th Feb 2004 03:02 UTC

Yeah, that's certainly a lot more convenient than making a gesture or pressing a button. And imagine how wonderful it would be if everyone walked around talking to their devices.
Bah, voice input is just a lousy idea.

Real 3d interfaces
by blah on Fri 20th Feb 2004 04:15 UTC

I have yet to see a competlling application for 3D interfaces. Do not confuse this with 3D data - this is useful, but flipping around my application window still seems pointless and certainly the Sun Looking Glass app seemed contrived.

RE : Reflekt
by You can call me Al on Fri 20th Feb 2004 05:06 UTC

<quote>This guy keeps writing this HOPEFUL drivel.... quite sad actually</quote>

Not nearly as sad as jumping on the flame train simply because you have nothing constructive to add! there are far more constructive ways to get noticed.

It's the way he envisages things to come, get it?

It's is strangely ironic that your nick itself opposes looking to the future.

If you want to make a point that something is blatently wrong then that's fine, but to come out with moronic comments simply because you don't agree with his point of view is ridiculous!

@Sabon

If you didn't read the article then you have no business replying in this thread imo.

I've said it previous and i'll say it again, these articles have been an interesting diversion from the run of the mill OS News articles.

Goggles
by Smartpatrol on Fri 20th Feb 2004 07:55 UTC

Goggles or glasses i think are the best idea. I wear Sunglasses all the time during the day outside. It would be nice if i had a means to display information on the edge of my field of vision that if i needed it i could glance to the side and view it without completely blocking out my sight. i imagine something like little billboards that i can minimize and maximize as needed. The best part of goggles/glasses is you can always take them off!

I do tend to agree that 3D displays are just for show we really need to come up with better ways to manage the huge amount of information we generate. I feel Google albiet better the most is a clusmy tool to access information.

Contradictions
by B. Adams on Fri 20th Feb 2004 10:09 UTC

Again, like in his previous articles, the author depends on false assumptions and contradicts himself.

For one he says that the desktop PC will disappear from the average household (he actually worded it much stronger than that in his previous article, but he seems to have come down from that position somewhat), but at the same time he acknowledges that an immersive, world-wide virtual community will require massive computing power to be available to everyone. Reeks of a powerful desktop PC to me, or at least something very similar to such a device.

His emphasis on 3D environments is somewhat odd, since that particular avenue has already been explored and deemed inappropriate. I expect more from augmented reality, as said in earlier replies.

Vision.
by cheezwog on Fri 20th Feb 2004 20:14 UTC

Slightly ot...
There is another system for artificial vision that requires a much less invasive method.
A 16x16 panel of solenoids is worn like a shirt, so they covered an area of the patients back. Differences in light intensity produced variations of pressure.

After a while, blind people can 'see' using this quite well. Just because sighted people use the optic nerve and visual cortex to process visual information, it does not mean that it's the only way.

The same thing may apply here... The whole point of 3D is to increase the amount of data the computer can communicate to us. Perhaps a touch screen that generated different textures and sensations so you could feel the data?

3D GUI being superior?
by Jonathan Thompson on Fri 20th Feb 2004 22:11 UTC

Well, the author clearly doesn't recognize the fact that not everybody processes what they see, hear, feel and sense in other ways in the same way. What makes it "obvious" for a 3D user interface? It might be obvious to one user, because they think and interpret the world in that manner, but others do not. For example, in the US they've started migrating to iconic road signs (something I suspect is more common and has existed for a longer time in the rest of the world) and to be perfectly honest, I have a hard time figuring out what they really mean. Why? I'm far from stupid; I just don't visualize their graphic representations to mean the same thing as I'd use to describe what they mean. The same thing applies to many graphic icons in computer programs. That's at least one reason why tooltips were invented!

The same thing is true of the commandline: what works and makes sense for one person is not obvious to another. However, some people process most of what they do in a more verbal manner than visual. This does not make them freaks or uneducated, just different.

What it comes down to is that computers will work best when both types of interfaces are available, both verbal as well as visual in nature.

Of course, besides verbal and visual, there are other senses: let's not neglect those! Why limit ourselves to 3D when other dimensions (sight, sound, touch, pressure, temperature) exist that can communicate information? cheezwog hinted at it with that pressure vest. There are many people that will be even further left out of the use of computers if 3D is the only real interface supported by computers, due to some limitation beyond their control. Not everybody has the equal power of control or sensation that everyone else does, and computers need to acommodate that reality.