Ever since the iPhone crashed AT&T’s network, it has become common cause that the increase in mobile data, or rather data that is transmitted over the air, is going to grow. A lot. Networks are already straining to cope. But that’s not the big problem, says Andrew Herbert, director of Microsoft’s Cambridge Lab. The real problem, he says, is latency. Latency is the delay between sending a signal and receiving an acknowledgment of receipt. The time it takes light to circumnavigate the Earth, about 133 milliseconds, is a fundamental constraint on network speed.
I am sure we will see some interesting ideas to solve it. Maybe increase latency but manage it in some way [intelligent routing] that once a response is established some sort of burst streaming occurs. I don’t know. I know jack-squat about hardware capabilities…
well for sure we can’t beat 133ms to go around the world. Cause when we do, we’ll have discovered a whole lot of more incredible things than having a low ping for FPS =)
Packet delay and latency can indeed ruin the experience (thanks alicedsl.de for bringing such enjoyable things ^^)
Edited 2010-06-29 19:49 UTC
There is no real solution to the latency problem because it is not a design limitation but rather a reality limitation. That speed of light is constant produces both relativity and this latency problem.
Unless you allow for wormholes, which had better not be unleashed on Earth.
There had always been one, and only one, “solution” to the latency problem, and that is to assume latency exists within some bounds and work around it. Talking about latency always brings up throughput, which, in the best case scenario, can be traded off against each other. The only solution is to always trade in favour of latency (by using batched requests, forms, MVC client side views updating) and simply just neglect throughput. The reason is that throughput can be solved with upgrades in technology, but not latency.
Sadly, almost nothing follows that principle. X-windows has every event ping around like mad. Http, on the other hand, is one good design, though it can still be improved upon.
But, I do not mean that throughput is unimportant. The sages have produced diffs and patches, even bindiff and binpatches, that incur latencies that are more than justified. I just mean that sending cursor movements around is just not sane design. And if Microsoft was not so brain dead as to require video adapters be as brain dead as they were, we would be bindiffing screen refreshes now. Having variable screen refreshes would be much easier on the video cards for simple 2D stuff, and text scrolling would be real simple. (Of course, 3D rendering would just be copied wholesale as it is now.)
Oh, come on, don’t you dream of tachyon-powered optical fibers which can transmit information in the past ?
I mean, a -10ms ping… That’d be something cool =p
Seriously, why would signals have to go all around the Earth ? If networks are properly designed and use extensive caching of information, this should not ever be needed…
Of course. However, that is not a real solution either. Somehow, people who share the same language tend to converse. Anglophones therefore have it real bad, whereas the South Koreans cannot spare a thought to the internet outside of their small island (other than those who can also speak English there, which is a rather big cohort).
Caching is already widely used, so I was not lamenting on that. Your point still stands, but then that will still be sad because of global “news-like” sites like facebook and twitter. Caching does them not much good.
Then again, you do see that caching reduces latency and increases throughput at the same time, don’t you? It is a definite win-win for both bottlenecks, so it is widely used. I still have trouble understanding how the forefathers emphasized throughput over latency in the first place. It is not like we did not already know that latency is the real killer. (Or maybe they really had trained themselves to have huge patience.)
This reminds be of another thing, and that is disc space. I once heard of and skimmed through a treatise on how to cherry-pick the files to cache, because caching is to trade disc space and traffic analysis via processor time for throughput and latency, so they actually went to measure and see how aggressive to cache. It turns out that all four resources were scarce. I suppose a repeat is required now that disc space had become much more available.
Which brings me back to the original point — latency is only “solved” by prioritising it. Batching, caching and most importantly, asynchronous communications must be given first class status. Synchronous communication, which is the norm in single computer mode, is the worst abuser of latency. We need to seriously learn from IMAP, which is really good design. (And I even think we should implement synchronous communications over asynchronous, so that it will always be slower than asynchronous, in order to force developers to use asynchronous exchange as much as possible.)
EDIT: <insert rant about commenting and voting. Wants to vote the above up.>
Edited 2010-06-30 09:35 UTC
I’m not sure I understand this. Shame on me non-native English speaker. However, if it is about real-time communications (like phone calls, games, chat…), then you do have a point. No network organization or caching trick will speed those up, we do need tachyons in our optical fibers for that.
What could be a potential solution to that can be found in modern processors : if memory is updated, it sends a signal to the processor, which in turns notes in its internal memory that the updated part of memory is “dirty” and that the cached version cannot be used anymore.
A web version would be to equip web servers with such “dirty bits”. Once a website updates its content, it sends a signal to the main web servers to notify that caches including it should be reloaded. Said servers then notify their regional childs, and so on until clients are informed of the update too.
Then, when a client wants to read content from the web, it polls its internal caches. If it does have a valid cached version, it reads it. If it does not have a cached version or if its cached version is invalidated, it asks the ISP for the updated version. The ISP then checks its own internal caches. And so on…
The sole drawback I see to this approach is that one should make sure that it does not become a major bandwidth eater. To achieve this result, several options could be experimented.
–Limit update frequency : Define a maximum portion of the available bandwidth which can be dedicated to caching (say, 1/10). If some cache-related updates can eat up more bandwidth, delay them.
–Do not send unneeded updates : Keep track of which pages each client has cached, and only send him updates related to those pages. There may be some privacy concerns about this though.
In the olde days ™, people did not use much the web for real-time purposes. Simply because they couldn’t : what can you do at 7KB/s ? Except chatting on IRC maybe, but then latency is not really an issue. However, they did have issues with large files (I remember downloading a 30MB Quicktime update for my grandmother’s eMac over a 40K connexion. It was nightmarish).
At the time, people wanted bitrate, not latency. The need for short latency appeared later, when things like online games and VoIP became more widespread…
Anyway it’s not that bad, because we *will* need of that bandwidth in order to keep caches consistent across the entire web ! ^^’
That’s true too. I think scheduling-related questions should be re-examined too : it’s sad that nowadays, computers and networks are at the same time incredibly powerful and incredibly sluggish. Is shows how badly computing/networking resources are distributed across programs/clients…
I like how those problematics networks have to face nowadays are similar to those you find at a much lower level in hardware or OS layers, and how similar the proposed solutions are too. This could mean that if one of those computing domain find a solution to those latency problems, it could be applied in other areas too !
It is just me and my convoluted non-native English. I simply mean that people from all nations that read English will tend to want to read from facebook, for example. (I’m in Singapore right now, and we routinely read both Chinese and English websites, and mind you, I mean from all over the world — UK, US, CN, TW, HK to name the most important few.) Koreans, however, tend to only focus on Korean content, so their traffic is much more cache-able, being geographically bound. There simply is no point caching a little-used website. It is also impractical to cache English websites in the whole world. (I mean, try caching youtube and you are already screwed.)
It actually had been tried, and the real drawback is that it is not possible to enumerate who you want to update. If a South Korean tourist goes to South Africa for this World Cup and surfs from there for a few times, should the South African ISP cache that traffic? How would the South Korean web server tell the South African ISP to update the cache?
Update propagation really only works if you can know beforehand where the updates will propagate to. (Push mail is one such — it is a shame it took so extremely long, and for a proprietary solution to have won out over the stagnated bazaar.)
There is an alternative that is already very much in use, and that is to cache only moderately large files, like 3MiB pdfs to 50MiB oggs, and to use the modification dates. For any eligible file that is fetched for the first time, the file will be stored on the ISP cache servers. Then, subsequent requests will get much higher throughput. This model does not improve latency because each request will still trigger updating (which is only to compare modification dates). However, once you take into account batching (allowing the server to update a single file only once every five minutes), then for very high profile files, the latency and throughput savings will be much higher.
Oh no! The situation will always be the same! With every improvement in throughput, there will always be another behemoth to choke up the pipe. First, it was everything. Then, it is pics. Then mp3s came along, and behind them, we have youtube vids. Now we have complete DVDs of stuff zipping around the web. It won’t ever stop until we have holographic movies downloaded before we click. (remember prefetching?)
I mean, throughput hogging is a never-ending battle. We even know that we can get better at throughput. It is latency that we don’t have any weapons to deal with. It is a wonder to me how throughput could have trumped latency concerns during those formative years _even_ though the sages already knew about the problem when they designed the systems. (Or probably they only knew about it many years after the disaster that is X-windows. See UNIX-haters handbook for that delightful story.)
PS: you mean not bitrate, but throughput, and that is connection, not connexion. Where we can use precise and accurate language, we had better do so, which is the reason why we talk about velocity and momentum instead of the mess that our forefathers bickered about (momentum referred to speed, velocity, momentum, energy and so much more during Newton’s time, it is scary to read.)
Can you please explain me what the differences are ?
intriguing question.
AFAIK, bitrate is usually talked about in media codecs while you normally talk about throughput vs latency in networking.
From wikipedia on bitrate, we see that throughput is more of “what you manage to put through” (obviously) against bitrate which is how much bits you send per second. This distinction is only there because you usually send a lot of rubbish along with the stuff you want to send.
The same article talks about goodput, which is even more to-the-point, but I’ve never even heard of it before (probably difficult to reliably measure anyway).
So, in a nutshell, the difference is in convention and the overheads. (Convention as in we simply just always talk about throughput with regards to latency, which is a bad reason)
“(by using batched requests, forms, MVC client side views updating)”
Yes, that is partly what I was trying to say, sort of. But I was thinking along the lines of adding some of that responsibility to the routers somehow – and tuning the client/server to allow for more latency but expect [somewhat complete] bursts of data.
But again, I haven’t been involved in network management for 20 years so I haven’t a clue. Holy cow, I am getting old. It’s weird how it sneaks up on you.
How can that be optimal? The problem is caused in the application/protocol level, and your solution is in the gateway level. Caching is more likely useful at the gateway level, but that does not solve the root cause of the problem, which is the fact that application developers assume low latency when it just is not required to have such assumption.
Or maybe more concisely: There is no fire that can continue to burn with the fuel stolen. Instead of trying to fight the huge forest fire by dumping water on it by the bucket, you should be trying to remove the wood from below the fire. (See how the Australians do it by pre-burning a perimeter to the fire so that it extinguishes itself.)
Oh I know and I agree! I was … well … looking for a shortcut. [wipes egg off face] I guess I was thinking with smart routers that can store and forward could also manage the traffic a little better… but that would just be putting off the inevitable for not much longer.
We do that here [Arizona] as well…in addition to that we light fires on purpose then managing them to remove potential fuel in a natural fire.
Don’t worry, was not wanting to throw eggs. I’m sure it isn’t even rotten.
You’d never know. NAT was meant to be a short term stop gap measure that ushers in IPv6, but see what that did. The real frustrating moments are when we already know the right thing to do, and we see implementers ignore the optimal solution. (i.e. we _know_ that latency should be given priority, but later systems simply just ignore. Or that MVC was already very well known, and my proposal is the obvious logical progression that I’m sure someone else already thought of, yet how many years has it been? Registry, binary blobs anyone?)
Speaking of binary blobs, whose great idea is it to depend upon them? Did we not have problems with them before? Can people not see how stupid it is that linux, with all the complaints of incompatible hardware, work with so much more stuff on 64bit than Windows? Are humans doomed to never learn from history? As always, one thing leads to another — if we do not learn from history, then probability of WW3 is almost one…
Prelude> “pre-burning a perimeter” == “light fires on purpose then managing them to remove potential fuel in a natural fire”
True
(Yes, I’m learning Haskell again. And I mean those two statements are equivalent.)
Another thought could be to have one-way pipelines in congested regions. Tho’ at some point everything has to come back together.
Maybe routers with microbuffers that store packets with similar destinations until it has a chance to get all of its requests out…
I give up.
[gotta go fix a production bug]
Edited 2010-06-30 16:24 UTC
Unlike article.
Other than the one quote about latency, there is no real meat in the original article. Where is the original article in the first place?
And what nonsense about Bing’s new AdPredictor? Ain’t Google already doing something along that line?
Furthermore, the latency problem is described to have been solved by “having the processor wait for something to happen” which is definitely something like what Microsoft would be doing with the x86 improvements. Utter Rubbish. How did I miss out on that in the first place?