Hide full article
Years ago David Cheriton at Stanford University taught me something that seemed obvious at the time - if you have a network link with low bandwidth then it's easy to put several in parallel to make a combined link with higher bandwidth, but if you have a network link with bad latency then no amount of money can turn any number of parallel links into a combined link with good latency. Many years have passed, and these facts seem lost on the most companies making networking hardware and software for the home. I think the time has come to explain it.
Speed & Capacity -- Even smart people have trouble grasping the implications of latency on throughput. Part of the problem is the misleading use of the word "faster." Would you say a Boeing 747 is three times faster than a Boeing 737? Of course not. They both cruise at around 500 miles per hour. The difference is that the 747 carries 500 passengers where as the 737 only carries 150. The Boeing 747 is three times bigger than the Boeing 737, not faster.
If you were in a hurry to get to London, you'd take the Concorde, which cruises around 1,350 miles per hour. It seats only 100 passengers though, so it's the smallest of the three. Size and speed are not the same thing.
On the other hand, if you had to transport 1,500 people and you only had one airplane to do it, the 747 could do it in three trips while the 737 would take ten. So, you might say the Boeing 747 can transport large numbers of people three times faster than a Boeing 737, but you would never say that a Boeing 747 is three times faster than a Boeing 737.
That's one problem with communications devices today. Manufacturers say speed when they mean capacity. The other problem is that as far as end-users are concerned, the main thing they want to do is transfer large files more quickly. It may seem to make sense that a high-capacity, slow link would be the best thing for the job. What end users don't see is that in order to manage that file transfer, their computers are sending dozens of little control messages back and forth. Computer communication differs from television or radio broadcasting in the interactivity of the communication, and interactivity depends on back-and-forth messages.
The phrase "high-capacity, slow link" above probably looks odd to you. It looks odd even to me. We've been used to wrong thinking for so long that correct thinking looks odd now. How can a high-capacity link be a slow link? High-capacity means fast, right? It's odd how that's not true in other areas. If someone talks about a high-capacity oil tanker, do you immediately assume it's a fast ship? If someone talks about a large-capacity truck, do you immediately assume it's faster than a small sports car?
We must start making this distinction again in communications. When someone tells us that a modem has a speed of 28.8 Kbps we have to remember that 28.8 Kbps is its capacity, not its speed. Speed is a measure of distance divided by time, and "bits" is not a measure of distance.
But there's more to perceived throughput than issues of speed and capacity, namely latency. Many people know that when you buy a hard disk you should check its seek time. The maximum transfer rate is something you might also be concerned with, but seek time is more important. Why does no one think to ask about a modem's seek time? Latency is the same thing as seek time: the minimum time between asking for a piece of data and getting it, just like the seek time of a disk, and it's just as important.
Monkey On Your Back -- Once you have bad latency you're stuck with it. If you want to transfer a large file over your modem it might take several minutes. The less data you send, the less time it takes, but there's a limit. No matter how small the amount of data, for any particular network device there's always a minimum time that you can never beat. That's called the latency of the device. For a typical Ethernet connection the latency is usually about 0.3 ms (milliseconds, or thousandths of a second). For a typical modem link, ping and traceroute tests show the latency is typically about 100 ms, about 300 times worse than Ethernet.
If you wanted to send ten characters (at eight bits per character) over your 33 Kbps modem link you might think it would take:
80 bits / 33000 bits per second = 2.4 ms
Unfortunately, it doesn't. It takes 102.4 ms because of the 100 ms latency introduced by the modems at each end of the link.
If you want to send a large amount of data, say 100K, then that takes 25 seconds, and the 100 ms latency isn't very noticeable, but for smaller amounts of data, say 100 bytes, the latency overwhelms the transmission time.
Why would you care about this? Why do small pieces of data matter? For most end-users it's the time it takes to transfer big files that annoys them, not small files, so they don't even think about latency when buying products. In fact, if you look at the boxes modems come in, they proudly proclaim "28.8 Kbps" and "33.6 Kbps", but they don't mention latency at all.
What most people don't realize is that computers must exchange hundreds of little control messages in the process of transferring big files, so the performance of small data packets directly affects the performance of everything else on the network.
Now, imagine you live in a world where the only network connection you can get to your house is a modem running over a telephone line. Your modem has a latency of 100 ms, but you're doing something that needs lower latency. Maybe you're trying to do audio over the network. 100 ms may not sound like much, but it's enough to cause a noticeable delay and echo in voice communications, which makes conversation difficult. Maybe you're playing an interactive game over the network. The game only sends tiny amounts of data, but that 100 ms delay makes the interactivity of the game decidedly sluggish.
What can you do about this? Absolutely nothing. You could compress the data, but that won't help: the data was already small, and that 100 ms latency is still there. You could install 80 phone lines in parallel and simultaneously send a single bit over each phone line, but that 100 ms latency is still there.
In other words, once you have a device with bad latency there's nothing you can do except replace the device with one that has good latency.
Modem Latency -- Current consumer devices have appallingly bad latency. A typical Ethernet card has a latency less than 1 ms. The Internet backbone as a whole also has very good latency. Here's a real example:
- The distance from Stanford in California to MIT in Boston is 4320 km
- The speed of light in vacuum is 300 * 10^6 m/s
- The speed of light in fibre is 60 percent of the speed of light in vacuum
- The speed of light in fibre is 300 * 10^6 m/s * 0.6 = 180 * 10^6 m/s
- The one-way delay to MIT is 4320 km / 180 * 10^6 m/s = 24 ms
- The round-trip time to MIT and back is 48 ms
- The current ping time from Stanford to MIT over today's Internet is about 85 ms:
- 84.5 ms / 48 ms = 1.76
- The hardware of the Internet can currently achieve speed of light + 76 percent
So the Internet is doing pretty well. It may get better with time, but we know it can never beat the speed of light. In other words, that 85 ms round-trip time to MIT might reduce a bit, but it's never going to beat 48 ms. The speed can improve a bit, but it isn't going to double. We're already within a factor of two of the theoretical optimum. I think that's pretty good - not many technologies can make that claim.
Compare this with a modem. Suppose you're 18 km from your Internet service provider. At the speed of light in fibre (or the speed of electricity in copper, which is about the same) the latency should be:
18000 / (180 * 10^6 m/s) = 0.1 ms
Although modems vary, the latency over your modem is anywhere from 75 ms to about 130 ms. Modems are currently operating at a level that's more than 1,000 times worse than the speed of light. And, of course, latency cuts both ways. If a one-way trip using a typical modem has a latency of about 130 ms, then the round-trip delay is about 260 ms.
Of course no modem link will ever have a latency of 0.1 ms. I'm not expecting that. The important issue is the total end-to-end transmission delay for a packet - the time from the moment the transmitting software sends the packet to the moment the last bit of the packet is delivered to the software at the receiving end. The total end-to-end transmission delay is made up of fixed latency (including the speed-of-light propagation delay), plus the transmission time. For a 36 byte packet the transmission time is 10 ms (the time it takes to send 288 bits at a rate of 28.8 Kbps). When the actual transmission time is only 10 ms, working to make the latency 0.1 ms would be silly. All that's needed is that the latency should be relatively small compared to the transmission time. About 5 ms would be a sensible latency target for a modem that has a transmission rate of 28.8 Kbps.
Understanding Transmission Delay -- At each hop, overall transmission time has two components: per-byte transmission time and fixed overhead. Per-byte transmission time is easy to calculate, since it depends only on the raw transmission rate. The fixed overhead comes from sources like software overhead, hardware overhead, and speed of light delay.
For modems, the distance is typically short, so speed of light delay should be negligible. However, the data rate is low, so it takes a long time to send each byte. The per-byte transmission time should account for most of the time taken to send the packet. To send 100 bytes over a 28.8 Kbps modem should take:
100 bytes * 8 bits per byte / 28800 bits per second = 28 ms
That means the round-trip should be twice that, or 56 ms. In reality it's often more like 260 ms. What's going on? Two other factors contribute to the overall time.
First, modems are often connected via serial ports. Many modem users assume that if they connect their 28.8 Kbps modem to their serial port at 38.4 Kbps they won't limit their performance, because 38.4 is greater than 28.8. It's true that the serial port won't limit throughput, but it will add delay, and delay, once added, never goes away. So, sending 100 bytes down the serial port to the modem should take:
100 bytes * 10 bits per byte / 38400 bps = 26 ms
Second, modems try to group data into blocks. The modem will wait for about 50 ms to see if more data is coming that it could add to the block, before it starts to send the data it already has. Let's see what the total time is now:
26 ms (100 bytes down serial port to modem)
50 ms (modem's fixed waiting time)
28 ms (transmission time over telephone line at 28.8 Kbps)
26 ms (100 bytes up serial port at receiving end)
Thus, the total time is 130 ms each way, or 260 ms for the round-trip. To make things worse, imagine that the 100 bytes in question are used by an interactive game being played by two players. If both players are connected to their respective Internet service providers by modem, then the total player-to-player round-trip delay is 520 ms, which is hopeless for any tightly-coupled interactivity, and this is reflected in the state of today's networked computer games. Can we do anything to improve this?
Improving Latency -- One thing to notice is that the 38.4 Kbps serial connection between the computer and the modem, which many people don't think of as being the bottleneck, turns out to be responsible for 52 ms of the delay. In fact, it's the single biggest contributor - almost twice as much as the actual communication over the telephone line. What can we do about this? If you can connect the modems at both ends at 115.2 Kbps instead of 38.4 Kbps, the serial port delay can be reduced to 9 ms at each end. Better still, if you can use an internal modem on a card instead of one connected through a serial port, the delay can be eliminated entirely, leaving a round-trip delay of only 156 ms.
Having eliminated the serial port delay, the next biggest contributor to delay is the fixed 50 ms overhead built into the modem itself. Why is there a fixed 50 ms overhead? The reason is that modern modems offer lots of "features" - namely, compression and automatic error correction. To get effective compression and error correction, modems must work on blocks of data, which means characters are corralled in a buffer until the modem has received a block big enough to work on efficiently. While the characters accumulate in the modem's buffer, they're not being sent over the phone line. Imagine you're sending a small amount of data, 100 bytes. That's not enough for the modem to work on effectively, so it would like a bigger block. After you have sent the 100 bytes to the modem, it waits to see if more characters arrive. After some time - about 50 ms - it decides no more characters are coming, so it compresses and ships what it has. That 50 ms the modem spends hoping for more data is unrecoverable, wasted time.
Modems were originally designed with remote terminal access in mind. They were meant to take characters - typed by a user on one end and transmitted by a mainframe on the other - and group them into little blocks to send. The only indication that a user had finished typing (or that the mainframe had finished responding) was a pause in the data stream. No one told the modem when no more characters would be coming for a while, so it had to guess.
This is no longer the case. Most people use modems to connect to the Internet, not old mainframes, and Internet traffic is made up of discrete packets, not a continuous stream of characters.
There's a simple fix for this problem. We could make modems aware that they are sending Internet packets. When a modem sees the PPP (Point to Point Protocol) End-Of-Packet character (0x7E), it could realize that the packet is complete and immediately begin compressing and sending the block of data it has, without pausing for 50 ms. This simple fix would eliminate the 50 ms fixed overhead, and should allow us to achieve a 56 ms round-trip delay over a modem PPP connection - almost five times better than what typical modems achieve today.
[Tune in next week as Stuart explains how bandwidth and latency interact, and how software can try to cope with the latency problem.]
