Hide full article
As you may remember from back in April, we had to move a number of TidBITS servers from Geoff's suddenly darkened broadband connection to the relatively poky 56 Kbps frame relay connection at our old house in Seattle. We did what we could to cache articles on our main Web server at digital.forest for fast access, but there was no question that some TidBITS services were significantly impacted by the massive traffic in and out of my line for about a month.
You can read more about how Geoff coped with his loss of connectivity in "Surviving Your ISP's Darkest Hour" in TidBITS-588, but the product that made the month-long episode possible from my connection was a PacketShaper, a truly magical network device made by Packeteer. PacketShapers are expensive for casual needs, but for anyone who's serious about monitoring and maintaining Internet performance, a PacketShaper is a thing of wonder.
<http://db.tidbits.com/article/06494>
<http://www.packeteer.com/products/packetshaper/>
Adam, Network Dodo -- This story starts before Geoff's broadband service went dark. Something was causing my Internet connection to turn into sludge. Technically speaking, when I would perform a ping test, which sends a single packet to a computer on the Internet and times how long it takes to return, it was taking 3,000 to 5,000 milliseconds rather than the more common 100 milliseconds. I could tell - Web pages loaded incredibly slowly, and Internet services in general felt unresponsive. As I said, sludge.
The problem was sporadic, which made testing difficult. Over the period of several months, I managed to document it a bit and got my ISP to verify that my network wasn't being attacked. The only other cause that I could think of was a problem in the telephone company switching hardware somewhere, so I called Qwest. A technician there looked at my traffic (luckily I was able to get him while the problem was occurring) and noted that the slowdown was only apparent in outgoing traffic from my connection. Then the lights went off in my head: it wasn't a network problem at all - it was my SE/30 running LetterRip Pro! Whenever one of the translated issues of TidBITS came in, LetterRip Pro promptly packaged up the 30K file and sent it to between 700 and 1,300 people. That's a lot of data, and even though the SE/30 may seem woefully underpowered in comparison to today's machines, it was more than capable of flooding the 56 Kbps Internet connection.
I reduced the severity of the problem by cutting down the number of outgoing connections that LetterRip Pro could make at any one time. But I was still bothered - it somehow didn't feel right that my little SE/30 could so easily overwhelm my Internet connection. I was sure there had to be a more elegant solution, and I knew just the friend to ask - Richard Ford, a networking enthusiast who was the Open Transport product manager before leaving Apple for Packeteer. Indeed, when I explained the problem to him, Richard offered to bring over a PacketShaper so I could see what was going on and "shape" the traffic to make the problem disappear. I'd seen one of the purple pizza box PacketShapers in action only briefly once before, when Richard used one at MacHack to prevent Hotline users from hogging all the bandwidth on a 256 Kbps ISDN connection, so I was intrigued to take a closer look.
<http://db.tidbits.com/article/05470>
PacketShaper to the Rescue -- Installing the PacketShaper required a bit of reconfiguration on my network so it could sit between my router and the rest of the network, since all traffic has to pass through the PacketShaper for it to work its magic. That done, we launched a Web browser to access the device's Web-based interface and logged in via a clever reverse lookup mechanism that lets you connect to a new PacketShaper even before it has an IP number. A few settings later, the PacketShaper was ready to go.
The first thing we did was check out the network efficiency graph, which showed that my network efficiency for Internet traffic was often as low as 65 to 75 percent. That's awful, since it means that one of every three or four packets was being lost somewhere and had to be retransmitted, reducing the effective throughput of the connection. The reason in my case was that computer operating systems are tuned to operate on high-speed Ethernet local area networks (LANs), and that simply didn't mesh with my slow 56 Kbps Internet connection. Even with faster Internet connections, such imbalances between the LAN and the Internet are common, so Packeteer developed a technology called "rate control" for the PacketShaper. Since the PacketShaper sees and can touch every packet to and from the Internet, the rate control setting lets it massage the connection such that packets aren't being lost nearly as often.
Apologies in advance for even approaching the overhyped information superhighway metaphor, but if you think of the Internet as a freeway and the interconnections between it and individual LANs as on-ramps, the PacketShaper's rate control feature acts a bit like those metering traffic lights that allow only one car onto the freeway every few seconds, rather than letting everyone try to merge as fast as possible. Though the metering slows every car entering the freeway down a bit, the end result is that traffic moves more smoothly, increasing the overall throughput of the freeway. The same is true in the network world, where metering the packet traffic via rate control smooths out the burstiness and improves overall throughput by reducing packet collisions (which, thankfully, don't result in injuries, gawkers, or insurance claims).
After turning on the PacketShaper's rate control feature, my network efficiency immediately jumped to between 95 and 99 percent, and the connection felt a bit more sprightly.
The next step was to let the PacketShaper run for a while and see what more it could determine about my Internet traffic. We queued up some messages for TidBITS Talk and went to dinner. By the time we came back, the PacketShaper had detected a number of different types of Internet traffic - SMTP, HTTP, FTP, POP, Timbuktu Pro, and so on - based on ports used and peeking into packet headers. It can detect most common types of Internet traffic automatically, and you can manually identify types it doesn't know about. It had also tracked every packet in and out and graphed the results so we could slice and dice the data in numerous ways. A look at an area graph of the most common protocols showed that outgoing SMTP - all those TidBITS Talk messages and TidBITS translation issues - was indeed the culprit in my network woes. Less-expensive versions of the PacketShaper are limited to monitoring traffic and displaying the results; they can be useful in diagnostic situations, but in this case, I wanted to solve my problem as well and that called for the PacketShaper's "shaping" capabilities - partitions and policies (monitoring-only PacketShapers can be upgraded to add shaping capabilities later).
Return for a moment to the freeway analogy. You can think of a partition as a separate carpool lane that's restricted to cars with three or more occupants. The PacketShaper can create a partition based on a variety of criteria - destination of traffic, source of traffic, type of traffic, and so on - and permanently carve out some amount of bandwidth for that traffic, just like a carpool is permanently carved out of a freeway (though the PacketShaper doesn't allow cheating). I could have, for instance, set up a partition that permanently carved 10 Kbps out of my 56 Kbps connection for outgoing SMTP. That would have been silly, though, since there are plenty of times when nothing else needed the remaining bandwidth, so there was no reason not to give the full 56 Kbps to outgoing SMTP. Partitions are most useful for services like voice that require only a certain amount of bandwidth or for situations where a Web hosting firm might want to limit the amount of bandwidth available to individual clients.
Instead of carving out a lane on the freeway, a policy simply changes the priority of different types of traffic, much like a police car or fire engine does by turning on its siren and lights. Since all the normal cars pull over, the emergency vehicle doesn't have to contend with congestion and can travel more quickly. So in the PacketShaper, we gave sirens to incoming HTTP (Web pages I was viewing) or incoming FTP (files I was downloading) so they had a higher priority than outgoing SMTP (TidBITS Talk messages or translations that were being sent). The amount of fast computation that this entails is mind-boggling, which is part of the reason why the PacketShaper is a dedicated box rather than a piece of software that could be distracted by something else happening in the operating system.
With this simple policy in place, my Internet performance troubles simply disappeared. The PacketShaper silently watched all the traffic, and as long as I didn't need to use the connection for some other reason, it allowed outgoing SMTP to take as much bandwidth as it wanted. As soon as I requested a Web page or started a download, though, the PacketShaper throttled outgoing SMTP traffic back to make sure the HTTP and FTP packets arrived at my Mac as fast as possible. Since outgoing SMTP isn't an interactive service where a person would notice the slowdown, the only liability is that outgoing messages were delivered a bit more slowly.
Rush Hour -- As much as I was grateful that the PacketShaper solved my problem, I couldn't pretend that I was in any way stressing the PacketShaper's capabilities. That opportunity came when Geoff's broadband connection went dark and five servers migrated to my house. Suddenly, the PacketShaper had a lot more types and quantity of traffic to squeeze through a too-small connection.
Most important was outgoing HTTP, of course, since we wanted people doing searches in the TidBITS article database and browsing the TidBITS Talk archive to have the best possible performance. We also increased the priority for outgoing POP3, to make it easier for Geoff to pick up his mail via a 19.2 Kbps modem connection. And over the next week, I twiddled policy settings for other services like Timbuktu Pro, FileMaker, and others to which we needed occasional access at high speed, but which seldom accounted for much traffic.
I also set up some custom reports that showed me graphs of what sort of traffic was being sent and received, and I took to refreshing those constantly to make sure the connection was holding up. It was astonishing. It went from a spiky usage pattern that corresponded to LetterRip Pro sending messages or me downloading big files to one where the network usage was pegged at the maximum 56 Kbps without any respite at all. Unfortunately, I was too busy at the time to think about saving the graphs for this article, so the best ones I could find came after we moved the servers back to Geoff's reactivated ISDN connection.
<http://www.tidbits.com/resources/601/>
The PacketShaper provided hard numbers too, so I saw that whereas a normal day might send 70 MB of outgoing traffic, the first Tuesday (our highest traffic day) after adding Geoff's servers transmitted over 320 MB for the 24 hour period. That's a lot of data for such a small connection.
Interface Annoyances -- The internal code is what matters the most in a product like the PacketShaper, of course, and you can see that Packeteer hasn't focused nearly as much effort on the interface. The PacketShaper's basic functionality is all accessible via a Web-based interface, which makes sense as a way of making it work across many different platforms. Packeteer's engineers chose to rely heavily on JavaScript, and the result is that you basically can't use the Macintosh version of Internet Explorer to manage the PacketShaper. Netscape (at least Netscape Communicator 4.7) on the Mac works for the most part, though resizing a window forces a refresh in Netscape, which in turn moves you to a specific, but usually unwanted page in the PacketShaper interface). Netscape also suffers from a font display issue that makes the text so small that it's almost unreadable. The sad fact is that the vast majority of Packeteer's customers use Windows or Unix, so making sure things work on the Mac has been a low priority, though one that Richard continually champions.
From a basic functionality viewpoint, though, the Web-based interface does provide quite a lot of flexibility. You can easily change the time ranges for graphs to show the last 3 hours of traffic or the last 4 days or even the last 2 months. And setting policies and partitions is relatively obvious, though I found that explanation in the extensive manual was often necessary to understand exactly what I was doing.
However, as much as the graphical interface is welcome, anyone who gets serious about the PacketShaper will learn its Unix-like command-line interface. Not only is it far faster, but you can create almost any arbitrary query against the PacketShaper's store of data. Want to know exactly what machine a spike of FTP traffic went to at a specific point in time? The PacketShaper knows, and can tell you, but only if you can construct the appropriate command. The command-line interface also comes in handy when you're trying to make a number of changes at once.
Who's It For? Realistically, the PacketShaper is not for end users or for those with modest Internet connection needs. It shines in situations where the demands on an Internet or wide area network (WAN) connection are too great and where the cost of buying more bandwidth is prohibitive or perhaps just unnecessary. The best example of this are international WAN connections, which are generally 56 Kbps to 384 Kbps frame relay connections, and satellite Internet connections, where installing a PacketShaper lets an organization avoid or put off paying for upgraded bandwidth. More generally, anytime users in organization (such as large companies, universities, and ISPs) regularly complain about WAN or Internet performance, a PacketShaper can help network administrators see exactly what's happening on the network and manipulate it for optimal efficiency.
Colleges and universities often have extremely high-bandwidth Internet connections, but over the last few years, they've seen their bandwidth needs increasing radically thanks in part to what are essentially dormitory-based peer-to-peer server farms running Napster and its descendents. With the PacketShaper, an administrator can simply lower the priority of song swapping traffic so it doesn't interfere with more important traffic. Plus, as Napster's legal woes have reduced usage significantly, the PacketShaper can tell network administrators which competing services are taking over the undiminished music-sharing traffic from Napster.
A PacketShaper is also a godsend in situations where you pay for bandwidth based on usage, something that's common outside of the U.S. When every gigabyte of data transferred per month has a specific cost, you want to know what sort of data you're paying for and where it's coming and going. A PacketShaper can tell you all that and help you make sure your bandwidth budget isn't being eaten up by music sharing or streaming video.
Finally, Web hosting firms often use the PacketShaper to provide a guaranteed minimum amount of bandwidth to clients. That ensures, for instance, that an extremely popular site won't slow traffic to other co-located sites. Plus, PacketShaper policies can ensure that non-interactive traffic, like SMTP email from a large mailing list, won't obstruct traffic that a person is waiting to receive - Web pages or a videoconferencing stream, for instance.
Pricing -- Like many high-end network devices, a PacketShaper isn't the kind of thing you pick up off the shelf in CompUSA. Prices vary based on whether the unit can monitor or shape the traffic and by the size of your connection, so you can expect prices to range from $3,500 to $34,000 (the unit I used was in the $3,500 category, whereas someone with a T1 Internet connection would be looking at more like $8,000). If you're interested in learning more, Packeteer's Web site has information, along with a form you can fill out to have a salesperson give you a call (or you can contact the local office nearest you).
<http://www.packeteer.com/moreinfo/>
<http://www.packeteer.com/company/offices.cfm>
Obviously, I can't recommend that everyone run out and buy a PacketShaper, but it's definitely worth a look for folks who are responsible for a large and/or heavily used Internet connection. That's particularly true if you've been considering adding more bandwidth but balking at the additional monthly expense. Between its rate control and prioritization capabilities, the PacketShaper can help you make sufficiently better use of your existing connection that you might be able to postpone adding more bandwidth, which - despite what some pundits say - still isn't free.
