Hide full article
I run a number of Macintosh-based Internet servers, and for the most part, these servers are stable. Crashes aren't frequent, but they do happen often enough to be a nuisance, particularly on Web servers that need to run all the time. Worse, several of my servers are a 45-minute drive away, in an office building I can't access after business hours. The laws of the universe state that crashes tend to happen on Friday evenings so that the server can be unavailable for as long as possible.
In part, I work around the laws of the universe by using crash-detection devices that can watch for crashes and restart a hung Mac without human interference. These devices connect to the ADB port (and sometimes the power outlet) and communicate with special software to determine if the Mac is operational. When they determine a crash has occurred, they restart the Mac, either by turning it off and on again, or by sending a keyboard restart over ADB - just as though you had pressed Command-Control-Power.
The main device I've used over the years to watch for crashes and restart Macs automatically is the PowerKey Pro from Sophisticated Circuits. A pair of PowerKey Pros watch our main servers in Seattle, and a third kept an eye on our internal file server until recently, when the time came to test several newcomers. On our internal file server, I installed the new crash-detection device from Sophisticated Circuits called Rebound. On our SE/30-based mailing list host I installed a crash-detection peripheral from Kernel Productions, called Lazarus. I never received another competitor, MacCoach from the Belgian company Neuron Data Systems, but I'll try to include information about it when possible.
<http://www.sophisticated.com/products/ powerkey.html>
<http://www.sophisticated.com/products/ rebound.html>
<http://www.kernel.com/kernel/lazarus.html>
<http://www.neuronsys.com/Products/>
Although I've found ways all that these devices can be fooled into restarting a Mac that hasn't crashed, the most important fact is that none have so far failed to restart a crashed Mac. That said, how do these products compare?
Hardware -- The PowerKey Pro comes in two different models, the 200 and the 600. The two units are physically similar, each offering six power outlets, a telephone jack, and an ADB plug. The 200 lacks the phone tone control and manual switches of the 600, but the 200 has 160 Joules of surge protection, whereas the 600 can take an add-on surge protector. (Omitting surge protection from the PowerKey Pro 600 is intentional, since people who buy the PowerKey Pro 600 also often have a UPS - uninterruptible power supply - and many UPS manufacturers don't recommend plugging surge protectors into a UPS.) The important feature for monitoring servers comes with the software, an optional $39.95 component called the Server Restart Option (SRO) that's currently bundled with the 600. Because the PowerKey Pro offers six power outlets and has space for the telephone jack, the ADB port, and its own power cord, it's quite a bit larger than the other devices I evaluated.
The PowerKey Pro's telephone jack bears mention. Before the SRO became available, we plugged the PowerKey Pro into a telephone line and created events that would restart the server if we called and let it ring 12 times. The PowerKey Pro can also detect distinctive ringing, so you could set up an alternate number for your main phone line and let the PowerKey Pro only detect calls specifically to the alternate number.
The minuscule Rebound is a stark contrast to the beefy PowerKey Pro. It's a sleek three-inch long black ADB cable tipped with a bright yellow ball. It fits anywhere in your ADB chain, so you can move it around. TidBITS Technical Editor Geoff Duncan ran into some "problems" with the Rebound's physical design - you'll enjoy reading his report and Sophisticated Circuits' response.
<http://www.quibble.com/geoff/rebound/>
Lazarus falls in between the previous two in size, since it's a black plastic box with a pair of power plugs (one for the cord to the wall, the other for a cord to the Mac), and a pair of ADB ports so it can sit in your ADB chain. The Lazarus device I received was an early unit and had a distinct hand-made feel.
MacCoach closely resembles Rebound, with a boxy connector replacing the yellow ball on one end of the ADB cable.
Restart Method -- Function follows form with these devices. The PowerKey Pro, with its six power outlets, restarts a Mac by cutting power to the Mac's power outlet. If the Mac isn't plugged into one of the switchable outlets in the PowerKey Pro 200, then the PowerKey software instead tries a keyboard restart, which is the same as pressing Command-Control-Power on most (but not all) Macs. Ideally, the PowerKey Pro should first try Command-Control-Power and if that fails, toggle power to the Mac, since toggling power can be stressful for the Mac's power supply and other components. This is especially true if an application crashes on startup, causing the Mac to restart constantly until someone intervenes.
The tiny Rebound can perform only keyboard restarts, limiting its range and effectiveness slightly. Some older Macintosh models don't support keyboard restarts, so the Rebound doesn't work for them. In addition, a crash could totally wipe out ADB, at which point a keyboard restart would fail. Luckily, such severe crashes are extremely unusual.
Like the PowerKey Pro, Lazarus toggles power to restart the Mac, so it works on all desktop Macs. One small advantage Lazarus has over the PowerKey Pro is that international users can use Lazarus with no trouble (as I understand it, the female end of power plugs is standardized, whereas the male end varies by country). The PowerKey Pro's power cable is permanently attached, so it would need an adapter in other countries. Neither Rebound nor MacCoach worry about power plugs, being ADB devices.
None of these devices are likely to work well on PowerBooks. PowerBook servers probably still have batteries installed, so they can withstand short power outages. I think of an internal PowerBook battery as a sort of built-in UPS. Thus, neither the PowerKey Pro nor Lazarus could restart a PowerBook by toggling power. In addition, PowerBooks reportedly have different ADB controller chips than desktop Macs, which eliminates MacCoach and Rebound (Sophisticated Circuits thought Rebound might work on the PowerBook 520/540 series and older, but not on newer PowerBooks). The MacCoach information explicitly says it won't work on PowerBooks or some 68K Macs.
Crash Detection -- All of these devices use essentially the same method of detecting crashes. They rely on software that regularly "pings" the hardware to check that the Mac is still alive, in essence continually resetting a timer. If that timer runs out because the hardware fails to receive pings for a user-specified amount of time, the device restarts the Mac.
The PowerKey Pro and Rebound use a faceless background application loaded from the Extensions folder to perform the pinging, whereas Lazarus relies on a normal application launched from the Startup Items folder. It's possible to quit the Lazarus application and lose protection, but it's also easier to turn off Lazarus's protection for maintenance or troubleshooting. MacCoach reportedly relies on a driver that loads very early, but may not be as sensitive as an application.
The PowerKey Pro, Rebound, and Lazarus can make sure specific applications remain active. Applications must explicitly support the PowerKey Pro and Rebound, whereas Lazarus can watch any application to make sure it's still active (as can the 2.0 version of the MacCoach software, reportedly). The list of server applications supporting the PowerKey Pro and Rebound currently includes WebSTAR, LetterRip Pro, Newsstand, Quid Pro Quo, WebServer 4D, WebSiphon, CommuniGate, TeleFinder, and PageSentry. Applications support the PowerKey Pro and Rebound by continually resetting a hardware timer. An advantage of the PowerKey Pro/Rebound method is that applications can restart the Mac if they detect low memory situations or other potential problems before they cause a crash.
For multiple server setups, the PowerKey Pro can work with Maxum's PageSentry Pro monitoring software to restart a number of servers. You plug the PowerKey Pro's ADB cable into the Mac running PageSentry, then plug the servers into the PowerKey Pro's switchable power outlets. When PageSentry detects that a server isn't responding, it toggles power to that server. My servers aren't even in the same area code, so this feature doesn't help me, but it's useful for many people running multiple servers.
<http://www.maxum.com/pagesentry/>
Be careful when deciding which applications Lazarus should watch. There's a button to add all the active applications to Lazarus's list, but when I accidentally added Email Admin (a LetterRip Pro processor), I ran into troubl, since Email Admin launches when LetterRip Pro receives its first message, not at startup. After startup, when all the other applications launched, Lazarus noticed that Email Admin wasn't running, and it restarted the Mac repeatedly until I noticed.
I prefer the method used by Karl Pottie's Keep It Up, a $22 shareware application that watches applications and when they quit or crash, tries to relaunch them, restarting the Mac only if relaunching fails.
<http://www.vl-brabant.be/mac/kiu.html>
Restart Next Week -- The second part of this article will discuss the documentation, logging features, interface, and pricing of these restart devices.
