An old joke says that to be successful, a college needs to provide a winning football team for alumni, sex for undergraduates, and parking for the staff. It's one of those self-deprecating jokes where the punchline depends on the particular point the speaker is trying to make. Living in Ithaca, home to Cornell University and Ithaca College, I've heard the joke in various forms ("well, one out three isn't terrible") numerous times. As I returned from the successful MacDesign Conference and started to plan for the intimate gathering of a few of my closest friends that could be Macworld Boston this year, I was struck by the level to which the basic thrust of the joke in fact applies to industry conferences as well. For any conference, there are only a few groups of people for whom certain things must be true for success to be achieved, with the main groups being attendees, exhibitors, speakers, and press.
So let's have some fun and see if we can put together a scorecard to rate conferences from a variety of perspectives (obviously, you may only rate a conference for the category in which you have experience). For each of the items below, assign +1 point to the conference if success was achieved in that area; give it 0 points if it was neither successful or unsuccessful (or not applicable, as in the case of a session rating at Macworld Expo if you didn't attend the sessions); and allot -1 point if the conference flopped in that department. In extreme situations, I'll allow +2 or -2 points for truly great things and utter disasters. I'll report back on my rating for Macworld Boston in a few weeks, and if someone wants to whip up a snazzy Web form for calculating and recording these conference ratings for posterity, tell me when it's done and we'll publish the link.
Attendees -- Nominally, attendees are of course the most important audience, since without them, the conference has no reason to exist. But a conference must do a great deal to please attendees.
Cost/value. There are different sorts of conferences, of course, ranging from the more-or-less free to those that cost more than $1,000. A high cost on its own shouldn't hurt a conference's rating, so instead assign your rating based on the value you received for the cost you paid. Make sure to include travel, hotel, and food costs in your value calculation.
Time/place. A successful conference should schedule itself to be in an interesting, easily accessible city at a time of year when being in that city is enjoyable. Rate the conference on its overall timing (points off for intersecting with holidays); offering reasonable travel and lodging prices; and the city having good hotels and restaurants available. For instance, I've avoided conferences in Austin (because, at the time, it was difficult and expensive to fly there from Seattle) and in Houston (because it was August). No one has tried to lure me to a city like Chicago or Minneapolis in January, but those would also be tough decisions (particularly given that I would have just returned from Macworld Expo in San Francisco). I'm sure residents of these cities will immediately write to tell me how wonderful their cities are in these months, but that's missing the point: the time and location of a conference has to seem attractive to attendees who may not live in that city; there's a reason all-expense-paid junkets are traditionally to warm resorts in the winter.
Logistics. Is it easy to sign up for the conference (preferably online), pick up or purchase your badge and conference materials in person, and move between the hotel and the conference center every day? Include in this rating your opinion of general policies such as admission of children and whether or not such policies were well-explained in advance.
Breadth and depth of exhibitors. For conferences like Macworld Expo or Comdex whose success revolves around the trade show floor, attendees want a lot of exhibitors, and they want exhibitors whose products interest them. I once went to Internet World in New York City, and although there were a ton of exhibitors, it took me only a few hours to visit all those that interested me. At Macworld Expo in San Francisco, though, I'm often hard-pressed to see all the exhibitors within the four days of the conference (admittedly, that's in large part because I have many other commitments at Macworld Expo). And Comdex, the giant PC trade show in Las Vegas in November, has fallen on hard times, such that a recent report about why this year's Comdex has been canceled commented that recent shows have been filled primarily with Asian PC component manufacturers, and there are only so many hard drive enclosures one can examine.
Product support. Most of these items are under the control of conference organizers, but I'm aiming this one at exhibitors. Although it's of course important to bring marketing staff to a trade show booth, the information those people can provide is often available on the company's Web site. I'd definitely increase this rating for shows where exhibitors cut down on marketers and bring a few more tech support engineers (labelling both groups clearly!) since many attendees find the value of a trade show is being able to talk directly with experts about problems they have. Include in your rating how well the staffers follow up after the show on any promises they make.
Session Quality. Almost all conferences - particularly training-based conferences like MacDesign or the O'Reilly Mac OS X Conference - now offer a significant number of talks by industry experts, and they're key to attracting attendees who would otherwise have a hard time justifying the time away from work. Of course, you probably wouldn't even attend a conference with sessions that didn't sound interesting, so base your rating here on how good a job the presenters do, if the sessions are at an appropriate level, whether or not the sessions meet their descriptions, and if the session logistics work out well (all the sessions in the same building, no overlap between similar sessions, enough comfortable seating in rooms of a reasonable temperature, well-executed audio-visual support, session materials, and so on).
Keynote. All conferences have keynote talks that in some way set the stage for the rest of the conference. In the past, Macworld Expo always scored big with Steve Jobs's keynotes, and MacHack's late-night multi-hour keynotes have often proved to be more interesting than the norm. A good keynote is definitely worth points, but a boring and self-promotional talk by some unknown and unpracticed industry executive can be painful.
Free wireless Internet access. It's tough to escape the office for many people, but being able to check email quickly and for free greatly reduces the stress of being away. Conferences should always make sure wireless Internet access is readily available, though there is a question if it should be available during conference sessions. On the one hand, there were sessions at the O'Reilly Mac OS X Conference I wouldn't have gone to at all if I hadn't been able to stay online at the same time, but on the other hand, I paid more attention to those talks I did attend at the MacDesign Conference because I couldn't access the Internet from the session rooms. Bonus points for any conference that coordinates attendees collaborating on session notes with SubEthaEdit.
Great deals. It's less true now, but in the earlier days of Macworld Expo, people used to hold off purchases until they arrived at the show and saw what amazing deals were being offered. Attendees love the feeling of being able to score a huge discount on an expensive product as a way of justifying the cost and time of attendance.
Freebies. Everyone loves getting something for nothing (or at least having it seem that way), and a smart conference planner will make sure attendees feel lucky. At larger shows, this job often falls to exhibitors, who are smart to raffle off products or give away clever tchotchkes (but clever is important - it's better to pass on the freebies entirely than to spend money on one that's bad or boring). To pick up points here, smaller conferences must provide their own giveaways, ranging from t-shirts to mugs to donated products from relevant vendors. Bonus points go to conferences or exhibitors that put some thought into their freebies. My recommendation: avoid t-shirts entirely unless you're willing to create a truly amazing design, and if you do give away t-shirts, bring a variety of sizes, since not everyone can wear the default extra-large size.
Snacks. Obviously, providing snacks at a show as large as Macworld Expo isn't feasible. (Exhibitors can pick up the slack at individual booths; WordPerfect showed their mastery of demonstrations by tossing small bags of peanut M&Ms to audience members who responded to questions during a demo. Almost everyone got a bag, and the M&Ms gave people a small sugar rush and something to do with their hands during the talk. As a bonus, the talk was more interactive, as the audience competed for the M&Ms in small ways and had to pay attention to catch the bags thrown out by the presenter.) My experience is that attendees like snack breaks in the middle of the morning and the middle of the afternoon during session-based conferences. Unfortunately, such snacks usually must be provided by the hotel, which means you get the canonical sugary cookies and muffins - I'd assign extra points to any conference that offers fruit, vegetables, and other healthier snacks.
Fun. Of all the conferences I've been to, the ones that have tried throughout to provide entertainment to the attendees - in addition to the serious focus of the conference - have seemed the most successful. MacHack (now renamed ADHOC) always took the crown in this respect, organized as it is by a group of attendees, but Scott Kelby and the team that put on the recent MacDesign Conference also did a bang-up job in this respect, providing some humorous moments during the keynote, throwing a party on the first night of the conference, and running an absolutely hilarious mock game show based on Who Wants to Be a Millionaire? on the second night. That's worth points.
Community. Larger conferences have a hard time fostering a sense of community because there are simply too many people, but other smaller conferences, such as MacHack and the O'Reilly Mac OS X Conference, have done a great job of creating lounge areas where attendees can meet, talk, check email, and generally interact with one another. Since a huge amount of what goes on in the Mac industry is about personal networking, any effort to bring attendees together is worthwhile, and worth a point or two.
Exhibitors -- As much as every conference must focus on attendees, since they're the heart of any show, many conferences also provide space for industry companies to exhibit their products. Obviously, a trade show floor with exhibitors is the focus of a large, general show like Macworld Expo, but even MacDesign and the O'Reilly Mac OS X Conference have had small expositions. And since exhibitors pay to have booths, it's important for the conference organizers to meet their needs as well.
Booth cost. For an exhibitor, attending a conference is a pure business decision, and the first question any exhibitor will ask is: "How much does booth space cost?" Given the astonishing prices that some conferences charge for booth space, a reasonable price goes a long way to ensuring a happy exhibitor.
Location. Although it's certainly important to locate the conference in a city that's good for attendees, exhibitors pay even more attention to the costs involved. For a small company, airfare, hotel, and food for a number of employees adds up fast. When Macworld Expo was in New York, a number of well-known Macintosh companies either side-stepped the show or settled for reduced presence because New York can be so expensive.
Hassle factor. Organizing a booth at a trade show is hard work, and anything that makes it more difficult is a bad thing. Horror stories abound among people who run booths at Macworld Expos, mostly surrounding interactions with the venue's staff, who often play petty power games surrounding who is allowed to do what (and we're talking incredibly petty, as in vendors being "punished" with poor service for failing to let the venue staff plug in computers, for instance). The less hassle, the better, and the higher the rating.
Attendees. As much as attendees want to see a lot of interesting exhibitors at a trade show, exhibitors want to see a lot of qualified attendees. Needless to say, it's a feedback loop, since more people will attend a trade show if the exhibitor list is long and includes relevant companies, but if there aren't many exhibitors, those that do show may be disappointed in either the number or interest level of attendees. In the end, the conference organizers are responsible for attracting the attendees, and their level of success will tie directly to their rating in this area.
Sales. Attendees should not just be interested, they should be interested in buying. Purchasing exhibit space and paying for travel and lodging is expensive, and if an exhibitor can sell enough product to break even or even make money, that's a good thing. Some conferences don't allow sales, but in my mind that's self-defeating. If attendees want to buy (and given that these are industry conferences, they probably will), they should be allowed to.
Speakers -- In recent years, as the rise of the corporate Web site full of product information has made the big trade show floor with lots of exhibitors less compelling, conferences have focused more on training sessions and other talks by industry experts. Speaking as someone who presents at conferences regularly, there are a number of things that can be done to make speakers happy and ensure good sessions.
Payment. Traveling to and staying at a conference isn't cheap, and if a speaker has to pay her own expenses, as happens at Macworld Expo, it's often difficult to justify the trip. Waiving attendance fees is also essential, although it's meaningful only to those speakers who would definitely have attended the conference anyway. MacHack has traditionally offered free attendance to anyone who gives a session, but the community-driven nature of MacHack makes such a reward both entirely appropriate and all that's necessary under the circumstances. My take is that a free conference pass counts only for 0 points, whereas covering expenses as well increases that to +1 point. To move to +2 points and elicit the best sessions, an honorarium is necessary. Most Macintosh industry experts earn their livings writing books, consulting, or doing high-end design and illustration, and for such people, time is money. Or, to put it another way, time away from work preparing for and presenting at a conference is time that can't be spent earning income, so the effort level isn't what it could be. MacDesign paid its speakers for each session they gave, and from what I could tell, that honorarium went a long way toward increasing preparation and improving the quality of the talks for attendees. By the way, claiming that the speaker will get lots of useful exposure doesn't cut it for any established expert.
Moderators. The MacDesign conference was the first multi-track conference at which I've spoken that had a staff member introduce each speaker and stay in the room to assist with any audio-visual problems and other logistics like giving away prizes at the end of the talk. It wasn't the first time I've been introduced to an audience, but I hadn't realized until that point just how helpful that introduction was for starting a talk without needing to introduce yourself and explain briefly, without sounding like you're bragging, why you're qualified to stand at the podium. Self-introductions are embarrassing for most of us, and being introduced by a moderator not only made getting started easier, it also allowed the conference organizers to boast about what impressive speakers they had assembled. I'd definitely give +1 to any conference that offers introductions.
Logistics. A significant aspect of what prevents me from speaking at as many events as I'm invited to is the stress of dealing with travel and lodging logistics. Just clearing the time from my schedule, figuring out exactly when to travel and where to stay, and making all the arrangements can destroy an entire work day. Some logistics are unavoidable, of course, but the more that the conference organizers can remove from the shoulders of the speakers, the happier those speakers will be. Years ago, when Thunder Lizard Productions was organizing a series of Adobe Internet Conferences, a woman named Marci Eversole handled the conference logistics. To this day, I don't know how she did it, but hotel staff would bend over backward for her, and awkward items like transportation from the hotel to the airport at the end of the conference just happened. Every conference should have a Marci Eversole handling logistics, and if they do, a +1 or +2 rating is warranted.
Press -- My last audience usually attends the larger trade shows, but members of the press can show up at any conference, and the coverage they provide during and after the conference can prove extremely helpful for drawing future attendees. Keep in mind that members of the press are also attendees, so they can include many of the attendee categories when rating a conference.
Press registration. Some conference organizers put onerous requirements on press registration to avoid giving free admittance to anyone who claims an occasionally updated Web page counts as a publication. Some restrictions aren't unreasonable, but the requirements should be clear and geared toward minimizing the number of people trying to scam a free pass. Include in this rating the logistics of getting your press pass - we've had some sticky moments at Macworld Expo trying to pick up our press passes in time to get into the keynote.
News events. It's tricky for smaller conferences to arrange news events, but if important companies can be convinced to make announcements at the show, that goes a long way toward giving journalists something to cover. The more news items from a conference, the more coverage will occur, and the more likely journalists are to give the conference a good rating.
Press room. When covering a conference live, journalists need both a place where they can interview attendees or exhibitors, and a quiet place to write. Some conferences provide two separate rooms for this purpose - it's a good idea. The press room should offer wireless Internet access (as well as a few wired ports, just in case) so journalists can research stories and submit their pieces via email. Ideally the press room should provide both tables and couches, since the standard banquet tables and chairs provided by hotels and convention centers are about as non-ergonomic as could be imagined; it's often easier to write on a laptop while sitting at a couch anyway.
Food. Members of the press are always on the run from one meeting to another, and being able to stop in the press room to grab a bite to eat at breakfast or lunch goes a long way toward improving my mood (and my opinion of the conference). I seldom have time to wait in line to buy lunch, and even meetings that involve a meal can be stressful since they're often sandwiched between meetings that might run over.
Adding Up Your Results -- I'm sure there are other criteria on which you could rate a conference, so feel free to add a wildcard category for something that's not included above. For instance, if the conference does a good job of allowing people to attend remotely, that might be worth an extra point or two.
When you're done, feel free to post the results to TidBITS Talk along with any other comments you may have so others can take your experience into consideration when considering future attendance at that particular conference.