Hide full article
It's funny, but at TidBITS hardly a day goes by that we don't hear about some new piece of "beta" software. Maybe it's a utility program, a Web browser, a plug-in, or a major commercial application. Whatever the product, it's in "beta" and someone wants us to write about it - as a service to our readers, of course.
Before the widespread popularity of the Internet, "beta" software was a mysterious and fabled thing. If you knew the right people, got on the right lists, and went to the right trade shows, someone might quietly ask you to test some forthcoming software. More often than not, you had to sign a non-disclosure agreement, and you certainly weren't allowed to talk about the software until it was officially released. "Public betas" were virtually unheard of, and even long-standing power users had never head the term. "Beta?" they'd ask. "What's that mean?"
With the advent of the Internet, however, the "beta" stage of software development is being redefined, and frankly, I'm not happy about it. What does "beta" mean, and why are so many software developers racing to distribute software they admit isn't finished, and may be unstable and dangerous to use? What's going on here?
What Beta (Really) Means -- Although there's no universal methodology for software development, serious software products generally go through a series of development phases. As you might expect, there are as many variations on these stages and terms as there are software development efforts. However, an average application goes through these basic stages:
Design and prototyping: In this stage, the program's designers decide upon the purpose, specification, functionality, and interface of a product, along with its basic feature set, interface, and technology. Will the product use Apple Guide? What Macs will it run on? Questions like these are addressed, and some "proof of concept" programming usually begins. This phrase can go on for months, or be essentially indistinguishable from the next stage.
Development releases (or builds): In this, usually the longest portion of a product's development, the major features are coded, assembled, tested, and fixed. Usually, these releases are numbered and may be referred to as development releases ("d7" or "dr7"), test releases ("tr7"), or simply builds ("build 7"). The numbers assist in tracking which release contains a particular problem, fix, or feature. Products usually have several - even hundreds- of development releases, depending on the product's complexity.
Code complete (or alpha): Definitions vary, but the "alpha" phase of software development usually indicates that all a program's features have been coded and are testable. Although a button or menu item might move, appear, or disappear, no major features remain to be implemented. During alpha, developers focus on fixing bugs and making the product as stable as possible. A product may go through dozens of alpha releases, depending on its complexity and the nature of problems found.
Beta: When a program reaches the beta stage, its developers have found and eliminated all known, severe bugs (note that it may well be impossible to fix - or even find - all severe bugs). Remaining bugs might be isolated - for instance, only occurring under System 7.0.1 on a PowerBook 100 running an ancient version of CopyDoubler. Remaining bugs may also be out of the developer's control, possibly because they are caused by another program or the operating system. The goal of the beta phase is to see if the product runs in a stable and reliable fashion on a wide range of machines; to achieve this, the release is distributed to a wider audience than development or alpha releases. During beta, bug fixes or code changes should be carefully considered before being implemented in order to avoid accidently de-stabilizing the program. Theoretically, programmers should mostly sit on their hands during beta, while the software testers do everything conceivable to break the product. Also in theory, a product should have few beta releases.
Release: The final phase of the development process varies widely. Some products enter a Final Candidate ("fc") phase after beta, where the product is frozen for a pre-determined period of time and continues to undergo rigorous testing. If any problems are found that must be fixed, the Final Candidate process begins all over again. Some products ship once a sufficiently stable beta has been achieved, and that beta becomes the shipping product. Some development teams use a combination, where the final beta becomes the "golden master," which is sent for manufacturing, but may be withdrawn if any must-fix problems turn up prior to actual distribution.
From this, it follows that a beta release theoretically has the following three properties:
1) The product's technology requirements (both hardware and software) are fixed, and have been for some time.
2) The product is code and feature complete. No features remain to be implemented, and all features are present and testable.
3) The product has no known severe outstanding bugs the developers plan to fix or work around.
Take a moment to think about beta releases of products you may have used recently, and compare them to the three points above. Notice anything different?
What Beta Means Now -- I'll be the first to admit the development cycle outline above is idyllic, and doesn't account for many complicated forces affecting software development. Competing products often force changes in feature sets, and marketing or distribution deadlines may cut short any of the phases above. Similarly, there are inherently unknowable factors: I once worked on a product whose lead programmer wasn't able to work for nearly three months due to a case of pneumonia.
Nonetheless, in many cases - particularly with Mac Internet software - "beta" doesn't mean anything close to what it used to. We've seen programs in public beta that not only contain innumerable known bugs the developers are aware of and plan to fix, but also accumulate major new features through subsequent releases. Similarly, we've seen products that change fundamental system and technology requirements during beta - details which should have been etched in stone long before. Beta often means what "alpha" or even "development build" used to mean.
The Race to Beta -- The emergence of an Internet software industry (and the Internet itself) are changing the beta process in two fundamental ways First, Internet betas are fast, widespread, and (often) cheap. Conducting a beta using traditional media - floppy disks or CD-ROM - is comparatively expensive and (more importantly) time consuming. Master disks must be duplicated and physically delivered to a list of testers, who may not even be representative of the product's potential users. In contrast, conducting a beta test via the Internet eliminates time delays as well as duplication and delivery costs, but more importantly reaches a wide range of users. In many cases, it also simplifies communication with beta testers (who all presumably can use email). Handled properly, a public Internet beta can significantly contribute to the quality of a product, even though it requires more person-hours to process the higher volume of bug reports and feedback. Done well, I think a public Internet beta is in the best interest of both software developers and end users who feel the need to explore the bleeding edge.
However, there's a flip side. Public betas usually receive a lot of attention, particularly on mailing lists and newsgroups, as well as computer industry publications including Macworld, MacWEEK, Wired, and, yes, TidBITS (we often feel pressure to report on beta releases being talked about online before we consider the products worthy of real consideration). What's more, beta releases are usually subject to less scrutiny than a shipping product: severe bugs in a beta are often played down or set aside, while new features and capabilities are hyped. Bugs - even flagrant, long-standing bugs - are excused because the product is "just a beta - what did you expect?"
This situation is a marketer's dream come true. Marketers used to approach the Internet community with "I don't care what you say about me, just get my URL right." Now, it's "Here's my URL, and it doesn't matter what you say about me - it's just a beta!" Not only do the products get publicity, but the company doesn't have to take as much heat about bugs and incompatibilities.
Beta Backlash -- The problem with using public beta testing for promotional and visibility purposes is that quality of the product is compromised. Many products labelled "beta" today are still in active, even furious development, with their programmers adding and removing features, changing the interface, and doing serious work to underlying, fundamental code.
I hate to say it, but that's not beta. It's not even alpha.
Careful readers will note this article hasn't used the names of any specific products or companies. It certainly could have. The point here isn't to criticize particular companies or products, or to praise others. The goal is to point out that the process of software development is undergoing a fundamental change, and users affected by this shift should be aware of the competing (and often conflicting) dynamics behind that change. In the next year, I expect the term "beta" will fade from usage, to be replaced by various phrases using the word "preview." My advice is to think seriously before using pre-release software for which developers and companies assume no responsibility, and to back up your data early and often.
And the next time someone says "It's a beta, what did you expect?", tell them: "software that's feature-complete and has no known serious bugs." That's what beta means.
