THE JOEL TEST.

As a "pipsqueak" junior programmer at Microsoft, Joel Spolsky saw a few death-march projects and occasionally locked horns with control-freak bosses. But most of the time, Spolsky says, Microsoft provided a supportive environment "where things are done at the lowest level and most managers act like their job is to run around the room, moving furniture out of the way, so people can concentrate on their work."

Spolsky now runs a well-respected contract programming firm, Fog Creek Software, so he spends much of his time on management issues like productivity, product quality, and recruiting. But he still believes the way to create first-class software is to build "a great place to work" and let programmers have the greatest possible control over their own projects.

Spolsky's notion of a "great place" isn't an unsupervised playground, though. In fact, he argues that the best development groups typically are run with considerable discipline and formal structure. As a way for managers and investors to measure this discipline, Spolsky has put together a dozen checklist questions called The Joel Test. "A score of 12 is perfect, 11 is tolerable, but 10 or lower and you've got serious problems," he warns. "The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full time."

n Do you use source control? "If you don't have source control, you're going to stress out trying to get programmers to work together," says Spolsky. "The other neat thing about source control systems is that the code itself is checked out on every programmer's hard drive--I've never heard of a project using source control that lost a lot of code."

n Can you make a build in one step? "If it takes 20 steps to compile the code, run the installation builder, etc., you're going to go crazy and you're going to make silly mistakes." Good teams typically create a single script that automates the whole build process, Spolsky says-- which is especially valuable when a product is almost ready to ship and programmers are racing to fix the "last" bug.

n Do you make daily builds? Simple mistakes--for instance, forgetting to add a source file to the code repository--often cause new builds to break, Spolsky notes, leaving the whole team in limbo until the problem is fixed. "One good way to insure that breakages are fixed right away is to do the daily build every afternoon at, say, lunchtime. If the build failed, you fix it, but everybody can keep on working with the pre-build, unbroken version of the source."

n Do you have a bug database? "Lots or programmers think they can hold the bug list in their heads. Nonsense," says Spolsky. "A minimal useful bug database must include the following data--complete steps to reproduce the bug, expected behavior, observed (buggy) behavior, who it's assigned to, and whether it has been fixed or not."

n Do you fix bugs before writing new code? Fixing bugs immediately is "basically trivial," says Spolsky, but the cost and effort rise dramatically when bugs are fixed later in the development cycle. Since it's hard to predict how much time it takes to fix a bug, moreover, "a schedule with a lot of bugs remaining to be fixed is unreliable. If all that's left is new code, your schedule will be stunningly more accurate."

n Do you have an up-to-date schedule? "Programmers are notoriously crabby about making schedules," Spolsky notes. "OIt will be done when it's done!' they scream at the business people." That's an unacceptable attitude, he argues: Programmers need to be shown how their progress affects decisions and time lines for the rest of the business. And if the schedule slips, it's usually best "to pick the least important features and cut them" rather than let the project run into overtime.

n Do you have a spec? "Writing specs is like flossing: Everybody agrees that it's a good thing, but nobody does it," Spolsky quips. "Software that wasn't built from a spec usually winds up badly designed and the schedule gets out of control." Often, specs are overlooked because programmers are "reluctant writers" who'd rather express their ideas in code than create documents, he says. One solution is to persuade programmers to take writing courses; if that's impractical, make spec writing part of the program manager's job. "In either case, you should enforce the simple rule, Ono code without spec.'"

n Do programmers have quiet working conditions? Giving programmers offices with walls and doors isn't just a luxury, Spolsky argues--it's an investment in productivity. "We all know that knowledge workers work best by getting Oin the zone,' where they are fully concentrated on their work and fully tuned out of their environment. This is when they get all of their productive work done," says Spolsky. "The trouble is, noise, phone calls, going out for lunch, having to drive five minutes to Starbucks for coffee, and interruptions by coworkers--especially interruptions by coworkers--all knock you out of the zone. If you're in a noisy bullpen environment like the type that caffeinated dotcoms love to create, your productivity will plunge."

n Do you use the best tools money can buy? "Top notch development teams don't torture their programmers. Even minor frustrations caused by using underpowered tools add up, making programmers grumpy and unhappy. And a grumpy programmer is an unproductive programmer."

n Do you have testers? "If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products or you're wasting money by having $100/hour programmers do work that can be done by $30/hour testers. Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't recognize it."

n Do new candidates write code during their interview? "Every day, programmers are hired on the basis of an impressive resume or because the interviewer enjoyed chatting with them. Or they are asked trivia questions which could be answered by looking at the documentation," says Spolsky. "Please, just stop doing this. Do whatever you want during interviews, but make the candidate write some code."

n Do you do hallway usability testing? "A hallway usability test is where you grab the next person who passes by and force them to try to use the code you just wrote. If you do this to five people, you'll learn about 95% of the usability problems in your code."

Joel Spolsky, founder, Fog Creek Software, 217 E. 31st St., New York, N.Y. 10016; 646/424-1712. E-mail: spolsky@fogcreek.com.

Print Get the Mag Weekly Updates