I really liked Malcolm Gladwell’s Outliers. The whole book was good, but one story in particular resonated with me. Gladwell was talking about why some airlines have better safety records than others. The biggest part of this, he said was communication, especially in a crisis. This is one of those “soft” skills that can be hard to measure or understand. All of this is buried in our culture so it’s hard to even notice our own problems, much less fix them. One of the positive examples he gave involved a large jumbo jet full of fuel making an emergency landing. He described several things the crew did to achieve a smooth landing. But the most important thing they did was wake the second crew (it was supposed to be a long international flight) so they’d have two more sets of eyes. And they weren’t just sitting there. They divvied up the responsibilities among the two pilots and the two copilots. More important, each one constantly making a report, keeping the others up to date.

We recently had some problems in our server room. No lives were at stake, but our customers and employees sure did want everything fixed ASAP. Mike and I each received a panicky phone call at about the same time. The first thing we both did was Skype each other. We’ve been working together for a while, so we’ve got that part down.

Mike and I started working through our checklist pretty fast. (And we made some notes for the future. After the panic we went back and cleaned up the check list.) Eventually we grabbed Dave. Dave’s only recently started working in the server room, so he wasn’t part of the usual routine. But as Trade-Ideas keeps growing, no one can be an expert on everything. Three heads can be better than two. As we keep growing I’m sure Dave will be doing more and more work in the server room. In any case, I’m glad he was available when we needed him.

I made a point to reassure Mike that he was in the lead. That was something else I learned from Outliers. Many of the bad airplane stories all came down to the exact same problem. The copilot saw something but was too intimidated to tell the pilot. Officially Mike reports to me (and he is very quiet), but he is the most knowledgeable in this area. We were all doing our parts. But I wanted him in the lead, and I needed him to know it.

When we first hired Mike our goal was to do twice as much technical work. I quickly realized how talented Mike was, and that gave me the confidence to go on vacation. It’s also good to bounce ideas off each other. We don’t do that as much as we should. (I hate to bother people with my work when they are busy with their own work.) That last part, bouncing ideas off someone, is essential in a crisis. Of course it’s good to have more experts, and more people can do more work at once. But the most important part of working together is that it gives us the sense of confidence that is required to stay cool and make good decisions under pressure. Often you don’t know everything you should, but you have to make a decision anyway.

My advice for handling a crisis: Obviously
• be prepared,
• don’t panic,
• do a post mortem, even if it’s informal.
But most important, get the brain trust together, get everyone focused on the problem at hand, and keep talking.

Gladwell used plane crashes just as an example. His real point was that we need to recognize and overcome the problems in our own culture. As programmers we are used to working alone. We like the ability to stop and think without interruption for hours if not days at a time. Sometimes we have big projects and have to work in teams. But even then we divide the work up into small pieces. You know you’ve done a good job dividing the work if each person can work on his own piece without bothering anyone else very often. But that’s not always what you need. A crisis calls for more communication, and that means getting over our culture of being lone wolves.

— Philip Smolen