Saturday, April 12, 2008

Debug by smell

I had lunch with my first development team a while back. They had all worked together at Control Data before I joined their team. When I meet with them, it's usually as a gathering of more Control Data people and they reminisce about the old Control Data computers and tell a bunch of Seymour Cray and CDC stories. It's a lot of fun to hear how computing used to be and how similar it is to today.
Often they talk about the size of the CDC supercomputers and how things were done on them back in the day.
At the last meeting, some of them talked about the dedicated team of electricians that were on hand for the computers and what a tough job it must have been to find and fix faulty wires.
To give a little perspective to this responsibility, I would recommend checking out Mark Richards' book Core Memory. Great book, BTW. Look at the pictures. Ok, see the wires? It was someone's job to go through those and determine which ones are faulty when there were problems.
I found out how the electricians did it.
My buddy Keith who I see at Caribou Coffee was an electrician for CDC. I asked him how they did it and the answer is pretty surprising. He told me that they would find it literally by smell. Usually when they'd have a wiring issue it wasn't the wire, but a solenoid. When a solenoid would burn out it would give off a distinct smell. The electricians could then replace the bad solenoid and carry on. He said the wires almost never went bad themselves.
Kind of gives new meaning to the term code smell.

No comments: