A while back--long enough ago that I'm probably getting details wrong--someone told me how the Scoobies tackle a puzzle. They set the puzzle out where everyone in the team can look at it. They look at it. But instead of everyone blurting out their ideas at once, they just kind of keep them in mind. After a while, they go around and folks talk about hypotheses. The intent: share some hypotheses, resist the temptation to go haring off after the first dang fool notion to pop into someone's head. Our brains do that--someone says "Hey maybe it's Morse code?" and, bam, everyone looks at that puzzle through the lens of Morse code. It's hard to break out of that Morse-ish point of view even when you've intellectually convinced yourself that it couldn't be Morse code, no way. It's as if someone said "Don't think of an elephant."
The technique is that someone says "Maybe it's Morse code," but before everyone looks back at the puzzle someone else says, "...or maybe, gee, I think it's semaphore", then your brains are less likely to get stuck in the Morse code track/trap. Everyone pipes up with their ideas before looking back at the puzzle.
So I was mighty interested when Ducky Sherwood mentioned a similar computer-debugging technique in her blog.
If you aren't a computer programmer, you might be surprised to find out that debugging a program is like solving a puzzle. It's a stretch, but... If you're looking at a puzzle, you're staring at something like, say the seeming nonsense 208120'19 238120 1985 19194; you're thinking There's a bazillion possibilities about what message is encoded here; there's a bazillion possibilities about how it could be encoded; how do I narrow down the possibilities? How do I test ideas? When you're programming, you're thinking In this web shopping-cart program, I expected taxRate to be .08, but instead it's 0.2; there's a bazillion places where we could have accidentally clobbered that value or skipped setting it; how do I track down exactly where things went wrong?
Ducky had read that programmers tend to fall into mental traps. Why does taxRate have the wrong value? It must be a memory corruption bug! You've run into a couple of memory corruption bugs lately, and you're soooo sure this must be another one. So you waste a couple of hours running the program under a heavy-duty memory-corruption-bug-finding tool. Meanwhile, you totally ignore the fact that this shopping cart belongs to your first-ever customer from Puerto Rico and your database of local tax rates has the wrong value for Puerto Rico.
Ducky decided on a new approach to debugging: before diving into the code, make up three hypotheses about what the problem is.
... After a binge of reading Andrew Ko papers last week, I decided to start forcing myself to write down three hypotheses every time I had to make a guess as to why something happened.
In my next substantive coding session, there were four bugs that I worked on. For two of them, I thought of two hypotheses quickly, but then was stumped for a moment as to what I could put for a third… so I put something highly unlikely. In once case, for example, I hypothesized a bug in code that I hadn’t touched in weeks.
Guess what? In both of those cases, it was the “far-fetched” hypothesis that turned out to be true! For example, there was a bug in the code that I hadn’t touched in weeks: I had not updated it to match some code that I’d recently refactored. ...
--"false hypotheses"
Those papers she mentions--Ducky researches programmer productivity. She's not just making up this three-hypotheses approach out of thin air. She's basing it on some research, though apparently the research itself is not so easy to find, as she points out in a later blog post:
I finally got my hands on the dead-trees (i.e. uncorrupted) version of the Klahr/Dunbar article that I posted about earlier, and it didn’t say anywhere how long people in the hypothesizing group spent on coming up with hypotheses. However, I was able to track down David Klahr, and emailed to ask him how long they hypothesizing group spent hypothesizing. He graciously and quickly replied that it was only a few minutes. So if we make a wild guess that “a few” works out to an average of about four minutes, then the hypothesizing group took an average of about 10.2 minutes, while the non-hypothesizing group took an average of 19.4 minutes — so the hypothesizing group is still twice as fast as the non-hypothesizing group. ...
--"Hypothesizing first makes you more productive"
So there's support for this notion of coming up with a few hypotheses before trying one of them out. And notice that this research mentions groups of people, not just lone programmers. Teams of people... hmm...
The important world-saving question here is of course: How to apply this to team-based puzzle-solving games? How do you convince folks to not blurt out their ideas in the first 30 seconds? This activity attracts plenty of competitive people. If I look at a puzzle and yell out "I think it's Morse code!" before anyone else on my team does... and if the puzzle is Morse code, then I get to strut as we walk back to the van, right? I just proved I'm a puzzling stud, right?
Or maybe I was the first to blurt out Morse because I'm a fan of Morse, I want to see it everywhere, it's always the first thing I look for. If I spotted Morse while my semaphore-loving teammate thought Is it semaphore?... Oh, I guess not maybe that's not cause for strutting. If the puzzle isn't Morse and the whole team wastes half-an-hour barking up the wrong tree, that's no good. It's all very well that I can say "Well, I just spoke up with a theory; other folks could have spoken up with their theories. I didn't gag them or anything." but maybe by speaking up so early, I caught their brains in the Morse trap. Maybe I should have kept quiet, tried applying Morse code for a minute on my own, let my team-mates consider other possibilities.
How long to sit and ponder quietly? How do you decide when to share ideas with the group? If you have three ideas, how do you decide which to work on first? I don't know.
Now that I think back, I vaguely remember that the reason I heard about this technique is that Alexandra Dixon wanted us Mystic Fish to use it--she had us look at a puzzle quietly, then share hypotheses. But I think that when we looked at the next puzzle, we were back to our blurty ways. (Or maybe the next puzzle was such a stumper such that no-one had any good ideas and everyone had kept quiet out of ignorance, not out of technique? So then when we saw the next-next puzzle, we had forgotten why we'd kept quiet before?.... Oh, vague vague memory.) So this technique does take discipline; it doesn't seem to come naturally, at least not to Team Mystic Fish, and I haven't seen other teams do it, either.
Labels: puzzlehunts, research, teams