MITMH 2020: 2019 Nov

Careful: I'm pretty blasé about spoilers for this year-old hunt. If you haven't already solved the puzzles you wanted to solve, maybe don't keep reading.

Every so often, fearless leader Corey sent out a "State of the Hunt" email. Here are some excerpts from November's emails to let you know what the writing team was doing. (And to remind me, as I write this, what we were doing.)

Overall: Since BTS2, our work has shifted from planning to doing. Most of the uncertain details have been sorted out; we now just need to put in the hours to make it all happen. Puzzles are being made production-ready; costumes are being bought and assembled; coins are being pressed; etc. We’re on a good trajectory to have our Hunt ready for January, but we still have a lot of work to do.

Story: We have the full narrative written, including the plot reveal points (eg, when Penny visits the team). All characters have been cast along with doubles. The intro skit will run a video of Penny Park “from years ago”; that video is being recorded this weekend. End game discussion is happening now (eg, #runaround).

Puzzles: Several puzzles are still in writing; these are all due for BTS3. We’re expecting to need urgent testing for some puzzles starting around Thanksgiving.

Server: The feedback from BTS2 drove us to create a final feature list for pennypark.fun [the hunt web server]. We’re aiming for the site and admin console to be feature complete by BTS3.

Events: We have 5 events planned: Midway, Character breakfast, Line [the event-idea from which we salvaged Build-a-Wand workshop], Parade, and Fireworks. Completing each event earns a PennyPass; completing all events earns a pressed penny. Line has been heavily revised in the past two weeks, and BTS2 provided a lot of useful feedback on Midway. We’re aiming for some of these events to test again during BTS3.

Registration: We’ll start sign-ups in about a week. We’re setting up our official contact accounts for teams now.

Requests for help:

Testing. We have an urgent need to test revisions to Domino Maze and First you visit… .

BTS3. Our third and final big test solve will be December 7–8. We’ll test the final set of puzzles (two inner lands and two outer lands) that weekend; please RSVP on Google Calendar.

Hunt Ops jobs. Nearly all the jobs are filled but we could still use an Unattached Hunter Attacher and second person to play Penny for team visits. Contact Linda if interested.

November started with Big Test Solve 2 (BTS2). That is, it started with a weekend-long testsolving party.

As Producton Czar (team lead for puzzle HTMLifiers), I was worried about last-minute puzzles coming in. In the week leading up to the test, there were still about a half-dozen puzzles that didn't even exist as a draft. What would I do if they all showed up the night before hunt? There wouldn't be time to HTMLify all of them.

As it turned out, that wasn't an issue. Those puzzles weren't ready the day before the test; they weren't ready the evening before the test. For the test, those puzzles just said "please ask HQ for the latest version of this hot-off-the-presses puzzle." I think(?) for each puzzle, the authors had come up with something by test-day morning to hand over to HQ so we could, in turn, hand it over to testers as needed. (But maybe not all puzzles even had that? I don't remember. Anyhow, it was no longer Production Czar's problem; it was Testing Czar's problem to figure out how to get two clean testsolves for these stragglers…  Though, I guess once that happened, those puzzles were Production Czar's problem, after all.)

As with BTS1, I didn't have much to do for BTS2. Due to my production/HTMLification duties, I was at least somewhat spoiled on all of the puzzles. So were fearless leader Corey and editor-in-chief Yar. There were a few hours Sunday morning when I was the only HQ person awake, so I handled the full load of handing out puzzles and hints and such. That sounds pretty impressive until you remember that most of our testsolvers were sleeping, too.

Once other folks had woken up, I went home and went to bed; I found out how the test had ended later.

This playtest was pretty darned useful. We had two testing teams. In addition to the expected Team Left Out folks, there was also a big group of Microsoft Puzzlehunt nerds up in Washington State. I'm not sure why they were up for playtesting. Perhaps they were too sensible to fly to Boston in January but where in the mood for a weekend's worth of gnarly puzzles? Anyhow, I'm glad they testsolved.

In hindsight, we crammed too many puzzles into this big test solve. This testsolve went long. Although our testsolvers were used to weekend-long hunts and we'd announced it as a weekend-long testsolve, both test-teams lost most of their steam about 18 hours in and didn't fully recover. To get teams through, we had to give out plenty of hints. That's not terrible—we wanted to test our hint system. But for puzzles that had been hinted, we couldn't apply BTS solves towards our "two clean testsolves before a puzzle is ready" rule. Team Left Out didn't make it all the way through the Creative Pictures Studio round—testsolvers didn't solve its meta until Wednesday. Instead of three BTSs (each covering one third of the hunt), I wish we'd done four BTSs.

[Picture: nerds gathered around long table] [Picture: nerds solving Domino Maze]

During BTS2, a testsolver asked for a hint on a puzzle. Fortunately, the puzzle's author had written some hints ahead of time. Unfortunately, none of those hints answered the testsolver's specific question.

Yar knew the puzzle pretty well and knew where he could find the notes from a previous testsolve. So Yar was able to write up the answer.

Our puzzle hints were part of our "production" puzzle data. Folks on the production team knew how to edit a puzzle's hints, but nobody else did. If a puzzle author or a knowledgable person (such as Yar) had good idea for a hint, they had to work with someone on the production team to update the puzzle. Furthermore, updating a puzzle during a hunt was somewhat dicey: it wasn't likely to trigger any hunt server bug, but it wasn't 100% unlikely, either.

Yar had worked hard to crank out that hint; but if another team asked that same question, we wouldn't have an easy way to find Yar's answer—we wouldn't have saved it with the puzzle's data.

So I had the bright idea: Instead of keeping hints with the puzzle data, we should have a separate Google Doc for each puzzle's hints. That way, any team member could edit hints without help from a production nerd or risking crashing the server or what-have-you.

(As it turns out, this was a good idea, but not for the reasons I predicted. It wasn't puzzle authors editing those hints. It was mostly Wei-Hwa Huang (and maybe Nina Hinrichs? I forget) going on a pre-hunt hint-writing spree. Editing those docs online was a lot easier than going through the production updating process.)

Good news: I'd figured out a way to improve our process. Bad news: Somebody had to copy all of those hints out of puzzle data and into a bunch of new Google docs. That took a fair amount of effort.

Then it was time to prepare for Big Test Solve 3 (BTS3). There was a fair amount of puzzle production (HTMLification) work leading up to this goal, but not as much as there had been for BTS2. This late in the year, a sizable fraction of puzzles had already been completed. There were still some BTS-deadline-driving puzzles taking their time, but not as many as we'd seen for BTS1 and 2.

I asked each production-team member to HTMLify a puzzle, but if they were busy with other things, I didn't panic—we kept up just fine.

Around Thanksgiving, half of our puzzles had made it out of testing (i.e., had two clean testsolves) and thus moved through "the pipeline" to production (HTMLification) or beyond.

This was much later in the year than optimistically scheduled.

We continued to have a tough time finding folks to test puzzles that had appeared in Big Test Solves. If a written-at-the-last-minute tough puzzle appeared in a Big Test Solve, probably many Left Out folks would saw it in that context. Thus, it was difficult to find unspoiled folks to test those puzzles later.

I don't have a great solution to this problem.

A problem that had been bubbling on the back burner but boiled over as our efforts shifted from first-drafts to final-polish: We didn't have a single "source of truth" for puzzle titles—which was bad because that's how we kept track of puzzles.

When Tech Czar Doug wanted to fetch the latest versions of our puzzles to put on the hunt web server, he'd fetch them by name. But puzzles sometimes changed names. E.g., when I started writing one mini-runaround puzzle, we hadn't picked a theme yet. It visited a couple of periodic table-related artworks, so I tentatively titled it "Table Hopping". Eventually, we figured out that this puzzle would be a theme park attraction in the Enchanted Forest section of Penny Park—a land of fairy tales. Thus, editor-in-chief Yar let me know that this puzzle would henceforth be known as Rumpelstiltskin's Cottage.

The hunt web server, unable to read Yar's mind, didn't know that the puzzle's title had changed. I changed the title of the puzzle on my computer; and when I uploaded future versions, they had that title. When Doug told the web server to fetch the newest-version puzzles, it would see that table_hopping hadn't changed recently and think that puzzle was up to date; it didn't look for a rumpelstiltskin puzzle because it didn't know there was such a puzzle.

Fortunately, I knew this was an issue. So I told Doug, Doug told the web server to look for rumpelstiltskin instead of table_hopping, and soon all was right with the world. But not everyone knew this was an issue.

Just about everyone knew about Puzzletron; some folks knew about Natalie's spreadsheet; some folks knew about production data. Few folks knew how to change their puzzle's title everywhere; many folks thought they knew how to change their puzzle's title everywhere. It was a mess that we never 100% figured out.

Later, we had a similar problem with the credits for each puzzle. Puzzletron had a list of authors, but only knew about people with puzzletron accounts. If your heraldry-expert friend consulted on one puzzle but didn't set up a Puzzletron account, Puzzletron wouldn't give them credit for authoring that puzzle. To work around this problem, we kept another set of credits for each puzzle in its production data. But someone didn't realize this and set up an online spreadsheet to give folks yet another place to specify credits… which Doug then had to somehow get onto the web site.

I wish we'd figured out "one source of truth" and then let everyone know what we'd figured out. Ah well. Maybe next time.

Dec [>]

| comment? | | home |