New: Puzzle Hunts are Everywhere: Simple Website Monitor

Waiting for the bus, Jonas asked me: "Why did you start beeping during that tech talk?"

People at work occasionally start beeping. We're an internet company with many servers. When servers have problems, system administrators' pagers start going off. But I'm not a system administrator. I'm a technical writer. When I go on vacation, I don't tell people "In case of emergency, you can reach me at ____", I write "In case of a documentation emergency (ha ha), you can reach me at _____". And it's funny. At least, it was funny the first time.

And the people at this tech talk probably weren't likely to get paged. It was a tech talk on an open-source library of computational geometry functions. You don't really see these people getting paged with... I dunno, some errant line segment getting into a data set, coincidentally totally vertical, its infinite slope causing numbers to spin out of whack or...

I don't know where I'm going with that. That doesn't happen.

I 'fessed up. That pager signal didn't come from a work system. I'd set up a computer program to monitor coed astronomy's web site. They were going to host a puzzle hunt and their web site would announce when sign-ups were possible. I wanted to know when that happened: if their game didn't have many slots for teams, I wanted to sign up before those slots filled up.

It's pretty easy to set up this kind of monitoring if you have a Unix machine on the internet. (I bet that very few people will be interested in this blog post. Half of the bay area puzzlehunters are software developers who will wonder why I'm describing something so obvious; the rest don't program and are about to get scared away when I show them source code. But maybe I'll show them that, if they're going to get into programming, that this is a pretty achievable task to take on.)

On a Unix machine, you can set up a "cron job". You can tell the machine to run a program once every few minutes. (You can set up cron jobs to run at strange times. I set up this cron job to run on prime-numbered minute offsets from the hour--because I was in a silly mood.)

What program did I set up? I set up a simple python script. This script checked coed astronomy's web page and compared its contents to a copy I'd downloaded earlier. If it noticed a change, it mailed my pager. Like I said, this script is simple, if you know what you're looking for. Python is a nice language to use because, for any given task, someone has probably written a library of functions to help you. The bad news is that it can find a while to find the library that you want. In this case, I wanted urllib2 to fetch the web page contents.

import os
import urllib2

# download the page
pagecontents = urllib2.urlopen("").read() 

# compare it to previously-saved file, "golden" copy of the page
# contents which I downloaded earlier.
goldenfile = open("/home/lahosken/golden.html")
goldencontents =

# If there's a difference, OMG page me by sending mail to my pager
if goldencontents != pagecontents:
  os.popen("mail -s IT_IS_HAPPENING < /dev/null")

This script watches one page. I've done stranger things to watch a web site, running wget in spider mode and then recursively diffing the resulting directory to a previously-generated "golden" directory tree. And there are stranger things.

Anyhow, my pager went off during a tech talk. And it kept going off because I was so busy signing up for the game that I didn't immediately shut the script down. But I eventually did. Hopefully, everyone at the tech talk thought I was a pager-wearing badass entrusted to rescue foundering servers. But Jonas wasn't fooled. Maybe none of them were.

Labels: ,

Posted 2008-03-08