: New:

I'm sufficiently vain to set a Google alert so I find out when I'm mentioned someplace new on the World Wide Web. Thus I found out I'm in the Acknowledgments section of a recent academic paper. Oh, neat!

Some nerds at a company called Scale A.I. set up a new benchmark for AIs. Here, a "benchmark" is a set of questions and answers; ask the same questions to several AIs, grade those AIs based on how many answers they get right. This new benchmark was unusual: They gathered together puzzles from places like the MIT Mystery Hunt and Puzzled Pint. I, with esteemed puzzle editor Neal Tibrewala, made some Puzzled Pint puzzles, thus the acknowledgment. (Some question the utility of this benchmark; is it useful to measure a chatbot's wordplay ability if customers don't care about that and instead want the chatbot's advice on, say, setting a tariff rate on an uninhabited set of islands in the Antarctic? This is a valid point. One nice thing about puzzles: They have unambiguously correct answers. Grading the answers to questions like "What tariff should we apply on these penguins?" isn't so straightforward.)

I chuckled. I wanted to keep up with whatever other silly things these "Scale" people came up with. I looked around to see if they had a blog I could follow. I found their blog! I saw some recent articles:

When a bunch of AIs try to solve Mystery Hunt puzzles and get 0.00% correct, it's a fun chuckle. But when Israel's military uses AI to figure out who to kill and ends up murdering aid workers and reporters, nobody's laughing.

Whew! Who knew looking over a vanity alert would just lead to a reminder that there are people playing at keeping me safe by means of scattershot murder. Here's hoping they fail at the scattershot murder part.

Tags: puzzle scene trust the machine

lahosken@gmail.com

Tags