Syllables from Phonemes: Nothing is E-Z

I want to write puzzles that use word-sounds. And by "write," I mean I want a computer to do all the hard work while I stare off into space and think about burritos. But word sounds are tricky for computers. You can't just look at the text of the word. E.g., LEMON and DEMON ought to rhyme, but don't. You kinda need a room full of students to key in all the English words' sounds "by hand."

I like to imagine that's how the CMU Pronouncing Dictionary came into existence; a room full of students muttering to themselves and typing. The result was a big text file, of which I'll copy a fraction here:

REACTS  R IY0 AE1 K T S
READ  R EH1 D
READ(1)  R IY1 D
READ'S  R IY1 D Z
READABILITY  R IY2 D AH0 B IH1 L IH0 T IY0
READABLE  R IY1 D AH0 B AH0 L

Each line looks like WORD (two spaces) PH ON EM ES OF WO RD, where vowel phonemes are further, uhm, decorated with numbers indicating stress. (Useful for poetry-rhyming: UNHURRIED and WANT TO READ both end with an -eed sound, but the different stress levels would make for an awkward couplet.) Since READ can be pronounced like RED or REED, there's an entry for READ and another for READ(1).

This is a pretty awesome resource, and I've been using it to good effect. Of course, since I have engineer-brain, I don't think much about the parts that work well, and instead stare off into space and think about the parts that don't.

I want to know the syllable-breaks in words. When a computer sees the line

READABLE  R IY1 D AH0 B AH0 L

...It doesn't know whether that word is REE-DUH-BULL or REED-UHB-UHL or what-have-you. Maybe there's an automatic way to figure it out, but when I tried googling [phoneme syllable], I bumped into academic linguistics papers. When one bumps into academic papers, one tends to assume one's reading about not-so-easy problems.

I want syllable breaks. Both for puzzles and for, uhm, not-so-great rhymes. Who can forget the classic

To find a rhyme for silver
Or any "rhymeless" rhyme
Requires only will, ver-
Bosity and time.

How would you find this rhyme in the CMU dictionary? (You wouldn't; it doesn't have the word VERBOSITY. But if it did, how would you find it?)

VERB  V ER1 B
VERBAL  V ER1 B AH0 L

In VERB, the B is in the same syllable as the "ER"; but in VERBAL, the B wanders over to the next syllable, clinging to AL. But you can't just say "If there's a following vowel, then the consonants in between wander over." In VENTRAL, the N stays in the first syllable, while the T R travels to AL.

Of course, there's the "easy" solution of asking a bunch of CMU students to mark the syllable breaks in words; or better yet, find out that someone else has already done this. I haven't had any luck finding such a thing, though. Perhaps because I don't know the name of what I'm looking for; perhaps because it doesn't exist.

Permalink
& Comments

Link: Can you code better than a fourth grader?

What if we're talking about a Vietnamese fourth grader? Neil Fraser went to Vietnam, and since he's a programming/educating nerd, he checked out the local computer science programs. He didn't just check out fourth graders, but also older kids; they're pretty impressive.

Permalink
& Comments

Book Report: Java Concurrency in Practice

I work with the Scala programming language but Scala runs on the JVM, the Java Virtual Machine. This is pretty important. Java turned out to be an icky programming language, but some smart folks have written some darned good APIs for it. Since Scala runs on the JVM, it can use those APIs. So when I want a data structure for my multi-threaded Scala-Finagle web server, I use a Scala ConcurrentMap but that's basically a wrapper around a Java ConcurrentHashMap.

Java Concurrency in Practice is a book by the Java folks who designed ConcurrentHashMap and all those other tasty Java ConcurrentThingies. I didn't finish reading it, though I liked the part that I read. You know how the scary aspect of programming Design Pattern books is when tyro programmers want to use all the design patterns they learned right away? Java Concurrency in Practice made me want to ~~spawn threa~~ start Executors all over the place, when... I'm not really writing that kind of code right now. At work and at home, I work on code where some geniuses have already figured out the multi-threaded web server framework, have already designed the ConcurrentThingies data structures for me to build on top of. All the changes I found myself thinking about were just complecting; I should use the code the geniuses wrote. I put the book down; I know where to find it again if I need it.

Permalink
& Comments

This tech talk about election software http://www.youtube.com/watch?v=RBSiuVGQECs shows politicians can't cooperate. In theory, it's a UX programmer talking about how he and other geeks worked with the Obama campaign to develop software to help folks organize. It's pretty interesting even if you think of getting Obama elected as misdirected effort.

If you're a computer programmer, working with non-programmers is tough. Apparently, folks don't come much more non-programmerish than politicos. Only… it's not that they weren't programmers. They are generally mired in the past.

Early on, our hero jokes about why the programmers used post-its on a wall to schedule their work: it's not that they didn't know more sophisticated tools. But they needed to get politicos involved with prioritizing/scheduling—and those folks don't do planning, issue tracking, none of that, nope. Post-it-based schedule on a wall was longer-term thinking than they were used to. At first it seems like a throwaway anecdote illustrating the difficulty of nerds getting along with non-nerds.

But then you hear about how political campaigns have operated historically. If politicians had ever tried to gather opinions from the electorate by anything more sophisticated than a multiple-choice poll, they would have been overwhelmed with the data. They didn't really have tools to gather it, measure it, make sense of it. It's 2013, the world has figured out something about how to conduct useful surveys, but these folks are still living in the 80s. They rely on hierarchy, rely on news bubbling up through the ranks. If most voters in the nation disagree with you on the economy, you might figure that out… but you might not. Obama's software made more things measurable; not that many more. From an outsider's perspective, it still sounds like they were stumbling around kind of blind. But the campaign was getting a lot more feedback than it was used to. (Well, probably the same amount of feedback; but maybe more signal and less noise in the mix; less distorted by games of telephone.)

Who knows? Maybe in another 20 years, politicians will listen to the people.

Yeah, I know, be careful what you wish for, right?

Permalink
& Comments

App Engine Programmers: Go just got practical (versus just plain fun)

tl;dr someone wrote some code that showed me how to speed up a game, so I'm happy.

If you're a Pythonic App Engine programmer, then you know that AppStats and NDB make your life a lot easier. AppStats watches your running program and gives you pretty whaddyacallem-Gantt-charts-or-something showing where the time goes. Since your program's time probably goes towards inefficient data storing, the next thing you want is NDB, which lets you write your program in a sensible way and automagically stores things efficiently. But golang App Engine doesn't have those things, right? Well, hang on.

Matt Jibson wrote an AppStats for Go. As with the Python AppStats, you can tweak your program a little, get your program to do that one slow thing, then look at a chart showing where the time went.

I just used it to speed up the way that The Octothorpean Order puzzlehunt game website thing checks puzzle answers. In general, the program behind The Octothorpean Order is written more to be readable than fast. It's a side project; if I can't quickly understand the code picking it up after having set it aside for a couple of weeks, then I'm in trouble. And for the most part, that's fine. Not-fast ain't necessarily slow. But the answer-checking function was slow. Thanks to AppStats, I could see which of my readable-instead-of-fast bits actually mattered, and then rewrite that part.

Those slow parts were, of course, in the data-storing things. (Did the team solve the puzzle? Did they unlock new puzzles? If so, let's store the fact that they unlocked those puzzles, one… puzzle… at… a… time (instead of all at once).) I reworked just enough of my code to fix the worst of these. But if I were starting over today, I'd look into the same author's Goon library—which ports NDB's auto-caching features to Go. (It doesn't have Python NDB's nice asynchronous support; if I were a better Go programmer, maybe I'd know whether it was easy to fake this with goroutines or impossible because of the lack of async golang datastore APIs and multi-threaded goroutine... Anyhow.)

When explaining why he wrote these things, Jibson said The two killer features of the python runtime on app engine are NDB and appstats. I have been refusing to make a serious app with the go runtime because of this lack. It sounds kinda breathless, but he's basically right; you need a way to find out where your program's slow or there's a temptation to optimize prematurely until you're wading in code spaghetti all the time; caching-I-don't-have-to-think-about solves many problems. I'm sure glad he ported AppStats.

Permalink
& Comments

I have infinity words for you Developers Developers Developers Developers Developers Developers Developers Developers Developers Developers Developers Developers Developers Developers... ...