Non-programmers might not realize it, but some computer program source code is even harder to read than the rest. Some of this code is so messy that an experienced programmer looks at it and says "I have no idea what is going on here. Maybe I could figure it out, but... what's on TV?" This book talks about some general principles of writing readable code; and there are examples to illustrate good and bad code.

This book is from the 1970s. The examples are in the FORTRAN and PL/I programming languages. They are in an old FORTRAN--I think FORTRAN has changed a bit since then. I think this book uses an old dialect. I'm not really sure, though. I don't know FORTRAN nor do I know PL/I. Actually, that was a problem with this book. The book had "before" and "after" examples to show how to "clean up" code to make it more readable. I couldn't always understand the "after" examples. Well, I could, but only after the head-scratching I associate with my attempts to read poorly-written code. For example, DO 2 I=1,N After looking at some other examples, I think that means "Loop N times over the block of code that starts here and ends with the line of code labeled "2".

As it was... I think this book might be of more interest to the historian than to the programmer.

It's a bunch of essays by Paul Graham about software development and other kinds of development. Some of these essays are interesting, some are irritating. They're interesting because Paul has a well-spoken, cranky take on many topics. They're irritating because he attributes his opinions to all computers geeks. "We hackers know this." "We hackers think that." There were plenty of times when I thought "I agree with this sentiment", but there were also plenty of times when I thought "What do you mean 'we'?"

I don't believe in absolute Quality with a capital Q. I don't believe it's the genius of creative people is to discern that Quality. If I wanted to read about Quality, I could go read Zen and the Art of Motorcycle maintenance. In that book, the Quality-delusion is attributed to an insane person, not to me. Thus, Zen and the Art of Motorcycle Maintenance is less irritating than Hackers and Painters. I'm not a Libertarian. I'm not. I'm glad that other languages have stolen cool features from LISP, but I don't especially want to code in LISP ever again. There are plenty of places where foreach makes more sense than recursion does; I'm glad your language is optimized to not blow stack on tail recursion, but if your language supported a foreach loop, you wouldn't have to worry about that @^*% in the first place.

Now that I go back and flip through the book, looking for something nice to say about it, I keep thinking "Oh, this was just a rant" and "Someone else said it better". So I'm not sure what good things I can say about the book--which is strange because I did enjoy reading it. I guess I enjoyed it because it touches on topics that interest me.

Yes, I should be writing about BATH4 and MSPH1[23]. But that would require effort. But link posts are easy. So I'll link to a video of Douglas Crockford talking about the good parts of Javascript.

I've been programming in Javascript lately. I approached the language with some trepidation. I'd seen plenty of bad Javascript. But I'd also heard from co-workers that the language itself was pretty good. Maybe I'm late to the party--plenty of folks program Javascript nowadays. But I think there are still some who dodge the language, having done a scream-and-run based on past follies. If you tried JS a while back and hated it, you might want to watch Crockford's talk--he talks about how to use JS's good parts while steering clear of the warts.

(Am I the only one who checked the coedastronomy site in case they meant March 3 Greenwich time?)

I can post an admission that I'm half-done with a handful of projects, but I don't have to like it. I finish projects! Or I give them up! (Right now, I am making a "chop" hand gesture to emphasize my willingness to give up on stalled projects.) So after posting that blog item, I forced myself to get my act together. (Right now I am gritting and baring my teeth to illustrate my renewed strength of purpose.)

Thus: a page of photos of San Francisco's 17th Street. A couple of weeks back, when it stopped raining, I walked the length of 17th Street. I snapped a bunch of photos. And then for two weeks, I didn't get around to captioning/uploading them. Instead, I just groused about not enjoying being in the middle of projects. Today, I finally finished slapping some captions on. Allez-oupload! I will now stop worrying about those photos.

Last night, I wasn't doing photos. Last night, I finished off my Erlang experiment. I was trying to learn about Erlang concurrency. And sure enough, concurrency is indeed easy with Erlang. Here's my program's, uhm, central dispatch control loop thingy:

queue(Migrant) ->
    receive
 TradedList ->
     RanList = cull_unhealthy(TradedList),
     {NewMigrant, NewList} = judge(Migrant, RanList),
     spawn(critter, run_n_trades, [self(), 
       clear_state(NewList), 500]),
     queue(NewMigrant)
    end.

That "spawn" spawns a new thread, a thread that executes a function called run_n_trades. That "receive" receives a message.

run_n_trades(Queue_PID, List, 0) ->
    Queue_PID ! List;
run_n_trades(Queue_PID, List, N) ->
    run_n_trades(Queue_PID, run_trades(List), N-1).

It's not obvious from this code, but run_n_trades does a lot of data crunching and then sends the results back to the, uhm, central dispatch control loop thingy. (It's that mysterious Queue_PID ! List blob.) That's sending back the data structure that the queue will receive. How does this message-passing benefit me? Well, I actually had two threads doing big data crunching at the same time. Each one would crunch, crunch, crunch, then send results back to the queue. The queue could combine their answers. (In this case, the "combine" was allowing one of the genetic-algorithm "critters" to migrate from one batch of critters to... whichever batch was next passed back to the queue. (But this parenthetical remark probably doesn't make much sense unless you're looking at the whole program, which isn't really interesting enough to be worth it.))

What did I learn from all this?

• Erlang is not so bad. I am not fond of languages designed by people in love with recursion. Among those, Erlang is not so frustrating as many.
• Erlang concurrency is indeed easy. If I were writing a program whose main challenge was coordinating many threads, watching a little data on each thread, I'd be glad to have Erlang in my bag of tricks. Because those problems can be really hard, and Erlang has nice language structures for these.
• Erlang was not a great choice for my sample program's purpose: yet another genetic-algorithm prisoner's dilemma fun-fest. Part of Erlang's safety comes from discouraging you from changing the value of variables. Instead, you're supposed to create new variables whose values don't change. (Is "variables" even the right word?) So if you're moving around little structs, there's some extra copies but you don't sweat it much. But if you make many tiny changes to a big array of structs... Erlang isn't a great choice.
• But it does suggest some ways to make thread-safe programming safer in, you know, real programming languages like C++. Maybe you make a rule saying that cross-thread messages pass in copies, not originals. That's one extra copy, but one extra copy maybe isn't so bad. And at least everyone knows which thread is responsible for which data structure.

Am I rambling? Sorry, I'm rambling. I'm just so happy that I have an excuse to stop thinking about Erlang now that I did what I set out to do. And I'm glad I finally uploaded those photos. So... I'm rambling. You shouldn't have to listen to me ramble. Here, go look at photos instead.

(Visiting the doctor is good for you. Today, I visited a cardiologist to make sure that my recent hospital visit was Really No Big Deal. Thus, I missed the last bus to work and worked from home today. Thus, I'm not trapped in this evening's huge peninsula traffic jam. Three cheers for modern medicine. What? Oh, right, some more chapters from Beautiful Code.)

A Spoonful of Sewage / Bryan Cantrill

Thread scheduling algorithms. When I hear that I'm going to have to think about a thread scheduling algorithm--especially a thread scheduling algorithm that isn't quite working--I gulp nervously and tug at my collar. This essay about thread scheduling algorithms; it desribes a bumpy ride. I learned from it.

Oh, I keep looking for things to quote from this essay--but things from the middle don't make much sense unless you've read the beginning. There's some talk about debugging multi-threaded problems... Oh, just go read it. Oh, here's a bit that makes some sense:

The essence of the problem is this: for user-level locks, we normally keep track of the state associated with the lock (e.g., whether or not there's a waiter) at user level--that information is considered purely advisory by the kernel. (There are several situations in which the waiters can't be trusted, and the kernel knows not to trust it in those situations.)

To implement priority inheritance for user-level locks, however, one must become much more precise about ownership; the ownership must be tracked the same way we track ownership for kernel-level synchronization primitives. That is, when we're doing the complicated thread lock dance in turnstile_interlock(), we can't be doing loads from user-level memory to determine ownership. The nasty implication of this is that the kernel-level state tracking the ownership of the user-level lock must itself be protected by a lock, and that (in-kernel) lock must itself implement priority inheritance to avoid a potential inversion.

It also has some implicit good advice for code reviewers towards the end.

Distributed Programming with MapReduce / Jeff Dean & Sanjay Ghemawat

I skipped this chapter. Hey, give me a break. I hear about this stuff all the frickin' time.

Beautiful Concurrency / Simon Peyton-Jones

People who work with Haskell keep telling me how self-documenting the code is. They never convince me--probably because they make the mistake of showing me some Haskell code. This essay didn't convince me, either. Maybe because it showed me some Haskell code and then rushed along, thinking I'd understand it.

nTimes :: Int -> IO () -> IO ()
nTimes 0 do_this = return ()
nTimes n do_this = do { do_this; n_times (n-1) do_this }

Erlang folks also pull this kind of no-for-loops nonsense, but they don't keep telling me I should be able to read it by instinct. What does a loop have to do with IO? Does IO mean what it usually means? Do I care anymore?

Jones just wrote this essay. It's not his fault that other Haskell fans have soured me on the language. Maybe I should give this essay another chance. Nnnergh. I just don't have the willpower, not today.

Syntactic Abstraction / R. Kent Dybvig

I can't blame this one on the language. I thought I understood Scheme before I read this essay. But it turns out I don't know as much as I thought I did. Or I know the wrong dialect. Or something. Yoy. And this chapter was giving me freshman software engineering class Metacircular Evaluator flashbacks. I gave up on this chapter partway through.

This guy Hans Boehm came and gave a talk at work today about upcoming C++ support for threads. That's support built into the language. It sounds like sometime in the next few years, we will have atomic<int> . That is to say that C++ will support concurrency, you'll be able to create objects that only one process/processor/whatever can mess with at a time. Up until now, it's been fun to mock people who have opinions about concurrency in programming languages, "Enjoy the Erlang!" and all that, but soon there will be no escape.

Actually, I was thinking about concurrency earlier, when I was reading this paper that's been going around, Some thoughts on security after ten years of qmail 1.0. Some things he doesn't say so well. But there is a nice list of things that a root program can do to run another program in a sandbox:

The jpegtopnm program reads a JPEG file, a compressed image, as input. It uncompresses the image, produces a bitmap as output, and exits. Right now this program is trusted: its bugs can compromise security. Let’s see how we can fix that.

Imagine running the jpegtopnm program in an “extreme sandbox” that doesn’t let the program do anything other than read the JPEG file from standard input, write the bitmap to standard output, and allocate a limited amount of memory. Existing UNIX tools make this sandbox tolerably easy for root to create:

• Prohibit new files, new sockets, etc., by setting the current and maximum RLIMIT_NOFILE limits to 0.
• Prohibit filesystem access: chdir and chroot to an empty directory.
• Choose a uid dedicated to this process ID. This can be as simple as adding the process ID to a base uid, as long as other system-administration tools stay away from the same uid range.
• Ensure that nothing is running under the uid: fork a child to run setuid(targetuid), kill(-1,SIGKILL), and _exit(0), and then check that the child exited normally.
• Prohibit kill(), ptrace(), etc., by setting gid and uid to the target uid.
• Prohibit fork(), by setting the current and maximum RLIMIT_NPROC limits to 0.
• Set the desired limits on memory allocation and other resource allocation.
• Run the rest of the program.

At this point, unless there are severe operating-system bugs, the program has no communication channels other than its initial file descriptors.

Up until now, the phrase "chroot jail" was one of those things that I read with only a vague sense of understanding. And folks kept saying "It's not enough to set up the chroot, there's more to it", but they never seemed to list the other things to do. But now that I have a list of a few things, I can probably search the web for pages and code that mentions these things and get a nice survey.

But something I hadn't caught onto before--this chroot stuff is all about spawning programs. I guess you end up with multiple programs all running at the same time. It's a concurrent programming model, I guess. Actually, now that I look at some code samples that deal with chroot this and RLIMIT that, this stuff doesn't look so easy.

Rob Pike gave a talk about Newsqueak a little while back. Newsqueak is a language that makes it pretty easy to spawn off little programlets--there are these objects that are kinda like function pointers. And you can do this thing where you kind of set up a thingy that invokes one of these functions and blocks/waits for its return value. And I thought it would be nice if I could do something like that, but maybe first set some flag on that function-pointer-like-thingy that means "run this function in jail".

Newsqueak-like message-passing concurrency programming... it lives on in other languages these days--like Erlang. Erlang? Erlang. Jeez, maybe I should look at Erlang, stop making fun of it, see what I can learn from it. Learn from Erlang. So this is how low I've sunk.

Next thing you know, I'll be associating with LARPers. Asking Furries to share their wisdom.

Oh, now I can't stop shuddering.

Maybe I'll put off studying Erlang until C++'s atomic<int> comes along.

Still pretty busy with game stuff--applications, puzzle ideas. It's been far too hectic for the last few days. Sometime yesterday afternoon, things turned a corner and I started to dig out from under. This evening I even started to catch up on personal mail. 180 pieces of personal mail. OK, I'm not caught up yet. But things feel soooo less hectic now. As part of the game application, I helped make a video. The guy with a video camera lives on Valencia, so Dwight and I headed over to Valencia to make this movie. Which is kind of a lame seque to say that I'm about to paste in this previously-composed book report about the novel Valencia. So sue me. I'm going to bed. I'm sleepy. It's been a heck of a week.

This novel is by Michelle Tea, but I've never heard her speak, so instead I'm going to start out by talking about Gerald Sussman, who I did hear speak a few weeks back. He's one of the authors of SICP, the Structure and Interpretation of Computer Programs. If you are a computer science geek, you may know this as the MIT intro-to-computer-science textbook. We used it at Cal, too. I remember that it was full of strange things to think about: the metacircular evaluator springs to mind. An interesting topic, but one that makes a naive frosh think, "What is the point of this?" Years later, I'd have some more perspective. Ah, he wanted to expose us to this concept and that concept, thinking that they'd come in handy later. Too bad that, at the time, I just thought that the textbook authors were crazy.

Sussman came to talk at work. He was worried about the state of computer progamming today. People creating code libraries, people designing APIs--they were all doing it wrong. There were these complicated interfaces for getting these gobbets of code to interact with each other. He sought inspiration from biological systems. In DNA, there's a reason that arms are like legs, but different. There is some DNA that controls how all limbs grow. But there is some DNA that is "triggered" based on where a body feature, uhm, sprouts from. Thus our legs are a lot like our arms, but are set up to be thicker, capable of supporting our weight all the time.

Modern software libraries aren't set up to fit together so easily. Sussman wanted components that could be combined more easily. To illustrate this, he put a diagram up on the board. It would be good if there was some way that in one stage of computing, a component could look at many thingies, figuring out something about each of these thingies; and then it would be good if there was some way to combine all of these results. So if you wanted to compute the sum of squares of many numbers, then you could say "well, for each number, the many-thingy operation is: compute the square; then when that's done, to combine the answer, we want to add those numbers together". To compute the average height of NBA athletes, you would say "for each athlete, the many-thingy operation is: extract the height from the data that we have about each athlete; then when that's done, to compute the answer, we want to add up all of those numbers and then divide by the number of athletes". This framework should exist; people writing software libraries should design those librarys' interface functions to "snap" into such a framework.

He had a few of these frameworks in mind, not just the many-operations-combined, but that's the one he showed. If he wanted to show us an idea that hadn't taken hold out in the world, that was the wrong framework to show. It was MapReduce. Folks at work use it all the time. I used it a couple of weeks ago for a project where I didn't have to--I could have used a loop. A year ago, I would have used a loop. But no, I used MapReduce. It was just the easiest way to solve the problem. I am not the sharpest programmer at work, not by a long shot. I have spent little time in the ivory tower of academia. But I'd been exposed to this idea for a while, and it had sunk in. As more coders get exposed to this stuff, they'll use it. I'd heard that the ideas for MapReduce came from the dread buried knowledge of the Lisp hackers, and of course Sussman is one of those. Anyhow, I can't claim that all of computer science has advanced to the point that Sussman dreams of, but... Perhaps we are closer than he thought.

After the talk, he took questions. I think that MIT's intro-to-CS course changed recently. Sussman's book will, presumably, be replaced by something else. I get the impression that various afficianados of novelty and fans of stability had been nattering at each other about whether or not this change was good. Maybe that's why someone asked about the future of education. I forget what the question was. But Sussman decided to talk a bit about the future of education. He pointed out that right now kids go to university because it's the ticket to a job. But jobs keep going away, and will do so more rapidly. As machines get better at tasks, it's not so important that humans do things. As the necessities of life become cheaper, we will have more idlers. If these people don't need jobs, how can we motivate them to educate themselves?

I thought about Michelle Tea's novel Valencia. This book is full of alternative youth falling in love, altering mental states, falling out of love, wandering city streets, wandering the nation. Occasionally someone holds down a job for a while, but most people seem to drift along without such. It would not be correct to call this book a lesbian version of a collection of Aaron Cometbus stories; but you suspect that these writers' characters would understand each other pretty well. These people drop out; when they have jobs, those jobs do not exercise their educations. That tattooed bartender's degree in comparative literature has not helped her career.

Sussman thought of this as an edutainment problem: how do you fool young people into learning if their education doesn't help them? I'm thinking, "Maybe that world is closer than you think." I don't have a great solution to this problem. I figure: if education becomes irrelevant, maybe we shouldn't continue shoving it down people's throats.

This weekend has been hectic. I attended a Game Control summit. I haven't listened to the audio I recorded. It could be interesting; it could be white noise. I stopped by the start of the Chinese New Years Treasure Hunt; post-hunt reports say that this one was less suck-tastic than the previous one I visited. Today, I worked on making a video with an unlikely number of Freunds. I guess I'm not going to find the time to write about any of this in the next few days. And my Chicago photos are lurking in a corner somewhere. And.. and... and after this weekend, I think I need a second weekend to recover. Meanwhile, please enjoy this pre-recorded book report about The Best Software Writing.

I finally got around to reading this collection of belles e-lettres. I felt a little silly paying for a book that basically printed articles which were available for free online. Then again, I was thankful to the editor for digging through lots of crappy writing about software development in order to unearth the good ones. So I bought a copy, and hopefully a nickel of royalties and karma will trickle over in the right direction.

Oh, the book. The book is about programming and projects and working together. That "editor" link up there links to the introduction. It also has a list of the articles if you want to seek them out and read them online.

A little script runs over this website's visit logs each night, generating a pretty report. I think I wrote the original back in 1999. I rewrote it last night. Python instead of Perl now. Sorted and clustered logs of errors. Some errors indicate that I've broken the web site; I want to fix those. Some errors indicate that someone on MySpace decided to use one of my photos as their background image without my permission; those errors are fine. The want-to-fix errors were getting lost in a sea of MySpace crud. So I re-wrote the script, tinkering, getting things the way I wanted them.

The pay-off so far? I finally noticed that I failed to upload a photo that someone sent me. Yes, the page that failed to display that graphic--I added that page back in October. Sorry about the broken image. Anyhow, it's there now.

This is a short book report. As if that wasn't bad enough, it's about computer programming. So maybe I should start out by relaying the story one of my relatives told tonight at dinner. It's a story about back when he worked at the Pentagon. He was in the 902nd military intelligence group. Every day, he'd receive a stack of reports from the FBI. He had a list of army people in the Pentagon, each of which was interested in certain things. So he'd spend a while each day reading over each of these FBI reports and figuring out which people would be interested in which reports. Then he'd make copies of these reports and distribute them. It was a tricky job, requiring attention to detail--except that he suspected that no-one really read these reports. One day, he nerved himself up and instead of carefully reading the reports, he just redistributed them randomly. It took less time, and he had more time to just walk around and goof off. You might think this was a foolish move--but he was young and foolish.

There's no denouement to this story. It turns out that nobody was reading those reports. He never bothered to read the reports again; he kept on distributing them randomly. Eventually, he was rewarded for his efforts with a different task.

In this spirit of not-reading things and then encouraging other people to read them, I recommend Programming Ruby. I didn't finish this book, but I recommend it. It's about a programming language. By reading the first part of the book, I was able to decide whether I wanted to use Ruby for an upcoming project or stick with Python.

This book talks a lot about anonymous program blocks, but doesn't mention lambda functions. And after a while, you wonder why there are no lambda functions, and you think that all Ruby programmers are nuts. But then the book finally mentions lambda functions after all. Ruby does have them. Ruby programmers aren't nuts.

I could write more about this book, but I think I need to go lie down and sleep off the effects of this dinner. Happy holidays, everybody.