: New: Milestone: 20 Million Hits

Wow, it's the site's 20 millionth hit. Looking at the apache log line, I see:

207.46.12.178 - - [25/Dec/2011:22:21:57 -0400] "GET /new/atom.xml HTTP/1.1" 200 15073 "-" "msnbot-UDiscovery/2.0b (+http://search.msn.com/msnbot.htm)"

The user agent string makes me think this might be Microsoft bot... and it was coming from a machine on the search.msn.com, so... yeah. It was looking at atom.xml, a page that tells computers when I've updated my blog. Microsoft checks this special page... uhm, more than 60 times yesterday. If you're wondering why my millionth-update logs are usually by bots, that's because bots tend to visit often compared to humans. Humans have figured out that I update this blog once every few days. But bots are optimistic, they keep checking.

The humans still visit this site about as much as they used top; but the bots don't come so much. If you remember back to the misty past of 2010, there was one of these "million" posts coming along once every few months. I skipped making a 19-million post because that 19-millionth hit came so soon after the 18 millionth hit that I hadn't thought of anything to say in the meantime. But then something changed. Traffic slowed down. And it took many months to go from 19 million to 20 million. But it happened; here we are.

I don't know what robot[s] stopped coming to this site last year, what slowed down the rate of hits so much. I don't keep the old logs, that would feel creepy.

Maybe we can get an idea of the robots that visited the site yesterday by looking at the most "popular" user agent strings that show up in yesterday's logs.

I don't see Yahoo Slurp on this list. Wasn't there once a bot named Yahoo Slurp? I kind of remember that. Did Yahoo slurp stop crawling? Maybe it stopped crawling in mid-2011, and that would explain the robot-traffic slowdown on my site?

Anyhow, if you're a human and you're reading this, you're more precious to me than all the bots combined, sure. (But sometimes the bots are kind of interesting.) Thanks for reading!

Tags: million site

web+comment@lahosken.san-francisco.ca.us
blog comments powered by Disqus

Updates:

Tags