Posted on by Dave

stressed


Happy Friday friends, boring entry for non-techies ahead… I’m afraid. Before you read it, I’d like to say that after writing it – I made some simple changes to my access files, and I’ve been having stellar results thus far. Keep your fingers crossed, maybe I have a working blog again.

If you’re reading this right now, consider yourself one of the lucky few who got through to what’s becoming an ever hard-to-reach sounds familiar. I apologize for the server problems folks, I assure you they are not of my making – in fact I’ve been have trouble finding something to pin the blame on. Right now, I’ve chatted several times with my hosting service’s tech support – and been assured that the server I’m on is functioning just fine – so the problem either lies with my pages themselves, or with the amount of traffic I’m sustaining (and thus asking the server to fulfill). Armed with this, I pored over the server usage logs to see just what might be the culprit.

Suspect #1 is blog comment spam, which has grown to a staggering amount (~3k pieces per day). All this comment spam activity is scripted and targeted at the same piece of PHP code, meaning it’s making numerous simultaneous database requests all day long. Since none of the spam makes it to the main page, I’m happy – but there’s no way to stop the database traffic without turning off commenting altogether, or implementing a login scheme – neither of which are attractive to actual commenters. Suspect #2 is the Coppermine application I use to power my image galleries, as the slowdown coincided somewhat with its introduction. I’ve often wondered if its code is super inefficient or sloppy, but the only way to test is to shut the galleries down for a month and watch the results – undesirable Suspect #3 is my own star-power. That’s right, I may just be too popular for my current hardware. I know, hard to believe – right? Let’s examine the evidence:

Below you’ll see my monthly bandwidth usage statistics from October of last year through February this year. You’ll notice that I was fairly solid around ~1-1.5GB of monthly transfer for an entire year there, up until October 2005 where you’ll start noticing some significant growth through February this year. In February, I hit 4GB transfer on the month – that’s a substantial difference from my ~1GB past year.

Now let’s take a look at March and April of this year. After a fairly smooth rise to the 4GBs of February, something happens and site usage freakin’ takes off over the past two months. Doubling over the course of March to a whopping 8GB and then increasing another gig in April to 9GB.

You think it’s a coincidence that Keaton was born on February 27th? That entry certainly generated some increased interest in the blog, and perhaps even bumped up my regular readership to some degree… But what are people looking at? Digging deeper below, they’re looking primarily at this very page. Viewed ~20,000 times in the month of April. However, some percentage of that, maybe 10%, is me writing and refreshing; and my blog-specific stats program clocked “unique” visitors at ~9,000, meaning half of those 20,000 hits were repeat customers. That’s impressive to me, especially since it doesn’t count web spiders and other forms of bots or indexers (which, incidentally, made up about 12% of April’s 8GB of transfer). The data below also shows that my image gallery is the clear runner-up to sounds familiar, being viewed 9,000+ times on the month.

Yeah, so what?, you may ask. 10GB monthly and 20,000 hits is peanuts, really. Especially when my hosting package allows me 250GB on the month. So why am I so bogged down at less than 10% of my allotment? I still have no idea, and I’m actively pursuing it with my host. As soon as I can find some relief, I’ll make the changes – I’m even willing to spend more money monthly if it comes down to it. As of now, I’ve taken some aggressive steps to block some greedy bots and spiders from aggressively crawling my domain – hopefully it’ll give some relief. Thanks for sticking with me through the crapiness… I appreciate your patience.

Before I close the topic, however, some random stats I enjoyed:

  • 83% of you come to my blog via a bookmark, while 13% arrived via a search engine.
  • Of you 13% who arrived via search engine, most of you came from a Google image search for “spider bites.” The next biggest chunk of you were searching on the term “crumbelievable.”
  • If you were referred to my blog, it was likely from myspace.com. Although I don’t understand this, and can’t find a link from myspace to me, that site accounts for 27% of my non-spam referral traffic.
  • In April, 13% of pharaohweb.com’s 9,828 unique visitors bookmarked a page (that’s 1,324 people).
  • 23 people visited my pages from a WebTV console. They still make those?
  • 80% of you are here and gone in 30sec, only 8% stick around as long as 2min. On the other hand, 212 of you were here for more than an hour. 30sec… what the eff is that?

I had big plans to write something here at the end with a little more meat, but I don’t have it in me. Until Monday, I aced another week. Love you.

Also written on this day...

Leave a Reply

Your email address will not be published. Required fields are marked *