Here’s an update for those of you who may have read my post a couple months back about my bandwidth dilemma. Basically I’ve been getting hit with a healthy dose of traffic to my site, both good and bad. Enough so that I have to keep increasing my bandwidth costs to my host.
Well, the good news is I think I’ve made some headway as to controlling the amount of bandwidth that’s been sucked up by “bad traffic” — basically I’d consider that anybody or anything that sucks bandwidth out of your website for no good reason or benefit to you.
I found the most offensive search spider was this one called Sogou out of China. It looks like it’s trying to be a Google clone. When I checked over my logs…I found that day after day it kept returning to my site and grabbing images over and over again. It’s ok for a search engine to come index my site every now and again, but to come suck it all out on an ongoing daily basis is obscene. There’s no reason why I only have less than 500MB of content on my site but it sucks out 6GB of traffic. BASTARD!
Anyway, I’ve been blocking the IPs going to Sogou and it seems to have reduced that “spider” line from close to 9GB down to 3GB without affecting my real legitimate traffic.
Those of you with heavy image websites (e.g., artists, photographers, designers, etc.) I’d recommend taking a regular check on your logs and see if you’re getting nailed by this irritating bot.
LoweStudio Inc was founded by me, Raphael Lowe, a Honolulu based illustrator and web designer. I presently design and manage a few dozen commercial websites, but my main focus these days is acting as the head developer for Kapiolani Community College and teaching