{"id":1871,"date":"2009-10-19T11:17:38","date_gmt":"2009-10-19T21:17:38","guid":{"rendered":"http:\/\/raphaellowe.com\/portal\/?p=1871"},"modified":"2009-10-19T11:17:38","modified_gmt":"2009-10-19T21:17:38","slug":"search-engine-shenanigans","status":"publish","type":"post","link":"https:\/\/raphaellowe.com\/portal\/search-engine-shenanigans\/","title":{"rendered":"Search engine shenanigans"},"content":{"rendered":"

\"robot_stat2\"<\/p>\n

Here’s an update for those of you who may have read my post a couple months back about my bandwidth dilemma<\/a>. Basically I’ve been getting hit with a healthy dose of traffic to my site, both good and bad. Enough so that I have to keep increasing my bandwidth costs to my host.<\/p>\n

Well, the good news is I think I’ve made some headway as to controlling the amount of bandwidth that’s been sucked up by “bad traffic” — basically I’d consider that anybody or anything that sucks bandwidth out of your website for no good reason or benefit to you.<\/p>\n

I found the most offensive search spider was this one called Sogou<\/a> out of China. It looks like it’s trying to be a Google clone. When I checked over my logs…I found that day after day it kept returning to my site and grabbing images over and over again. It’s ok for a search engine to come index my site every now and again, but to come suck it all out on an ongoing daily basis is obscene. There’s no reason why I only have less than 500MB of content on my site but it sucks out 6GB of traffic.\u00a0 BASTARD!<\/p>\n

Anyway, I’ve been blocking the IPs going to Sogou and it seems to have reduced that “spider” line from close to 9GB down to 3GB without affecting my real legitimate traffic.<\/p>\n

Those of you with heavy image websites (e.g., artists, photographers, designers, etc.) I’d recommend taking a regular check on your logs and see if you’re getting nailed by this irritating bot.<\/p>\n","protected":false},"excerpt":{"rendered":"

Here’s an update for those of you who may have read my post a couple months back about my bandwidth dilemma. Basically I’ve been getting hit with a healthy dose of traffic to my site, both good and bad. Enough so that I have to keep increasing my bandwidth costs to my host. Well, the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,57],"tags":[251,252,250],"class_list":["post-1871","post","type-post","status-publish","format-standard","hentry","category-business","category-rants","tag-bot","tag-search-engines","tag-sogou"],"_links":{"self":[{"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/posts\/1871"}],"collection":[{"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/comments?post=1871"}],"version-history":[{"count":1,"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/posts\/1871\/revisions"}],"predecessor-version":[{"id":1872,"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/posts\/1871\/revisions\/1872"}],"wp:attachment":[{"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/media?parent=1871"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/categories?post=1871"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/raphaellowe.com\/portal\/wp-json\/wp\/v2\/tags?post=1871"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}