Who's online? Mostly spiders. - Scroll Saw Woodworking & Crafts Message Board

kerry_nf · 03-15-2007, 10:41 PM

Hey guys, I was always impressed by the number of guests browsing the forum until I clicked "Quick Links" and "Who's Online"...the majority of the guests are indexing spiders, mostly from Yahoo.

I know spiders need to crawl website content to keep their indexes up to date but their has to be a way for forum software logic to exclude these "guests" from the "Who's Online" list.

...a simple robots.txt file with the appropriate parameters limits their presense.

PuzzledMoose · 03-15-2007, 10:55 PM

LOL - just looked and apart from the Slurp Spiders - what a great name! - there are also now a lot of people reading Quick Links and whose online - now look what you've done, Keri ...

kerry_nf · 03-15-2007, 11:07 PM

LOL! Oh no, I have created a loop...we'll all perish at the mercy of the "slurpies" Aaaarrrgh!!! lol

DukeNukem · 03-16-2007, 02:50 AM

LOL all I can say

.Jerry

kyscroller · 03-16-2007, 02:55 AM

I like the google spider.

JimSawyer · 03-16-2007, 03:50 PM

So what exactly are spiders and what do they do

BobD · 03-16-2007, 03:52 PM

The spiders are sent out by search engines such as Google and Yahoo to check the content of different sites. That way if someone googles "drilling holes for scroll sawing" it will bring up the site.

I'll check with Rob about the tweak you suggested, Kerry, but I know he's swamped right now...it might be a while before he can implement it!

Bob

Mackem · 03-16-2007, 06:31 PM

I like the "bot" spider, it's got my imagination working overtime.lol.

03-15-2007, 10:41 PM	#1
kerry_nf Senior Member Join Date: Nov 2006 Posts: 727	Who's online? Mostly spiders. Hey guys, I was always impressed by the number of guests browsing the forum until I clicked "Quick Links" and "Who's Online"...the majority of the guests are indexing spiders, mostly from Yahoo. I know spiders need to crawl website content to keep their indexes up to date but their has to be a way for forum software logic to exclude these "guests" from the "Who's Online" list. ...a simple robots.txt file with the appropriate parameters limits their presense. __________________ Check me out on the web: http://www.kerrysworld.com

03-15-2007, 10:55 PM	#2
PuzzledMoose Newly Customized Moose Join Date: Oct 2006 Location: Truro, Nova Scotia Posts: 2,603	LOL - just looked and apart from the Slurp Spiders - what a great name! - there are also now a lot of people reading Quick Links and whose online - now look what you've done, Keri ... __________________ Ian Scrolling with a Dewalt 788

03-15-2007, 11:07 PM	#3
kerry_nf Senior Member Join Date: Nov 2006 Posts: 727	LOL! Oh no, I have created a loop...we'll all perish at the mercy of the "slurpies" Aaaarrrgh!!! lol __________________ Check me out on the web: http://www.kerrysworld.com

03-16-2007, 02:50 AM	#4
DukeNukem Senior Member Join Date: Feb 2007 Posts: 1,172	LOL all I can say .Jerry __________________ Don't worry be scrolling

03-16-2007, 03:50 PM	#6
JimSawyer Laying into Inlay Join Date: Feb 2007 Location: Lincoln, RI Posts: 1,191	So what exactly are spiders and what do they do __________________ Jim The limits of the imagination are imaginary. No task is too tedious for Art. Rock and Scroll My Website Featherwood Woodcrafts

03-16-2007, 02:55 AM	#5
kyscroller Senior Member Join Date: Feb 2007 Posts: 125	I like the google spider.

03-16-2007, 03:52 PM	#7
BobD Technical Editor Join Date: Aug 2004 Location: Lebanon, Pa Posts: 3,160 Blog Entries: 2	The spiders are sent out by search engines such as Google and Yahoo to check the content of different sites. That way if someone googles "drilling holes for scroll sawing" it will bring up the site. I'll check with Rob about the tweak you suggested, Kerry, but I know he's swamped right now...it might be a while before he can implement it! Bob __________________ www.WoodCarvingIllustrated.com www.FoxChapelPublishing.com www.ScrollSawer.com Shopsmith, Hawk G4

03-16-2007, 06:31 PM	#8
Mackem Senior Member Join Date: Dec 2006 Location: sunderland. england Posts: 736	I like the "bot" spider, it's got my imagination working overtime.lol. __________________ kevin/pitbull.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
best online store in uk for wood ?	invasion1973	Wood and Materials	2	02-07-2007 07:44 PM
Online ticket Problem.	Rolf	Publisher Feedback	3	09-19-2006 03:43 PM
100% pure Tung oil online	ozarkhillbilly	Wood Finishing and Painting	1	12-02-2005 06:09 PM