Keith Devens .com - Weblog: Inktomi is the worst spider ever

Tuesday, September 16, 2003

Inktomi is the worst spider ever

Inktomi is the worst spider ever. Not only is it still looking for pages I removed from my site years ago (and redirected to new locations years ago), but it continues to look for pages that have never existed on my site. Argh!

Permalink | 1 Comment | id 4247

10:15 (utc-4)

Comments

Yeah Inktomi is the same for me. It seems to have a problem following links, especially ones beginning with a slash '/' because I'm getting strange 404 errors for pages that have never existed too.

∴ Jason Davies | 19-Sep-2003 7:10am est | http://www.netspade.com/ | #2939

Feel free to post a comment below. Please see my comment policy.

Formatting Rules (No HTML):

**bold**, *italic*, _underlined_, --strikeout--
"text"="url" creates a link, and URLs are auto-highlighted
Blockquote: Like e-mail, begin paragraph with > (greater-than sign)
Lists: begin paragraph with *,-, or + (unordered), or # (ordered)
Code block: ?!code:language=perl|php|sql|javascript|etc.{\n}...{\n}?!/code

RSS feed RSS feed for Keith's Weblog
Atom feed Atom feed for Keith's Weblog
Weblog archive
Recent comments
on 6 posts