SadHOO – Deadlinks update
- This topic has 9 replies, 3 voices, and was last updated 22 years, 9 months ago by
.
-
Posts
-
We have finally managed to check all the links in SadHOO and get rid of the deadlinks! (All 1100) of them.
We now have (thanks to NewKate) a program that checks all the links periodically and deleted those that are dead or too slow to load.
SadHOO is now the most up-to-date and accurate independant sci fi directory in the world!
Cool huh? [img]images/smiles/icon_biggrin.gif[/img]
SADZilla is online!
Ohmygod competition!
[img]images/smiles/icon_smile.gif[/img]
(Robozilla is the robot who trawls the DMOZ Open Directory looking for 404s and removing themn pending checking by a human editor – [url=http://dmoz.org/profiles/robozilla.html]http://dmoz.org/profiles/robozilla.html[/url] )
Ah haa!
Actually, NewKate wrote a teriffic program to check the links, I wonder what the ither search engines have.
It’s a real problem. When you get over a few hundred links you can spend hours each weak finding and deleting dead ones.
The Links 2 system I was using was great at organising and searching the links, bit the cgi scrip for checking them fell over after only about 300 urls!
The new directory that she’s working on will be much better and faster and more interactive.
Gossamer Threads write some nice scripts.
So that’s where my link went.
The link to my skins site is now missing: [url=http://www.geocities.com/thesinistermrx/index.html]http://www.geocities.com/thesinistermrx/index.html[/url]
Whereas the Texx Lexx link is still up. It still works but it only refers people to my new site: [url=http://xevofr2d2.tripod.com/]http://xevofr2d2.tripod.com/[/url]
quote:
Originally posted by Xev of R2D2:
So that’s where my link went.The link to my skins site is now missing: [url=http://www.geocities.com/thesinistermrx/index.html]http://www.geocities.com/thesinistermrx/index.html[/url]
Whereas the Texx Lexx link is still up. It still works but it only refers people to my new site: [url=http://xevofr2d2.tripod.com/]http://xevofr2d2.tripod.com/[/url]
If a webpage returns a 404 (missing page) error for the url that is entered in the database, it will automatically wipe the site from the index.
Normally this would be ok. But if the site goes down while the check is taking place (even for only a few minutes) it will be still be wiped.
The only solution I’m afraid is to re-enter it into SadHOO.
The other link didn’t return a 404 error. Ie. the page was still there, it just re-directed the user to the new url. We considered including redirects in the deletion process, but thought that would be unfair.
Please don’t feel singled out Xev of R2D2, it’s happened to a few peeps. But now that the database is ‘clean’ such upsets shouldn’t happen too often. Please re-submit.
Actually I blame Yahoo. When a site has too much bandwidth being used from people downloading stuff, they will often take it down temporarily. In fact I recieved a notice like that this past week. At least Tripod doesn’t do that.
Maybe I’ll submit the new one as well.
ThanX.
The other option is to move deadlinks to a holding area to be reasessed after some time has passed or the be looked over by a human (on DMOZ submissions are all checked before being entered, and 404s found by robozilla are put back into the unreviewed area for checking)
- You must be logged in to reply to this topic.