|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical Re: Spammer web harvesting tool countermeasures
"Deepak Jain" <[email protected]> said the following at 10/30/97 6:56 PM: >And wouldn't we, in turn, see some kind of problems arise with legitimate >search engines because of this? If you downloaded it and looked at it, you would have noticed that it follows search engine guidelines by adding the appropriate <META> tag to the HTML as well as the fact, that you can also use the robots.txt file to block it. Of course this also breaks down if spammer robots actually follow the rules...but how many of those do you think that there are? ;-) -jon Jon (no h) S. Stevens Web Engineer [email protected] Clear Ink and The Internet Weather Report <http://www.clearink.com/> | <http://www.internetweather.com/>
|