Robots

Robots (crawlers, spiders):



• Software which retrieve web resources for indexing, auditing, validation purposes
• Do robots breach copyright?
• robots protocol can be used to instruct robots not to retrieve resource. However the protocol lacks granularity.

Previous slide

Next slide

Back to first slide

View graphic version