Site Update Notification Project – Web Crawler and Deep Web Research

http://www.siteupdatenotification.com/

The Site Update Notification project aims to deliver a mechanism for web masters to automatically publish information about changes to their website (changes in content and in presentation and layout) directly to those interested in keeping up-to-date with their publications. This could be Search Engines, Page Monitors, Bookmark Managers, Offline Browsers, Website Mirroring, Web Archives or others. This is an alternative to the traditional crawling of sites that download pages and following hyperlinks to related pages (crawling the web) and distributes the burden from a centralised (albeit distributed) web spider and increases efficiency by reducing bandwidth requirements across the internet. The Site Update Notification System seeks to: a) address the problem with crawler scalability; b) reduce the crawler bandwidth and index load; c) reduce the time that a web site’s change takes to be available in a search engine’s index; d) address the problem crawlers currently face in accessing the Deep Web; and e) provide an infrastructure for agent data aggregation and delivery to subscribers that reduces the bandwidth of individual web sites. This has been added to the tools section of Research Resources Subject Tracer™ Inform,ation Blog. This has been added to Deep Web Research Subject Tracer™ Information Blog. This has been added to Bot Research Subject Tracer™ Information Blog.