Build a Web Spider on Linux – A Simple Spider and Scraper Collects Internet Content
Web spiders are software agents that traverse the Internet gathering, filtering, and potentially aggregating information for a user. Using common scripting languages and their collection of Web modules, you can easily develop Web spiders. This article shows you how to build spiders and scrapers for Linux® to crawl a Web site and gather information, stock data, in this case. This has been added to Bot Research Subject Tracer™ Information Blog.