Bot and Intelligent Agent Research Resources and Sites

Bot and Intelligent Agent Research Resources and Sites
[Download Bot and Intelligent Agent Research Resources White Paper Link Compilation]

Bot and Intelligent Agent Research Resources and Sites Resources ( is a Subject Tracer™ Information Blog developed and created by the Virtual Private Library™. It is designed to monitor bot and intelligent agent resources on the World Wide Web and listed below. We always welcome suggestions of additional sites and resources to be added to this comprehensive listing and please submit by clicking here. This site has been developed and maintained by Marcus P. Zillman, M.S., A.M.H.A.; Internet expert, author, keynote speaker, and consultant. His latest white papers include Searching the Internet, Academic and Scholar Search Engines and Sources, and Knowledge Discovery Resources 2015. All of his Subject Tracer™ Information Blogs and his white papers are available from  His latest monthly column is available by clicking here. Subscribe to his free monthly Awareness Watch™ Newsletter. Learn more by clicking here.

Bot and Intelligent Agent Research Resources and Sites:

1st Spot

80legs – Powerful and Economical Service Platform for Crawling and Processing Web Content

Agent Construction Tools



Agent Model Yields Leadership

Agent Portal AI


AgentSheets – Authoring Tool to Create Agents


Android World – Speech Interface for Apps and Devices

Applied Soft Computing

Article Search API – New York Times Articles 1981 to Present

artoo.js – The Client-Side Scraping Companion

B.4.1 Search Robots – The Robots.txt File

BotHunter – Passive Network Monitoring Tool

Bots, Blogs and News Aggregators

Build a Web Spider on Linux – A Simple Spider and Scraper Collects Internet Content

Cetus Links – Mobile Agents


Clara – Digital Employee That Schedules Meetings

Common Crawl – Web Crawl Data Accessed and Analyzed By Everyone

Connotate – Intelligent Agent Technology and Competitive Intelligence Tools

cQuery – Content Query Engine

CrawlTrack – Your Web Statistics Tool

Data Mining Resources

Dataminr – Real-time Information Discovery

DataparkSearch Engine – Full-Featured Open Source Web-Based Search Engine

Deep Web Research

Design of a Parallel and Distributed Web Search Engine by Salvatore Orlando, Raffaele Perego, and Fabrizio Silvestri

Dictionary of Algorithms and Data Structures

DiffBot – Get Data From Web Pages Automatically                                                                                                                                                                                            

Eliza – The Original ChatterBot

FAME (Facilitating Agents in Multiculture Exchange) Project

File Information Tool Set (FITS)

Foundation for Intelligent Physical Agents


GeneSys Middleware

Google Guide

IBM Watson Services

iMacros™ – Browser Based Macro Recorder and Intelligent Agent

Imagination Engines

Indexing Robot Crawler Checklist

Information Retrieval Intelligence

Institute for Human and Machine Cognition (IHMC)

Intellexer – Custom Built Search Engines, Knowledge Management Tools, Natural Language Processing

Intelligent Information Systems Research Laboratory

International Journal of Agent-Oriented Software Engineering (IJAOSE)


Kngine – Semantic Search and Answer Engine

Knowledge Discovery

LAIR – Laboratory of Applied Informatics Research

List of User-Agents (Spiders, Robots, Crawler, Browser)

Minimal-Intelligence Agents for Bargaining Behaviors in Market-Based Environments by Dave Cliff and Janet Bruten

MIT Media Lab: Software Agents

Modelling and Mining of Network Information Systems

Mozenda Web Agent Builder – Web Data Extraction



Neurence – Intelligent Information Platform

Open Source Web Information Retrieval (OSWIR05)

Oxyus Search Engine

ResearchKit Framework – Medical Research Apps

Robo Brain – Large-scale Computational System That Learns From Publicly Available Internet Resources

RoboEarth – A Worldwide Web for Robots

Robots.txt Checker

Robots.Txt Checker – Validator for Robots.txt Files

Robots.Txt – Robots Exclusion Standards

Scrapple – A Framework For Creating Web Scrapers and Web Crawlers

Searchbots – Uniquely Searching the Internet

Search Engine Robots

Search Engine Watch News

Search Tools – Information Guides and News

SeerSuite – CiteSeerX Toolkit

Semantic Web


Siri – Your Virtual Personal Assistant

Smarter Bots

SocialBuzzBot – The Business and Social Intelligence Search Engine for Information Discovery                                                                

SocSciBot – Social Sciences Link Analysis Research

Spidering Hacks [book]

Spinn3r: RSS Content, News Feeds, News Content, News Crawler and Web Crawler APIs

STACKS – Social Media Tracker, Analyzer, & Collector Toolkit at Syracuse

Structure and Interpretation of Computer Programs – Video Lectures by Hal Abelson and Gerald Jay Sussman

Supybot, A Superb Python IRC Bot

Swoogle – Semantic Bot

TBot – Windows Live Messenger Translation Bot

TextRunner Search – Searches Hundreds of Millions of Assertions Extracted from 500 Million High-Quality Web Pages

The Intelligent Software Agents Lab

The Lemur Toolkit – Language Modeling and Information Retrieval Research

The Search Engine Project (TSEP)

The Simon Lavern Page

The Web Robots Pages

TSEP – The Search Engine Project

UMBC AgentWeb

UMBC eBiquity

Webbot – the W3C libwww Robot

Web Curator Tool (WCT)

Web Data Extractors – White Paper Link Compilation

Web Intelligence Consortium

Web IR & IE

WolframAlpha Computational Knowledge Engine – Trillions of Pieces of Curated Data and Millions of Lines of Algorithms

Zakta – Personal and Social Deep Web Search Engine

©2015 Marcus P. Zillman, M.S., A.M.H.A.

Sign up for Awareness Watch Newsletter

* = required field

Browse Categories