Internet Bot project is a crawler which can help users to search required information in short time by searching records in database. Main task of crawler is to update data to database based on categories and provide results using data mining search techniques.


Web indexing is a important task for web crawler to update links from world wide web and update to database.  There is different method to update website data for indexing. Google search provides web portal for indexing website links in a fast way. There are few software which can index website links.  There are some issues in crawling procedure; there is no specific permission option to not crawl any website data.  Users data is accessed without any permissions.

Project Introduction:

Search is a common method used by any user to get specific accurate information by crawling entire web.   At present search is used in mobiles also for example apps search from play store and each website is also providing search option to find required information.

Crawler algorithm is developed in java which will update database with latest links from World Wide Web and index them by dividing in to categories. When user search for data based on that result algorithm will search in database and display results.

Software requirements:

Client – side Scripting                 : Javascript

Programming Language                    : Java

Backend Database                        : Oracle/SQL Server/MY SQL/MS Access

Download Internet Bot Project source code and project report.