A PRACTICAL APPROACH TO WORKING OF WEB SEARCH ENGINE

The main purpose of this paper is to present the anatomy of a large scale Hypertext Transfer Protocol (HTTP) based Web search engine by using the system architecture of large search engines such as Google, Bing, and Yahoo as a prototype. The paper walks you through the search engine processes from the point of entering the search query into the text box to retrieval of the relevant web pages to the user with a practical approach. To demonstrate whole system of search engine working a Web crawler is developed and implemented in java V 1.7.0_03, MySQL5.1, Apache Tomcat V6 were used as database and server respectively. The paper describes in detail the basic tasks a search engine performs. The process is too complex and many agents like crawler, indexer, and repository are involved in performing their jobs. An overview of each agent is provided in a practical way. [2] We have various types of search engines like Crawler-Based Search Engines, Directories, Hybrid Search Engines, Meta Search Engines, and Specialty Search Engines. The paper covers general purpose Crawler-Based Search Engine only.