Implementing Mobile Crawler Using JINI Technology to Reduce Network Traffic and Bandwidth Consumption

To search any information on the web users extensively use the search engines. As the growth of the World Wide Web exceeded all expectations, the search engines rely on web crawlers to maintain the index of billions of pages for e cient searching. The web crawlers have to interact with millions of hosts and retrieve the pages continuously to keep the index up-to-date. According to literature survey, most of the network traffic and bandwidth is consumed by web crawlers so instead of using them, we are proposing mobile crawler developed using JINI technology with the help of remote page selection, filtration and compression at web servers and not search engine.