Open Access Journal

ISSN : 2394-2320 (Online)

International Journal of Engineering Research in Computer Science and Engineering (IJERCSE)

Monthly Journal for Computer Science and Engineering

Open Access Journal

International Journal of Engineering Research in Computer Science and Engineering (IJERCSE)

Monthly Journal for Computer Science and Engineering

ISSN : 2394-2320 (Online)

Web Crawler: A Crawler for Efficiently Retrieving Relevant Data

Author : Prabhu Alamkare 1 ShrikantGiri 2 Shubham Bardiya 3 PradeepGite 4

Date of Publication :7th April 2016

Abstract: The World Wide Web is a rapidly growing and changing information source. Due to the dynamic nature of the Web, it becomes harder to find relevant and recent information. WebCrawler are one of the most crucial part of the search engines to collect pages from the Web. WebCrawler is to download most relevant web pages from such a large web is still a major challenge in the field of Information Retrieval Systems. WebCrawler uses two-stage framework. In the first stage, WebCrawler performs site-based searching for visiting a large number of pages. In the second stage, WebCrawler achieves fast in-site searching by extracting most relevant links with an adaptive link-ranking. To achieve more accurate results WebCrawler ranks websites to prioritize highly relevant ones.

Reference :

    1. WenwenLia, ChaoweiYanga and ChongjunYangb,"An active crawler for discovering geospatial Web services and their distribution pattern –Vol. 24, No. 8, August 2010.
    2. MahmudurRahman,"Search Engines going beyond Keyword Search",School of Computing and Information Sciences Florida International University, Miami, FL 33199,Volume 75 - No. 17, August 2013.
    3. Trupti V. Udapure, Ravindra D. Kale, Rajesh C. Dharmik,"Study of Web Crawler and its Different Types” ISSN: 2278-8727Volume 16, Issue 1, Ver. VI (Feb. 2014)
    4. A.B. Gil, S. Rodríguez, F. de la Prieta and De Paz J.F,"Personalization on E-Content Retrieval Based on Semantic Web Services
    5. Pavalam S M, S V Kashmir Raja, Felix K Akorli3 and Jawahar M,"A Survey of Web Crawler Algorithms" National University of Rwanda Huye, RWANDA,Vol. 8, Issue 6, No 1, November 2011.
    6. Ms. Pallavi Wadibhasme1, Prof. NitinShivale ,"Survey on – Self Adaptive Focused Crawler,Issue 6, No 1, November 2013.
    7. Paolo Boldi_ Bruno Codenotti† Massimo Santini‡ SebastianoVigna“ A Scalable Fully Distributed Web Crawler”
    8. Tiffany Ya TANG and Gordon MCCALL Smart Recommendation for an Evolving E-Learning System
    9. Feng Zhao, Jingyu Zhou, Chang Nie, Heqing SmartCrawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces. IEEE Transactions on Services Computing Volume: PP Year: 2015
    10. RajashreeShettar, Dr.Shobha G. “Web Crawler On Client Machine” Proceedings of the International MultiConference of Engineers and Computer Scientists 2008 Vol II IMECS 2008, 19-21 March, 2008.

Recent Article