Date of Publication :22nd August 2017

Abstract: Nowadays the Internet is part of life because of any information is easily available on the Internet. It has a large size of information; hence the high efficiency and get relevant information are challenging issue due to the changing nature of the deep web. As crawler plays important role in such cases. So we proposed such crawler which provides efficient and extracts relevant information from web. The smart crawler contains two-phases as site locating and in-site exploring. We developed smart crawler using K-means clustering methods. Clustering makes a group of similar data items known as clusters. Here we describe K-means clustering techniques. The most famous clustering method is K-means methods which divide data items in K clusters and provide better result with high efficiency. Also we compare the result of existing system and smart crawler using Kmeans provide an efficient harvesting rate of deep websites within the least amount of time.

Reference :