A Survey on Distributed Web Crawler Assignment Techniques
A Survey on Distributed Web Crawler Assignment Techniques
Masoud Vahidi1 Amir Minbashi2 Mehrdad Masoumi3 Sepideh Kolahdooz4
1) University of Tehran, Kish campus, Iran:
2) Islamic Azad University, Qazvin Branch, Iran
3) Islamic Azad University, Qazvin Branch, Iran :
4) University of Tehran, Kish campus, Iran:
IOI Code
:
XYZG-ZAAFE
Publication :
2nd International Conference on Science and Engineering(2icesconf.com)
Abstract :
In this paper, we study the techniques of assigning objectives to distributed web crawlers and evaluate their strengths and weaknesses. Considering the fact that the Internet is growing, it is better to use parallel methods in crawling the web. In this paper, we study methods of assigning to distributed crawlers, after which we choose one of them as future research works
Keywords : social networks social education semantic web e learning semantic web in social education