I's have tried using Java based webharvest to crawl wikipedia.org. But it is getting stuck and does not crawl further. How do I get everything on Wikipedia to use for non-profit or education purposes?
You can download availale database Wikipedia dump from:
http://download.wikimedia.org/. Take this dump on your local machine and then use it for whatever educational purposes you want.
Java based crawling tool: Webharvest is amazing. http://web-harvest.sourceforge.net/. What problems did you have?
1 year 51 weeks ago
You can download availale database Wikipedia dump from:
http://download.wikimedia.org/. Take this dump on your local machine and then use it for whatever educational purposes you want.
Java based crawling tool: Webharvest is amazing. http://web-harvest.sourceforge.net/. What problems did you have?