open source webcrawler in .NET



Does any one have any suggestion on any open source webcrawler .NET. I want to crawl all web pages for a web site. For all the crawled web pages, I want to store them into local file directory.



Use this open source web crawler: Arachnode.net. This should work.