I’ve been working on a tool to scrape urls. The idea is to have a tool that checks my sites for invalid code, urls and images. Below some tools and ideas that I might consider using Wget can be used to spider a website (list url’s, not download) with the following command: wget -r –spider… Read More