Use depth-first and breadth-first search to find dead links
The following script is to catch dead links (return 404 HTTP status code) under a domain name. Depth-first Search (DFS) After getting the root path, first, take out all the sub-paths, and use urllib to verify these paths. If the verification fails, record them, otherwise use the sub-path as the root path to traverse its sub-paths. Breadth-First Search… Read More »