Have a setting during crawl to only load links to html/js/css files rather than a huge exlude list
We don't have a master list of filetypes on a 80,000+ page website, crawls keep failing as they find yet another new filetype. Can we have a setting that prevents the crawl from following links to anything that <b>isn't</b> a .html|.js|.css |.xml|.gz ? A whitelist makes much more sense than blacklist. What is wasp going to do once it's loaded an eps file anyway?
I’m closing this feature request as WASP for Firefox will eventually be sunset in favour of the new and enhanced Chrome version.
That makes sense – keep the exclude concept or maybe change it to "track only links to files with extension .htm, .html, .asp, .aspx, .php, .net, etc.