Cloudfeaster

Added

* add comprehensive artifact storage for spiders run by the CircleCI
workflow generated by generate-circleci-config.py

Changed

* remove debugging statement from run-all-spiders.sh

Removed

* Nothing

Added

* Nothing

Changed

* fix: generate-circleci-config.py had oustanding problems from Python 2.7 -> 3.7
* usability: improve usability of run-all-spiders.sh and run-spider.sh in spider repos
* docs: fix docker image badge in main README.md

Removed

* Nothing

Added

* add --verbose command line argument to docker_image_integration_tests.sh
* add --verbose and --debug command line options to run-sample.sh
* add CrawlDebugger and use in sample spiders - start of improving debugging
* setting CLF_DEBUG can now be used to generate spiderLog and chromeDriverLog in spider output
* add CrawlResponse.SC_UNKNOWN
* when a spider fails all attempts are made to take a screenshot of the browser window

Changed

* spiderArgs in crawl results now crawlArgs
* run-spider.sh now accepts full file name of spider rather than just base name -
so run-spider.sh xe_exchange_rates is now run-spider.sh xe_exchange_rates.py
* python-dateutil 2.8.0 -> 2.8.1
* dev-env v0.5.25 -> v0.6.6
* :MATERIAL CHANGE: Python 2.7 -> Python 3.7

Removed

* Nothing

Added

* Nothing

Changed

* Nothing

Removed

* remove [Snyk](https://snyk.io/) from CI pipeline & docs

Added

* add more BeautifulSoup and Scrapy doc references
* dev-env 0.5.21 -> 0.5.25
* add [Codecov](https://codecov.io) upload to CircleCI pipeline

Changed

* SpiderCrawler has chromedriver_log_file allowing
callers access to the [ChromeDriver debug logs](http://chromedriver.chromium.org/logging)
when the debug property for SpiderCrawler is set to True

Removed

* _debug property in crawl response under all circumstances

Added

* Nothing

Changed

* fix logging of CLF_CHROME value

Removed

* Nothing