Wpull

===================

* Fixes occasional error on chunked transfer encoding. Thanks to ivan for reporting.
* Fixes handling links with newlines found in HTML pages. Newlines are now stripped in links when scraping pages to better handle HTML soup.

===================

* Fixes another case of ``AssertionError`` on ``url_item.is_processed`` when robots.txt was enabled.
* Fixes crash if a malformed gzip response was received.
* Fixes ``--span-hosts`` to be implicitly enabled (as with ``--no-robots``) if ``--recursive`` is not supplied. This behavior unconditionally allows downloading a single file without specifying any options. It is what a user intuitively expects.

===================

* Improves performance on database operations. CPU usage should be less intensive.

===================

* Fixes handling of "204 No Content" responses.
* Fixes ``AssertionError`` on ``url_item.is_processed`` when robots.txt was enabled.
* Fixes PhantomJS page scrolling to be consistent.
* Lengthens PhantomJS viewport to ensure lazy-load images are properly triggered.
* Lengthens PhantomJS paper size to reduce excessive fragmentation of blocks.

=================

* Implements ``--phantomjs-scroll`` and ``--phantomjs-wait``.
* Implements saving HTML and PDF snapshots (including inside WARC file). Disable with ``--no-phantomjs-snapshot``.
* API: Adds PhantomJSController.

===================

* Fixes missing dependencies and files in ``setup.py``.
* For PhantomJS:

* Fixes capturing HTTPS connections .
* Fixes statistics counter.
* Supports very basic scraping of HTML. See Usage section.