Changelogs » Exoskeleton



* Add ability to create jobs: In essence a convenient way to store the current URL while parsing a large set of results. In case the program is terminated, now it is easy to pick up at the same page.


* Require Python 3.6 (Debian 10 "Buster" just moved to Python 3.7 with the 10.3 update. Ubuntu stable uses 3.6.)
* Use transactions and rollbacks to ensure dbms integrity (i.e. changes to the database schema).


* If the user tries to add an URL which is already processed or in the queue, any new labels get associated with that item.


* Assign labels to pages in the queue / the filemaster
* Store MIME type / Media type
* Some settings cannot be set at init anymore, but are now stored in the database instead.
* Small but breaking changes to the database schema
* Change default file hash version from SHA1 to SHA256


* Eliminate race condition
* Prepare labels


* Add statistics on a per host basis.
* Improve error handling.


* Check if a file / URL has already been processed before adding it to the queue.


* *Breaking Change*: Changed database structure in order to manage multiple versions of a file in different locations (local / Cloud / ...).
* Using database triggers to increment / decrement the number of versions saved.
* Add file name prefix
* Allow database access without passphrase
* Improved error handling
* Extended documentation (especially for the [Database Structure](Database-Scripts/
* Introduced unit tests


* Removed f-strings to be compatible with Python 3.5 as used by Debian 9.
* Add URL specific delays (for example in case of timeouts / 7)
* minor bug fixes


* Initial public release