Luigi

Fixed
luigi
- append is vulnerable of xss 2391

Added
luigi:
- Add google-auth-httplib2 as dependency (2384)
- Add new company user (2388)
- Allow release of resources during task running. (2346)
luigi.contrib:
- Add parameterized backoff limit (2375)

Fixed
luigi:
- Typo in running luigi documentation (2387)
- Skip running coverage on unreasonable files
- Pass kwargs to `discovery.build()` when instantiating `GSCClient`. (2291)
- Removed message 'No Instance(s) Available.' in Windows when starting … 2294

Added
luigi:
- Added generated data files to `.gitignore` (2367)
luigi.contrib:
- Add possibility to specify `Redshift` column compression (2343)

Changed
luigi:
- Show status message button in worker tab when only progress is set (2344)
- Verbose worker error logging (2353)
luigi.contrib:
- Replace `oauth2client` by `google-auth` (2361)

Fixed
luigi:
- Fix unicode formatting (2339)
luigi.contrib:
- Fix `contrib.docker_runner` exit code check (2356)

Added

luigi:
- A button to show errors for disabled tasks in the visualizer (2266)
- `OptionalParameter` parameter class (2295)

luigi.contrib:
- Support for the Docker Python SDK (2158)

Changed

luigi:
- Change the logging status of prune messages from info to debug (2254) - reduce repetitiveness of logging
- Speed up the visualizer so that a refresh doesn’t take minutes when number of tasks gets into the millions (2239)
- Hide "re-enable" and "forgive failures" buttons on success (2281)
- Convert `date`s to `datetime`s for `DateHourParameter` (2285)
- Handle multiprocessing with request sessions (2290) - Fixes bug with "RPCError: Received null response from remote scheduler"
- Replaced `param_args` with dynamic property with deprecation message
- Check task equality using `param_kwargs` instead of `param_args`

Removed

luigi.contrib:
- Copying of Avro field documentation by `BigQueryLoadAvro` (2269) - said copying is done by BigQuery natively now

Fixed

luigi:
- Remove invalid entrypoint luigi.tools.migrate (2257)
- Preserve filter on server on reload (2273)
- Fix MRO on tasks using `util.requires` (2307)

luigi.contrib:
- Fix Python 3 TypeError in `contrib.hive.HiveTableTarget.exists` (2323)

Luigi 2.7.1 is mostly bug fixes and small feature additions.

* Standardize Redshift credential usage across Redshift tasks: https://github.com/spotify/luigi/pull/2068
* BigQueryLoadAvro now handles complex Avro types: https://github.com/spotify/luigi/pull/2224
* Support for user-specified number of paralleled scheduled processes:
https://github.com/spotify/luigi/pull/2205
* ECS support for non-default cluster: https://github.com/spotify/luigi/pull/2045
* BigQuery ExtractTask support: https://github.com/spotify/luigi/pull/2134
* Support for autocommitting queries within Redshift and Postgres, allowing VACUUM statement execution: https://github.com/spotify/luigi/pull/2242
* High Availability (HA) support with WebHdfsClient using multiple namenodes: https://github.com/spotify/luigi/pull/2230
* Column mapping support for Redshift S3CopyToTable using the column list: https://github.com/spotify/luigi/pull/2245

There have also been some other feature additions, bugfixes, and docfixes. See all commits [here](https://github.com/spotify/luigi/compare/2.7.0...2.7.1).

- _Removed deprecations!_ `luigi.{postgres, s3}` are removed, use `luigi.contrib.{..}` instead https://github.com/spotify/luigi/pull/2181
- As of the bullet above. We now believe all contrib and core modules are well seperated by either being under either `luigi.xyx` or `luigi.contrib.xyz`. This should simplify to move modules into seperate repositories, which is future work.
- Luigi finally has a progress bar! https://github.com/spotify/luigi/pull/2108
- Scheduling improvements / bug fixes: https://github.com/spotify/luigi/pull/2208 https://github.com/spotify/luigi/pull/2207
- Luigi can now run PySpark in *cluster mode* https://github.com/spotify/luigi/pull/2197
- The batch email notifier shouldn't stall any more on failures. https://github.com/spotify/luigi/pull/2177
- A global panic "stop scheduling button" https://github.com/spotify/luigi/pull/2145
- Fixed error emails from workers: https://github.com/spotify/luigi/pull/2191
- My personal favorite PR, as it removes many deprecation warnings when running the tests: https://github.com/spotify/luigi/pull/2130

There have been a lot of other bugfixes, docsfixes, cleanups and added testcases! See all commits
[here](https://github.com/spotify/luigi/compare/2.6.2...2.7.0).