Gallery-dl

Changes
- [e621] split `e621` extractors from `danbooru` module ([3425](https://github.com/mikf/gallery-dl/issues/3425))
- [deviantart] remove mature scraps warning ([3691](https://github.com/mikf/gallery-dl/issues/3691))
- [deviantart] use `/collections/all` endpoint for favorites ([3666](https://github.com/mikf/gallery-dl/issues/3666), [#3668](https://github.com/mikf/gallery-dl/issues/3668))
- [newgrounds] update default image and audio archive IDs to prevent ID overlap ([3681](https://github.com/mikf/gallery-dl/issues/3681))
- rename `--ignore-config` to `--config-ignore`
Extractors
- [catbox] add `file` extractor ([3570](https://github.com/mikf/gallery-dl/issues/3570))
- [deviantart] add `search` extractor ([538](https://github.com/mikf/gallery-dl/issues/538), [#1264](https://github.com/mikf/gallery-dl/issues/1264), [#2954](https://github.com/mikf/gallery-dl/issues/2954), [#2970](https://github.com/mikf/gallery-dl/issues/2970), [#3577](https://github.com/mikf/gallery-dl/issues/3577))
- [deviantart] add `gallery-search` extractor ([1695](https://github.com/mikf/gallery-dl/issues/1695))
- [deviantart] support `fxdeviantart.com` URLs (3740)
- [e621] implement `notes` and `pools` metadata extraction ([3425](https://github.com/mikf/gallery-dl/issues/3425))
- [gelbooru] add `favorite` extractor ([3704](https://github.com/mikf/gallery-dl/issues/3704))
- [imagetwist] support `phun.imagetwist.com` and `imagehaha.com` domains ([3622](https://github.com/mikf/gallery-dl/issues/3622))
- [instagram] add `user` metadata field ([3107](https://github.com/mikf/gallery-dl/issues/3107))
- [manganelo] update and fix metadata extraction
- [manganelo] support mobile-only chapters
- [mangasee] extract `author` and `genre` metadata ([3703](https://github.com/mikf/gallery-dl/issues/3703))
- [misskey] add `misskey` extractors ([3717](https://github.com/mikf/gallery-dl/issues/3717))
- [pornpics] add `gallery` and `search` extractors ([263](https://github.com/mikf/gallery-dl/issues/263), [#3544](https://github.com/mikf/gallery-dl/issues/3544), [#3654](https://github.com/mikf/gallery-dl/issues/3654))
- [redgifs] support v3 URLs ([3588](https://github.com/mikf/gallery-dl/issues/3588). [#3589](https://github.com/mikf/gallery-dl/issues/3589))
- [redgifs] add `collection` extractors ([3427](https://github.com/mikf/gallery-dl/issues/3427), [#3662](https://github.com/mikf/gallery-dl/issues/3662))
- [shopify] support ohpolly.com ([440](https://github.com/mikf/gallery-dl/issues/440), [#3596](https://github.com/mikf/gallery-dl/issues/3596))
- [szurubooru] add `tag` and `post` extractors ([3583](https://github.com/mikf/gallery-dl/issues/3583), [#3713](https://github.com/mikf/gallery-dl/issues/3713))
- [twitter] add `transform` option
Options
- [postprocessor:metadata] add `sort` and `separators` options
- [postprocessor:exec] implement archive options ([3584](https://github.com/mikf/gallery-dl/issues/3584))
- add `--config-create` command-line option ([2333](https://github.com/mikf/gallery-dl/issues/2333))
- add `--config-toml` command-line option to load config files in TOML format
- add `output.stdout`, `output.stdin`, and `output.stderr` options ([1621](https://github.com/mikf/gallery-dl/issues/1621), [#2152](https://github.com/mikf/gallery-dl/issues/2152), [#2529](https://github.com/mikf/gallery-dl/issues/2529))
- add `hash_md5` and `hash_sha1` functions ([3679](https://github.com/mikf/gallery-dl/issues/3679))
- implement `globals` option to enable defining custom functions for `eval` statements
- implement `archive-pragma` option to use SQLite PRAGMA statements
- implement `actions` to trigger events on logging messages ([3338](https://github.com/mikf/gallery-dl/issues/3338), [#3630](https://github.com/mikf/gallery-dl/issues/3630))
- implement ability to load external extractor classes
- `-X/--extractors` command-line options
- `extractor.modules-sources` config option
Fixes
- [bunkr] fix extraction ([3636](https://github.com/mikf/gallery-dl/issues/3636), [#3655](https://github.com/mikf/gallery-dl/issues/3655))
- [danbooru] send gallery-dl User-Agent ([3665](https://github.com/mikf/gallery-dl/issues/3665))
- [deviantart] fix crash when handling deleted deviations in status updates ([3656](https://github.com/mikf/gallery-dl/issues/3656))
- [fanbox] fix crash with missing images ([3673](https://github.com/mikf/gallery-dl/issues/3673))
- [imagefap] update `gallery` URLs ([3595](https://github.com/mikf/gallery-dl/issues/3595))
- [imagefap] fix infinite pagination loop ([3594](https://github.com/mikf/gallery-dl/issues/3594))
- [imagefap] fix metadata extraction
- [oauth] use default name for browsers without `name` attribute
- [pinterest] unescape search terms ([3621](https://github.com/mikf/gallery-dl/issues/3621))
- [pixiv] fix `--write-tags` for `"tags": "original"` ([3675](https://github.com/mikf/gallery-dl/issues/3675))
- [poipiku] warn about incorrect passwords ([3646](https://github.com/mikf/gallery-dl/issues/3646))
- [reddit] update `videos` option ([3712](https://github.com/mikf/gallery-dl/issues/3712))
- [soundgasm] rewrite ([3578](https://github.com/mikf/gallery-dl/issues/3578))
- [telegraph] fix extraction when images are not in `<figure>` elements ([3590](https://github.com/mikf/gallery-dl/issues/3590))
- [tumblr] raise more detailed errors for dashboard-only blogs ([3628](https://github.com/mikf/gallery-dl/issues/3628))
- [twitter] fix some `original` retweets not downloading ([3744](https://github.com/mikf/gallery-dl/issues/3744))
- [ytdl] fix `--parse-metadata` ([3663](https://github.com/mikf/gallery-dl/issues/3663))
- [downloader:ytdl] prevent exception on empty results
Improvements
- [downloader:http] use `time.monotonic()`
- [downloader:http] update `_http_retry` to accept a Python function ([3569](https://github.com/mikf/gallery-dl/issues/3569))
- [postprocessor:metadata] speed up JSON encoding
- replace `json.loads/dumps` with direct calls to `JSONDecoder.decode/JSONEncoder.encode`
- improve `option.Formatter` performance
Removals
- [nitter] remove `nitter.pussthecat.org`

Additions
- [booru] add `url` option
- [danbooru] extend `metadata` option ([3505](https://github.com/mikf/gallery-dl/issues/3505))
- [deviantart] add extractor for status updates ([3539](https://github.com/mikf/gallery-dl/issues/3539), [#3541](https://github.com/mikf/gallery-dl/issues/3541))
- [deviantart] add support for `/deviation/` and `fav.me` URLs ([3558](https://github.com/mikf/gallery-dl/issues/3558), [#3560](https://github.com/mikf/gallery-dl/issues/3560))
- [kemonoparty] extract `hash` metadata for discord files ([3531](https://github.com/mikf/gallery-dl/issues/3531))
- [lexica] add `search` extractor ([3567](https://github.com/mikf/gallery-dl/issues/3567))
- [mastodon] add `num` and `count` metadata fields ([3517](https://github.com/mikf/gallery-dl/issues/3517))
- [nudecollect] add `image` and `album` extractors ([2430](https://github.com/mikf/gallery-dl/issues/2430), [#2818](https://github.com/mikf/gallery-dl/issues/2818), [#3575](https://github.com/mikf/gallery-dl/issues/3575))
- [wikifeet] add `gallery` extractor ([519](https://github.com/mikf/gallery-dl/issues/519), [#3537](https://github.com/mikf/gallery-dl/issues/3537))
- [downloader:http] add signature checks for `.blend`, `.obj`, and `.clip` files ([3535](https://github.com/mikf/gallery-dl/issues/3535))
- add `extractor.retry-codes` option
- add `-O/--postprocessor-option` command-line option ([3565](https://github.com/mikf/gallery-dl/issues/3565))
- improve `write-pages` output
Fixes
- [bunkr] fix downloading `.mkv` and `.ts` files ([3571](https://github.com/mikf/gallery-dl/issues/3571))
- [fantia] send `X-CSRF-Token` headers ([3576](https://github.com/mikf/gallery-dl/issues/3576))
- [generic] fix regex for non-src image URLs ([3555](https://github.com/mikf/gallery-dl/issues/3555))
- [hiperdex] update domain ([3572](https://github.com/mikf/gallery-dl/issues/3572))
- [hotleak] fix video URLs ([3516](https://github.com/mikf/gallery-dl/issues/3516), [#3525](https://github.com/mikf/gallery-dl/issues/3525), [#3563](https://github.com/mikf/gallery-dl/issues/3563), [#3581](https://github.com/mikf/gallery-dl/issues/3581))
- [instagram] always show `cursor` value after errors ([3440](https://github.com/mikf/gallery-dl/issues/3440))
- [instagram] update API domain, headers, and csrf token handling
- [oauth] show `client-id`/`api-key` values ([3518](https://github.com/mikf/gallery-dl/issues/3518))
- [philomena] match URLs with www subdomain
- [sankaku] update URL pattern ([3523](https://github.com/mikf/gallery-dl/issues/3523))
- [twitter] refresh guest tokens ([3445](https://github.com/mikf/gallery-dl/issues/3445), [#3458](https://github.com/mikf/gallery-dl/issues/3458))
- [twitter] fix search pagination ([3536](https://github.com/mikf/gallery-dl/issues/3536), [#3534](https://github.com/mikf/gallery-dl/issues/3534), [#3549](https://github.com/mikf/gallery-dl/issues/3549))
- [twitter] use `"browser": "firefox"` by default ([3522](https://github.com/mikf/gallery-dl/issues/3522))

Additions
- [downloader:http] add `validate` option
Fixes
- [kemonoparty] fix regression from commit 473bd380 ([3519](https://github.com/mikf/gallery-dl/issues/3519))

Additions
- [danbooru] extract `uploader` metadata ([3457](https://github.com/mikf/gallery-dl/issues/3457))
- [deviantart] initial implementation of username & password login for `scraps` ([1029](https://github.com/mikf/gallery-dl/issues/1029))
- [fanleaks] add `post` and `model` extractors ([3468](https://github.com/mikf/gallery-dl/issues/3468), [#3474](https://github.com/mikf/gallery-dl/issues/3474))
- [imagefap] add `folder` extractor ([3504](https://github.com/mikf/gallery-dl/issues/3504))
- [lynxchan] support `bbw-chan.nl` ([3456](https://github.com/mikf/gallery-dl/issues/3456), [#3463](https://github.com/mikf/gallery-dl/issues/3463))
- [pinterest] support `All Pins` boards ([2855](https://github.com/mikf/gallery-dl/issues/2855), [#3484](https://github.com/mikf/gallery-dl/issues/3484))
- [pinterest] add `domain` option ([3484](https://github.com/mikf/gallery-dl/issues/3484))
- [pixiv] implement `metadata-bookmark` option ([3417](https://github.com/mikf/gallery-dl/issues/3417))
- [tcbscans] add `chapter` and `manga` extractors ([3189](https://github.com/mikf/gallery-dl/issues/3189))
- [twitter] implement `syndication=extended` ([3483](https://github.com/mikf/gallery-dl/issues/3483))
- implement slice notation for `range` options ([918](https://github.com/mikf/gallery-dl/issues/918), [#2865](https://github.com/mikf/gallery-dl/issues/2865))
- allow `filter` options to be a list of expressions
Fixes
- [behance] use delay between requests ([2507](https://github.com/mikf/gallery-dl/issues/2507))
- [bunkr] fix URLs returned by API ([3481](https://github.com/mikf/gallery-dl/issues/3481))
- [fanbox] return `imageMap` files in order ([2718](https://github.com/mikf/gallery-dl/issues/2718))
- [imagefap] use delay between requests ([1140](https://github.com/mikf/gallery-dl/issues/1140))
- [imagefap] warn about redirects to `/human-verification` ([1140](https://github.com/mikf/gallery-dl/issues/1140))
- [kemonoparty] reject invalid/empty files ([3510](https://github.com/mikf/gallery-dl/issues/3510))
- [myhentaigallery] handle whitespace before title tag ([3503](https://github.com/mikf/gallery-dl/issues/3503))
- [poipiku] fix extraction for a different warning button style ([3493](https://github.com/mikf/gallery-dl/issues/3493), [#3460](https://github.com/mikf/gallery-dl/issues/3460))
- [poipiku] warn about login requirements
- [telegraph] fix file URLs ([3506](https://github.com/mikf/gallery-dl/issues/3506))
- [twitter] fix crash when using `expand` and `syndication` ([3473](https://github.com/mikf/gallery-dl/issues/3473))
- [twitter] apply tweet type checks before uniqueness check ([3439](https://github.com/mikf/gallery-dl/issues/3439), [#3455](https://github.com/mikf/gallery-dl/issues/3455))
- [twitter] force `https://` for TwitPic URLs ([#3449](https://github.com/mikf/gallery-dl/issues/3449))
- [ytdl] adapt to yt-dlp changes
- update and improve documentation ([3453](https://github.com/mikf/gallery-dl/issues/3453), [#3462](https://github.com/mikf/gallery-dl/issues/3462), [#3496](https://github.com/mikf/gallery-dl/issues/3496))

Additions
- [2chen] support `.club` URLs ([3406](https://github.com/mikf/gallery-dl/issues/3406))
- [deviantart] extract sta.sh URLs from `text_content` ([3366](https://github.com/mikf/gallery-dl/issues/3366))
- [deviantart] add `/view` URL support ([3367](https://github.com/mikf/gallery-dl/issues/3367))
- [e621] implement `threshold` option to control pagination ([3413](https://github.com/mikf/gallery-dl/issues/3413))
- [fapello] add `post`, `user` and `path` extractors ([3065](https://github.com/mikf/gallery-dl/issues/3065), [#3360](https://github.com/mikf/gallery-dl/issues/3360), [#3415](https://github.com/mikf/gallery-dl/issues/3415))
- [imgur] add support for imgur.io URLs ([3419](https://github.com/mikf/gallery-dl/issues/3419))
- [lynxchan] add generic extractors for lynxchan imageboards ([3389](https://github.com/mikf/gallery-dl/issues/3389), [#3394](https://github.com/mikf/gallery-dl/issues/3394))
- [mangafox] extract more metadata ([3167](https://github.com/mikf/gallery-dl/issues/3167))
- [pixiv] extract `date_url` metadata ([3405](https://github.com/mikf/gallery-dl/issues/3405))
- [soundgasm] add `audio` and `user` extractors ([3384](https://github.com/mikf/gallery-dl/issues/3384), [#3388](https://github.com/mikf/gallery-dl/issues/3388))
- [webmshare] add `video` extractor ([2410](https://github.com/mikf/gallery-dl/issues/2410))
- support Firefox containers for `--cookies-from-browser` ([3346](https://github.com/mikf/gallery-dl/issues/3346))
Fixes
- [2chen] fix file URLs
- [bunkr] update domain ([3391](https://github.com/mikf/gallery-dl/issues/3391))
- [exhentai] fix pagination
- [imagetwist] fix extraction
- [imgth] rewrite
- [instagram] prevent post `date` overwriting file `date` ([3392](https://github.com/mikf/gallery-dl/issues/3392))
- [khinsider] fix metadata extraction
- [komikcast] update domain and fix extraction
- [reddit] increase `id-max` default value ([3397](https://github.com/mikf/gallery-dl/issues/3397))
- [seiga] raise error when redirected to login page ([3401](https://github.com/mikf/gallery-dl/issues/3401))
- [sexcom] fix video URLs ([3408](https://github.com/mikf/gallery-dl/issues/3408), [#3414](https://github.com/mikf/gallery-dl/issues/3414))
- [twitter] update `search` pagination ([544](https://github.com/mikf/gallery-dl/issues/544))
- [warosu] fix and update
- [zerochan] update for layout v3
- restore paths for archived files ([3362](https://github.com/mikf/gallery-dl/issues/3362), [#3377](https://github.com/mikf/gallery-dl/issues/3377))
- use `util.NONE` as `keyword-default` default value ([3334](https://github.com/mikf/gallery-dl/issues/3334))
Removals
- [foolslide] remove `kireicake`
- [kissgoddess] remove module

Additions
- [artstation] add `pro-first` option ([3273](https://github.com/mikf/gallery-dl/issues/3273))
- [artstation] add `max-posts` option ([3270](https://github.com/mikf/gallery-dl/issues/3270))
- [fapachi] add `post` and `user` extractors ([3339](https://github.com/mikf/gallery-dl/issues/3339), [#3347](https://github.com/mikf/gallery-dl/issues/3347))
- [inkbunny] provide additional metadata ([3274](https://github.com/mikf/gallery-dl/issues/3274))
- [nitter] add `retweets` option ([3278](https://github.com/mikf/gallery-dl/issues/3278))
- [nitter] add `videos` option ([3279](https://github.com/mikf/gallery-dl/issues/3279))
- [nitter] support `/i/web/` and `/i/user/` URLs ([3310](https://github.com/mikf/gallery-dl/issues/3310))
- [pixhost] add `gallery` support ([3336](https://github.com/mikf/gallery-dl/issues/3336), [#3353](https://github.com/mikf/gallery-dl/issues/3353))
- [weibo] add `count` metadata field ([3305](https://github.com/mikf/gallery-dl/issues/3305))
- [downloader:http] add `retry-codes` option ([3313](https://github.com/mikf/gallery-dl/issues/3313))
- [formatter] implement `S` format specifier to sort lists ([3266](https://github.com/mikf/gallery-dl/issues/3266))
- implement `version-metadata` option ([3201](https://github.com/mikf/gallery-dl/issues/3201))
Fixes
- [2chen] fix extraction ([3354](https://github.com/mikf/gallery-dl/issues/3354), [#3356](https://github.com/mikf/gallery-dl/issues/3356))
- [bcy] fix JSONDecodeError ([3321](https://github.com/mikf/gallery-dl/issues/3321))
- [bunkr] fix video downloads ([3326](https://github.com/mikf/gallery-dl/issues/3326), [#3335](https://github.com/mikf/gallery-dl/issues/3335))
- [bunkr] use `media-files` servers for more file types
- [itaku] remove `Extreme` rating ([3285](https://github.com/mikf/gallery-dl/issues/3285), [#3287](https://github.com/mikf/gallery-dl/issues/3287))
- [hitomi] apply format check for every image ([3280](https://github.com/mikf/gallery-dl/issues/3280))
- [hotleak] fix UnboundLocalError ([3288](https://github.com/mikf/gallery-dl/issues/3288), [#3293](https://github.com/mikf/gallery-dl/issues/3293))
- [nitter] sanitize filenames ([3294](https://github.com/mikf/gallery-dl/issues/3294))
- [nitter] retry downloads on 404 ([3313](https://github.com/mikf/gallery-dl/issues/3313))
- [nitter] set `hlsPlayback` cookie
- [patreon] fix `403 Forbidden` errors ([3341](https://github.com/mikf/gallery-dl/issues/3341))
- [patreon] improve `campaign_id` extraction ([3235](https://github.com/mikf/gallery-dl/issues/3235))
- [patreon] update API query parameters
- [pixiv] preserve `tags` order ([3266](https://github.com/mikf/gallery-dl/issues/3266))
- [reddit] use `dash_url` for videos ([3258](https://github.com/mikf/gallery-dl/issues/3258), [#3306](https://github.com/mikf/gallery-dl/issues/3306))
- [twitter] fix error when using user IDs for suspended accounts
- [weibo] fix bug with empty `playback_list` ([3301](https://github.com/mikf/gallery-dl/issues/3301))
- [downloader:http] fix potential `ZeroDivisionError` ([3328](https://github.com/mikf/gallery-dl/issues/3328))
Removals
- [lolisafe] remove `zz.ht`