Release 2021.09.02

Partially revert "[build] Add homebrew taps (#827 )"
Show a more useful error in older python versions
2026-01-11 17:31:31 +00:00 · 2021-09-02 04:43:38 +05:30 · 2021-09-02 04:43:38 +05:30 · 2021-09-02 03:52:08 +05:30 · 2021-09-02 02:26:27 +05:30 · 2021-09-02 02:25:16 +05:30
400 changed files with 5224 additions and 1841 deletions
--- a/.github/ISSUE_TEMPLATE/1_broken_site.md
+++ b/.github/ISSUE_TEMPLATE/1_broken_site.md
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.02. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.10. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running yt-dlp version **2021.08.02**
+- [ ] I've verified that I'm running yt-dlp version **2021.08.10**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.08.02
+ [debug] yt-dlp version 2021.08.10
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/.github/ISSUE_TEMPLATE/2_site_support_request.md
+++ b/.github/ISSUE_TEMPLATE/2_site_support_request.md
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.02. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.10. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,9 +29,10 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running yt-dlp version **2021.08.02**
+- [ ] I've verified that I'm running yt-dlp version **2021.08.10**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
+- [ ] The provided URLs do not contain any DRM to the best of my knowledge
 - [ ] I've searched the bugtracker for similar site support requests including closed ones


--- a/.github/ISSUE_TEMPLATE/3_site_feature_request.md
+++ b/.github/ISSUE_TEMPLATE/3_site_feature_request.md
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.02. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.10. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running yt-dlp version **2021.08.02**
+- [ ] I've verified that I'm running yt-dlp version **2021.08.10**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones


--- a/.github/ISSUE_TEMPLATE/4_bug_report.md
+++ b/.github/ISSUE_TEMPLATE/4_bug_report.md
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.02. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.10. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,9 +29,10 @@ Carefully read and work through this check list in order to prevent the most com
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

- [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running yt-dlp version **2021.08.02**
+- [ ] I'm reporting a bug unrelated to a specific site
+- [ ] I've verified that I'm running yt-dlp version **2021.08.10**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
+- [ ] The provided URLs do not contain any DRM to the best of my knowledge
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
 - [ ] I've read bugs section in FAQ
@@ -46,7 +47,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.08.02
+ [debug] yt-dlp version 2021.08.10
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/.github/ISSUE_TEMPLATE/5_feature_request.md
+++ b/.github/ISSUE_TEMPLATE/5_feature_request.md
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.02. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.08.10. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a feature request
- [ ] I've verified that I'm running yt-dlp version **2021.08.02**
+- [ ] I've verified that I'm running yt-dlp version **2021.08.10**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones


--- a/.github/ISSUE_TEMPLATE/6_question.md
+++ b/.github/ISSUE_TEMPLATE/6_question.md
@@ -1,6 +1,6 @@
 ---
 name: Ask question
-about: Ask youtube-dl related question
+about: Ask yt-dlp related question
 title: "[Question]"
 labels: question
 assignees: ''
--- a/.github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md
+++ b/.github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md
@@ -32,6 +32,7 @@ Carefully read and work through this check list in order to prevent the most com
 - [ ] I've verified that I'm running yt-dlp version **%(version)s**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
+- [ ] The provided URLs do not contain any DRM to the best of my knowledge
 - [ ] I've searched the bugtracker for similar site support requests including closed ones


--- a/.github/ISSUE_TEMPLATE_tmpl/4_bug_report.md
+++ b/.github/ISSUE_TEMPLATE_tmpl/4_bug_report.md
@@ -29,9 +29,10 @@ Carefully read and work through this check list in order to prevent the most com
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

- [ ] I'm reporting a broken site support issue
+- [ ] I'm reporting a bug unrelated to a specific site
 - [ ] I've verified that I'm running yt-dlp version **%(version)s**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
+- [ ] The provided URLs do not contain any DRM to the best of my knowledge
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
 - [ ] I've read bugs section in FAQ
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -11,7 +11,7 @@
 - [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
 - [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)

-### In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
+### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
 - [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
 - [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

--- a/.github/workflows/quick-test.yml
+++ b/.github/workflows/quick-test.yml
@@ -27,5 +27,7 @@ jobs:
        python-version: 3.9
    - name: Install flake8
      run: pip install flake8
+    - name: Make lazy extractors
+      run: python devscripts/make_lazy_extractors.py yt_dlp/extractor/lazy_extractors.py
    - name: Run flake8
      run: flake8 .
--- a/.gitignore
+++ b/.gitignore
@@ -19,6 +19,8 @@ cookies.txt
 *.wav
 *.ape
 *.mkv
+*.flac
+*.avi
 *.swf
 *.part
 *.part-*
--- a/24
+++ b/24
@@ -22,7 +22,7 @@ Zocker1999NET
 nao20010128nao
 kurumigi
 bbepis
-animelover1984
+animelover1984/horahoradev
 Pccode66
 RobinD42
 hseg
@@ -78,3 +78,25 @@ pgaig
 PSlava
 stdedos
 u-spec-png
+Sipherdrakon
+kidonng
+smege1001
+tandy1000
+IONECarter
+capntrips
+mrfade
+ParadoxGBB
+wlritchi
+NeroBurner
+mahanstreamer
+alerikaisattera
+Derkades
+BunnyHelp
+i6t
+std-move
+Chocobozzz
+ouwou
+korli
+octotherp
+CeruleanSky
+zootedb0t
--- a/Changelog.md
+++ b/Changelog.md
@@ -19,6 +19,115 @@
 -->


+### 2021.09.02
+
+* **Native SponsorBlock** implementation by [nihil-admirari](https://github.com/nihil-admirari), [pukkandan](https://github.com/pukkandan)
+    * `--sponsorblock-remove CATS` removes specified chapters from file
+    * `--sponsorblock-mark CATS` marks the specified sponsor sections as chapters
+    * `--sponsorblock-chapter-title TMPL` to specify sponsor chapter template
+    * `--sponsorblock-api URL` to use a different API
+    * No re-encoding is done unless `--force-keyframes-at-cuts` is used
+    * The fetched sponsor sections are written to the infojson
+    * Deprecates: `--sponskrub`, `--no-sponskrub`, `--sponskrub-cut`, `--no-sponskrub-cut`, `--sponskrub-force`, `--no-sponskrub-force`, `--sponskrub-location`, `--sponskrub-args`
+* Split `--embed-chapters` from `--embed-metadata` (it still implies the former by default)
+* Add option `--remove-chapters` to remove arbitrary chapters by [nihil-admirari](https://github.com/nihil-admirari), pukkandan
+* Add option `--force-keyframes-at-cuts` for more accurate cuts when removing and splitting chapters by [nihil-admirari](https://github.com/nihil-admirari)
+* Let `--match-filter` reject entries early
+    * Makes redundant: `--match-title`, `--reject-title`, `--min-views`, `--max-views`
+* [lazy_extractor] Improvements (It now passes all tests)
+    * Bugfix for when plugin directory doesn't exist by [kidonng](https://github.com/kidonng)
+    * Create instance only after pre-checking archive
+    * Import actual class if an attribute is accessed
+    * Fix `suitable` and add flake8 test
+* [downloader/ffmpeg] Experimental support for DASH manifests (including live)
+    * Your ffmpeg must have [this patch](https://github.com/FFmpeg/FFmpeg/commit/3249c757aed678780e22e99a1a49f4672851bca9) applied for YouTube DASH to work
+* [downloader/ffmpeg] Allow passing custom arguments before `-i`
+
+* [BannedVideo] Add extractor by [smege1001](https://github.com/smege1001), [blackjack4494](https://github.com/blackjack4494), [pukkandan](https://github.com/pukkandan)
+* [bilibili] Add category extractor by [animelover1984](https://github.com/animelover1984)
+* [Epicon] Add extractors by [Ashish0804](https://github.com/Ashish0804)
+* [filmmodu] Add extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
+* [GabTV] Add extractor by [Ashish0804](https://github.com/Ashish0804)
+* [Hungama] Fix `HungamaSongIE` and add `HungamaAlbumPlaylistIE` by [Ashish0804](https://github.com/Ashish0804)
+* [ManotoTV] Add new extractors by [tandy1000](https://github.com/tandy1000)
+* [Niconico] Add Search extractors by [animelover1984](https://github.com/animelover1984), [pukkandan](https://github.com/pukkandan)
+* [Patreon] Add `PatreonUserIE` by [zenerdi0de](https://github.com/zenerdi0de)
+* [peloton] Add extractor by [IONECarter](https://github.com/IONECarter), [capntrips](https://github.com/capntrips), [pukkandan](https://github.com/pukkandan)
+* [ProjectVeritas] Add extractor by [Ashish0804](https://github.com/Ashish0804)
+* [radiko] Add extractors by [nao20010128nao](https://github.com/nao20010128nao)
+* [StarTV] Add extractor for `startv.com.tr` by [mrfade](https://github.com/mrfade), [coletdjnz](https://github.com/coletdjnz)
+* [tiktok] Add `TikTokUserIE` by [Ashish0804](https://github.com/Ashish0804), [pukkandan](https://github.com/pukkandan)
+* [Tokentube] Add extractor by [u-spec-png](https://github.com/u-spec-png)
+* [TV2Hu] Fix `TV2HuIE` and add `TV2HuSeriesIE` by [Ashish0804](https://github.com/Ashish0804)
+* [voicy] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
+
+* [adobepass] Fix Verizon SAML login by [nyuszika7h](https://github.com/nyuszika7h), [ParadoxGBB](https://github.com/ParadoxGBB)
+* [afreecatv] Fix adult VODs by [wlritchi](https://github.com/wlritchi)
+* [afreecatv] Tolerate failure to parse date string by [wlritchi](https://github.com/wlritchi)
+* [aljazeera] Fix extractor by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [ATV.at] Fix extractor for ATV.at by [NeroBurner](https://github.com/NeroBurner), [coletdjnz](https://github.com/coletdjnz)
+* [bitchute] Fix test by [mahanstreamer](https://github.com/mahanstreamer)
+* [camtube] Remove obsolete extractor by [alerikaisattera](https://github.com/alerikaisattera)
+* [CDA] Add more formats by [u-spec-png](https://github.com/u-spec-png)
+* [eroprofile] Fix page skipping in albums by [jhwgh1968](https://github.com/jhwgh1968)
+* [facebook] Fix format sorting
+* [facebook] Fix metadata extraction by [kikuyan](https://github.com/kikuyan)
+* [facebook] Update onion URL by [Derkades](https://github.com/Derkades)
+* [HearThisAtIE] Fix extractor by [Ashish0804](https://github.com/Ashish0804)
+* [instagram] Add referrer to prevent throttling by [u-spec-png](https://github.com/u-spec-png), [kikuyan](https://github.com/kikuyan)
+* [iwara.tv] Extract more metadata by [BunnyHelp](https://github.com/BunnyHelp)
+* [iwara] Add thumbnail by [i6t](https://github.com/i6t)
+* [kakao] Fix extractor
+* [mediaset] Fix extraction for some videos by [nyuszika7h](https://github.com/nyuszika7h)
+* [Motherless] Fix extractor by [coletdjnz](https://github.com/coletdjnz)
+* [Nova] fix extractor by [std-move](https://github.com/std-move)
+* [ParamountPlus] Fix geo verification by [shirt](https://github.com/shirt-dev)
+* [peertube] handle new video URL format by [Chocobozzz](https://github.com/Chocobozzz)
+* [pornhub] Separate and fix playlist extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
+* [reddit] Fix for quarantined subreddits by [ouwou](https://github.com/ouwou)
+* [ShemarooMe] Fix extractor by [Ashish0804](https://github.com/Ashish0804)
+* [soundcloud] Refetch `client_id` on 403
+* [tiktok] Fix metadata extraction
+* [TV2] Fix extractor by [Ashish0804](https://github.com/Ashish0804)
+* [tv5mondeplus] Fix extractor by [korli](https://github.com/korli)
+* [VH1,TVLand] Fix extractors by [Sipherdrakon](https://github.com/Sipherdrakon)
+* [Viafree] Fix extractor and extract subtitles by [coletdjnz](https://github.com/coletdjnz)
+* [XHamster] Extract `uploader_id` by [octotherp](https://github.com/octotherp)
+* [youtube] Add `shorts` to `_VALID_URL`
+* [youtube] Add av01 itags to known formats list by [blackjack4494](https://github.com/blackjack4494)
+* [youtube] Extract error messages from HTTPError response by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Fix subtitle names
+* [youtube] Prefer audio stream that YouTube considers default
+* [youtube] Remove annotations and deprecate `--write-annotations` by [coletdjnz](https://github.com/coletdjnz)
+* [Zee5] Fix extractor and add subtitles by [Ashish0804](https://github.com/Ashish0804)
+
+* [aria2c] Obey `--rate-limit`
+* [EmbedSubtitle] Continue even if some files are missing
+* [extractor] Better error message for DRM
+* [extractor] Common function `_match_valid_url`
+* [extractor] Show video id in error messages if possible
+* [FormatSort] Remove priority of `lang`
+* [options] Add `_set_from_options_callback`
+* [SubtitleConvertor] Fix bug during subtitle conversion
+* [utils] Add `parse_qs`
+* [webvtt] Fix timestamp overflow adjustment by [fstirlitz](https://github.com/fstirlitz)
+* Bugfix for `--replace-in-metadata`
+* Don't try to merge with final extension
+* Fix `--force-overwrites` when using `-k`
+* Fix `--no-prefer-free-formats` by [CeruleanSky](https://github.com/CeruleanSky)
+* Fix `-F` for extractors that directly return url
+* Fix `-J` when there are failed videos
+* Fix `extra_info` being reused across runs
+* Fix `playlist_index` not obeying `playlist_start` and add tests
+* Fix resuming of single formats when using `--no-part`
+* Revert erroneous use of the `Content-Length` header by [fstirlitz](https://github.com/fstirlitz)
+* Use `os.replace` where applicable by; paulwrubel
+* [build] Add homebrew taps `yt-dlp/taps/yt-dlp` by [nao20010128nao](https://github.com/nao20010128nao)
+* [build] Fix bug in making `yt-dlp.tar.gz`
+* [docs] Fix some typos by [pukkandan](https://github.com/pukkandan), [zootedb0t](https://github.com/zootedb0t)
+* [cleanup] Replace improper use of tab in trovo by [glenn-slayden](https://github.com/glenn-slayden)
+
+
 ### 2021.08.10

 * Add option `--replace-in-metadata`
@@ -30,7 +139,7 @@
 * Add compat-option `no-keep-subs`
 * [adobepass] Add MSO Cablevision by [Jessecar96](https://github.com/Jessecar96)
 * [BandCamp] Add BandcampMusicIE by [Ashish0804](https://github.com/Ashish0804)
-* [blackboardcollaborate] Add new extractor by [Ashish0804](https://github.com/Ashish0804)
+* [blackboardcollaborate] Add new extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
 * [eroprofile] Add album downloader by [jhwgh1968](https://github.com/jhwgh1968)
 * [mirrativ] Add extractors by [nao20010128nao](https://github.com/nao20010128nao)
 * [openrec] Add extractors by [nao20010128nao](https://github.com/nao20010128nao)
--- a/4
+++ b/4
@@ -110,7 +110,7 @@ _EXTRACTOR_FILES = $(shell find yt_dlp/extractor -iname '*.py' -and -not -iname
 yt_dlp/extractor/lazy_extractors.py: devscripts/make_lazy_extractors.py devscripts/lazy_load_template.py $(_EXTRACTOR_FILES)
 	$(PYTHON) devscripts/make_lazy_extractors.py $@

-yt-dlp.tar.gz: README.md yt-dlp.1 completions Changelog.md AUTHORS
+yt-dlp.tar.gz: yt-dlp README.md supportedsites.md yt-dlp.1 completions Changelog.md AUTHORS
 	@tar -czf $(DESTDIR)/yt-dlp.tar.gz --transform "s|^|yt-dlp/|" --owner 0 --group 0 \
 		--exclude '*.DS_Store' \
 		--exclude '*.kate-swp' \
@@ -124,7 +124,7 @@ yt-dlp.tar.gz: README.md yt-dlp.1 completions Changelog.md AUTHORS
 		devscripts test \
 		Changelog.md AUTHORS LICENSE README.md supportedsites.md \
 		Makefile MANIFEST.in yt-dlp.1 completions \
-		setup.py setup.cfg yt-dlp
+		setup.py setup.cfg yt-dlp yt_dlp

 AUTHORS: .mailmap
 	git shortlog -s -n | cut -f2 | sort > AUTHORS
--- a/README.md
+++ b/README.md
@@ -39,7 +39,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
    * [Subtitle Options](#subtitle-options)
    * [Authentication Options](#authentication-options)
    * [Post-processing Options](#post-processing-options)
-    * [SponSkrub (SponsorBlock) Options](#sponskrub-sponsorblock-options)
+    * [SponsorBlock Options](#sponsorblock-options)
    * [Extractor Options](#extractor-options)
 * [CONFIGURATION](#configuration)
    * [Authentication with .netrc file](#authentication-with-netrc-file)
@@ -62,9 +62,9 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
 # NEW FEATURES
 The major new features from the latest release of [blackjack4494/yt-dlc](https://github.com/blackjack4494/yt-dlc) are:

-* **[SponSkrub Integration](#sponskrub-sponsorblock-options)**: You can use [SponSkrub](https://github.com/yt-dlp/SponSkrub) to mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
+* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API

-* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))
+* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples))

 * **Merged with youtube-dl [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)**: (v2021.06.06) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)

@@ -78,7 +78,7 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/
    * Partial workaround for throttling issue
    * Redirect channel's home URL automatically to `/video` to preserve the old behaviour
    * `255kbps` audio is extracted from youtube music if premium cookies are given
-    * Youtube music Albums, channels etc can be downloaded
+    * Youtube music Albums, channels etc can be downloaded ([except self-uploaded music](https://github.com/yt-dlp/yt-dlp/issues/723))

 * **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[:PROFILE]`

@@ -88,9 +88,9 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/

 * **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats

-* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, SlingTV MSO, Cablevision MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip, ParamountPlusSeries, ScienceChannel, Utreon, OpenRec, BandcampMusic, blackboardcollaborate, eroprofile albums, mirrativ
+* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, SlingTV MSO, Cablevision MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip, ParamountPlusSeries, ScienceChannel, Utreon, OpenRec, BandcampMusic, blackboardcollaborate, eroprofile albums, mirrativ, BannedVideo, bilibili categories, Epicon, filmmodu, GabTV, HungamaAlbum, ManotoTV, Niconico search, Patreon User, peloton, ProjectVeritas, radiko, StarTV, tiktok user, Tokentube, voicy, TV2HuSeries

-* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer, nbcolympics, ParamountPlus, Newgrounds, 
+* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer, nbcolympics, ParamountPlus, Newgrounds, SAML Verizon login, Hungama, afreecatv, aljazeera, ATV, bitchute, camtube, CDA, eroprofile, facebook, HearThisAtIE, iwara, kakao, Motherless, Nova, peertube, pornhub, reddit, tiktok, TV2, TV2Hu, tv5mondeplus, VH1, Viafree, XHamster

 * **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details

@@ -151,6 +151,7 @@ yt-dlp is not platform specific. So it should work on your Unix box, on Windows

 You can install yt-dlp using one of the following methods:
 * Download the binary from the [latest release](https://github.com/yt-dlp/yt-dlp/releases/latest) (recommended method)
+* With Homebrew, `brew install yt-dlp/taps/yt-dlp`
 * Use [PyPI package](https://pypi.org/project/yt-dlp): `python3 -m pip install --upgrade yt-dlp`
 * Use pip+git: `python3 -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp.git@release`
 * Install master branch: `python3 -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp`
@@ -174,9 +175,16 @@ sudo aria2c https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o
 sudo chmod a+rx /usr/local/bin/yt-dlp
 ```

+macOS or Linux users that are using Homebrew (formerly known as Linuxbrew for Linux users) can also install it by:
+
+```
+brew install yt-dlp/taps/yt-dlp
+```
+
 ### UPDATE
 You can use `yt-dlp -U` to update if you are using the provided release.
 If you are using `pip`, simply re-run the same command that was used to install the program.
+If you have installed using Homebrew, run `brew upgrade yt-dlp/taps/yt-dlp`

 ### DEPENDENCIES
 Python versions 3.6+ (CPython and PyPy) are supported. Other versions and implementations may or may not work correctly.
@@ -186,7 +194,6 @@ On windows, [Microsoft Visual C++ 2010 SP1 Redistributable Package (x86)](https:

 While all the other dependancies are optional, `ffmpeg` and `ffprobe` are highly recommended
 * [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging seperate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. Licence [depends on the build](https://www.ffmpeg.org/legal.html)
-* [**sponskrub**](https://github.com/faissaloo/SponSkrub) - For using the [sponskrub options](#sponskrub-sponsorblock-options). Licenced under [GPLv3+](https://github.com/faissaloo/SponSkrub/blob/master/LICENCE.md)
 * [**mutagen**](https://github.com/quodlibet/mutagen) - For embedding thumbnail in certain formats. Licenced under [GPLv2+](https://github.com/quodlibet/mutagen/blob/master/COPYING)
 * [**pycryptodome**](https://github.com/Legrandin/pycryptodome) - For decrypting various data. Licenced under [BSD2](https://github.com/Legrandin/pycryptodome/blob/master/LICENSE.rst)
 * [**websockets**](https://github.com/aaugustin/websockets) - For downloading over websocket. Licenced under [BSD3](https://github.com/aaugustin/websockets/blob/main/LICENSE)
@@ -195,6 +202,7 @@ While all the other dependancies are optional, `ffmpeg` and `ffprobe` are highly
 * [**rtmpdump**](http://rtmpdump.mplayerhq.hu) - For downloading `rtmp` streams. ffmpeg will be used as a fallback. Licenced under [GPLv2+](http://rtmpdump.mplayerhq.hu)
 * [**mplayer**](http://mplayerhq.hu/design7/info.html) or [**mpv**](https://mpv.io) - For downloading `rstp` streams. ffmpeg will be used as a fallback. Licenced under [GPLv2+](https://github.com/mpv-player/mpv/blob/master/Copyright)
 * [**phantomjs**](https://github.com/ariya/phantomjs) - Used in extractors where javascript needs to be run. Licenced under [BSD3](https://github.com/ariya/phantomjs/blob/master/LICENSE.BSD)
+* [**sponskrub**](https://github.com/faissaloo/SponSkrub) - For using the now **deprecated** [sponskrub options](#sponskrub-options). Licenced under [GPLv3+](https://github.com/faissaloo/SponSkrub/blob/master/LICENCE.md)
 * Any external downloader that you want to use with `--downloader`

 To use or redistribute the dependencies, you must agree to their respective licensing terms.
@@ -213,7 +221,7 @@ Once you have all the necessary dependencies installed, just run `py pyinst.py`.
 You can also build the executable without any version info or metadata by using:

    pyinstaller.exe yt_dlp\__main__.py --onefile --name yt-dlp
-    
+
 Note that pyinstaller [does not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment

 **For Unix**:
@@ -248,9 +256,9 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     extractor
    --default-search PREFIX          Use this prefix for unqualified URLs. For
                                     example "gvsearch2:" downloads two videos
-                                     from google videos for youtube-dl "large
-                                     apple". Use the value "auto" to let
-                                     youtube-dl guess ("auto_warning" to emit a
+                                     from google videos for the search term
+                                     "large apple". Use the value "auto" to let
+                                     yt-dlp guess ("auto_warning" to emit a
                                     warning when guessing). "error" just throws
                                     an error. The default value "fixup_error"
                                     repairs broken URLs, but emits an error if
@@ -273,7 +281,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --no-mark-watched                Do not mark videos watched (default)
    --no-colors                      Do not emit color codes in output
    --compat-options OPTS            Options that can help keep compatibility
-                                     with youtube-dl and youtube-dlc
+                                     with youtube-dl or youtube-dlc
                                     configurations by reverting some of the
                                     changes made in yt-dlp. See "Differences in
                                     default behavior" for details
@@ -317,10 +325,6 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     specify range: "--playlist-items
                                     1-3,7,10-13", it will download the videos
                                     at index 1, 2, 3, 7, 10, 11, 12 and 13
-    --match-title REGEX              Download only matching titles (regex or
-                                     caseless sub-string)
-    --reject-title REGEX             Skip download for matching titles (regex or
-                                     caseless sub-string)
    --max-downloads NUMBER           Abort after downloading NUMBER files
    --min-filesize SIZE              Do not download any videos smaller than
                                     SIZE (e.g. 50k or 44.6m)
@@ -335,10 +339,6 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --dateafter DATE                 Download only videos uploaded on or after
                                     this date. The date formats accepted is the
                                     same as --date
-    --min-views COUNT                Do not download any videos with less than
-                                     COUNT views
-    --max-views COUNT                Do not download any videos with more than
-                                     COUNT views
    --match-filter FILTER            Generic video filter. Any field (see
                                     "OUTPUT TEMPLATE") can be compared with a
                                     number or a string using the operators
@@ -351,7 +351,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     filters can be checked with "&". Use a "\"
                                     to escape "&" or quotes if needed. Eg:
                                     --match-filter "!is_live & like_count>?100
-                                     & description~=\'(?i)\bcats \& dogs\b\'"
+                                     & description~='(?i)\bcats \& dogs\b'"
                                     matches only videos that are not live, has
                                     a like count more than 100 (or the like
                                     field is not available), and also has a
@@ -439,9 +439,12 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     (Alias: --external-downloader)
    --downloader-args NAME:ARGS      Give these arguments to the external
                                     downloader. Specify the downloader name and
-                                     the arguments separated by a colon ":". You
-                                     can use this option multiple times to give
-                                     different arguments to different downloaders
+                                     the arguments separated by a colon ":". For
+                                     ffmpeg, arguments can be passed to
+                                     different positions using the same syntax
+                                     as --postprocessor-args. You can use this
+                                     option multiple times to give different
+                                     arguments to different downloaders
                                     (Alias: --external-downloader-args)

 ## Filesystem Options:
@@ -500,9 +503,6 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --write-info-json                Write video metadata to a .info.json file
                                     (this may contain personal information)
    --no-write-info-json             Do not write video metadata (default)
-    --write-annotations              Write video annotations to a
-                                     .annotations.xml file
-    --no-write-annotations           Do not write video annotations (default)
    --write-playlist-metafiles       Write playlist metadata in addition to the
                                     video metadata when using --write-info-json,
                                     --write-description etc. (default)
@@ -541,8 +541,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --cache-dir DIR                  Location in the filesystem where youtube-dl
                                     can store some downloaded information (such
                                     as client ids and signatures) permanently.
-                                     By default $XDG_CACHE_HOME/youtube-dl or
-                                     ~/.cache/youtube-dl
+                                     By default $XDG_CACHE_HOME/yt-dlp or
+                                     ~/.cache/yt-dlp
    --no-cache-dir                   Disable filesystem caching
    --rm-cache-dir                   Delete all filesystem cache files

@@ -668,11 +668,6 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     bestvideo+bestaudio), output to given
                                     container format. One of mkv, mp4, ogg,
                                     webm, flv. Ignored if no merge is required
-    --allow-unplayable-formats       Allow unplayable formats to be listed and
-                                     downloaded. All video post-processing will
-                                     also be turned off
-    --no-allow-unplayable-formats    Do not allow unplayable formats to be
-                                     listed or downloaded (default)

 ## Subtitle Options:
    --write-subs                     Write subtitle file
@@ -738,24 +733,23 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     and the arguments separated by a colon ":"
                                     to give the argument to the specified
                                     postprocessor/executable. Supported PP are:
-                                     Merger, ExtractAudio, SplitChapters,
+                                     Merger, ModifyChapters, SplitChapters,
+                                     ExtractAudio, VideoRemuxer, VideoConvertor,
                                     Metadata, EmbedSubtitle, EmbedThumbnail,
                                     SubtitlesConvertor, ThumbnailsConvertor,
-                                     VideoRemuxer, VideoConvertor, SponSkrub,
                                     FixupStretched, FixupM4a, FixupM3u8,
                                     FixupTimestamp and FixupDuration. The
                                     supported executables are: AtomicParsley,
-                                     FFmpeg, FFprobe, and SponSkrub. You can
-                                     also specify "PP+EXE:ARGS" to give the
-                                     arguments to the specified executable only
-                                     when being used by the specified
-                                     postprocessor. Additionally, for
-                                     ffmpeg/ffprobe, "_i"/"_o" can be appended
-                                     to the prefix optionally followed by a
-                                     number to pass the argument before the
-                                     specified input/output file. Eg: --ppa
-                                     "Merger+ffmpeg_i1:-v quiet". You can use
-                                     this option multiple times to give
+                                     FFmpeg and FFprobe. You can also specify
+                                     "PP+EXE:ARGS" to give the arguments to the
+                                     specified executable only when being used
+                                     by the specified postprocessor.
+                                     Additionally, for ffmpeg/ffprobe, "_i"/"_o"
+                                     can be appended to the prefix optionally
+                                     followed by a number to pass the argument
+                                     before the specified input/output file. Eg:
+                                     --ppa "Merger+ffmpeg_i1:-v quiet". You can
+                                     use this option multiple times to give
                                     different arguments to different
                                     postprocessors. (Alias: --ppa)
    -k, --keep-video                 Keep the intermediate video file on disk
@@ -769,11 +763,15 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --no-embed-subs                  Do not embed subtitles (default)
    --embed-thumbnail                Embed thumbnail in the video as cover art
    --no-embed-thumbnail             Do not embed thumbnail (default)
-    --embed-metadata                 Embed metadata including chapter markers
-                                     (if supported by the format) to the video
-                                     file (Alias: --add-metadata)
-    --no-embed-metadata              Do not write metadata (default)
+    --embed-metadata                 Embed metadata to the video file. Also adds
+                                     chapters to file unless --no-add-chapters
+                                     is used (Alias: --add-metadata)
+    --no-embed-metadata              Do not add metadata to file (default)
                                     (Alias: --no-add-metadata)
+    --embed-chapters                 Add chapter markers to the video file
+                                     (Alias: --add-chapters)
+    --no-embed-chapters              Do not add chapter markers (default)
+                                     (Alias: --no-add-chapters)
    --parse-metadata FROM:TO         Parse additional metadata like title/artist
                                     from other fields; see "MODIFYING METADATA"
                                     for details
@@ -821,27 +819,51 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     files. See "OUTPUT TEMPLATE" for details
    --no-split-chapters              Do not split video based on chapters
                                     (default)
+    --remove-chapters REGEX          Remove chapters whose title matches the
+                                     given regular expression. This option can
+                                     be used multiple times
+    --no-remove-chapters             Do not remove any chapters from the file
+                                     (default)
+    --force-keyframes-at-cuts        Force keyframes around the chapters before
+                                     removing/splitting them. Requires a
+                                     reencode and thus is very slow, but the
+                                     resulting video may have fewer artifacts
+                                     around the cuts
+    --no-force-keyframes-at-cuts     Do not force keyframes around the chapters
+                                     when cutting/splitting (default)

-## SponSkrub (SponsorBlock) Options:
-[SponSkrub](https://github.com/yt-dlp/SponSkrub) is a utility to
-    mark/remove sponsor segments from downloaded YouTube videos using
+## SponsorBlock Options:
+Make chapter entries for, or remove various segments (sponsor,
+    introductions, etc.) from downloaded YouTube videos using the
    [SponsorBlock API](https://sponsor.ajay.app)

-    --sponskrub                      Use sponskrub to mark sponsored sections.
-                                     This is enabled by default if the sponskrub
-                                     binary exists (Youtube only)
-    --no-sponskrub                   Do not use sponskrub
-    --sponskrub-cut                  Cut out the sponsor sections instead of
-                                     simply marking them
-    --no-sponskrub-cut               Simply mark the sponsor sections, not cut
-                                     them out (default)
-    --sponskrub-force                Run sponskrub even if the video was already
-                                     downloaded
-    --no-sponskrub-force             Do not cut out the sponsor sections if the
-                                     video was already downloaded (default)
-    --sponskrub-location PATH        Location of the sponskrub binary; either
-                                     the path to the binary or its containing
-                                     directory
+    --sponsorblock-mark CATS         SponsorBlock categories to create chapters
+                                     for, separated by commas. Available
+                                     categories are all, sponsor, intro, outro,
+                                     selfpromo, interaction, preview,
+                                     music_offtopic. You can prefix the category
+                                     with a "-" to exempt it. See 
+                                     https://wiki.sponsor.ajay.app/index.php/Segment_Categories
+                                     for description of the categories. Eg:
+                                     --sponsorblock-query all,-preview
+    --sponsorblock-remove CATS       SponsorBlock categories to be removed from
+                                     the video file, separated by commas. If a
+                                     category is present in both mark and
+                                     remove, remove takes precedence. The syntax
+                                     and available categories are the same as
+                                     for --sponsorblock-mark
+    --sponsorblock-chapter-title TEMPLATE
+                                     The title template for SponsorBlock
+                                     chapters created by --sponsorblock-mark.
+                                     The same syntax as the output template is
+                                     used, but the only available fields are
+                                     start_time, end_time, category, categories,
+                                     name, category_names. Defaults to
+                                     "[SponsorBlock]: %(category_names)l"
+    --no-sponsorblock                Disable both --sponsorblock-mark and
+                                     --sponsorblock-remove
+    --sponsorblock-api URL           SponsorBlock API location, defaults to
+                                     https://sponsor.ajay.app

 ## Extractor Options:
    --extractor-retries RETRIES      Number of retries for known extractor
@@ -1051,6 +1073,15 @@ Available only when used in `--print`:

 - `urls` (string): The URLs of all requested formats, one in each line
 - `filename` (string): Name of the video file. Note that the actual filename may be different due to post-processing. Use `--exec echo` to get the name after all postprocessing is complete
+ 
+Available only in `--sponsorblock-chapter-title`:
+
+ - `start_time` (numeric): Start time of the chapter in seconds
+ - `end_time` (numeric): End time of the chapter in seconds
+ - `categories` (list): The SponsorBlock categories the chapter belongs to
+ - `category` (string): The smallest SponsorBlock category the chapter belongs to
+ - `category_names` (list): Friendly names of the categories
+ - `name` (string): Friendly name of the smallest category

 Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).

@@ -1175,7 +1206,9 @@ Format selectors can also be grouped using parentheses, for example if you want

 ## Sorting Formats

-You can change the criteria for being considered the `best` by using `-S` (`--format-sort`). The general format for this is `--format-sort field1,field2...`. The available fields are:
+You can change the criteria for being considered the `best` by using `-S` (`--format-sort`). The general format for this is `--format-sort field1,field2...`.
+
+The available fields are:

 - `hasvid`: Gives priority to formats that has a video stream
 - `hasaud`: Gives priority to formats that has a audio stream
@@ -1203,9 +1236,11 @@ You can change the criteria for being considered the `best` by using `-S` (`--fo
 - `br`: Equivalent to using `tbr,vbr,abr`
 - `asr`: Audio sample rate in Hz

-Note that any other **numerical** field made available by the extractor can also be used. All fields, unless specified otherwise, are sorted in descending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a preferred value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two preferred values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB.
+All fields, unless specified otherwise, are sorted in descending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a preferred value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two preferred values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB.

-The fields `hasvid`, `ie_pref`, `lang` are always given highest priority in sorting, irrespective of the user-defined order. This behaviour can be changed by using `--force-format-sort`. Apart from these, the default order used is: `quality,res,fps,codec:vp9.2,size,br,asr,proto,ext,hasaud,source,id`. Note that the extractors may override this default order, but they cannot override the user-provided order.
+The fields `hasvid` and `ie_pref` are always given highest priority in sorting, irrespective of the user-defined order. This behaviour can be changed by using `--force-format-sort`. Apart from these, the default order used is: `lang,quality,res,fps,codec:vp9.2,size,br,asr,proto,ext,hasaud,source,id`. The extractors may override this default order, but they cannot override the user-provided order.
+
+Note that the default has `codec:vp9.2`; i.e. `av1` is not prefered

 If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all respects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`.

@@ -1341,7 +1376,7 @@ The metadata obtained the the extractors can be modified by using `--parse-metad

 `--replace-in-metadata FIELDS REGEX REPLACE` is used to replace text in any metadata field using [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax). [Backreferences](https://docs.python.org/3/library/re.html?highlight=backreferences#re.sub) can be used in the replace string for advanced use.

-The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or a template (with same syntax as [output template](#output-template)) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields.
+The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or an [output template](#output-template) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields.

 Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.

@@ -1405,7 +1440,7 @@ The following extractors use this feature:
    * `include_live_dash`: Include live dash formats (These formats don't download properly)
    * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side).
    * `max_comments`: Maximum amount of comments to download (default all).
-    * `max_comment_depth`: Maximum depth for nested comments. YouTube supports depths 1 or 2 (default). 
+    * `max_comment_depth`: Maximum depth for nested comments. YouTube supports depths 1 or 2 (default).

 * **funimation**
    * `language`: Languages to extract. Eg: `funimation:language=english,japanese`
@@ -1439,6 +1474,10 @@ While these options are redundant, they are still expected to be used due to the
    -e, --get-title                  --print title
    -g, --get-url                    --print urls
    -j, --dump-json                  --print "%()j"
+    --match-title REGEX              --match-filter "title ~= (?i)REGEX"
+    --reject-title REGEX             --match-filter "title !~= (?i)REGEX"
+    --min-views COUNT                --match-filter "view_count >=? COUNT"
+    --max-views COUNT                --match-filter "view_count <=? COUNT"


 #### Not recommended
@@ -1454,7 +1493,6 @@ While these options still work, their use is not recommended since there are oth
    --hls-prefer-ffmpeg              --downloader "m3u8:ffmpeg"
    --list-formats-old               --compat-options list-formats (Alias: --no-list-formats-as-table)
    --list-formats-as-table          --compat-options -list-formats [Default] (Alias: --no-list-formats-old)
-    --sponskrub-args ARGS            --ppa "sponskrub:ARGS"
    --youtube-skip-dash-manifest     --extractor-args "youtube:skip=dash" (Alias: --no-youtube-include-dash-manifest)
    --youtube-skip-hls-manifest      --extractor-args "youtube:skip=hls" (Alias: --no-youtube-include-hls-manifest)
    --youtube-include-dash-manifest  Default (Alias: --no-youtube-skip-dash-manifest)
@@ -1466,6 +1504,8 @@ These options are not intended to be used by the end-user

    --test                           Download only part of video for testing extractors
    --youtube-print-sig-code         For testing youtube signatures
+    --allow-unplayable-formats       List unplayable formats also
+    --no-allow-unplayable-formats    Default


 #### Old aliases
@@ -1487,6 +1527,18 @@ These are aliases that are no longer documented for various reasons
    --write-srt                      --write-subs
    --yes-overwrites                 --force-overwrites

+#### Sponskrub Options
+Support for [SponSkrub](https://github.com/faissaloo/SponSkrub) has been deprecated in favor of `--sponsorblock`
+
+    --sponskrub                      --sponsorblock-mark all
+    --no-sponskrub                   --no-sponsorblock
+    --sponskrub-cut                  --sponsorblock-remove all
+    --no-sponskrub-cut               --sponsorblock-remove -all
+    --sponskrub-force                Not applicable
+    --no-sponskrub-force             Not applicable
+    --sponskrub-location             Not applicable
+    --sponskrub-args                 Not applicable
+
 #### No longer supported
 These options may no longer work as intended

@@ -1496,6 +1548,8 @@ These options may no longer work as intended
    --no-call-home                   Default
    --include-ads                    No longer supported
    --no-include-ads                 Default
+    --write-annotations              No supported site has annotations now
+    --no-write-annotations           Default

 #### Removed
 These options were deprecated since 2014 and have now been entirely removed
--- a/devscripts/lazy_load_template.py
+++ b/devscripts/lazy_load_template.py
@@ -1,20 +1,25 @@
-#!/usr/bin/env python3
 # coding: utf-8
-from __future__ import unicode_literals
-
 import re


-class LazyLoadExtractor(object):
+class LazyLoadMetaClass(type):
+    def __getattr__(cls, name):
+        return getattr(cls._get_real_class(), name)
+
+
+class LazyLoadExtractor(metaclass=LazyLoadMetaClass):
    _module = None
+    _WORKING = True

    @classmethod
-    def ie_key(cls):
-        return cls.__name__[:-2]
+    def _get_real_class(cls):
+        if '__real_class' not in cls.__dict__:
+            mod = __import__(cls._module, fromlist=(cls.__name__,))
+            cls.__real_class = getattr(mod, cls.__name__)
+        return cls.__real_class

    def __new__(cls, *args, **kwargs):
-        mod = __import__(cls._module, fromlist=(cls.__name__,))
-        real_cls = getattr(mod, cls.__name__)
+        real_cls = cls._get_real_class()
        instance = real_cls.__new__(real_cls)
        instance.__init__(*args, **kwargs)
        return instance
--- a/devscripts/make_lazy_extractors.py
+++ b/devscripts/make_lazy_extractors.py
@@ -16,23 +16,28 @@ if os.path.exists(lazy_extractors_filename):
    os.remove(lazy_extractors_filename)

 # Block plugins from loading
-os.rename('ytdlp_plugins', 'ytdlp_plugins_blocked')
+plugins_dirname = 'ytdlp_plugins'
+plugins_blocked_dirname = 'ytdlp_plugins_blocked'
+if os.path.exists(plugins_dirname):
+    os.rename(plugins_dirname, plugins_blocked_dirname)

 from yt_dlp.extractor import _ALL_CLASSES
 from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor

-os.rename('ytdlp_plugins_blocked', 'ytdlp_plugins')
+if os.path.exists(plugins_blocked_dirname):
+    os.rename(plugins_blocked_dirname, plugins_dirname)

 with open('devscripts/lazy_load_template.py', 'rt') as f:
    module_template = f.read()

+CLASS_PROPERTIES = ['ie_key', 'working', '_match_valid_url', 'suitable', '_match_id', 'get_temp_id']
 module_contents = [
-    module_template + '\n' + getsource(InfoExtractor.suitable) + '\n',
-    'class LazyLoadSearchExtractor(LazyLoadExtractor):\n    pass\n']
+    module_template,
+    *[getsource(getattr(InfoExtractor, k)) for k in CLASS_PROPERTIES],
+    '\nclass LazyLoadSearchExtractor(LazyLoadExtractor):\n    pass\n']

 ie_template = '''
 class {name}({bases}):
-    _VALID_URL = {valid_url!r}
    _module = '{module}'
 '''

@@ -53,14 +58,17 @@ def get_base_name(base):


 def build_lazy_ie(ie, name):
-    valid_url = getattr(ie, '_VALID_URL', None)
    s = ie_template.format(
        name=name,
        bases=', '.join(map(get_base_name, ie.__bases__)),
-        valid_url=valid_url,
        module=ie.__module__)
+    valid_url = getattr(ie, '_VALID_URL', None)
+    if valid_url:
+        s += f'    _VALID_URL = {valid_url!r}\n'
+    if not ie._WORKING:
+        s += '    _WORKING = False\n'
    if ie.suitable.__func__ is not InfoExtractor.suitable.__func__:
-        s += '\n' + getsource(ie.suitable)
+        s += f'\n{getsource(ie.suitable)}'
    if hasattr(ie, '_make_valid_url'):
        # search extractors
        s += make_valid_template.format(valid_url=ie._make_valid_url())
@@ -98,7 +106,7 @@ for ie in ordered_cls:
        names.append(name)

 module_contents.append(
-    '_ALL_CLASSES = [{0}]'.format(', '.join(names)))
+    '\n_ALL_CLASSES = [{0}]'.format(', '.join(names)))

 module_src = '\n'.join(module_contents) + '\n'

--- a/devscripts/update-formulae.py
+++ b/devscripts/update-formulae.py
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+from __future__ import unicode_literals
+
+import json
+import os
+import re
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from yt_dlp.compat import compat_urllib_request
+
+
+# usage: python3 ./devscripts/update-formulae.py <path-to-formulae-rb> <version>
+# version can be either 0-aligned (yt-dlp version) or normalized (PyPl version)
+
+filename, version = sys.argv[1:]
+
+normalized_version = '.'.join(str(int(x)) for x in version.split('.'))
+
+pypi_release = json.loads(compat_urllib_request.urlopen(
+    'https://pypi.org/pypi/yt-dlp/%s/json' % normalized_version
+).read().decode('utf-8'))
+
+tarball_file = next(x for x in pypi_release['urls'] if x['filename'].endswith('.tar.gz'))
+
+sha256sum = tarball_file['digests']['sha256']
+url = tarball_file['url']
+
+with open(filename, 'r') as r:
+    formulae_text = r.read()
+
+formulae_text = re.sub(r'sha256 "[0-9a-f]*?"', 'sha256 "%s"' % sha256sum, formulae_text)
+formulae_text = re.sub(r'url "[^"]*?"', 'url "%s"' % url, formulae_text)
+
+with open(filename, 'w') as w:
+    w.write(formulae_text)
--- a/supportedsites.md
+++ b/supportedsites.md
@@ -97,6 +97,7 @@
 - **Bandcamp:weekly**
 - **BandcampMusic**
 - **bangumi.bilibili.com**: BiliBili番剧
+ - **BannedVideo**
 - **bbc**: BBC
 - **bbc.co.uk**: BBC iPlayer
 - **bbc.co.uk:article**: BBC articles
@@ -118,6 +119,7 @@
 - **Bigflix**
 - **Bild**: Bild.de
 - **BiliBili**
+ - **Bilibili category extractor**
 - **BilibiliAudio**
 - **BilibiliAudioAlbum**
 - **BilibiliChannel**
@@ -153,7 +155,6 @@
 - **Camdemy**
 - **CamdemyFolder**
 - **CamModels**
- - **CamTube**
 - **CamWithHer**
 - **canalc2.tv**
 - **Canalplus**: mycanal.fr and piwiplus.fr
@@ -295,6 +296,8 @@
 - **Embedly**
 - **EMPFlix**
 - **Engadget**
+ - **Epicon**
+ - **EpiconSeries**
 - **Eporner**
 - **EroProfile**
 - **EroProfile:album**
@@ -316,6 +319,7 @@
 - **fc2**
 - **fc2:embed**
 - **Fczenit**
+ - **Filmmodu**
 - **filmon**
 - **filmon:channel**
 - **Filmweb**
@@ -353,6 +357,7 @@
 - **Funk**
 - **Fusion**
 - **Fux**
+ - **GabTV**
 - **Gaia**
 - **GameInformer**
 - **GameSpot**
@@ -408,6 +413,7 @@
 - **Huajiao**: 花椒直播
 - **HuffPost**: Huffington Post
 - **Hungama**
+ - **HungamaAlbumPlaylist**
 - **HungamaSong**
 - **Hypem**
 - **ign.com**
@@ -520,6 +526,9 @@
 - **MallTV**
 - **mangomolo:live**
 - **mangomolo:video**
+ - **ManotoTV**: Manoto TV (Episode)
+ - **ManotoTVLive**: Manoto TV (Live)
+ - **ManotoTVShow**: Manoto TV (Show)
 - **ManyVids**
 - **MaoriTV**
 - **Markiza**
@@ -658,6 +667,9 @@
 - **niconico**: ニコニコ動画
 - **NiconicoPlaylist**
 - **NiconicoUser**
+ - **nicovideo:search**: Nico video searches
+ - **nicovideo:search:date**: Nico video searches, newest first
+ - **nicovideo:search_url**: Nico video search URLs
 - **Nintendo**
 - **Nitter**
 - **njoy**: N-JOY
@@ -740,9 +752,12 @@
 - **parliamentlive.tv**: UK parliament videos
 - **Parlview**
 - **Patreon**
+ - **PatreonUser**
 - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
 - **PearVideo**
 - **PeerTube**
+ - **peloton**
+ - **peloton:live**: Peloton Live
 - **People**
 - **PerformGroup**
 - **periscope**: Periscope
@@ -783,6 +798,7 @@
 - **PornHd**
 - **PornHub**: PornHub and Thumbzilla
 - **PornHubPagedVideoList**
+ - **PornHubPlaylist**
 - **PornHubUser**
 - **PornHubUserVideosUpload**
 - **Pornotube**
@@ -790,6 +806,7 @@
 - **PornoXO**
 - **PornTube**
 - **PressTV**
+ - **ProjectVeritas**
 - **prosiebensat1**: ProSiebenSat.1 Digital
 - **puhutv**
 - **puhutv:serie**
@@ -806,6 +823,8 @@
 - **QuicklineLive**
 - **R7**
 - **R7Article**
+ - **Radiko**
+ - **RadikoRadio**
 - **radio.de**
 - **radiobremen**
 - **radiocanada**
@@ -956,6 +975,7 @@
 - **SRGSSR**
 - **SRGSSRPlay**: srf.ch, rts.ch, rsi.ch, rtr.ch and swissinfo.ch play sites
 - **stanfordoc**: Stanford Open ClassRoom
+ - **startv**
 - **Steam**
 - **Stitcher**
 - **StitcherShow**
@@ -1023,11 +1043,14 @@
 - **ThisAV**
 - **ThisOldHouse**
 - **TikTok**
+ - **tiktok:user**
 - **tinypic**: tinypic.com videos
 - **TMZ**
 - **TNAFlix**
 - **TNAFlixNetworkEmbed**
 - **toggle**
+ - **Tokentube**
+ - **Tokentube:channel**
 - **ToonGoggles**
 - **tou.tv**
 - **Toypics**: Toypics video
@@ -1050,10 +1073,11 @@
 - **Turbo**
 - **tv.dfb.de**
 - **TV2**
- - **tv2.hu**
 - **TV2Article**
 - **TV2DK**
 - **TV2DKBornholmPlay**
+ - **tv2play.hu**
+ - **tv2playseries.hu**
 - **TV4**: tv4.se and tv4play.se
 - **TV5MondePlus**: TV5MONDE+
 - **tv5unis**
@@ -1187,6 +1211,8 @@
 - **VODPl**
 - **VODPlatform**
 - **VoiceRepublic**
+ - **voicy**
+ - **voicy:channel**
 - **Voot**
 - **VootSeries**
 - **VoxMedia**
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -978,54 +978,31 @@ class TestYoutubeDL(unittest.TestCase):
            ydl.process_ie_result(copy.deepcopy(playlist))
            return ydl.downloaded_info_dicts

-        def get_ids(params):
-            return [int(v['id']) for v in get_downloaded_info_dicts(params)]
+        def test_selection(params, expected_ids):
+            results = [
+                (v['playlist_autonumber'] - 1, (int(v['id']), v['playlist_index']))
+                for v in get_downloaded_info_dicts(params)]
+            self.assertEqual(results, list(enumerate(zip(expected_ids, expected_ids))))

-        result = get_ids({})
-        self.assertEqual(result, [1, 2, 3, 4])
-
-        result = get_ids({'playlistend': 10})
-        self.assertEqual(result, [1, 2, 3, 4])
-
-        result = get_ids({'playlistend': 2})
-        self.assertEqual(result, [1, 2])
-
-        result = get_ids({'playliststart': 10})
-        self.assertEqual(result, [])
-
-        result = get_ids({'playliststart': 2})
-        self.assertEqual(result, [2, 3, 4])
-
-        result = get_ids({'playlist_items': '2-4'})
-        self.assertEqual(result, [2, 3, 4])
-
-        result = get_ids({'playlist_items': '2,4'})
-        self.assertEqual(result, [2, 4])
-
-        result = get_ids({'playlist_items': '10'})
-        self.assertEqual(result, [])
-
-        result = get_ids({'playlist_items': '3-10'})
-        self.assertEqual(result, [3, 4])
-
-        result = get_ids({'playlist_items': '2-4,3-4,3'})
-        self.assertEqual(result, [2, 3, 4])
+        test_selection({}, [1, 2, 3, 4])
+        test_selection({'playlistend': 10}, [1, 2, 3, 4])
+        test_selection({'playlistend': 2}, [1, 2])
+        test_selection({'playliststart': 10}, [])
+        test_selection({'playliststart': 2}, [2, 3, 4])
+        test_selection({'playlist_items': '2-4'}, [2, 3, 4])
+        test_selection({'playlist_items': '2,4'}, [2, 4])
+        test_selection({'playlist_items': '10'}, [])

        # Tests for https://github.com/ytdl-org/youtube-dl/issues/10591
-        # @{
-        result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
-        self.assertEqual(result[0]['playlist_index'], 2)
-        self.assertEqual(result[1]['playlist_index'], 3)
+        test_selection({'playlist_items': '2-4,3-4,3'}, [2, 3, 4])
+        test_selection({'playlist_items': '4,2'}, [4, 2])

-        result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
-        self.assertEqual(result[0]['playlist_index'], 2)
-        self.assertEqual(result[1]['playlist_index'], 3)
-        self.assertEqual(result[2]['playlist_index'], 4)
-
-        result = get_downloaded_info_dicts({'playlist_items': '4,2'})
-        self.assertEqual(result[0]['playlist_index'], 4)
-        self.assertEqual(result[1]['playlist_index'], 2)
-        # @}
+        # Tests for https://github.com/yt-dlp/yt-dlp/issues/720
+        # https://github.com/yt-dlp/yt-dlp/issues/302
+        test_selection({'playlistreverse': True}, [4, 3, 2, 1])
+        test_selection({'playliststart': 2, 'playlistreverse': True}, [4, 3, 2])
+        test_selection({'playlist_items': '2,4', 'playlistreverse': True}, [4, 2])
+        test_selection({'playlist_items': '4,2'}, [4, 2])

    def test_urlopen_no_file_protocol(self):
        # see https://github.com/ytdl-org/youtube-dl/issues/8227
--- a/test/test_download.py
+++ b/test/test_download.py
--- a/test/test_postprocessors.py
+++ b/test/test_postprocessors.py
@@ -6,6 +6,7 @@ from __future__ import unicode_literals
 import os
 import sys
 import unittest
+
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

 from yt_dlp import YoutubeDL
@@ -15,6 +16,7 @@ from yt_dlp.postprocessor import (
    FFmpegThumbnailsConvertorPP,
    MetadataFromFieldPP,
    MetadataParserPP,
+    ModifyChaptersPP
 )


@@ -68,3 +70,461 @@ class TestExec(unittest.TestCase):
        self.assertEqual(pp.parse_cmd('echo', info), cmd)
        self.assertEqual(pp.parse_cmd('echo {}', info), cmd)
        self.assertEqual(pp.parse_cmd('echo %(filepath)q', info), cmd)
+
+
+class TestModifyChaptersPP(unittest.TestCase):
+    def setUp(self):
+        self._pp = ModifyChaptersPP(YoutubeDL())
+
+    @staticmethod
+    def _sponsor_chapter(start, end, cat, remove=False):
+        c = {'start_time': start, 'end_time': end, '_categories': [(cat, start, end)]}
+        if remove:
+            c['remove'] = True
+        return c
+
+    @staticmethod
+    def _chapter(start, end, title=None, remove=False):
+        c = {'start_time': start, 'end_time': end}
+        if title is not None:
+            c['title'] = title
+        if remove:
+            c['remove'] = True
+        return c
+
+    def _chapters(self, ends, titles):
+        self.assertEqual(len(ends), len(titles))
+        start = 0
+        chapters = []
+        for e, t in zip(ends, titles):
+            chapters.append(self._chapter(start, e, t))
+            start = e
+        return chapters
+
+    def _remove_marked_arrange_sponsors_test_impl(
+            self, chapters, expected_chapters, expected_removed):
+        actual_chapters, actual_removed = (
+            self._pp._remove_marked_arrange_sponsors(chapters))
+        for c in actual_removed:
+            c.pop('title', None)
+            c.pop('_categories', None)
+        actual_chapters = [{
+            'start_time': c['start_time'],
+            'end_time': c['end_time'],
+            'title': c['title'],
+        } for c in actual_chapters]
+        self.assertSequenceEqual(expected_chapters, actual_chapters)
+        self.assertSequenceEqual(expected_removed, actual_removed)
+
+    def test_remove_marked_arrange_sponsors_CanGetThroughUnaltered(self):
+        chapters = self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, chapters, [])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithSponsors(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 20, 'sponsor'),
+            self._sponsor_chapter(30, 40, 'preview'),
+            self._sponsor_chapter(50, 60, 'sponsor')]
+        expected = self._chapters(
+            [10, 20, 30, 40, 50, 60, 70],
+            ['c', '[SponsorBlock]: Sponsor', 'c', '[SponsorBlock]: Preview/Recap',
+             'c', '[SponsorBlock]: Sponsor', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_UniqueNamesForOverlappingSponsors(self):
+        chapters = self._chapters([120], ['c']) + [
+            self._sponsor_chapter(10, 45, 'sponsor'), self._sponsor_chapter(20, 40, 'selfpromo'),
+            self._sponsor_chapter(50, 70, 'sponsor'), self._sponsor_chapter(60, 85, 'selfpromo'),
+            self._sponsor_chapter(90, 120, 'selfpromo'), self._sponsor_chapter(100, 110, 'sponsor')]
+        expected = self._chapters(
+            [10, 20, 40, 45, 50, 60, 70, 85, 90, 100, 110, 120],
+            ['c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Sponsor, Unpaid/Self Promotion',
+             '[SponsorBlock]: Sponsor',
+             'c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Sponsor, Unpaid/Self Promotion',
+             '[SponsorBlock]: Unpaid/Self Promotion',
+             'c', '[SponsorBlock]: Unpaid/Self Promotion', '[SponsorBlock]: Unpaid/Self Promotion, Sponsor',
+             '[SponsorBlock]: Unpaid/Self Promotion'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithCuts(self):
+        cuts = [self._chapter(10, 20, remove=True),
+                self._sponsor_chapter(30, 40, 'sponsor', remove=True),
+                self._chapter(50, 60, remove=True)]
+        chapters = self._chapters([70], ['c']) + cuts
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([40], ['c']), cuts)
+
+    def test_remove_marked_arrange_sponsors_ChapterWithSponsorsAndCuts(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 20, 'sponsor'),
+            self._sponsor_chapter(30, 40, 'selfpromo', remove=True),
+            self._sponsor_chapter(50, 60, 'interaction')]
+        expected = self._chapters([10, 20, 40, 50, 60],
+                                  ['c', '[SponsorBlock]: Sponsor', 'c',
+                                   '[SponsorBlock]: Interaction Reminder', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, expected, [self._chapter(30, 40, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithSponsorCutInTheMiddle(self):
+        cuts = [self._sponsor_chapter(20, 30, 'selfpromo', remove=True),
+                self._chapter(40, 50, remove=True)]
+        chapters = self._chapters([70], ['c']) + [self._sponsor_chapter(10, 60, 'sponsor')] + cuts
+        expected = self._chapters(
+            [10, 40, 50], ['c', '[SponsorBlock]: Sponsor', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_ChapterWithCutHidingSponsor(self):
+        cuts = [self._sponsor_chapter(20, 50, 'selpromo', remove=True)]
+        chapters = self._chapters([60], ['c']) + [
+            self._sponsor_chapter(10, 20, 'intro'),
+            self._sponsor_chapter(30, 40, 'sponsor'),
+            self._sponsor_chapter(50, 60, 'outro'),
+        ] + cuts
+        expected = self._chapters(
+            [10, 20, 30], ['c', '[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Endcards/Credits'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_ChapterWithAdjacentSponsors(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 20, 'sponsor'),
+            self._sponsor_chapter(20, 30, 'selfpromo'),
+            self._sponsor_chapter(30, 40, 'interaction')]
+        expected = self._chapters(
+            [10, 20, 30, 40, 70],
+            ['c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion',
+             '[SponsorBlock]: Interaction Reminder', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithAdjacentCuts(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 20, 'sponsor'),
+            self._sponsor_chapter(20, 30, 'interaction', remove=True),
+            self._chapter(30, 40, remove=True),
+            self._sponsor_chapter(40, 50, 'selpromo', remove=True),
+            self._sponsor_chapter(50, 60, 'interaction')]
+        expected = self._chapters([10, 20, 30, 40],
+                                  ['c', '[SponsorBlock]: Sponsor',
+                                   '[SponsorBlock]: Interaction Reminder', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, expected, [self._chapter(20, 50, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithOverlappingSponsors(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 30, 'sponsor'),
+            self._sponsor_chapter(20, 50, 'selfpromo'),
+            self._sponsor_chapter(40, 60, 'interaction')]
+        expected = self._chapters(
+            [10, 20, 30, 40, 50, 60, 70],
+            ['c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Sponsor, Unpaid/Self Promotion',
+             '[SponsorBlock]: Unpaid/Self Promotion', '[SponsorBlock]: Unpaid/Self Promotion, Interaction Reminder',
+             '[SponsorBlock]: Interaction Reminder', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithOverlappingCuts(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 30, 'sponsor', remove=True),
+            self._sponsor_chapter(20, 50, 'selfpromo', remove=True),
+            self._sponsor_chapter(40, 60, 'interaction', remove=True)]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([20], ['c']), [self._chapter(10, 60, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingSponsors(self):
+        chapters = self._chapters([170], ['c']) + [
+            self._sponsor_chapter(0, 30, 'intro'),
+            self._sponsor_chapter(20, 50, 'sponsor'),
+            self._sponsor_chapter(40, 60, 'selfpromo'),
+            self._sponsor_chapter(70, 90, 'sponsor'),
+            self._sponsor_chapter(80, 100, 'sponsor'),
+            self._sponsor_chapter(90, 110, 'sponsor'),
+            self._sponsor_chapter(120, 140, 'selfpromo'),
+            self._sponsor_chapter(130, 160, 'interaction'),
+            self._sponsor_chapter(150, 170, 'outro')]
+        expected = self._chapters(
+            [20, 30, 40, 50, 60, 70, 110, 120, 130, 140, 150, 160, 170],
+            ['[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Intermission/Intro Animation, Sponsor', '[SponsorBlock]: Sponsor',
+             '[SponsorBlock]: Sponsor, Unpaid/Self Promotion', '[SponsorBlock]: Unpaid/Self Promotion', 'c',
+             '[SponsorBlock]: Sponsor', 'c', '[SponsorBlock]: Unpaid/Self Promotion',
+             '[SponsorBlock]: Unpaid/Self Promotion, Interaction Reminder',
+             '[SponsorBlock]: Interaction Reminder',
+             '[SponsorBlock]: Interaction Reminder, Endcards/Credits', '[SponsorBlock]: Endcards/Credits'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingCuts(self):
+        chapters = self._chapters([170], ['c']) + [
+            self._chapter(0, 30, remove=True),
+            self._sponsor_chapter(20, 50, 'sponsor', remove=True),
+            self._chapter(40, 60, remove=True),
+            self._sponsor_chapter(70, 90, 'sponsor', remove=True),
+            self._chapter(80, 100, remove=True),
+            self._chapter(90, 110, remove=True),
+            self._sponsor_chapter(120, 140, 'sponsor', remove=True),
+            self._sponsor_chapter(130, 160, 'selfpromo', remove=True),
+            self._chapter(150, 170, remove=True)]
+        expected_cuts = [self._chapter(0, 60, remove=True),
+                         self._chapter(70, 110, remove=True),
+                         self._chapter(120, 170, remove=True)]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([20], ['c']), expected_cuts)
+
+    def test_remove_marked_arrange_sponsors_OverlappingSponsorsDifferentTitlesAfterCut(self):
+        chapters = self._chapters([60], ['c']) + [
+            self._sponsor_chapter(10, 60, 'sponsor'),
+            self._sponsor_chapter(10, 40, 'intro'),
+            self._sponsor_chapter(30, 50, 'interaction'),
+            self._sponsor_chapter(30, 50, 'selfpromo', remove=True),
+            self._sponsor_chapter(40, 50, 'interaction'),
+            self._sponsor_chapter(50, 60, 'outro')]
+        expected = self._chapters(
+            [10, 30, 40], ['c', '[SponsorBlock]: Sponsor, Intermission/Intro Animation', '[SponsorBlock]: Sponsor, Endcards/Credits'])
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, expected, [self._chapter(30, 50, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_SponsorsNoLongerOverlapAfterCut(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 30, 'sponsor'),
+            self._sponsor_chapter(20, 50, 'interaction'),
+            self._sponsor_chapter(30, 50, 'selpromo', remove=True),
+            self._sponsor_chapter(40, 60, 'sponsor'),
+            self._sponsor_chapter(50, 60, 'interaction')]
+        expected = self._chapters(
+            [10, 20, 40, 50], ['c', '[SponsorBlock]: Sponsor',
+                               '[SponsorBlock]: Sponsor, Interaction Reminder', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, expected, [self._chapter(30, 50, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_SponsorsStillOverlapAfterCut(self):
+        chapters = self._chapters([70], ['c']) + [
+            self._sponsor_chapter(10, 60, 'sponsor'),
+            self._sponsor_chapter(20, 60, 'interaction'),
+            self._sponsor_chapter(30, 50, 'selfpromo', remove=True)]
+        expected = self._chapters(
+            [10, 20, 40, 50], ['c', '[SponsorBlock]: Sponsor',
+                               '[SponsorBlock]: Sponsor, Interaction Reminder', 'c'])
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, expected, [self._chapter(30, 50, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_ChapterWithRunsOfOverlappingSponsorsAndCuts(self):
+        chapters = self._chapters([200], ['c']) + [
+            self._sponsor_chapter(10, 40, 'sponsor'),
+            self._sponsor_chapter(10, 30, 'intro'),
+            self._chapter(20, 30, remove=True),
+            self._sponsor_chapter(30, 40, 'selfpromo'),
+            self._sponsor_chapter(50, 70, 'sponsor'),
+            self._sponsor_chapter(60, 80, 'interaction'),
+            self._chapter(70, 80, remove=True),
+            self._sponsor_chapter(70, 90, 'sponsor'),
+            self._sponsor_chapter(80, 100, 'interaction'),
+            self._sponsor_chapter(120, 170, 'selfpromo'),
+            self._sponsor_chapter(130, 180, 'outro'),
+            self._chapter(140, 150, remove=True),
+            self._chapter(150, 160, remove=True)]
+        expected = self._chapters(
+            [10, 20, 30, 40, 50, 70, 80, 100, 110, 130, 140, 160],
+            ['c', '[SponsorBlock]: Sponsor, Intermission/Intro Animation', '[SponsorBlock]: Sponsor, Unpaid/Self Promotion',
+             'c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Sponsor, Interaction Reminder',
+             '[SponsorBlock]: Interaction Reminder', 'c', '[SponsorBlock]: Unpaid/Self Promotion',
+             '[SponsorBlock]: Unpaid/Self Promotion, Endcards/Credits', '[SponsorBlock]: Endcards/Credits', 'c'])
+        expected_cuts = [self._chapter(20, 30, remove=True),
+                         self._chapter(70, 80, remove=True),
+                         self._chapter(140, 160, remove=True)]
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, expected_cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorOverlapsMultipleChapters(self):
+        chapters = (self._chapters([20, 40, 60, 80, 100], ['c1', 'c2', 'c3', 'c4', 'c5'])
+                    + [self._sponsor_chapter(10, 90, 'sponsor')])
+        expected = self._chapters([10, 90, 100], ['c1', '[SponsorBlock]: Sponsor', 'c5'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutOverlapsMultipleChapters(self):
+        cuts = [self._chapter(10, 90, remove=True)]
+        chapters = self._chapters([20, 40, 60, 80, 100], ['c1', 'c2', 'c3', 'c4', 'c5']) + cuts
+        expected = self._chapters([10, 20], ['c1', 'c5'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorsWithinSomeChaptersAndOverlappingOthers(self):
+        chapters = (self._chapters([10, 40, 60, 80], ['c1', 'c2', 'c3', 'c4'])
+                    + [self._sponsor_chapter(20, 30, 'sponsor'),
+                       self._sponsor_chapter(50, 70, 'selfpromo')])
+        expected = self._chapters([10, 20, 30, 40, 50, 70, 80],
+                                  ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c2', 'c3',
+                                   '[SponsorBlock]: Unpaid/Self Promotion', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutsWithinSomeChaptersAndOverlappingOthers(self):
+        cuts = [self._chapter(20, 30, remove=True), self._chapter(50, 70, remove=True)]
+        chapters = self._chapters([10, 40, 60, 80], ['c1', 'c2', 'c3', 'c4']) + cuts
+        expected = self._chapters([10, 30, 40, 50], ['c1', 'c2', 'c3', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_ChaptersAfterLastSponsor(self):
+        chapters = (self._chapters([20, 40, 50, 60], ['c1', 'c2', 'c3', 'c4'])
+                    + [self._sponsor_chapter(10, 30, 'music_offtopic')])
+        expected = self._chapters(
+            [10, 30, 40, 50, 60],
+            ['c1', '[SponsorBlock]: Non-Music Section', 'c2', 'c3', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_ChaptersAfterLastCut(self):
+        cuts = [self._chapter(10, 30, remove=True)]
+        chapters = self._chapters([20, 40, 50, 60], ['c1', 'c2', 'c3', 'c4']) + cuts
+        expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorStartsAtChapterStart(self):
+        chapters = (self._chapters([10, 20, 40], ['c1', 'c2', 'c3'])
+                    + [self._sponsor_chapter(20, 30, 'sponsor')])
+        expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c3'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutStartsAtChapterStart(self):
+        cuts = [self._chapter(20, 30, remove=True)]
+        chapters = self._chapters([10, 20, 40], ['c1', 'c2', 'c3']) + cuts
+        expected = self._chapters([10, 20, 30], ['c1', 'c2', 'c3'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorEndsAtChapterEnd(self):
+        chapters = (self._chapters([10, 30, 40], ['c1', 'c2', 'c3'])
+                    + [self._sponsor_chapter(20, 30, 'sponsor')])
+        expected = self._chapters([10, 20, 30, 40], ['c1', 'c2', '[SponsorBlock]: Sponsor', 'c3'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutEndsAtChapterEnd(self):
+        cuts = [self._chapter(20, 30, remove=True)]
+        chapters = self._chapters([10, 30, 40], ['c1', 'c2', 'c3']) + cuts
+        expected = self._chapters([10, 20, 30], ['c1', 'c2', 'c3'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorCoincidesWithChapters(self):
+        chapters = (self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4'])
+                    + [self._sponsor_chapter(10, 30, 'sponsor')])
+        expected = self._chapters([10, 30, 40], ['c1', '[SponsorBlock]: Sponsor', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutCoincidesWithChapters(self):
+        cuts = [self._chapter(10, 30, remove=True)]
+        chapters = self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4']) + cuts
+        expected = self._chapters([10, 20], ['c1', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorsAtVideoBoundaries(self):
+        chapters = (self._chapters([20, 40, 60], ['c1', 'c2', 'c3'])
+                    + [self._sponsor_chapter(0, 10, 'intro'), self._sponsor_chapter(50, 60, 'outro')])
+        expected = self._chapters(
+            [10, 20, 40, 50, 60], ['[SponsorBlock]: Intermission/Intro Animation', 'c1', 'c2', 'c3', '[SponsorBlock]: Endcards/Credits'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutsAtVideoBoundaries(self):
+        cuts = [self._chapter(0, 10, remove=True), self._chapter(50, 60, remove=True)]
+        chapters = self._chapters([20, 40, 60], ['c1', 'c2', 'c3']) + cuts
+        expected = self._chapters([10, 30, 40], ['c1', 'c2', 'c3'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_SponsorsOverlapChaptersAtVideoBoundaries(self):
+        chapters = (self._chapters([10, 40, 50], ['c1', 'c2', 'c3'])
+                    + [self._sponsor_chapter(0, 20, 'intro'), self._sponsor_chapter(30, 50, 'outro')])
+        expected = self._chapters(
+            [20, 30, 50], ['[SponsorBlock]: Intermission/Intro Animation', 'c2', '[SponsorBlock]: Endcards/Credits'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_CutsOverlapChaptersAtVideoBoundaries(self):
+        cuts = [self._chapter(0, 20, remove=True), self._chapter(30, 50, remove=True)]
+        chapters = self._chapters([10, 40, 50], ['c1', 'c2', 'c3']) + cuts
+        expected = self._chapters([10], ['c2'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
+
+    def test_remove_marked_arrange_sponsors_EverythingSponsored(self):
+        chapters = (self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4'])
+                    + [self._sponsor_chapter(0, 20, 'intro'), self._sponsor_chapter(20, 40, 'outro')])
+        expected = self._chapters([20, 40], ['[SponsorBlock]: Intermission/Intro Animation', '[SponsorBlock]: Endcards/Credits'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
+
+    def test_remove_marked_arrange_sponsors_EverythingCut(self):
+        cuts = [self._chapter(0, 20, remove=True), self._chapter(20, 40, remove=True)]
+        chapters = self._chapters([10, 20, 30, 40], ['c1', 'c2', 'c3', 'c4']) + cuts
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, [], [self._chapter(0, 40, remove=True)])
+
+    def test_remove_marked_arrange_sponsors_TinyChaptersInTheOriginalArePreserved(self):
+        chapters = self._chapters([0.1, 0.2, 0.3, 0.4], ['c1', 'c2', 'c3', 'c4'])
+        self._remove_marked_arrange_sponsors_test_impl(chapters, chapters, [])
+
+    def test_remove_marked_arrange_sponsors_TinySponsorsAreIgnored(self):
+        chapters = [self._sponsor_chapter(0, 0.1, 'intro'), self._chapter(0.1, 0.2, 'c1'),
+                    self._sponsor_chapter(0.2, 0.3, 'sponsor'), self._chapter(0.3, 0.4, 'c2'),
+                    self._sponsor_chapter(0.4, 0.5, 'outro')]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([0.3, 0.5], ['c1', 'c2']), [])
+
+    def test_remove_marked_arrange_sponsors_TinyChaptersResultingFromCutsAreIgnored(self):
+        cuts = [self._chapter(1.5, 2.5, remove=True)]
+        chapters = self._chapters([2, 3, 3.5], ['c1', 'c2', 'c3']) + cuts
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([2, 2.5], ['c1', 'c3']), cuts)
+
+    def test_remove_marked_arrange_sponsors_TinyChaptersResultingFromSponsorOverlapAreIgnored(self):
+        chapters = self._chapters([1, 3, 4], ['c1', 'c2', 'c3']) + [
+            self._sponsor_chapter(1.5, 2.5, 'sponsor')]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([1.5, 3, 4], ['c1', '[SponsorBlock]: Sponsor', 'c3']), [])
+
+    def test_remove_marked_arrange_sponsors_TinySponsorsOverlapsAreIgnored(self):
+        chapters = self._chapters([2, 3, 5], ['c1', 'c2', 'c3']) + [
+            self._sponsor_chapter(1, 3, 'sponsor'),
+            self._sponsor_chapter(2.5, 4, 'selfpromo')
+        ]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([1, 3, 4, 5], [
+                'c1', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion', 'c3']), [])
+
+    def test_make_concat_opts_CommonCase(self):
+        sponsor_chapters = [self._chapter(1, 2, 's1'), self._chapter(10, 20, 's2')]
+        expected = '''ffconcat version 1.0
+file 'file:test'
+outpoint 1.000000
+file 'file:test'
+inpoint 2.000000
+outpoint 10.000000
+file 'file:test'
+inpoint 20.000000
+'''
+        opts = self._pp._make_concat_opts(sponsor_chapters, 30)
+        self.assertEqual(expected, ''.join(self._pp._concat_spec(['test'] * len(opts), opts)))
+
+    def test_make_concat_opts_NoZeroDurationChunkAtVideoStart(self):
+        sponsor_chapters = [self._chapter(0, 1, 's1'), self._chapter(10, 20, 's2')]
+        expected = '''ffconcat version 1.0
+file 'file:test'
+inpoint 1.000000
+outpoint 10.000000
+file 'file:test'
+inpoint 20.000000
+'''
+        opts = self._pp._make_concat_opts(sponsor_chapters, 30)
+        self.assertEqual(expected, ''.join(self._pp._concat_spec(['test'] * len(opts), opts)))
+
+    def test_make_concat_opts_NoZeroDurationChunkAtVideoEnd(self):
+        sponsor_chapters = [self._chapter(1, 2, 's1'), self._chapter(10, 20, 's2')]
+        expected = '''ffconcat version 1.0
+file 'file:test'
+outpoint 1.000000
+file 'file:test'
+inpoint 2.000000
+outpoint 10.000000
+'''
+        opts = self._pp._make_concat_opts(sponsor_chapters, 20)
+        self.assertEqual(expected, ''.join(self._pp._concat_spec(['test'] * len(opts), opts)))
+
+    def test_quote_for_concat_RunsOfQuotes(self):
+        self.assertEqual(
+            r"'special '\'' '\'\''characters'\'\'\''galore'",
+            self._pp._quote_for_ffmpeg("special ' ''characters'''galore"))
+
+    def test_quote_for_concat_QuotesAtStart(self):
+        self.assertEqual(
+            r"\'\'\''special '\'' characters '\'' galore'",
+            self._pp._quote_for_ffmpeg("'''special ' characters ' galore"))
+
+    def test_quote_for_concat_QuotesAtEnd(self):
+        self.assertEqual(
+            r"'special '\'' characters '\'' galore'\'\'\'",
+            self._pp._quote_for_ffmpeg("special ' characters ' galore'''"))
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -62,6 +62,7 @@ from yt_dlp.utils import (
    parse_iso8601,
    parse_resolution,
    parse_bitrate,
+    parse_qs,
    pkcs1pad,
    read_batch_urls,
    sanitize_filename,
@@ -117,8 +118,6 @@ from yt_dlp.compat import (
    compat_getenv,
    compat_os_name,
    compat_setenv,
-    compat_urlparse,
-    compat_parse_qs,
 )


@@ -688,38 +687,36 @@ class TestUtil(unittest.TestCase):
        self.assertTrue(isinstance(data, bytes))

    def test_update_url_query(self):
-        def query_dict(url):
-            return compat_parse_qs(compat_urlparse.urlparse(url).query)
-        self.assertEqual(query_dict(update_url_query(
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'quality': ['HD'], 'format': ['mp4']})),
-            query_dict('http://example.com/path?quality=HD&format=mp4'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?quality=HD&format=mp4'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'system': ['LINUX', 'WINDOWS']})),
-            query_dict('http://example.com/path?system=LINUX&system=WINDOWS'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?system=LINUX&system=WINDOWS'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'fields': 'id,formats,subtitles'})),
-            query_dict('http://example.com/path?fields=id,formats,subtitles'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?fields=id,formats,subtitles'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'fields': ('id,formats,subtitles', 'thumbnails')})),
-            query_dict('http://example.com/path?fields=id,formats,subtitles&fields=thumbnails'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?fields=id,formats,subtitles&fields=thumbnails'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path?manifest=f4m', {'manifest': []})),
-            query_dict('http://example.com/path'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path?system=LINUX&system=WINDOWS', {'system': 'LINUX'})),
-            query_dict('http://example.com/path?system=LINUX'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?system=LINUX'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'fields': b'id,formats,subtitles'})),
-            query_dict('http://example.com/path?fields=id,formats,subtitles'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?fields=id,formats,subtitles'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'width': 1080, 'height': 720})),
-            query_dict('http://example.com/path?width=1080&height=720'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?width=1080&height=720'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'bitrate': 5020.43})),
-            query_dict('http://example.com/path?bitrate=5020.43'))
-        self.assertEqual(query_dict(update_url_query(
+            parse_qs('http://example.com/path?bitrate=5020.43'))
+        self.assertEqual(parse_qs(update_url_query(
            'http://example.com/path', {'test': '第二行тест'})),
-            query_dict('http://example.com/path?test=%E7%AC%AC%E4%BA%8C%E8%A1%8C%D1%82%D0%B5%D1%81%D1%82'))
+            parse_qs('http://example.com/path?test=%E7%AC%AC%E4%BA%8C%E8%A1%8C%D1%82%D0%B5%D1%81%D1%82'))

    def test_multipart_encode(self):
        self.assertEqual(
@@ -1285,9 +1282,15 @@ ffmpeg version 2.4.4 Copyright (c) 2000-2014 the FFmpeg ...'''), '2.4.4')
        self.assertTrue(match_str(r'x="foo \& bar" & x^=foo', {'x': 'foo & bar'}))

        # Example from docs
-        self.assertTrue(
-            r'!is_live & like_count>?100 & description~=\'(?i)\bcats \& dogs\b\'',
-            {'description': 'Raining Cats & Dogs'})
+        self.assertTrue(match_str(
+            r"!is_live & like_count>?100 & description~='(?i)\bcats \& dogs\b'",
+            {'description': 'Raining Cats & Dogs'}))
+
+        # Incomplete
+        self.assertFalse(match_str('id!=foo', {'id': 'foo'}, True))
+        self.assertTrue(match_str('x', {'id': 'foo'}, True))
+        self.assertTrue(match_str('!x', {'id': 'foo'}, True))
+        self.assertFalse(match_str('x', {'id': 'foo'}, False))

    def test_parse_dfxp_time_expr(self):
        self.assertEqual(parse_dfxp_time_expr(None), None)
--- a/test/test_write_annotations.py.disabled
+++ b/test/test_write_annotations.py.disabled
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -461,7 +461,7 @@ class YoutubeDL(object):
    ))

    params = None
-    _ies = []
+    _ies = {}
    _pps = {'pre_process': [], 'before_dl': [], 'after_move': [], 'post_process': []}
    _printed_messages = set()
    _first_webpage_request = True
@@ -475,7 +475,7 @@ class YoutubeDL(object):
        """Create a FileDownloader object with the given options."""
        if params is None:
            params = {}
-        self._ies = []
+        self._ies = {}
        self._ies_instances = {}
        self._pps = {'pre_process': [], 'before_dl': [], 'after_move': [], 'post_process': []}
        self._printed_messages = set()
@@ -497,6 +497,12 @@ class YoutubeDL(object):
            self.report_warning(
                'Python version %d.%d is not supported! Please update to Python 3.6 or above' % sys.version_info[:2])

+        if self.params.get('allow_unplayable_formats'):
+            self.report_warning(
+                'You have asked for unplayable formats to be listed/downloaded. '
+                'This is a developer option intended for debugging. '
+                'If you experience any issues while using this option, DO NOT open a bug report')
+
        def check_deprecated(param, option, suggestion):
            if self.params.get(param) is not None:
                self.report_warning('%s is deprecated. Use %s instead' % (option, suggestion))
@@ -514,11 +520,6 @@ class YoutubeDL(object):
        for msg in self.params.get('warnings', []):
            self.report_warning(msg)

-        if self.params.get('final_ext'):
-            if self.params.get('merge_output_format'):
-                self.report_warning('--merge-output-format will be ignored since --remux-video or --recode-video is given')
-            self.params['merge_output_format'] = self.params['final_ext']
-
        if self.params.get('overwrites') is None:
            self.params.pop('overwrites', None)
        elif self.params.get('nooverwrites') is not None:
@@ -630,11 +631,19 @@ class YoutubeDL(object):

    def add_info_extractor(self, ie):
        """Add an InfoExtractor object to the end of the list."""
-        self._ies.append(ie)
+        ie_key = ie.ie_key()
+        self._ies[ie_key] = ie
        if not isinstance(ie, type):
-            self._ies_instances[ie.ie_key()] = ie
+            self._ies_instances[ie_key] = ie
            ie.set_downloader(self)

+    def _get_info_extractor_class(self, ie_key):
+        ie = self._ies.get(ie_key)
+        if ie is None:
+            ie = get_info_extractor(ie_key)
+            self.add_info_extractor(ie)
+        return ie
+
    def get_info_extractor(self, ie_key):
        """
        Get an instance of an IE with name ie_key, it will try to get one from
@@ -832,6 +841,16 @@ class YoutubeDL(object):
        except UnicodeEncodeError:
            self.to_screen('Deleting existing file')

+    def raise_no_formats(self, info, forced=False):
+        has_drm = info.get('__has_drm')
+        msg = 'This video is DRM protected' if has_drm else 'No video formats found!'
+        expected = self.params.get('ignore_no_formats_error')
+        if forced or not expected:
+            raise ExtractorError(msg, video_id=info['id'], ie=info['extractor'],
+                                 expected=has_drm or expected)
+        else:
+            self.report_warning(msg)
+
    def parse_outtmpl(self):
        outtmpl_dict = self.params.get('outtmpl', {})
        if not isinstance(outtmpl_dict, dict):
@@ -1117,12 +1136,15 @@ class YoutubeDL(object):
            if age_restricted(info_dict.get('age_limit'), self.params.get('age_limit')):
                return 'Skipping "%s" because it is age restricted' % video_title

-            if not incomplete:
-                match_filter = self.params.get('match_filter')
-                if match_filter is not None:
-                    ret = match_filter(info_dict)
-                    if ret is not None:
-                        return ret
+            match_filter = self.params.get('match_filter')
+            if match_filter is not None:
+                try:
+                    ret = match_filter(info_dict, incomplete=incomplete)
+                except TypeError:
+                    # For backward compatibility
+                    ret = None if incomplete else match_filter(info_dict)
+                if ret is not None:
+                    return ret
            return None

        if self.in_download_archive(info_dict):
@@ -1165,31 +1187,24 @@ class YoutubeDL(object):
            ie_key = 'Generic'

        if ie_key:
-            ies = [self.get_info_extractor(ie_key)]
+            ies = {ie_key: self._get_info_extractor_class(ie_key)}
        else:
            ies = self._ies

-        for ie in ies:
+        for ie_key, ie in ies.items():
            if not ie.suitable(url):
                continue

-            ie_key = ie.ie_key()
-            ie = self.get_info_extractor(ie_key)
            if not ie.working():
                self.report_warning('The program functionality for this site has been marked as broken, '
                                    'and will probably not work.')

-            try:
-                temp_id = str_or_none(
-                    ie.extract_id(url) if callable(getattr(ie, 'extract_id', None))
-                    else ie._match_id(url))
-            except (AssertionError, IndexError, AttributeError):
-                temp_id = None
+            temp_id = ie.get_temp_id(url)
            if temp_id is not None and self.in_download_archive({'id': temp_id, 'ie_key': ie_key}):
                self.to_screen("[%s] %s: has already been recorded in archive" % (
                               ie_key, temp_id))
                break
-            return self.__extract_info(url, ie, download, extra_info, process)
+            return self.__extract_info(url, self.get_info_extractor(ie_key), download, extra_info, process)
        else:
            self.report_error('no suitable InfoExtractor for URL %s' % url)

@@ -1251,7 +1266,7 @@ class YoutubeDL(object):
                'extractor_key': ie.ie_key(),
            })

-    def process_ie_result(self, ie_result, download=True, extra_info={}):
+    def process_ie_result(self, ie_result, download=True, extra_info=None):
        """
        Take the result of the ie(may be modified) and resolve all unresolved
        references (URLs, playlist items).
@@ -1259,6 +1274,8 @@ class YoutubeDL(object):
        It will also download the videos if 'download'.
        Returns the resolved ie_result.
        """
+        if extra_info is None:
+            extra_info = {}
        result_type = ie_result.get('_type', 'video')

        if result_type in ('url', 'url_transparent'):
@@ -1449,7 +1466,7 @@ class YoutubeDL(object):

        # Save playlist_index before re-ordering
        entries = [
-            ((playlistitems[i - 1] if playlistitems else i), entry)
+            ((playlistitems[i - 1] if playlistitems else i + playliststart - 1), entry)
            for i, entry in enumerate(entries, 1)
            if entry is not None]
        n_entries = len(entries)
@@ -1514,7 +1531,7 @@ class YoutubeDL(object):
        max_failures = self.params.get('skip_playlist_after_errors') or float('inf')
        for i, entry_tuple in enumerate(entries, 1):
            playlist_index, entry = entry_tuple
-            if 'playlist_index' in self.params.get('compat_options', []):
+            if 'playlist-index' in self.params.get('compat_options', []):
                playlist_index = playlistitems[i - 1] if playlistitems else i
            self.to_screen('[download] Downloading video %s of %s' % (i, n_entries))
            # This __x_forwarded_for_ip thing is a bit ugly but requires
@@ -2050,7 +2067,8 @@ class YoutubeDL(object):
        if 'id' not in info_dict:
            raise ExtractorError('Missing "id" field in extractor result')
        if 'title' not in info_dict:
-            raise ExtractorError('Missing "title" field in extractor result')
+            raise ExtractorError('Missing "title" field in extractor result',
+                                 video_id=info_dict['id'], ie=info_dict['extractor'])

        def report_force_conversion(field, field_not, conversion):
            self.report_warning(
@@ -2151,11 +2169,12 @@ class YoutubeDL(object):
        else:
            formats = info_dict['formats']

+        info_dict['__has_drm'] = any(f.get('has_drm') for f in formats)
+        if not self.params.get('allow_unplayable_formats'):
+            formats = [f for f in formats if not f.get('has_drm')]
+
        if not formats:
-            if not self.params.get('ignore_no_formats_error'):
-                raise ExtractorError('No video formats found!')
-            else:
-                self.report_warning('No video formats found!')
+            self.raise_no_formats(info_dict)

        def is_wellformed(f):
            url = f.get('url')
@@ -2219,7 +2238,7 @@ class YoutubeDL(object):

        # TODO Central sorting goes here

-        if formats and formats[0] is not info_dict:
+        if not formats or formats[0] is not info_dict:
            # only set the 'formats' fields if the original info_dict list them
            # otherwise we end up with a circular reference, the first (and unique)
            # element in the 'formats' field in info_dict is info_dict itself,
@@ -2231,9 +2250,10 @@ class YoutubeDL(object):
        if self.params.get('list_thumbnails'):
            self.list_thumbnails(info_dict)
        if self.params.get('listformats'):
-            if not info_dict.get('formats'):
-                raise ExtractorError('No video formats found', expected=True)
-            self.list_formats(info_dict)
+            if not info_dict.get('formats') and not info_dict.get('url'):
+                self.to_screen('%s has no formats' % info_dict['id'])
+            else:
+                self.list_formats(info_dict)
        if self.params.get('listsubtitles'):
            if 'automatic_captions' in info_dict:
                self.list_subtitles(
@@ -2281,7 +2301,8 @@ class YoutubeDL(object):
        formats_to_download = list(format_selector(ctx))
        if not formats_to_download:
            if not self.params.get('ignore_no_formats_error'):
-                raise ExtractorError('Requested format is not available', expected=True)
+                raise ExtractorError('Requested format is not available', expected=True,
+                                     video_id=info_dict['id'], ie=info_dict['extractor'])
            else:
                self.report_warning('Requested format is not available')
                # Process what we can, even without any available formats.
@@ -2410,6 +2431,8 @@ class YoutubeDL(object):
            self.to_stdout(json.dumps(self.sanitize_info(info_dict)))

    def dl(self, name, info, subtitle=False, test=False):
+        if not info.get('url'):
+            self.raise_no_formats(info, True)

        if test:
            verbose = self.params.get('verbose')
@@ -2663,7 +2686,6 @@ class YoutubeDL(object):
                            os.remove(encodeFilename(file))
                        return None

-                    self.report_file_already_downloaded(existing_files[0])
                    info_dict['ext'] = os.path.splitext(existing_files[0])[1][1:]
                    return existing_files[0]

@@ -2718,7 +2740,7 @@ class YoutubeDL(object):
                        info_dict['protocol'] = _protocols.pop()
                    directly_mergable = FFmpegFD.can_merge_formats(info_dict)
                    if dl_filename is not None:
-                        pass
+                        self.report_file_already_downloaded(dl_filename)
                    elif (directly_mergable and get_suitable_downloader(
                            info_dict, self.params, to_stdout=(temp_filename == '-')) == FFmpegFD):
                        info_dict['url'] = '\n'.join(f['url'] for f in requested_formats)
@@ -2770,9 +2792,13 @@ class YoutubeDL(object):
                else:
                    # Just a single file
                    dl_filename = existing_file(full_filename, temp_filename)
-                    if dl_filename is None:
+                    if dl_filename is None or dl_filename == temp_filename:
+                        # dl_filename == temp_filename could mean that the file was partially downloaded with --no-part.
+                        # So we should try to resume the download
                        success, real_download = self.dl(temp_filename, info_dict)
                        info_dict['__real_download'] = real_download
+                    else:
+                        self.report_file_already_downloaded(dl_filename)

                dl_filename = dl_filename or temp_filename
                info_dict['__finaldir'] = os.path.dirname(os.path.abspath(encodeFilename(full_filename)))
@@ -2870,13 +2896,13 @@ class YoutubeDL(object):
            except UnavailableVideoError:
                self.report_error('unable to download video')
            except MaxDownloadsReached:
-                self.to_screen('[info] Maximum number of downloaded files reached')
+                self.to_screen('[info] Maximum number of downloads reached')
                raise
            except ExistingVideoReached:
-                self.to_screen('[info] Encountered a file that is already in the archive, stopping due to --break-on-existing')
+                self.to_screen('[info] Encountered a video that is already in the archive, stopping due to --break-on-existing')
                raise
            except RejectedVideoReached:
-                self.to_screen('[info] Encountered a file that did not match filter, stopping due to --break-on-reject')
+                self.to_screen('[info] Encountered a video that did not match filter, stopping due to --break-on-reject')
                raise
            else:
                if self.params.get('dump_single_json', False):
@@ -2905,6 +2931,8 @@ class YoutubeDL(object):
    @staticmethod
    def sanitize_info(info_dict, remove_private_keys=False):
        ''' Sanitize the infodict for converting to json '''
+        if info_dict is None:
+            return info_dict
        info_dict.setdefault('epoch', int(time.time()))
        remove_keys = {'__original_infodict'}  # Always remove this since this may contain a copy of the entire dict
        keep_keys = ['_type'],  # Always keep this to facilitate load-info-json
@@ -3003,9 +3031,9 @@ class YoutubeDL(object):
            if not url:
                return
            # Try to find matching extractor for the URL and take its ie_key
-            for ie in self._ies:
+            for ie_key, ie in self._ies.items():
                if ie.suitable(url):
-                    extractor = ie.ie_key()
+                    extractor = ie_key
                    break
            else:
                return
--- a/yt_dlp/init.py
+++ b/yt_dlp/init.py
@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 # coding: utf-8

-from __future__ import unicode_literals
+f'You are using an unsupported version of Python. Only Python versions 3.6 and above are supported by yt-dlp'  # noqa: F541

 __license__ = 'Public Domain'

@@ -13,7 +13,6 @@ import random
 import re
 import sys

-
 from .options import (
    parseOpts,
 )
@@ -110,14 +109,14 @@ def _real_main(argv=None):

    if opts.list_extractors:
        for ie in list_extractors(opts.age_limit):
-            write_string(ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie._WORKING else '') + '\n', out=sys.stdout)
+            write_string(ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie.working() else '') + '\n', out=sys.stdout)
            matchedUrls = [url for url in all_urls if ie.suitable(url)]
            for mu in matchedUrls:
                write_string('  ' + mu + '\n', out=sys.stdout)
        sys.exit(0)
    if opts.list_extractor_descriptions:
        for ie in list_extractors(opts.age_limit):
-            if not ie._WORKING:
+            if not ie.working():
                continue
            desc = getattr(ie, 'IE_DESC', ie.IE_NAME)
            if desc is False:
@@ -257,35 +256,7 @@ def _real_main(argv=None):
    else:
        date = DateRange(opts.dateafter, opts.datebefore)

-    def parse_compat_opts():
-        parsed_compat_opts, compat_opts = set(), opts.compat_opts[::-1]
-        while compat_opts:
-            actual_opt = opt = compat_opts.pop().lower()
-            if opt == 'youtube-dl':
-                compat_opts.extend(['-multistreams', 'all'])
-            elif opt == 'youtube-dlc':
-                compat_opts.extend(['-no-youtube-channel-redirect', '-no-live-chat', 'all'])
-            elif opt == 'all':
-                parsed_compat_opts.update(all_compat_opts)
-            elif opt == '-all':
-                parsed_compat_opts = set()
-            else:
-                if opt[0] == '-':
-                    opt = opt[1:]
-                    parsed_compat_opts.discard(opt)
-                else:
-                    parsed_compat_opts.update([opt])
-                if opt not in all_compat_opts:
-                    parser.error('Invalid compatibility option %s' % actual_opt)
-        return parsed_compat_opts
-
-    all_compat_opts = [
-        'filename', 'format-sort', 'abort-on-error', 'format-spec', 'no-playlist-metafiles',
-        'multistreams', 'no-live-chat', 'playlist-index', 'list-formats', 'no-direct-merge',
-        'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
-        'embed-thumbnail-atomicparsley', 'seperate-video-versions', 'no-clean-infojson', 'no-keep-subs',
-    ]
-    compat_opts = parse_compat_opts()
+    compat_opts = opts.compat_opts

    def _unused_compat_opt(name):
        if name not in compat_opts:
@@ -335,6 +306,7 @@ def _real_main(argv=None):
    opts.forceprint = opts.forceprint or []
    for tmpl in opts.forceprint or []:
        validate_outtmpl(tmpl, 'print template')
+    validate_outtmpl(opts.sponsorblock_chapter_title, 'SponsorBlock chapter title')

    if opts.extractaudio and not opts.keepvideo and opts.format is None:
        opts.format = 'bestaudio/best'
@@ -381,15 +353,34 @@ def _real_main(argv=None):
    if opts.getcomments and not printing_json:
        opts.writeinfojson = True

+    if opts.no_sponsorblock:
+        opts.sponsorblock_mark = set()
+        opts.sponsorblock_remove = set()
+    sponsorblock_query = opts.sponsorblock_mark | opts.sponsorblock_remove
+
+    if (opts.addmetadata or opts.sponsorblock_mark) and opts.addchapters is None:
+        opts.addchapters = True
+    opts.remove_chapters = opts.remove_chapters or []
+
    def report_conflict(arg1, arg2):
        warnings.append('%s is ignored since %s was given' % (arg2, arg1))

+    if (opts.remove_chapters or sponsorblock_query) and opts.sponskrub is not False:
+        if opts.sponskrub:
+            if opts.remove_chapters:
+                report_conflict('--remove-chapters', '--sponskrub')
+            if opts.sponsorblock_mark:
+                report_conflict('--sponsorblock-mark', '--sponskrub')
+            if opts.sponsorblock_remove:
+                report_conflict('--sponsorblock-remove', '--sponskrub')
+        opts.sponskrub = False
+    if opts.sponskrub_cut and opts.split_chapters and opts.sponskrub is not False:
+        report_conflict('--split-chapter', '--sponskrub-cut')
+        opts.sponskrub_cut = False
+
    if opts.remuxvideo and opts.recodevideo:
        report_conflict('--recode-video', '--remux-video')
        opts.remuxvideo = False
-    if opts.sponskrub_cut and opts.split_chapters and opts.sponskrub is not False:
-        report_conflict('--split-chapter', '--sponskrub-cut')
-        opts.sponskrub_cut = False

    if opts.allow_unplayable_formats:
        if opts.extractaudio:
@@ -416,12 +407,26 @@ def _real_main(argv=None):
        if opts.fixup and opts.fixup.lower() not in ('never', 'ignore'):
            report_conflict('--allow-unplayable-formats', '--fixup')
        opts.fixup = 'never'
+        if opts.remove_chapters:
+            report_conflict('--allow-unplayable-formats', '--remove-chapters')
+            opts.remove_chapters = []
+        if opts.sponsorblock_remove:
+            report_conflict('--allow-unplayable-formats', '--sponsorblock-remove')
+            opts.sponsorblock_remove = set()
        if opts.sponskrub:
            report_conflict('--allow-unplayable-formats', '--sponskrub')
        opts.sponskrub = False

    # PostProcessors
    postprocessors = []
+    if sponsorblock_query:
+        postprocessors.append({
+            'key': 'SponsorBlock',
+            'categories': sponsorblock_query,
+            'api': opts.sponsorblock_api,
+            # Run this immediately after extraction is complete
+            'when': 'pre_process'
+        })
    if opts.parse_metadata:
        postprocessors.append({
            'key': 'MetadataParser',
@@ -467,16 +472,7 @@ def _real_main(argv=None):
            'key': 'FFmpegVideoConvertor',
            'preferedformat': opts.recodevideo,
        })
-    # FFmpegMetadataPP should be run after FFmpegVideoConvertorPP and
-    # FFmpegExtractAudioPP as containers before conversion may not support
-    # metadata (3gp, webm, etc.)
-    # And this post-processor should be placed before other metadata
-    # manipulating post-processors (FFmpegEmbedSubtitle) to prevent loss of
-    # extra metadata. By default ffmpeg preserves metadata applicable for both
-    # source and target containers. From this point the container won't change,
-    # so metadata can be added here.
-    if opts.addmetadata:
-        postprocessors.append({'key': 'FFmpegMetadata'})
+    # If ModifyChapters is going to remove chapters, subtitles must already be in the container.
    if opts.embedsubtitles:
        already_have_subtitle = opts.writesubtitles and 'no-keep-subs' not in compat_opts
        postprocessors.append({
@@ -490,6 +486,33 @@ def _real_main(argv=None):
    # this was the old behaviour if only --all-sub was given.
    if opts.allsubtitles and not opts.writeautomaticsub:
        opts.writesubtitles = True
+    # ModifyChapters must run before FFmpegMetadataPP
+    remove_chapters_patterns = []
+    for regex in opts.remove_chapters:
+        try:
+            remove_chapters_patterns.append(re.compile(regex))
+        except re.error as err:
+            parser.error(f'invalid --remove-chapters regex {regex!r} - {err}')
+    if opts.remove_chapters or sponsorblock_query:
+        postprocessors.append({
+            'key': 'ModifyChapters',
+            'remove_chapters_patterns': remove_chapters_patterns,
+            'remove_sponsor_segments': opts.sponsorblock_remove,
+            'sponsorblock_chapter_title': opts.sponsorblock_chapter_title,
+            'force_keyframes': opts.force_keyframes_at_cuts
+        })
+    # FFmpegMetadataPP should be run after FFmpegVideoConvertorPP and
+    # FFmpegExtractAudioPP as containers before conversion may not support
+    # metadata (3gp, webm, etc.)
+    # By default ffmpeg preserves metadata applicable for both
+    # source and target containers. From this point the container won't change,
+    # so metadata can be added here.
+    if opts.addmetadata or opts.addchapters:
+        postprocessors.append({
+            'key': 'FFmpegMetadata',
+            'add_chapters': opts.addchapters,
+            'add_metadata': opts.addmetadata,
+        })
    # This should be above EmbedThumbnail since sponskrub removes the thumbnail attachment
    # but must be below EmbedSubtitle and FFmpegMetadata
    # See https://github.com/yt-dlp/yt-dlp/issues/204 , https://github.com/faissaloo/SponSkrub/issues/29
@@ -513,7 +536,10 @@ def _real_main(argv=None):
        if not already_have_thumbnail:
            opts.writethumbnail = True
    if opts.split_chapters:
-        postprocessors.append({'key': 'FFmpegSplitChapters'})
+        postprocessors.append({
+            'key': 'FFmpegSplitChapters',
+            'force_keyframes': opts.force_keyframes_at_cuts,
+        })
    # XAttrMetadataPP should be run after post-processors that may change file contents
    if opts.xattrs:
        postprocessors.append({'key': 'XAttrMetadata'})
--- a/yt_dlp/downloader/init.py
+++ b/yt_dlp/downloader/init.py
@@ -94,6 +94,10 @@ def _get_suitable_downloader(info_dict, params, default):
        if ed.can_download(info_dict, external_downloader):
            return ed

+    if protocol == 'http_dash_segments':
+        if info_dict.get('is_live') and (external_downloader or '').lower() != 'native':
+            return FFmpegFD
+
    if protocol in ('m3u8', 'm3u8_native'):
        if info_dict.get('is_live'):
            return FFmpegFD
--- a/yt_dlp/downloader/common.py
+++ b/yt_dlp/downloader/common.py
@@ -204,12 +204,12 @@ class FileDownloader(object):
        return filename + '.ytdl'

    def try_rename(self, old_filename, new_filename):
+        if old_filename == new_filename:
+            return
        try:
-            if old_filename == new_filename:
-                return
-            os.rename(encodeFilename(old_filename), encodeFilename(new_filename))
+            os.replace(old_filename, new_filename)
        except (IOError, OSError) as err:
-            self.report_error('unable to rename file: %s' % error_to_compat_str(err))
+            self.report_error(f'unable to rename file: {err}')

    def try_utime(self, filename, last_modified_hdr):
        """Try to set the last-modified time of the given file."""
--- a/yt_dlp/downloader/external.py
+++ b/yt_dlp/downloader/external.py
@@ -22,7 +22,7 @@ from ..utils import (
    cli_option,
    cli_valueless_option,
    cli_bool_option,
-    cli_configuration_args,
+    _configuration_args,
    encodeFilename,
    encodeArgument,
    handle_youtubedl_headers,
@@ -111,11 +111,10 @@ class ExternalFD(FileDownloader):
    def _valueless_option(self, command_option, param, expected_value=True):
        return cli_valueless_option(self.params, command_option, param, expected_value)

-    def _configuration_args(self, *args, **kwargs):
-        return cli_configuration_args(
-            self.params.get('external_downloader_args'),
-            [self.get_basename(), 'default'],
-            *args, **kwargs)
+    def _configuration_args(self, keys=None, *args, **kwargs):
+        return _configuration_args(
+            self.get_basename(), self.params.get('external_downloader_args'), self.get_basename(),
+            keys, *args, **kwargs)

    def _call_downloader(self, tmpfilename, info_dict):
        """ Either overwrite this or implement _make_cmd """
@@ -289,6 +288,7 @@ class Aria2cFD(ExternalFD):
        if info_dict.get('http_headers') is not None:
            for key, val in info_dict['http_headers'].items():
                cmd += ['--header', '%s: %s' % (key, val)]
+        cmd += self._option('--max-overall-download-limit', 'ratelimit')
        cmd += self._option('--interface', 'source_address')
        cmd += self._option('--all-proxy', 'proxy')
        cmd += self._bool_option('--check-certificate', 'nocheckcertificate', 'false', 'true', '=')
@@ -343,7 +343,7 @@ class HttpieFD(ExternalFD):


 class FFmpegFD(ExternalFD):
-    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'm3u8', 'm3u8_native', 'rtsp', 'rtmp', 'rtmp_ffmpeg', 'mms')
+    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'm3u8', 'm3u8_native', 'rtsp', 'rtmp', 'rtmp_ffmpeg', 'mms', 'http_dash_segments')
    can_download_to_stdout = True

    @classmethod
@@ -459,16 +459,15 @@ class FFmpegFD(ExternalFD):
            elif isinstance(conn, compat_str):
                args += ['-rtmp_conn', conn]

-        for url in urls:
-            args += ['-i', url]
+        for i, url in enumerate(urls):
+            args += self._configuration_args((f'_i{i + 1}', '_i')) + ['-i', url]

-        args += self._configuration_args() + ['-c', 'copy']
-        if info_dict.get('requested_formats'):
-            for (i, fmt) in enumerate(info_dict['requested_formats']):
-                if fmt.get('acodec') != 'none':
-                    args.extend(['-map', '%d:a:0' % i])
-                if fmt.get('vcodec') != 'none':
-                    args.extend(['-map', '%d:v:0' % i])
+        args += ['-c', 'copy']
+        if info_dict.get('requested_formats') or protocol == 'http_dash_segments':
+            for (i, fmt) in enumerate(info_dict.get('requested_formats') or [info_dict]):
+                stream_number = fmt.get('manifest_stream_number', 0)
+                a_or_v = 'a' if fmt.get('acodec') != 'none' else 'v'
+                args.extend(['-map', f'{i}:{a_or_v}:{stream_number}'])

        if self.params.get('test', False):
            args += ['-fs', compat_str(self._TEST_FILE_SIZE)]
@@ -491,9 +490,10 @@ class FFmpegFD(ExternalFD):
        else:
            args += ['-f', EXT_TO_OUT_FORMATS.get(ext, ext)]

+        args += self._configuration_args(('_o1', '_o', ''))
+
        args = [encodeArgument(opt) for opt in args]
        args.append(encodeFilename(ffpp._ffmpeg_filename_argument(tmpfilename), True))
-
        self._debug_cmd(args)

        proc = subprocess.Popen(args, stdin=subprocess.PIPE, env=env)
--- a/yt_dlp/downloader/hls.py
+++ b/yt_dlp/downloader/hls.py
@@ -254,8 +254,14 @@ class HlsFD(FragmentFD):
            def pack_fragment(frag_content, frag_index):
                output = io.StringIO()
                adjust = 0
+                overflow = False
+                mpegts_last = None
                for block in webvtt.parse_fragment(frag_content):
                    if isinstance(block, webvtt.CueBlock):
+                        extra_state['webvtt_mpegts_last'] = mpegts_last
+                        if overflow:
+                            extra_state['webvtt_mpegts_adjust'] += 1
+                            overflow = False
                        block.start += adjust
                        block.end += adjust

@@ -296,9 +302,9 @@ class HlsFD(FragmentFD):
                        extra_state.setdefault('webvtt_mpegts_adjust', 0)
                        block.mpegts += extra_state['webvtt_mpegts_adjust'] << 33
                        if block.mpegts < extra_state.get('webvtt_mpegts_last', 0):
-                            extra_state['webvtt_mpegts_adjust'] += 1
+                            overflow = True
                            block.mpegts += 1 << 33
-                        extra_state['webvtt_mpegts_last'] = block.mpegts
+                        mpegts_last = block.mpegts

                        if frag_index == 1:
                            extra_state['webvtt_mpegts'] = block.mpegts or 0
--- a/yt_dlp/downloader/http.py
+++ b/yt_dlp/downloader/http.py
@@ -238,7 +238,7 @@ class HttpFD(FileDownloader):
            while True:
                try:
                    # Download and write
-                    data_block = ctx.data.read(block_size if data_len is None else min(block_size, data_len - byte_counter))
+                    data_block = ctx.data.read(block_size if not is_test else min(block_size, data_len - byte_counter))
                # socket.timeout is a subclass of socket.error but may not have
                # errno set
                except socket.timeout as e:
--- a/yt_dlp/extractor/abcnews.py
+++ b/yt_dlp/extractor/abcnews.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .amp import AMPIE
 from .common import InfoExtractor
@@ -59,7 +58,7 @@ class AbcNewsVideoIE(AMPIE):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        display_id = mobj.group('display_id')
        video_id = mobj.group('id')
        info_dict = self._extract_feed_info(
--- a/yt_dlp/extractor/abcotvs.py
+++ b/yt_dlp/extractor/abcotvs.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..compat import compat_str
@@ -55,7 +54,7 @@ class ABCOTVSIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        site, display_id, video_id = re.match(self._VALID_URL, url).groups()
+        site, display_id, video_id = self._match_valid_url(url).groups()
        display_id = display_id or video_id
        station = self._SITE_MAP[site]

--- a/yt_dlp/extractor/acast.py
+++ b/yt_dlp/extractor/acast.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -80,7 +79,7 @@ class ACastIE(ACastBaseIE):
    }]

    def _real_extract(self, url):
-        channel, display_id = re.match(self._VALID_URL, url).groups()
+        channel, display_id = self._match_valid_url(url).groups()
        episode = self._call_api(
            '%s/episodes/%s' % (channel, display_id),
            display_id, {'showInfo': 'true'})
--- a/yt_dlp/extractor/adobepass.py
+++ b/yt_dlp/extractor/adobepass.py
@@ -1508,7 +1508,8 @@ class AdobePassIE(InfoExtractor):
                    # In general, if you're connecting from a Verizon-assigned IP,
                    # you will not actually pass your credentials.
                    provider_redirect_page, urlh = provider_redirect_page_res
-                    if 'Please wait ...' in provider_redirect_page:
+                    # From non-Verizon IP, still gave 'Please wait', but noticed N==Y; will need to try on Verizon IP
+                    if 'Please wait ...' in provider_redirect_page and '\'N\'== "Y"' not in provider_redirect_page:
                        saml_redirect_url = self._html_search_regex(
                            r'self\.parent\.location=(["\'])(?P<url>.+?)\1',
                            provider_redirect_page,
@@ -1516,7 +1517,8 @@ class AdobePassIE(InfoExtractor):
                        saml_login_page = self._download_webpage(
                            saml_redirect_url, video_id,
                            'Downloading SAML Login Page')
-                    else:
+                    elif 'Verizon FiOS - sign in' in provider_redirect_page:
+                        # FXNetworks from non-Verizon IP
                        saml_login_page_res = post_form(
                            provider_redirect_page_res, 'Logging in', {
                                mso_info['username_field']: username,
@@ -1526,6 +1528,26 @@ class AdobePassIE(InfoExtractor):
                        if 'Please try again.' in saml_login_page:
                            raise ExtractorError(
                                'We\'re sorry, but either the User ID or Password entered is not correct.')
+                    else:
+                        # ABC from non-Verizon IP
+                        saml_redirect_url = self._html_search_regex(
+                            r'var\surl\s*=\s*(["\'])(?P<url>.+?)\1',
+                            provider_redirect_page,
+                            'SAML Redirect URL', group='url')
+                        saml_redirect_url = saml_redirect_url.replace(r'\/', '/')
+                        saml_redirect_url = saml_redirect_url.replace(r'\-', '-')
+                        saml_redirect_url = saml_redirect_url.replace(r'\x26', '&')
+                        saml_login_page = self._download_webpage(
+                            saml_redirect_url, video_id,
+                            'Downloading SAML Login Page')
+                        saml_login_page, urlh = post_form(
+                            [saml_login_page, saml_redirect_url], 'Logging in', {
+                                mso_info['username_field']: username,
+                                mso_info['password_field']: password,
+                            })
+                        if 'Please try again.' in saml_login_page:
+                            raise ExtractorError(
+                                'Failed to login, incorrect User ID or Password.')
                    saml_login_url = self._search_regex(
                        r'xmlHttp\.open\("POST"\s*,\s*(["\'])(?P<url>.+?)\1',
                        saml_login_page, 'SAML Login URL', group='url')
--- a/yt_dlp/extractor/adobetv.py
+++ b/yt_dlp/extractor/adobetv.py
@@ -132,7 +132,7 @@ class AdobeTVIE(AdobeTVBaseIE):
    }

    def _real_extract(self, url):
-        language, show_urlname, urlname = re.match(self._VALID_URL, url).groups()
+        language, show_urlname, urlname = self._match_valid_url(url).groups()
        if not language:
            language = 'en'

@@ -178,7 +178,7 @@ class AdobeTVShowIE(AdobeTVPlaylistBaseIE):
    _process_data = AdobeTVBaseIE._parse_video_data

    def _real_extract(self, url):
-        language, show_urlname = re.match(self._VALID_URL, url).groups()
+        language, show_urlname = self._match_valid_url(url).groups()
        if not language:
            language = 'en'
        query = {
@@ -215,7 +215,7 @@ class AdobeTVChannelIE(AdobeTVPlaylistBaseIE):
            show_data['url'], 'AdobeTVShow', str_or_none(show_data.get('id')))

    def _real_extract(self, url):
-        language, channel_urlname, category_urlname = re.match(self._VALID_URL, url).groups()
+        language, channel_urlname, category_urlname = self._match_valid_url(url).groups()
        if not language:
            language = 'en'
        query = {
--- a/yt_dlp/extractor/adultswim.py
+++ b/yt_dlp/extractor/adultswim.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 import json
-import re

 from .turner import TurnerBaseIE
 from ..utils import (
@@ -89,7 +88,7 @@ class AdultSwimIE(TurnerBaseIE):
    }]

    def _real_extract(self, url):
-        show_path, episode_path = re.match(self._VALID_URL, url).groups()
+        show_path, episode_path = self._match_valid_url(url).groups()
        display_id = episode_path or show_path
        query = '''query {
  getShowBySlug(slug:"%s") {
--- a/yt_dlp/extractor/aenetworks.py
+++ b/yt_dlp/extractor/aenetworks.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .theplatform import ThePlatformIE
 from ..utils import (
@@ -170,7 +169,7 @@ class AENetworksIE(AENetworksBaseIE):
    }]

    def _real_extract(self, url):
-        domain, canonical = re.match(self._VALID_URL, url).groups()
+        domain, canonical = self._match_valid_url(url).groups()
        return self._extract_aetn_info(domain, 'canonical', '/' + canonical, url)


@@ -187,7 +186,7 @@ class AENetworksListBaseIE(AENetworksBaseIE):
            }))['data'][resource]

    def _real_extract(self, url):
-        domain, slug = re.match(self._VALID_URL, url).groups()
+        domain, slug = self._match_valid_url(url).groups()
        _, brand = self._DOMAIN_MAP[domain]
        playlist = self._call_api(self._RESOURCE, slug, brand, self._FIELDS)
        base_url = 'http://watch.%s' % domain
@@ -309,7 +308,7 @@ class HistoryPlayerIE(AENetworksBaseIE):
    _TESTS = []

    def _real_extract(self, url):
-        domain, video_id = re.match(self._VALID_URL, url).groups()
+        domain, video_id = self._match_valid_url(url).groups()
        return self._extract_aetn_info(domain, 'id', video_id, url)


--- a/yt_dlp/extractor/afreecatv.py
+++ b/yt_dlp/extractor/afreecatv.py
@@ -6,9 +6,11 @@ import re
 from .common import InfoExtractor
 from ..compat import compat_xpath
 from ..utils import (
+    date_from_str,
    determine_ext,
    ExtractorError,
    int_or_none,
+    unified_strdate,
    url_or_none,
    urlencode_postdata,
    xpath_text,
@@ -237,6 +239,7 @@ class AfreecaTVIE(InfoExtractor):
            r'nTitleNo\s*=\s*(\d+)', webpage, 'title', default=video_id)

        partial_view = False
+        adult_view = False
        for _ in range(2):
            query = {
                'nTitleNo': video_id,
@@ -245,6 +248,8 @@ class AfreecaTVIE(InfoExtractor):
            }
            if partial_view:
                query['partialView'] = 'SKIP_ADULT'
+            if adult_view:
+                query['adultView'] = 'ADULT_VIEW'
            video_xml = self._download_xml(
                'http://afbbs.afreecatv.com:8080/api/video/get_video_info.php',
                video_id, 'Downloading video info XML%s'
@@ -264,6 +269,9 @@ class AfreecaTVIE(InfoExtractor):
                partial_view = True
                continue
            elif flag == 'ADULT':
+                if not adult_view:
+                    adult_view = True
+                    continue
                error = 'Only users older than 19 are able to watch this video. Provide account credentials to download this content.'
            else:
                error = flag
@@ -309,8 +317,15 @@ class AfreecaTVIE(InfoExtractor):
                if not file_url:
                    continue
                key = file_element.get('key', '')
-                upload_date = self._search_regex(
-                    r'^(\d{8})_', key, 'upload date', default=None)
+                upload_date = unified_strdate(self._search_regex(
+                    r'^(\d{8})_', key, 'upload date', default=None))
+                if upload_date is not None:
+                    # sometimes the upload date isn't included in the file name
+                    # instead, another random ID is, which may parse as a valid
+                    # date but be wildly out of a reasonable range
+                    parsed_date = date_from_str(upload_date)
+                    if parsed_date.year < 2000 or parsed_date.year >= 2100:
+                        upload_date = None
                file_duration = int_or_none(file_element.get('duration'))
                format_id = key if key else '%s_%s' % (video_id, file_num)
                if determine_ext(file_url) == 'm3u8':
--- a/yt_dlp/extractor/aljazeera.py
+++ b/yt_dlp/extractor/aljazeera.py
@@ -1,7 +1,6 @@
 from __future__ import unicode_literals

 import json
-import re

 from .common import InfoExtractor

@@ -32,7 +31,7 @@ class AlJazeeraIE(InfoExtractor):
    BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/%s/%s_default/index.html?videoId=%s'

    def _real_extract(self, url):
-        post_type, name = re.match(self._VALID_URL, url).groups()
+        post_type, name = self._match_valid_url(url).groups()
        post_type = {
            'features': 'post',
            'program': 'episode',
@@ -40,7 +39,7 @@ class AlJazeeraIE(InfoExtractor):
        }[post_type.split('/')[0]]
        video = self._download_json(
            'https://www.aljazeera.com/graphql', name, query={
-                'operationName': 'SingleArticleQuery',
+                'operationName': 'ArchipelagoSingleArticleQuery',
                'variables': json.dumps({
                    'name': name,
                    'postType': post_type,
--- a/yt_dlp/extractor/alura.py
+++ b/yt_dlp/extractor/alura.py
@@ -42,8 +42,7 @@ class AluraIE(InfoExtractor):

    def _real_extract(self, url):

-        video_id = self._match_id(url)
-        course = self._search_regex(self._VALID_URL, url, 'post url', group='course_name')
+        course, video_id = self._match_valid_url(url)
        video_url = self._VIDEO_URL % (course, video_id)

        video_dict = self._download_json(video_url, video_id, 'Searching for videos')
--- a/yt_dlp/extractor/amcnetworks.py
+++ b/yt_dlp/extractor/amcnetworks.py
@@ -63,7 +63,7 @@ class AMCNetworksIE(ThePlatformIE):
    }

    def _real_extract(self, url):
-        site, display_id = re.match(self._VALID_URL, url).groups()
+        site, display_id = self._match_valid_url(url).groups()
        requestor_id = self._REQUESTOR_ID_MAP[site]
        page_data = self._download_json(
            'https://content-delivery-gw.svc.ds.amcn.com/api/v2/content/amcn/%s/url/%s'
--- a/yt_dlp/extractor/americastestkitchen.py
+++ b/yt_dlp/extractor/americastestkitchen.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 import json
-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -69,7 +68,7 @@ class AmericasTestKitchenIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        resource_type, video_id = re.match(self._VALID_URL, url).groups()
+        resource_type, video_id = self._match_valid_url(url).groups()
        is_episode = resource_type == 'episode'
        if is_episode:
            resource_type = 'episodes'
@@ -114,7 +113,7 @@ class AmericasTestKitchenSeasonIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        show_name, season_number = re.match(self._VALID_URL, url).groups()
+        show_name, season_number = self._match_valid_url(url).groups()
        season_number = int(season_number)

        slug = 'atk' if show_name == 'americastestkitchen' else 'cco'
--- a/yt_dlp/extractor/anvato.py
+++ b/yt_dlp/extractor/anvato.py
@@ -390,7 +390,7 @@ class AnvatoIE(InfoExtractor):
            'countries': smuggled_data.get('geo_countries'),
        })

-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        access_key, video_id = mobj.group('access_key_or_mcp', 'id')
        if access_key not in self._ANVACK_TABLE:
            access_key = self._MCP_TO_ACCESS_KEY_TABLE.get(
--- a/yt_dlp/extractor/aol.py
+++ b/yt_dlp/extractor/aol.py
@@ -4,13 +4,10 @@ from __future__ import unicode_literals
 import re

 from .yahoo import YahooIE
-from ..compat import (
-    compat_parse_qs,
-    compat_urllib_parse_urlparse,
-)
 from ..utils import (
    ExtractorError,
    int_or_none,
+    parse_qs,
    url_or_none,
 )

@@ -119,7 +116,7 @@ class AolIE(YahooIE):
                        'height': int(mobj.group(2)),
                    })
                else:
-                    qs = compat_parse_qs(compat_urllib_parse_urlparse(video_url).query)
+                    qs = parse_qs(video_url)
                    f.update({
                        'width': int_or_none(qs.get('w', [None])[0]),
                        'height': int_or_none(qs.get('h', [None])[0]),
--- a/yt_dlp/extractor/apa.py
+++ b/yt_dlp/extractor/apa.py
@@ -42,7 +42,7 @@ class APAIE(InfoExtractor):
                webpage)]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id, base_url = mobj.group('id', 'base_url')

        webpage = self._download_webpage(
--- a/yt_dlp/extractor/appletrailers.py
+++ b/yt_dlp/extractor/appletrailers.py
@@ -94,7 +94,7 @@ class AppleTrailersIE(InfoExtractor):
    _JSON_RE = r'iTunes.playURL\((.*?)\);'

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        movie = mobj.group('movie')
        uploader_id = mobj.group('company')

--- a/yt_dlp/extractor/archiveorg.py
+++ b/yt_dlp/extractor/archiveorg.py
@@ -9,8 +9,6 @@ from .youtube import YoutubeIE
 from ..compat import (
    compat_urllib_parse_unquote,
    compat_urllib_parse_unquote_plus,
-    compat_urlparse,
-    compat_parse_qs,
    compat_HTTPError
 )
 from ..utils import (
@@ -25,6 +23,7 @@ from ..utils import (
    merge_dicts,
    mimetype2ext,
    parse_duration,
+    parse_qs,
    RegexNotFoundError,
    str_to_int,
    str_or_none,
@@ -399,7 +398,7 @@ class YoutubeWebArchiveIE(InfoExtractor):
                    expected=True)
            raise
        video_file_url = compat_urllib_parse_unquote(video_file_webpage.url)
-        video_file_url_qs = compat_parse_qs(compat_urlparse.urlparse(video_file_url).query)
+        video_file_url_qs = parse_qs(video_file_url)

        # Attempt to recover any ext & format info from playback url
        format = {'url': video_file_url}
--- a/yt_dlp/extractor/arcpublishing.py
+++ b/yt_dlp/extractor/arcpublishing.py
@@ -86,7 +86,7 @@ class ArcPublishingIE(InfoExtractor):
        return entries

    def _real_extract(self, url):
-        org, uuid = re.match(self._VALID_URL, url).groups()
+        org, uuid = self._match_valid_url(url).groups()
        for orgs, tmpl in self._POWA_DEFAULTS:
            if org in orgs:
                base_api_tmpl = tmpl
--- a/yt_dlp/extractor/ard.py
+++ b/yt_dlp/extractor/ard.py
@@ -199,7 +199,7 @@ class ARDMediathekIE(ARDMediathekBaseIE):

    def _real_extract(self, url):
        # determine video id from url
-        m = re.match(self._VALID_URL, url)
+        m = self._match_valid_url(url)

        document_id = None

@@ -325,7 +325,7 @@ class ARDIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        display_id = mobj.group('id')

        player_url = mobj.group('mainurl') + '~playerXml.xml'
@@ -525,7 +525,7 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
        return self.playlist_result(entries, playlist_title=display_id)

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('video_id')
        display_id = mobj.group('display_id')
        if display_id:
--- a/yt_dlp/extractor/arkena.py
+++ b/yt_dlp/extractor/arkena.py
@@ -4,12 +4,12 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
-from ..compat import compat_urlparse
 from ..utils import (
    ExtractorError,
    float_or_none,
    int_or_none,
    parse_iso8601,
+    parse_qs,
    try_get,
 )

@@ -63,13 +63,13 @@ class ArkenaIE(InfoExtractor):
            return mobj.group('url')

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('id')
        account_id = mobj.group('account_id')

        # Handle http://video.arkena.com/play2/embed/player URL
        if not video_id:
-            qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
+            qs = parse_qs(url)
            video_id = qs.get('mediaId', [None])[0]
            account_id = qs.get('accountId', [None])[0]
            if not video_id or not account_id:
--- a/yt_dlp/extractor/arte.py
+++ b/yt_dlp/extractor/arte.py
@@ -6,11 +6,11 @@ import re
 from .common import InfoExtractor
 from ..compat import (
    compat_str,
-    compat_urlparse,
 )
 from ..utils import (
    ExtractorError,
    int_or_none,
+    parse_qs,
    qualities,
    try_get,
    unified_strdate,
@@ -49,7 +49,7 @@ class ArteTVIE(ArteTVBaseIE):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('id')
        lang = mobj.group('lang') or mobj.group('lang_2')

@@ -204,7 +204,7 @@ class ArteTVEmbedIE(InfoExtractor):
            webpage)]

    def _real_extract(self, url):
-        qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
+        qs = parse_qs(url)
        json_url = qs['json_url'][0]
        video_id = ArteTVIE._match_id(json_url)
        return self.url_result(
@@ -227,7 +227,7 @@ class ArteTVPlaylistIE(ArteTVBaseIE):
    }]

    def _real_extract(self, url):
-        lang, playlist_id = re.match(self._VALID_URL, url).groups()
+        lang, playlist_id = self._match_valid_url(url).groups()
        collection = self._download_json(
            '%s/collectionData/%s/%s?source=videos'
            % (self._API_BASE, lang, playlist_id), playlist_id)
--- a/yt_dlp/extractor/asiancrush.py
+++ b/yt_dlp/extractor/asiancrush.py
@@ -111,7 +111,7 @@ class AsianCrushIE(AsianCrushBaseIE):
    }]

    def _real_extract(self, url):
-        host, video_id = re.match(self._VALID_URL, url).groups()
+        host, video_id = self._match_valid_url(url).groups()

        if host == 'cocoro.tv':
            webpage = self._download_webpage(url, video_id)
@@ -161,7 +161,7 @@ class AsianCrushPlaylistIE(AsianCrushBaseIE):
            yield self._parse_video_data(video)

    def _real_extract(self, url):
-        host, playlist_id = re.match(self._VALID_URL, url).groups()
+        host, playlist_id = self._match_valid_url(url).groups()

        if host == 'cocoro.tv':
            webpage = self._download_webpage(url, playlist_id)
--- a/yt_dlp/extractor/atresplayer.py
+++ b/yt_dlp/extractor/atresplayer.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..compat import compat_HTTPError
@@ -75,7 +74,7 @@ class AtresPlayerIE(InfoExtractor):
        self._request_webpage(target_url, None, 'Following Target URL')

    def _real_extract(self, url):
-        display_id, video_id = re.match(self._VALID_URL, url).groups()
+        display_id, video_id = self._match_valid_url(url).groups()

        try:
            episode = self._download_json(
--- a/yt_dlp/extractor/atvat.py
+++ b/yt_dlp/extractor/atvat.py
@@ -4,6 +4,7 @@ from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..utils import (
    determine_ext,
+    dict_get,
    int_or_none,
    unescapeHTML,
 )
@@ -12,64 +13,62 @@ from ..utils import (
 class ATVAtIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?atv\.at/(?:[^/]+/){2}(?P<id>[dv]\d+)'
    _TESTS = [{
-        'url': 'http://atv.at/aktuell/di-210317-2005-uhr/v1698449/',
-        'md5': 'c3b6b975fb3150fc628572939df205f2',
+        'url': 'https://www.atv.at/bauer-sucht-frau-die-zweite-chance/folge-1/d3390693/',
+        'md5': 'c471605591009dfb6e6c54f7e62e2807',
        'info_dict': {
-            'id': '1698447',
+            'id': '3390684',
            'ext': 'mp4',
-            'title': 'DI, 21.03.17 | 20:05 Uhr 1/1',
+            'title': 'Bauer sucht Frau - Die zweite Chance Folge 1',
        }
    }, {
-        'url': 'http://atv.at/aktuell/meinrad-knapp/d8416/',
+        'url': 'https://www.atv.at/bauer-sucht-frau-staffel-17/fuenfte-eventfolge/d3339537/',
        'only_matching': True,
    }]

+    def _process_source_entry(self, source, part_id):
+        source_url = source.get('url')
+        if not source_url:
+            return
+        if determine_ext(source_url) == 'm3u8':
+            return self._extract_m3u8_formats(
+                source_url, part_id, 'mp4', 'm3u8_native',
+                m3u8_id='hls', fatal=False)
+        else:
+            return [{
+                'url': source_url,
+            }]
+
+    def _process_entry(self, entry):
+        part_id = entry.get('id')
+        if not part_id:
+            return
+        formats = []
+        for source in entry.get('sources', []):
+            formats.extend(self._process_source_entry(source, part_id) or [])
+
+        self._sort_formats(formats)
+        return {
+            'id': part_id,
+            'title': entry.get('title'),
+            'duration': int_or_none(entry.get('duration')),
+            'formats': formats
+        }
+
    def _real_extract(self, url):
        display_id = self._match_id(url)
        webpage = self._download_webpage(url, display_id)
        video_data = self._parse_json(unescapeHTML(self._search_regex(
-            [r'flashPlayerOptions\s*=\s*(["\'])(?P<json>(?:(?!\1).)+)\1',
-             r'class="[^"]*jsb_video/FlashPlayer[^"]*"[^>]+data-jsb="(?P<json>[^"]+)"'],
+            r'var\splaylist\s*=\s*(?P<json>\[.*\]);',
            webpage, 'player data', group='json')),
-            display_id)['config']['initial_video']
+            display_id)

-        video_id = video_data['id']
-        video_title = video_data['title']
-
-        parts = []
-        for part in video_data.get('parts', []):
-            part_id = part['id']
-            part_title = part['title']
-
-            formats = []
-            for source in part.get('sources', []):
-                source_url = source.get('src')
-                if not source_url:
-                    continue
-                ext = determine_ext(source_url)
-                if ext == 'm3u8':
-                    formats.extend(self._extract_m3u8_formats(
-                        source_url, part_id, 'mp4', 'm3u8_native',
-                        m3u8_id='hls', fatal=False))
-                else:
-                    formats.append({
-                        'format_id': source.get('delivery'),
-                        'url': source_url,
-                    })
-            self._sort_formats(formats)
-
-            parts.append({
-                'id': part_id,
-                'title': part_title,
-                'thumbnail': part.get('preview_image_url'),
-                'duration': int_or_none(part.get('duration')),
-                'is_live': part.get('is_livestream'),
-                'formats': formats,
-            })
+        first_video = video_data[0]
+        video_id = first_video['id']
+        video_title = dict_get(first_video, ('tvShowTitle', 'title'))

        return {
            '_type': 'multi_video',
            'id': video_id,
            'title': video_title,
-            'entries': parts,
+            'entries': (self._process_entry(entry) for entry in video_data),
        }
--- a/yt_dlp/extractor/audius.py
+++ b/yt_dlp/extractor/audius.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 import random
-import re

 from .common import InfoExtractor
 from ..utils import ExtractorError, try_get, compat_str, str_or_none
@@ -124,7 +123,7 @@ class AudiusIE(AudiusBaseIE):
    }

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        track_id = try_get(mobj, lambda x: x.group('track_id'))
        if track_id is None:
            title = mobj.group('title')
@@ -217,7 +216,7 @@ class AudiusPlaylistIE(AudiusBaseIE):

    def _real_extract(self, url):
        self._select_api_base()
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        title = mobj.group('title')
        # uploader = mobj.group('uploader')
        url = self._prepare_url(url, title)
--- a/yt_dlp/extractor/awaan.py
+++ b/yt_dlp/extractor/awaan.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re
 import base64

 from .common import InfoExtractor
@@ -22,7 +21,7 @@ class AWAANIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?(?:awaan|dcndigital)\.ae/(?:#/)?show/(?P<show_id>\d+)/[^/]+(?:/(?P<id>\d+)/(?P<season_id>\d+))?'

    def _real_extract(self, url):
-        show_id, video_id, season_id = re.match(self._VALID_URL, url).groups()
+        show_id, video_id, season_id = self._match_valid_url(url).groups()
        if video_id and int(video_id) > 0:
            return self.url_result(
                'http://awaan.ae/media/%s' % video_id, 'AWAANVideo')
@@ -154,7 +153,7 @@ class AWAANSeasonIE(InfoExtractor):

    def _real_extract(self, url):
        url, smuggled_data = unsmuggle_url(url, {})
-        show_id, season_id = re.match(self._VALID_URL, url).groups()
+        show_id, season_id = self._match_valid_url(url).groups()

        data = {}
        if season_id:
--- a/yt_dlp/extractor/azmedien.py
+++ b/yt_dlp/extractor/azmedien.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 import json
-import re

 from .common import InfoExtractor
 from .kaltura import KalturaIE
@@ -51,7 +50,7 @@ class AZMedienIE(InfoExtractor):
    _PARTNER_ID = '1719221'

    def _real_extract(self, url):
-        host, display_id, article_id, entry_id = re.match(self._VALID_URL, url).groups()
+        host, display_id, article_id, entry_id = self._match_valid_url(url).groups()

        if not entry_id:
            entry_id = self._download_json(
--- a/yt_dlp/extractor/baidu.py
+++ b/yt_dlp/extractor/baidu.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import unescapeHTML
@@ -33,7 +32,7 @@ class BaiduVideoIE(InfoExtractor):
            path, category, playlist_id), playlist_id, note)

    def _real_extract(self, url):
-        category, playlist_id = re.match(self._VALID_URL, url).groups()
+        category, playlist_id = self._match_valid_url(url).groups()
        if category == 'show':
            category = 'tvshow'
        if category == 'tv':
--- a/yt_dlp/extractor/bandcamp.py
+++ b/yt_dlp/extractor/bandcamp.py
@@ -294,7 +294,7 @@ class BandcampAlbumIE(BandcampIE):
                else super(BandcampAlbumIE, cls).suitable(url))

    def _real_extract(self, url):
-        uploader_id, album_id = re.match(self._VALID_URL, url).groups()
+        uploader_id, album_id = self._match_valid_url(url).groups()
        playlist_id = album_id or uploader_id
        webpage = self._download_webpage(url, playlist_id)
        tralbum = self._extract_data_attr(webpage, playlist_id)
--- a/yt_dlp/extractor/bannedvideo.py
+++ b/yt_dlp/extractor/bannedvideo.py
@@ -0,0 +1,165 @@
+from __future__ import unicode_literals
+
+import json
+
+from .common import InfoExtractor
+from ..utils import (
+    try_get,
+    int_or_none,
+    url_or_none,
+    float_or_none,
+    unified_timestamp,
+)
+
+
+class BannedVideoIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?banned\.video/watch\?id=(?P<id>[0-f]{24})'
+    _TESTS = [{
+        'url': 'https://banned.video/watch?id=5e7a859644e02200c6ef5f11',
+        'md5': '14b6e81d41beaaee2215cd75c6ed56e4',
+        'info_dict': {
+            'id': '5e7a859644e02200c6ef5f11',
+            'ext': 'mp4',
+            'title': 'China Discovers Origin of Corona Virus: Issues Emergency Statement',
+            'thumbnail': r're:^https?://(?:www\.)?assets\.infowarsmedia.com/images/',
+            'description': 'md5:560d96f02abbebe6c6b78b47465f6b28',
+            'upload_date': '20200324',
+            'timestamp': 1585087895,
+        }
+    }]
+
+    _GRAPHQL_GETMETADATA_QUERY = '''
+query GetVideoAndComments($id: String!) {
+    getVideo(id: $id) {
+        streamUrl
+        directUrl
+        unlisted
+        live
+        tags {
+            name
+        }
+        title
+        summary
+        playCount
+        largeImage
+        videoDuration
+        channel {
+            _id
+            title
+        }
+        createdAt
+    }
+    getVideoComments(id: $id, limit: 999999, offset: 0) {
+        _id
+        content
+        user {
+            _id
+            username
+        }
+        voteCount {
+            positive
+        }
+        createdAt
+        replyCount
+    }
+}'''
+
+    _GRAPHQL_GETCOMMENTSREPLIES_QUERY = '''
+query GetCommentReplies($id: String!) {
+    getCommentReplies(id: $id, limit: 999999, offset: 0) {
+        _id
+        content
+        user {
+            _id
+            username
+        }
+        voteCount {
+            positive
+        }
+        createdAt
+        replyCount
+    }
+}'''
+
+    _GRAPHQL_QUERIES = {
+        'GetVideoAndComments': _GRAPHQL_GETMETADATA_QUERY,
+        'GetCommentReplies': _GRAPHQL_GETCOMMENTSREPLIES_QUERY,
+    }
+
+    def _call_api(self, video_id, id, operation, note):
+        return self._download_json(
+            'https://api.infowarsmedia.com/graphql', video_id, note=note,
+            headers={
+                'Content-Type': 'application/json; charset=utf-8'
+            }, data=json.dumps({
+                'variables': {'id': id},
+                'operationName': operation,
+                'query': self._GRAPHQL_QUERIES[operation]
+            }).encode('utf8')).get('data')
+
+    def _extract_comments(self, video_id, comments, comment_data):
+        for comment in comment_data.copy():
+            comment_id = comment.get('_id')
+            if comment.get('replyCount') > 0:
+                reply_json = self._call_api(
+                    video_id, comment_id, 'GetCommentReplies',
+                    f'Downloading replies for comment {comment_id}')
+                comments.extend(
+                    self._parse_comment(reply, comment_id)
+                    for reply in reply_json.get('getCommentReplies'))
+
+        return {
+            'comments': comments,
+            'comment_count': len(comments),
+        }
+
+    @staticmethod
+    def _parse_comment(comment_data, parent):
+        return {
+            'id': comment_data.get('_id'),
+            'text': comment_data.get('content'),
+            'author': try_get(comment_data, lambda x: x['user']['username']),
+            'author_id': try_get(comment_data, lambda x: x['user']['_id']),
+            'timestamp': unified_timestamp(comment_data.get('createdAt')),
+            'parent': parent,
+            'like_count': try_get(comment_data, lambda x: x['voteCount']['positive']),
+        }
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        video_json = self._call_api(video_id, video_id, 'GetVideoAndComments', 'Downloading video metadata')
+        video_info = video_json['getVideo']
+        is_live = video_info.get('live')
+        comments = [self._parse_comment(comment, 'root') for comment in video_json.get('getVideoComments')]
+
+        formats = [{
+            'format_id': 'direct',
+            'quality': 1,
+            'url': video_info.get('directUrl'),
+            'ext': 'mp4',
+        }] if url_or_none(video_info.get('directUrl')) else []
+        if video_info.get('streamUrl'):
+            formats.extend(self._extract_m3u8_formats(
+                video_info.get('streamUrl'), video_id, 'mp4',
+                entry_protocol='m3u8_native', m3u8_id='hls', live=True))
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': video_info.get('title')[:-1],
+            'formats': formats,
+            'is_live': is_live,
+            'description': video_info.get('summary'),
+            'channel': try_get(video_info, lambda x: x['channel']['title']),
+            'channel_id': try_get(video_info, lambda x: x['channel']['_id']),
+            'view_count': int_or_none(video_info.get('playCount')),
+            'thumbnail': url_or_none(video_info.get('largeImage')),
+            'duration': float_or_none(video_info.get('videoDuration')),
+            'timestamp': unified_timestamp(video_info.get('createdAt')),
+            'tags': [tag.get('name') for tag in video_info.get('tags')],
+            'availability': self._availability(is_unlisted=video_info.get('unlisted')),
+            'comments': comments,
+            '__post_extractor': (
+                (lambda: self._extract_comments(video_id, comments, video_json.get('getVideoComments')))
+                if self.get_param('getcomments') else None)
+        }
--- a/yt_dlp/extractor/bbc.py
+++ b/yt_dlp/extractor/bbc.py
@@ -10,9 +10,7 @@ from .common import InfoExtractor
 from ..compat import (
    compat_etree_Element,
    compat_HTTPError,
-    compat_parse_qs,
    compat_str,
-    compat_urllib_parse_urlparse,
    compat_urlparse,
 )
 from ..utils import (
@@ -26,6 +24,7 @@ from ..utils import (
    js_to_json,
    parse_duration,
    parse_iso8601,
+    parse_qs,
    strip_or_none,
    try_get,
    unescapeHTML,
@@ -1410,7 +1409,7 @@ class BBCCoUkIPlayerPlaylistBaseIE(InfoExtractor):

    def _real_extract(self, url):
        pid = self._match_id(url)
-        qs = compat_parse_qs(compat_urllib_parse_urlparse(url).query)
+        qs = parse_qs(url)
        series_id = qs.get('seriesId', [None])[0]
        page = qs.get('page', [None])[0]
        per_page = 36 if page else self._PAGE_SIZE
--- a/yt_dlp/extractor/beatport.py
+++ b/yt_dlp/extractor/beatport.py
@@ -40,7 +40,7 @@ class BeatportIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        track_id = mobj.group('id')
        display_id = mobj.group('display_id')

--- a/yt_dlp/extractor/beeg.py
+++ b/yt_dlp/extractor/beeg.py
@@ -3,10 +3,10 @@ from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..compat import (
    compat_str,
-    compat_urlparse,
 )
 from ..utils import (
    int_or_none,
+    parse_qs,
    unified_timestamp,
 )

@@ -57,7 +57,7 @@ class BeegIE(InfoExtractor):
            query = {
                'v': 2,
            }
-            qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
+            qs = parse_qs(url)
            t = qs.get('t', [''])[0].split('-')
            if len(t) > 1:
                query.update({
--- a/yt_dlp/extractor/behindkink.py
+++ b/yt_dlp/extractor/behindkink.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import url_basename
@@ -24,7 +23,7 @@ class BehindKinkIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        display_id = mobj.group('id')

        webpage = self._download_webpage(url, display_id)
--- a/yt_dlp/extractor/bellmedia.py
+++ b/yt_dlp/extractor/bellmedia.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor

@@ -78,7 +77,7 @@ class BellMediaIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        domain, video_id = re.match(self._VALID_URL, url).groups()
+        domain, video_id = self._match_valid_url(url).groups()
        domain = domain.split('.')[0]
        return {
            '_type': 'url_transparent',
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -4,13 +4,16 @@ from __future__ import unicode_literals
 import hashlib
 import itertools
 import json
+import functools
 import re
+import math

 from .common import InfoExtractor, SearchInfoExtractor
 from ..compat import (
    compat_str,
    compat_parse_qs,
    compat_urlparse,
+    compat_urllib_parse_urlparse
 )
 from ..utils import (
    ExtractorError,
@@ -24,6 +27,7 @@ from ..utils import (
    unified_timestamp,
    unsmuggle_url,
    urlencode_postdata,
+    OnDemandPagedList
 )


@@ -140,7 +144,7 @@ class BiliBiliIE(InfoExtractor):
    def _real_extract(self, url):
        url, smuggled_data = unsmuggle_url(url, {})

-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('id_bv') or mobj.group('id')

        av_id, bv_id = self._get_video_id_set(video_id, mobj.group('id_bv') is not None)
@@ -535,6 +539,75 @@ class BilibiliChannelIE(InfoExtractor):
        return self.playlist_result(self._entries(list_id), list_id)


+class BilibiliCategoryIE(InfoExtractor):
+    IE_NAME = 'Bilibili category extractor'
+    _MAX_RESULTS = 1000000
+    _VALID_URL = r'https?://www\.bilibili\.com/v/[a-zA-Z]+\/[a-zA-Z]+'
+    _TESTS = [{
+        'url': 'https://www.bilibili.com/v/kichiku/mad',
+        'info_dict': {
+            'id': 'kichiku: mad',
+            'title': 'kichiku: mad'
+        },
+        'playlist_mincount': 45,
+        'params': {
+            'playlistend': 45
+        }
+    }]
+
+    def _fetch_page(self, api_url, num_pages, query, page_num):
+        parsed_json = self._download_json(
+            api_url, query, query={'Search_key': query, 'pn': page_num},
+            note='Extracting results from page %s of %s' % (page_num, num_pages))
+
+        video_list = try_get(parsed_json, lambda x: x['data']['archives'], list)
+        if not video_list:
+            raise ExtractorError('Failed to retrieve video list for page %d' % page_num)
+
+        for video in video_list:
+            yield self.url_result(
+                'https://www.bilibili.com/video/%s' % video['bvid'], 'BiliBili', video['bvid'])
+
+    def _entries(self, category, subcategory, query):
+        # map of categories : subcategories : RIDs
+        rid_map = {
+            'kichiku': {
+                'mad': 26,
+                'manual_vocaloid': 126,
+                'guide': 22,
+                'theatre': 216,
+                'course': 127
+            },
+        }
+
+        if category not in rid_map:
+            raise ExtractorError('The supplied category, %s, is not supported. List of supported categories: %s' % (category, list(rid_map.keys())))
+
+        if subcategory not in rid_map[category]:
+            raise ExtractorError('The subcategory, %s, isn\'t supported for this category. Supported subcategories: %s' % (subcategory, list(rid_map[category].keys())))
+
+        rid_value = rid_map[category][subcategory]
+
+        api_url = 'https://api.bilibili.com/x/web-interface/newlist?rid=%d&type=1&ps=20&jsonp=jsonp' % rid_value
+        page_json = self._download_json(api_url, query, query={'Search_key': query, 'pn': '1'})
+        page_data = try_get(page_json, lambda x: x['data']['page'], dict)
+        count, size = int_or_none(page_data.get('count')), int_or_none(page_data.get('size'))
+        if count is None or not size:
+            raise ExtractorError('Failed to calculate either page count or size')
+
+        num_pages = math.ceil(count / size)
+
+        return OnDemandPagedList(functools.partial(
+            self._fetch_page, api_url, num_pages, query), size)
+
+    def _real_extract(self, url):
+        u = compat_urllib_parse_urlparse(url)
+        category, subcategory = u.path.split('/')[2:4]
+        query = '%s: %s' % (category, subcategory)
+
+        return self.playlist_result(self._entries(category, subcategory, query), query, query)
+
+
 class BiliBiliSearchIE(SearchInfoExtractor):
    IE_DESC = 'Bilibili video search, "bilisearch" keyword'
    _MAX_RESULTS = 100000
--- a/yt_dlp/extractor/bitchute.py
+++ b/yt_dlp/extractor/bitchute.py
@@ -17,16 +17,16 @@ from ..utils import (
 class BitChuteIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?bitchute\.com/(?:video|embed|torrent/[^/]+)/(?P<id>[^/?#&]+)'
    _TESTS = [{
-        'url': 'https://www.bitchute.com/video/szoMrox2JEI/',
-        'md5': '66c4a70e6bfc40dcb6be3eb1d74939eb',
+        'url': 'https://www.bitchute.com/video/UGlrF9o9b-Q/',
+        'md5': '7e427d7ed7af5a75b5855705ec750e2b',
        'info_dict': {
            'id': 'szoMrox2JEI',
            'ext': 'mp4',
-            'title': 'Fuck bitches get money',
-            'description': 'md5:3f21f6fb5b1d17c3dee9cf6b5fe60b3a',
+            'title': 'This is the first video on #BitChute !',
+            'description': 'md5:a0337e7b1fe39e32336974af8173a034',
            'thumbnail': r're:^https?://.*\.jpg$',
-            'uploader': 'Victoria X Rave',
-            'upload_date': '20170813',
+            'uploader': 'BitChute',
+            'upload_date': '20170103',
        },
    }, {
        'url': 'https://www.bitchute.com/embed/lbb5G1hjPhw/',
--- a/yt_dlp/extractor/blackboardcollaborate.py
+++ b/yt_dlp/extractor/blackboardcollaborate.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import parse_iso8601
@@ -48,7 +47,7 @@ class BlackboardCollaborateIE(InfoExtractor):
    ]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        region = mobj.group('region')
        video_id = mobj.group('id')
        info = self._download_json(
--- a/yt_dlp/extractor/bokecc.py
+++ b/yt_dlp/extractor/bokecc.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..compat import compat_parse_qs
@@ -45,7 +44,7 @@ class BokeCCIE(BokeCCBaseIE):
    }]

    def _real_extract(self, url):
-        qs = compat_parse_qs(re.match(self._VALID_URL, url).group('query'))
+        qs = compat_parse_qs(self._match_valid_url(url).group('query'))
        if not qs.get('vid') or not qs.get('uid'):
            raise ExtractorError('Invalid URL', expected=True)

--- a/yt_dlp/extractor/bongacams.py
+++ b/yt_dlp/extractor/bongacams.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..compat import compat_str
@@ -22,7 +21,7 @@ class BongaCamsIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        host = mobj.group('host')
        channel_id = mobj.group('id')

--- a/yt_dlp/extractor/box.py
+++ b/yt_dlp/extractor/box.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 import json
-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -30,7 +29,7 @@ class BoxIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        shared_name, file_id = re.match(self._VALID_URL, url).groups()
+        shared_name, file_id = self._match_valid_url(url).groups()
        webpage = self._download_webpage(url, file_id)
        request_token = self._parse_json(self._search_regex(
            r'Box\.config\s*=\s*({.+?});', webpage,
--- a/yt_dlp/extractor/br.py
+++ b/yt_dlp/extractor/br.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 import json
-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -86,7 +85,7 @@ class BRIE(InfoExtractor):
    ]

    def _real_extract(self, url):
-        base_url, display_id = re.search(self._VALID_URL, url).groups()
+        base_url, display_id = self._match_valid_url(url).groups()
        page = self._download_webpage(url, display_id)
        xml_url = self._search_regex(
            r"return BRavFramework\.register\(BRavFramework\('avPlayer_(?:[a-f0-9-]{36})'\)\.setup\({dataURL:'(/(?:[a-z0-9\-]+/)+[a-z0-9/~_.-]+)'}\)\);", page, 'XMLURL')
--- a/yt_dlp/extractor/bravotv.py
+++ b/yt_dlp/extractor/bravotv.py
@@ -42,7 +42,7 @@ class BravoTVIE(AdobePassIE):
    }]

    def _real_extract(self, url):
-        site, display_id = re.match(self._VALID_URL, url).groups()
+        site, display_id = self._match_valid_url(url).groups()
        webpage = self._download_webpage(url, display_id)
        settings = self._parse_json(self._search_regex(
            r'<script[^>]+data-drupal-selector="drupal-settings-json"[^>]*>({.+?})</script>', webpage, 'drupal settings'),
--- a/yt_dlp/extractor/breakcom.py
+++ b/yt_dlp/extractor/breakcom.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from .youtube import YoutubeIE
@@ -41,7 +40,7 @@ class BreakIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        display_id, video_id = re.match(self._VALID_URL, url).groups()
+        display_id, video_id = self._match_valid_url(url).groups()

        webpage = self._download_webpage(url, display_id)

--- a/yt_dlp/extractor/brightcove.py
+++ b/yt_dlp/extractor/brightcove.py
@@ -11,7 +11,6 @@ from ..compat import (
    compat_etree_fromstring,
    compat_HTTPError,
    compat_parse_qs,
-    compat_urllib_parse_urlparse,
    compat_urlparse,
    compat_xml_parse_error,
 )
@@ -26,6 +25,7 @@ from ..utils import (
    js_to_json,
    mimetype2ext,
    parse_iso8601,
+    parse_qs,
    smuggle_url,
    str_or_none,
    try_get,
@@ -177,7 +177,7 @@ class BrightcoveLegacyIE(InfoExtractor):
            flashvars = {}

        data_url = object_doc.attrib.get('data', '')
-        data_url_params = compat_parse_qs(compat_urllib_parse_urlparse(data_url).query)
+        data_url_params = parse_qs(data_url)

        def find_param(name):
            if name in flashvars:
@@ -290,7 +290,7 @@ class BrightcoveLegacyIE(InfoExtractor):
        url = re.sub(r'(?<=[?&])(videoI(d|D)|idVideo|bctid)', '%40videoPlayer', url)
        # Change bckey (used by bcove.me urls) to playerKey
        url = re.sub(r'(?<=[?&])bckey', 'playerKey', url)
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        query_str = mobj.group('query')
        query = compat_urlparse.parse_qs(query_str)

@@ -549,7 +549,7 @@ class BrightcoveNewIE(AdobePassIE):
                    error.get('message') or error.get('error_subcode') or error['error_code'], expected=True)
            elif (not self.get_param('allow_unplayable_formats')
                    and sources and num_drm_sources == len(sources)):
-                raise ExtractorError('This video is DRM protected.', expected=True)
+                self.report_drm(video_id)

        self._sort_formats(formats)

@@ -595,7 +595,7 @@ class BrightcoveNewIE(AdobePassIE):
            'ip_blocks': smuggled_data.get('geo_ip_blocks'),
        })

-        account_id, player_id, embed, content_type, video_id = re.match(self._VALID_URL, url).groups()
+        account_id, player_id, embed, content_type, video_id = self._match_valid_url(url).groups()

        policy_key_id = '%s_%s' % (account_id, player_id)
        policy_key = self._downloader.cache.load('brightcove', policy_key_id)
--- a/yt_dlp/extractor/byutv.py
+++ b/yt_dlp/extractor/byutv.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -52,7 +51,7 @@ class BYUtvIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('id')
        display_id = mobj.group('display_id') or video_id

--- a/yt_dlp/extractor/c56.py
+++ b/yt_dlp/extractor/c56.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import js_to_json
@@ -31,7 +30,7 @@ class C56IE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url, flags=re.VERBOSE)
+        mobj = self._match_valid_url(url)
        text_id = mobj.group('textid')

        webpage = self._download_webpage(url, text_id)
--- a/yt_dlp/extractor/camtube.py
+++ b/yt_dlp/extractor/camtube.py
@@ -1,71 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-from .common import InfoExtractor
-from ..utils import (
-    int_or_none,
-    unified_timestamp,
-)
-
-
-class CamTubeIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:(?:www|api)\.)?camtube\.co/recordings?/(?P<id>[^/?#&]+)'
-    _TESTS = [{
-        'url': 'https://camtube.co/recording/minafay-030618-1136-chaturbate-female',
-        'info_dict': {
-            'id': '42ad3956-dd5b-445a-8313-803ea6079fac',
-            'display_id': 'minafay-030618-1136-chaturbate-female',
-            'ext': 'mp4',
-            'title': 'minafay-030618-1136-chaturbate-female',
-            'duration': 1274,
-            'timestamp': 1528018608,
-            'upload_date': '20180603',
-            'age_limit': 18
-        },
-        'params': {
-            'skip_download': True,
-        },
-    }]
-
-    _API_BASE = 'https://api.camtube.co'
-
-    def _real_extract(self, url):
-        display_id = self._match_id(url)
-
-        token = self._download_json(
-            '%s/rpc/session/new' % self._API_BASE, display_id,
-            'Downloading session token')['token']
-
-        self._set_cookie('api.camtube.co', 'session', token)
-
-        video = self._download_json(
-            '%s/recordings/%s' % (self._API_BASE, display_id), display_id,
-            headers={'Referer': url})
-
-        video_id = video['uuid']
-        timestamp = unified_timestamp(video.get('createdAt'))
-        duration = int_or_none(video.get('duration'))
-        view_count = int_or_none(video.get('viewCount'))
-        like_count = int_or_none(video.get('likeCount'))
-        creator = video.get('stageName')
-
-        formats = [{
-            'url': '%s/recordings/%s/manifest.m3u8'
-                   % (self._API_BASE, video_id),
-            'format_id': 'hls',
-            'ext': 'mp4',
-            'protocol': 'm3u8_native',
-        }]
-
-        return {
-            'id': video_id,
-            'display_id': display_id,
-            'title': display_id,
-            'timestamp': timestamp,
-            'duration': duration,
-            'view_count': view_count,
-            'like_count': like_count,
-            'creator': creator,
-            'formats': formats,
-            'age_limit': 18
-        }
--- a/yt_dlp/extractor/canalplus.py
+++ b/yt_dlp/extractor/canalplus.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -50,7 +49,7 @@ class CanalplusIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        site, display_id, video_id = re.match(self._VALID_URL, url).groups()
+        site, display_id, video_id = self._match_valid_url(url).groups()

        site_id = self._SITE_ID_MAP[site]

--- a/yt_dlp/extractor/canvas.py
+++ b/yt_dlp/extractor/canvas.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from .gigya import GigyaBaseIE
@@ -47,7 +46,7 @@ class CanvasIE(InfoExtractor):
    _REST_API_BASE = 'https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1'

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        site_id, video_id = mobj.group('site_id'), mobj.group('id')

        data = None
@@ -192,7 +191,7 @@ class CanvasEenIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        site_id, display_id = mobj.group('site_id'), mobj.group('id')

        webpage = self._download_webpage(url, display_id)
--- a/yt_dlp/extractor/cbsinteractive.py
+++ b/yt_dlp/extractor/cbsinteractive.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .cbs import CBSIE
 from ..utils import int_or_none
@@ -71,7 +70,7 @@ class CBSInteractiveIE(CBSIE):
    }

    def _real_extract(self, url):
-        site, display_id = re.match(self._VALID_URL, url).groups()
+        site, display_id = self._match_valid_url(url).groups()
        webpage = self._download_webpage(url, display_id)

        data_json = self._html_search_regex(
--- a/yt_dlp/extractor/cbssports.py
+++ b/yt_dlp/extractor/cbssports.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 # from .cbs import CBSBaseIE
 from .common import InfoExtractor
@@ -30,7 +29,7 @@ class CBSSportsEmbedIE(InfoExtractor):
    #     return self._extract_feed_info('dJ5BDC', 'VxxJg8Ymh8sE', filter_query, video_id)

    def _real_extract(self, url):
-        uuid, pcid = re.match(self._VALID_URL, url).groups()
+        uuid, pcid = self._match_valid_url(url).groups()
        query = {'id': uuid} if uuid else {'pcid': pcid}
        video = self._download_json(
            'https://www.cbssports.com/api/content/video/',
--- a/yt_dlp/extractor/ccma.py
+++ b/yt_dlp/extractor/ccma.py
@@ -3,7 +3,6 @@ from __future__ import unicode_literals

 import calendar
 import datetime
-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -61,7 +60,7 @@ class CCMAIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        media_type, media_id = re.match(self._VALID_URL, url).groups()
+        media_type, media_id = self._match_valid_url(url).groups()

        media = self._download_json(
            'http://dinamics.ccma.cat/pvideo/media.jsp', media_id, query={
--- a/yt_dlp/extractor/cda.py
+++ b/yt_dlp/extractor/cda.py
@@ -3,6 +3,7 @@ from __future__ import unicode_literals

 import codecs
 import re
+import json

 from .common import InfoExtractor
 from ..compat import (
@@ -19,6 +20,7 @@ from ..utils import (
    parse_duration,
    random_birthday,
    urljoin,
+    try_get,
 )


@@ -38,6 +40,8 @@ class CDAIE(InfoExtractor):
            'average_rating': float,
            'duration': 39,
            'age_limit': 0,
+            'upload_date': '20160221',
+            'timestamp': 1456078244,
        }
    }, {
        'url': 'http://www.cda.pl/video/57413289',
@@ -143,7 +147,7 @@ class CDAIE(InfoExtractor):
            b = []
            for c in a:
                f = compat_ord(c)
-                b.append(compat_chr(33 + (f + 14) % 94) if 33 <= f and 126 >= f else compat_chr(f))
+                b.append(compat_chr(33 + (f + 14) % 94) if 33 <= f <= 126 else compat_chr(f))
            a = ''.join(b)
            a = a.replace('.cda.mp4', '')
            for p in ('.2cda.pl', '.3cda.pl'):
@@ -173,18 +177,34 @@ class CDAIE(InfoExtractor):
                    video['file'] = video['file'].replace('adc.mp4', '.mp4')
            elif not video['file'].startswith('http'):
                video['file'] = decrypt_file(video['file'])
-            f = {
+            video_quality = video.get('quality')
+            qualities = video.get('qualities', {})
+            video_quality = next((k for k, v in qualities.items() if v == video_quality), video_quality)
+            info_dict['formats'].append({
                'url': video['file'],
-            }
-            m = re.search(
-                r'<a[^>]+data-quality="(?P<format_id>[^"]+)"[^>]+href="[^"]+"[^>]+class="[^"]*quality-btn-active[^"]*">(?P<height>[0-9]+)p',
-                page)
-            if m:
-                f.update({
-                    'format_id': m.group('format_id'),
-                    'height': int(m.group('height')),
-                })
-            info_dict['formats'].append(f)
+                'format_id': video_quality,
+                'height': int_or_none(video_quality[:-1]),
+            })
+            for quality, cda_quality in qualities.items():
+                if quality == video_quality:
+                    continue
+                data = {'jsonrpc': '2.0', 'method': 'videoGetLink', 'id': 2,
+                        'params': [video_id, cda_quality, video.get('ts'), video.get('hash2'), {}]}
+                data = json.dumps(data).encode('utf-8')
+                video_url = self._download_json(
+                    f'https://www.cda.pl/video/{video_id}', video_id, headers={
+                        'Content-Type': 'application/json',
+                        'X-Requested-With': 'XMLHttpRequest'
+                    }, data=data, note=f'Fetching {quality} url',
+                    errnote=f'Failed to fetch {quality} url', fatal=False)
+                if try_get(video_url, lambda x: x['result']['status']) == 'ok':
+                    video_url = try_get(video_url, lambda x: x['result']['resp'])
+                    info_dict['formats'].append({
+                        'url': video_url,
+                        'format_id': quality,
+                        'height': int_or_none(quality[:-1])
+                    })
+
            if not info_dict['duration']:
                info_dict['duration'] = parse_duration(video.get('duration'))

--- a/yt_dlp/extractor/ceskatelevize.py
+++ b/yt_dlp/extractor/ceskatelevize.py
@@ -147,9 +147,6 @@ class CeskaTelevizeIE(InfoExtractor):
                is_live = item.get('type') == 'LIVE'
                formats = []
                for format_id, stream_url in item.get('streamUrls', {}).items():
-                    if (not self.get_param('allow_unplayable_formats')
-                            and 'drmOnly=true' in stream_url):
-                        continue
                    if 'playerType=flash' in stream_url:
                        stream_formats = self._extract_m3u8_formats(
                            stream_url, playlist_id, 'mp4', 'm3u8_native',
@@ -158,6 +155,9 @@ class CeskaTelevizeIE(InfoExtractor):
                        stream_formats = self._extract_mpd_formats(
                            stream_url, playlist_id,
                            mpd_id='dash-%s' % format_id, fatal=False)
+                    if 'drmOnly=true' in stream_url:
+                        for f in stream_formats:
+                            f['has_drm'] = True
                    # See https://github.com/ytdl-org/youtube-dl/issues/12119#issuecomment-280037031
                    if format_id == 'audioDescription':
                        for f in stream_formats:
--- a/yt_dlp/extractor/channel9.py
+++ b/yt_dlp/extractor/channel9.py
@@ -96,7 +96,7 @@ class Channel9IE(InfoExtractor):
        return self.playlist_result(entries, video_id, title_text)

    def _real_extract(self, url):
-        content_path, rss = re.match(self._VALID_URL, url).groups()
+        content_path, rss = self._match_valid_url(url).groups()

        if rss:
            return self._extract_list(content_path, url)
--- a/yt_dlp/extractor/chilloutzone.py
+++ b/yt_dlp/extractor/chilloutzone.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re
 import json

 from .common import InfoExtractor
@@ -51,7 +50,7 @@ class ChilloutzoneIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('id')

        webpage = self._download_webpage(url, video_id)
--- a/yt_dlp/extractor/cinemax.py
+++ b/yt_dlp/extractor/cinemax.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .hbo import HBOBaseIE

@@ -23,7 +22,7 @@ class CinemaxIE(HBOBaseIE):
    }]

    def _real_extract(self, url):
-        path, video_id = re.match(self._VALID_URL, url).groups()
+        path, video_id = self._match_valid_url(url).groups()
        info = self._extract_info('https://www.cinemax.com/%s.xml' % path, video_id)
        info['id'] = video_id
        return info
--- a/yt_dlp/extractor/ciscolive.py
+++ b/yt_dlp/extractor/ciscolive.py
@@ -4,14 +4,11 @@ from __future__ import unicode_literals
 import itertools

 from .common import InfoExtractor
-from ..compat import (
-    compat_parse_qs,
-    compat_urllib_parse_urlparse,
-)
 from ..utils import (
    clean_html,
    float_or_none,
    int_or_none,
+    parse_qs,
    try_get,
    urlencode_postdata,
 )
@@ -145,7 +142,7 @@ class CiscoLiveSearchIE(CiscoLiveBaseIE):
            query['from'] += query['size']

    def _real_extract(self, url):
-        query = compat_parse_qs(compat_urllib_parse_urlparse(url).query)
+        query = parse_qs(url)
        query['type'] = 'session'
        return self.playlist_result(
            self._entries(query, url), playlist_title='Search query')
--- a/yt_dlp/extractor/cjsw.py
+++ b/yt_dlp/extractor/cjsw.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import (
@@ -30,7 +29,7 @@ class CJSWIE(InfoExtractor):
    }]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        program, episode_id = mobj.group('program', 'id')
        audio_id = '%s/%s' % (program, episode_id)

--- a/yt_dlp/extractor/clyp.py
+++ b/yt_dlp/extractor/clyp.py
@@ -1,12 +1,9 @@
 from __future__ import unicode_literals

 from .common import InfoExtractor
-from ..compat import (
-    compat_parse_qs,
-    compat_urllib_parse_urlparse,
-)
 from ..utils import (
    float_or_none,
+    parse_qs,
    unified_timestamp,
 )

@@ -44,7 +41,7 @@ class ClypIE(InfoExtractor):
    def _real_extract(self, url):
        audio_id = self._match_id(url)

-        qs = compat_parse_qs(compat_urllib_parse_urlparse(url).query)
+        qs = parse_qs(url)
        token = qs.get('token', [None])[0]

        query = {}
--- a/yt_dlp/extractor/cnbc.py
+++ b/yt_dlp/extractor/cnbc.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..utils import smuggle_url
@@ -57,7 +56,7 @@ class CNBCVideoIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        path, display_id = re.match(self._VALID_URL, url).groups()
+        path, display_id = self._match_valid_url(url).groups()
        video_id = self._download_json(
            'https://webql-redesign.cnbcfm.com/graphql', display_id, query={
                'query': '''{
--- a/yt_dlp/extractor/cnn.py
+++ b/yt_dlp/extractor/cnn.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from .turner import TurnerBaseIE
@@ -88,7 +87,7 @@ class CNNIE(TurnerBaseIE):
        return None

    def _real_extract(self, url):
-        sub_domain, path, page_title = re.match(self._VALID_URL, url).groups()
+        sub_domain, path, page_title = self._match_valid_url(url).groups()
        if sub_domain not in ('money', 'edition'):
            sub_domain = 'edition'
        config = self._CONFIG[sub_domain]
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -203,6 +203,7 @@ class InfoExtractor(object):
                                 width : height ratio as float.
                    * no_resume  The server does not support resuming the
                                 (HTTP or RTMP) download. Boolean.
+                    * has_drm    The format has DRM and cannot be downloaded. Boolean
                    * downloader_options  A dictionary of downloader options as
                                 described in FileDownloader
                    RTMP formats can also have the additional fields: page_url,
@@ -447,23 +448,31 @@ class InfoExtractor(object):
        self.set_downloader(downloader)

    @classmethod
-    def suitable(cls, url):
-        """Receives a URL and returns True if suitable for this IE."""
-
+    def _match_valid_url(cls, url):
        # This does not use has/getattr intentionally - we want to know whether
        # we have cached the regexp for *this* class, whereas getattr would also
        # match the superclass
        if '_VALID_URL_RE' not in cls.__dict__:
            cls._VALID_URL_RE = re.compile(cls._VALID_URL)
-        return cls._VALID_URL_RE.match(url) is not None
+        return cls._VALID_URL_RE.match(url)
+
+    @classmethod
+    def suitable(cls, url):
+        """Receives a URL and returns True if suitable for this IE."""
+        # This function must import everything it needs (except other extractors),
+        # so that lazy_extractors works correctly
+        return cls._match_valid_url(url) is not None

    @classmethod
    def _match_id(cls, url):
-        if '_VALID_URL_RE' not in cls.__dict__:
-            cls._VALID_URL_RE = re.compile(cls._VALID_URL)
-        m = cls._VALID_URL_RE.match(url)
-        assert m
-        return compat_str(m.group('id'))
+        return cls._match_valid_url(url).group('id')
+
+    @classmethod
+    def get_temp_id(cls, url):
+        try:
+            return cls._match_id(url)
+        except (IndexError, AttributeError):
+            return None

    @classmethod
    def working(cls):
@@ -586,12 +595,14 @@ class InfoExtractor(object):
                    if self.__maybe_fake_ip_and_retry(e.countries):
                        continue
                    raise
-        except ExtractorError:
-            raise
+        except ExtractorError as e:
+            video_id = e.video_id or self.get_temp_id(url)
+            raise ExtractorError(
+                e.msg, video_id=video_id, ie=self.IE_NAME, tb=e.traceback, expected=e.expected, cause=e.cause)
        except compat_http_client.IncompleteRead as e:
-            raise ExtractorError('A network error has occurred.', cause=e, expected=True)
+            raise ExtractorError('A network error has occurred.', cause=e, expected=True, video_id=self.get_temp_id(url))
        except (KeyError, StopIteration) as e:
-            raise ExtractorError('An extractor error has occurred.', cause=e)
+            raise ExtractorError('An extractor error has occurred.', cause=e, video_id=self.get_temp_id(url))

    def __maybe_fake_ip_and_retry(self, countries):
        if (not self.get_param('geo_bypass_country', None)
@@ -623,7 +634,7 @@ class InfoExtractor(object):
    @classmethod
    def ie_key(cls):
        """A string for getting the InfoExtractor with get_info_extractor"""
-        return compat_str(cls.__name__[:-2])
+        return cls.__name__[:-2]

    @property
    def IE_NAME(self):
@@ -1023,6 +1034,9 @@ class InfoExtractor(object):
            return self._downloader.params.get(name, default, *args, **kwargs)
        return default

+    def report_drm(self, video_id, partial=False):
+        self.raise_no_formats('This video is DRM protected', expected=True, video_id=video_id)
+
    def report_extraction(self, id_or_name):
        """Report information extraction."""
        self.to_screen('%s: Extracting information' % id_or_name)
@@ -1488,7 +1502,7 @@ class InfoExtractor(object):
        default = ('hidden', 'aud_or_vid', 'hasvid', 'ie_pref', 'lang', 'quality',
                   'res', 'fps', 'codec:vp9.2', 'size', 'br', 'asr',
                   'proto', 'ext', 'hasaud', 'source', 'format_id')  # These must not be aliases
-        ytdl_default = ('hasaud', 'quality', 'tbr', 'filesize', 'vbr',
+        ytdl_default = ('hasaud', 'lang', 'quality', 'tbr', 'filesize', 'vbr',
                        'height', 'width', 'proto', 'vext', 'abr', 'aext',
                        'fps', 'fs_approx', 'source', 'format_id')

@@ -1512,7 +1526,7 @@ class InfoExtractor(object):
            'ie_pref': {'priority': True, 'type': 'extractor'},
            'hasvid': {'priority': True, 'field': 'vcodec', 'type': 'boolean', 'not_in_list': ('none',)},
            'hasaud': {'field': 'acodec', 'type': 'boolean', 'not_in_list': ('none',)},
-            'lang': {'priority': True, 'convert': 'ignore', 'field': 'language_preference'},
+            'lang': {'convert': 'ignore', 'field': 'language_preference'},
            'quality': {'convert': 'float_none', 'default': -1},
            'filesize': {'convert': 'bytes'},
            'fs_approx': {'convert': 'bytes', 'field': 'filesize_approx'},
@@ -1751,9 +1765,7 @@ class InfoExtractor(object):

    def _sort_formats(self, formats, field_preference=[]):
        if not formats:
-            if self.get_param('ignore_no_formats_error'):
-                return
-            raise ExtractorError('No video formats found')
+            return
        format_sort = self.FormatSort()  # params and to_screen are taken from the downloader
        format_sort.evaluate_params(self._downloader.params, field_preference)
        if self.get_param('verbose', False):
@@ -1991,9 +2003,7 @@ class InfoExtractor(object):
        if '#EXT-X-FAXS-CM:' in m3u8_doc:  # Adobe Flash Access
            return formats, subtitles

-        if (not self.get_param('allow_unplayable_formats')
-                and re.search(r'#EXT-X-SESSION-KEY:.*?URI="skd://', m3u8_doc)):  # Apple FairPlay
-            return formats, subtitles
+        has_drm = re.search(r'#EXT-X-SESSION-KEY:.*?URI="skd://', m3u8_doc)

        def format_url(url):
            return url if re.match(r'^https?://', url) else compat_urlparse.urljoin(m3u8_url, url)
@@ -2039,6 +2049,7 @@ class InfoExtractor(object):
                'protocol': entry_protocol,
                'preference': preference,
                'quality': quality,
+                'has_drm': has_drm,
            } for idx in _extract_m3u8_playlist_indices(m3u8_doc=m3u8_doc)]

            return formats, subtitles
@@ -2572,11 +2583,9 @@ class InfoExtractor(object):
                        extract_Initialization(segment_template)
            return ms_info

-        skip_unplayable = not self.get_param('allow_unplayable_formats')
-
        mpd_duration = parse_duration(mpd_doc.get('mediaPresentationDuration'))
-        formats = []
-        subtitles = {}
+        formats, subtitles = [], {}
+        stream_numbers = {'audio': 0, 'video': 0}
        for period in mpd_doc.findall(_add_ns('Period')):
            period_duration = parse_duration(period.get('duration')) or mpd_duration
            period_ms_info = extract_multisegment_info(period, {
@@ -2584,12 +2593,8 @@ class InfoExtractor(object):
                'timescale': 1,
            })
            for adaptation_set in period.findall(_add_ns('AdaptationSet')):
-                if skip_unplayable and is_drm_protected(adaptation_set):
-                    continue
                adaption_set_ms_info = extract_multisegment_info(adaptation_set, period_ms_info)
                for representation in adaptation_set.findall(_add_ns('Representation')):
-                    if skip_unplayable and is_drm_protected(representation):
-                        continue
                    representation_attrib = adaptation_set.attrib.copy()
                    representation_attrib.update(representation.attrib)
                    # According to [1, 5.3.7.2, Table 9, page 41], @mimeType is mandatory
@@ -2599,8 +2604,8 @@ class InfoExtractor(object):
                    codecs = representation_attrib.get('codecs', '')
                    if content_type not in ('video', 'audio', 'text'):
                        if mime_type == 'image/jpeg':
-                            content_type = 'image/jpeg'
-                        if codecs.split('.')[0] == 'stpp':
+                            content_type = mime_type
+                        elif codecs.split('.')[0] == 'stpp':
                            content_type = 'text'
                        else:
                            self.report_warning('Unknown MIME type %s in DASH manifest' % mime_type)
@@ -2642,8 +2647,10 @@ class InfoExtractor(object):
                            'format_note': 'DASH %s' % content_type,
                            'filesize': filesize,
                            'container': mimetype2ext(mime_type) + '_dash',
+                            'manifest_stream_number': stream_numbers[content_type]
                        }
                        f.update(parse_codecs(codecs))
+                        stream_numbers[content_type] += 1
                    elif content_type == 'text':
                        f = {
                            'ext': mimetype2ext(mime_type),
@@ -2661,6 +2668,8 @@ class InfoExtractor(object):
                            'acodec': 'none',
                            'vcodec': 'none',
                        }
+                    if is_drm_protected(adaptation_set) or is_drm_protected(representation):
+                        f['has_drm'] = True
                    representation_ms_info = extract_multisegment_info(representation, adaption_set_ms_info)

                    def prepare_template(template_name, identifiers):
@@ -2847,9 +2856,6 @@ class InfoExtractor(object):
        """
        if ism_doc.get('IsLive') == 'TRUE':
            return [], {}
-        if (not self.get_param('allow_unplayable_formats')
-                and ism_doc.find('Protection') is not None):
-            return [], {}

        duration = int(ism_doc.attrib['Duration'])
        timescale = int_or_none(ism_doc.get('TimeScale')) or 10000000
@@ -2940,6 +2946,7 @@ class InfoExtractor(object):
                        'acodec': 'none' if stream_type == 'video' else fourcc,
                        'protocol': 'ism',
                        'fragments': fragments,
+                        'has_drm': ism_doc.find('Protection') is not None,
                        '_download_params': {
                            'stream_type': stream_type,
                            'duration': duration,
--- a/yt_dlp/extractor/commonprotocols.py
+++ b/yt_dlp/extractor/commonprotocols.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals

-import re

 from .common import InfoExtractor
 from ..compat import (
@@ -72,4 +71,4 @@ class ViewSourceIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        return self.url_result(re.match(self._VALID_URL, url).group('url'))
+        return self.url_result(self._match_valid_url(url).group('url'))
--- a/yt_dlp/extractor/condenast.py
+++ b/yt_dlp/extractor/condenast.py
@@ -222,7 +222,7 @@ class CondeNastIE(InfoExtractor):
        }

    def _real_extract(self, url):
-        video_id, player_id, target, url_type, display_id = re.match(self._VALID_URL, url).groups()
+        video_id, player_id, target, url_type, display_id = self._match_valid_url(url).groups()

        if video_id:
            return self._extract_video({
--- a/yt_dlp/extractor/corus.py
+++ b/yt_dlp/extractor/corus.py
@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import re

 from .theplatform import ThePlatformFeedIE
 from ..utils import (
@@ -96,7 +95,7 @@ class CorusIE(ThePlatformFeedIE):
    }

    def _real_extract(self, url):
-        domain, video_id = re.match(self._VALID_URL, url).groups()
+        domain, video_id = self._match_valid_url(url).groups()
        site = domain.split('.')[0]
        path = self._SITE_MAP.get(site, site)
        if path != 'series':
@@ -131,7 +130,7 @@ class CorusIE(ThePlatformFeedIE):
            formats.extend(self._parse_smil_formats(
                smil, smil_url, video_id, namespace))
        if not formats and video.get('drm'):
-            self.raise_no_formats('This video is DRM protected.', expected=True)
+            self.report_drm(video_id)
        self._sort_formats(formats)

        subtitles = {}
--- a/yt_dlp/extractor/crackle.py
+++ b/yt_dlp/extractor/crackle.py
@@ -176,8 +176,8 @@ class CrackleIE(InfoExtractor):
                    'width': mfs_info['width'],
                    'height': mfs_info['height'],
                })
-        if not formats and has_drm and not ignore_no_formats:
-            raise ExtractorError('The video is DRM protected', expected=True)
+        if not formats and has_drm:
+            self.report_drm(video_id)
        self._sort_formats(formats)

        description = media.get('Description')
--- a/yt_dlp/extractor/crunchyroll.py
+++ b/yt_dlp/extractor/crunchyroll.py
@@ -413,7 +413,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
        return subtitles

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        mobj = self._match_valid_url(url)
        video_id = mobj.group('id')

        if mobj.group('prefix') == 'm':
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
pukkandan	9ee4f0bb5b	Release 2021.09.02	2021-09-02 04:43:38 +05:30
pukkandan	be4d9f4cd9	Partially revert "[build] Add homebrew taps (#827 )"	2021-09-02 04:43:38 +05:30
pukkandan	347182a0cd	Show a more useful error in older python versions	2021-09-02 03:52:08 +05:30
pukkandan	a7429aa9fa	[youtube] Fix subtitle names	2021-09-02 02:26:27 +05:30
Nil Admirari	7a340e0df3	Native SponsorBlock implementation and related improvements (#360 ) SponsorBlock options: * The fetched sponsor sections are written to infojson * `--sponsorblock-remove` removes specified chapters from file * `--sponsorblock-mark` marks the specified sponsor sections as chapters * `--sponsorblock-chapter-title` to specify sponsor chapter template * `--sponsorblock-api` to use a different API Related improvements: * Split `--embed-chapters` from `--embed-metadata` * Add `--remove-chapters` to remove arbitrary chapters * Add `--force-keyframes-at-cuts` for more accurate cuts when removing and splitting chapters Deprecates all `--sponskrub` options Authored by: nihil-admirari, pukkandan	2021-09-02 02:25:16 +05:30
ouwou	f0e5366335	[reddit] Fix for quarantined subreddits (#848 ) Authored by: ouwou	2021-09-02 00:24:31 +05:30
nyuszika7h	49ca8db06b	[mediaset] Fix extraction for more videos (#852 ) Closes #851 Authored by: nyuszika7h	2021-09-02 00:23:19 +05:30
nyuszika7h	ee57a19d84	[mediaset] Fix extraction for some videos (#850 ) This was broken by #564 Closes #849 Authored by: nyuszika7h	2021-09-01 21:09:15 +05:30
octotherp	908b56eaf7	[XHamster] Extract `uploader_id` (#844 ) Authored by: octotherp	2021-09-01 18:58:25 +05:30
u-spec-png	1461d7bef2	[Tokentube] Add extractor (#842 ) Closes #800 Authored by: u-spec-png	2021-09-01 18:40:25 +05:30
pukkandan	8a2d992389	[facebook] Fix format sorting Closes #795	2021-09-01 09:17:52 +05:30
pukkandan	8e25d624df	[EmbedSubtitle] Continue even if some files are missing	2021-09-01 08:51:22 +05:30
coletdjnz	e88dabb35e	[Viafree] Fix extractor and extract subtitles (#828 ) Authored by: coletdjnz Fixes #820	2021-08-31 22:31:11 +00:00
BunnyHelp	8eb7ba82ca	[iwara.tv] Extract more metadata (#829 ) Authored-by: BunnyHelp	2021-09-01 00:59:30 +05:30
Luc Ritchie	b2eeee0ce0	[afreecatv] Tolerate failure to parse date string (#832 ) Authored by: wlritchi	2021-08-30 21:37:34 +05:30
Luc Ritchie	875cfb8cbc	[afreecatv] Fix adult VODs (#831 ) Original PR: https://github.com/ytdl-org/youtube-dl/pull/28405 Fixes https://github.com/ytdl-org/youtube-dl/issues/26622, https://github.com/ytdl-org/youtube-dl/issues/26926 Authored by: wlritchi	2021-08-30 21:05:48 +05:30
The Hatsune Daishi	b8773e63f0	[build] Add homebrew taps (#827 ) https://github.com/yt-dlp/homebrew-taps Closes: #754, #770 Authored by: nao20010128nao	2021-08-30 20:07:43 +05:30
u-spec-png	05664a2f7b	[CDA] Add more formats (#805 ) Fixes: #791, https://github.com/ytdl-org/youtube-dl/issues/29844 Authored by: u-spec-png	2021-08-30 19:37:03 +05:30
pukkandan	2ee6389bef	[build] Fix bug in making `yt-dlp.tar.gz`	2021-08-30 08:28:49 +05:30
coletdjnz	62cdaaf0e2	[StarTV] Add extractor for startv.com.tr (#815 ) Authored-by: mrfade, coletdjnz Related: https://github.com/ytdl-org/youtube-dl/issues/22715	2021-08-29 22:29:42 +00:00
coletdjnz	419508eabb	[Motherless] Fix extractor (#809 ) Authored-by: coletdjnz Fixes #806, https://github.com/ytdl-org/youtube-dl/issues/29626	2021-08-29 22:22:57 +00:00
Sipherdrakon	54153fb71b	[VH1,TVLand] Fix extractors (#784 ) Fixes #745 but not #713 Authored by: Sipherdrakon	2021-08-30 03:20:58 +05:30
zenerdi0de	1dd6d9ca9d	[Patreon] Add PatreonUserIE (#573 ) Authored by: zenerdi0de	2021-08-30 03:17:50 +05:30
IONECarter	356ac009d3	[peloton] Add extractor (#192 ) Authored by: IONECarter, capntrips, pukkandan	2021-08-30 03:13:59 +05:30
coletdjnz	9a292a620c	[ATV.at] Fix extractor for ATV.at (#816 ) Authored-by: NeroBurner, coletdjnz Fixes https://github.com/ytdl-org/youtube-dl/issues/29079	2021-08-29 21:34:39 +00:00
coletdjnz	7e55872286	[camtube] remove extractor (#810 ) Co-authored-by: alerikaisattera	2021-08-29 21:11:03 +00:00
std-move	2fc14b9925	[Nova] fix extractor (#807 ) Fixes: https://github.com/ytdl-org/youtube-dl/issues/27840 Authored by: std-move	2021-08-29 07:04:42 +05:30
Ashish	58f68fe703	[TV2Hu] Fix `TV2HuIE` and add `TV2HuSeriesIE` (#804 ) Closes #799 Authored by: Ashish0804	2021-08-29 06:44:22 +05:30
animelover1984	abafce59a1	[Niconico] Add Search extractors (#672 ) Authored by: animelover1984, pukkandan	2021-08-28 07:07:13 +05:30
pukkandan	2e7781a93c	[docs] Fix some typos Closes #677, #774	2021-08-28 02:20:40 +05:30
Ashish	bc36bc36a1	[ShemarooMe] Fix extractor (#798 ) Closes #797 Authored by: Ashish0804	2021-08-27 20:39:13 +05:30
Paul Wrubel	d75201a873	Use `os.replace` where applicable (#793 ) When using ```py os.remove(encodeFilename(filename)) os.rename(encodeFilename(temp_filename), encodeFilename(filename)) ``` the `os.remove` need not be atomic and so can be executed arbitrarily compared to the immediately following rename call. It is better to use `os.replace` instead Authored by: paulwrubel	2021-08-27 07:57:20 +05:30
pukkandan	691d5823d6	[aria2c] Obey `--rate-limit`	2021-08-27 00:59:36 +05:30
pukkandan	c311988d19	[youtube] Improve `26e8e04454` The streams of the same itag may have slightly different size/bitrate	2021-08-26 08:27:29 +05:30
pukkandan	26e8e04454	[youtube] Prefer audio stream that YouTube considers default Fixes: https://github.com/ytdl-org/youtube-dl/issues/29864 Related: https://github.com/clsid2/mpc-hc/issues/1268	2021-08-26 08:08:34 +05:30
pukkandan	198e3a04c9	[FormatSort] Remove priority of `lang`	2021-08-26 08:08:33 +05:30
Robin	61bfacb233	[facebook] Update onion URL (#788 ) Authored by: Derkades	2021-08-25 20:31:43 +05:30
Ashish	85a0021fb3	[ProjectVeritas] Add extractor (#790 ) https://github.com/ytdl-org/youtube-dl/issues/26749 Authored by: Ashish0804	2021-08-25 20:17:58 +05:30
Ashish	7a45a1590b	[Epicon] Add extractors (#789 ) Authored by: Ashish0804	2021-08-25 19:33:32 +05:30
CeruleanSky	1c36c1f320	Fix `--no-prefer-free-formats` (#787 ) Authored by: CeruleanSky	2021-08-25 17:19:05 +05:30
pukkandan	e0493e90fc	fix bug in `88acdbc269`	2021-08-25 10:26:09 +05:30
The Hatsune Daishi	1931a55ee8	[radiko] Add extractors (#731 ) https://github.com/ytdl-org/youtube-dl/issues/29840 Authored by: nao20010128nao	2021-08-25 10:18:27 +05:30
i6t	63b1ad0f05	[iwara] Add thumbnail (#781 ) Authored by: i6t	2021-08-25 03:06:15 +05:30
coletdjnz	0bb1bc1b10	[youtube] Remove annotations and deprecate `--write-annotations` (#765 ) Closes #692 Authored by: coletdjnz	2021-08-24 09:22:40 +05:30
pukkandan	45842107b9	fix bug in `6251555f1c` :ci skip	2021-08-24 06:23:21 +05:30
pukkandan	6251555f1c	[downloader/ffmpeg] Support for DASH manifests (experimental) Closes #159	2021-08-24 05:52:00 +05:30
pukkandan	330690a214	[downloader/ffmpeg] Allow passing custom arguments before -i Closes #686	2021-08-24 04:24:12 +05:30
tandy1000	91d4b32bb6	[ManotoTV] Add new extractors (#767 ) Authored by: tandy1000	2021-08-24 00:15:46 +05:30
pukkandan	a181cd0c60	[facebook] Fix metadata extraction Original PR: https://github.com/ytdl-org/youtube-dl/pull/29796 Closes #453, https://github.com/ytdl-org/youtube-dl/issues/29421, https://github.com/ytdl-org/youtube-dl/issues/23627, https://github.com/ytdl-org/youtube-dl/issues/23180, https://github.com/ytdl-org/youtube-dl/issues/14156 Authored by: kikuyan	2021-08-23 22:07:00 +05:30
Ashish	ea81966e64	[TV2] Fix extractor (#766 ) Closes #764 Authored by: Ashish0804	2021-08-23 21:32:33 +05:30
Ashish	2acf2ce5cb	[GabTV] Add extractor (#768 ) Closes #499 Authored by: Ashish0804	2021-08-23 21:30:39 +05:30
Ashish	f7f18f905c	[tiktok] Add TikTokUserIE (#756 ) Authored-by: Ashish0804, pukkandan	2021-08-23 20:12:23 +05:30
pukkandan	4f8b70b593	[TikTok] Fix metadata extraction	2021-08-23 19:31:28 +05:30
MinePlayersPE	e43e9f3c2c	[aljazeera] Fix extractor (#763 ) Closes #762, https://github.com/ytdl-org/youtube-dl/issues/29517 Authored by: MinePlayersPE	2021-08-23 15:24:15 +05:30
pukkandan	71dd5d4a00	[peertube] handle new video URL format Closes #722, https://github.com/ytdl-org/youtube-dl/issues/29782 Original PR: https://github.com/ytdl-org/youtube-dl/pull/29475 Authored by: Chocobozzz	2021-08-23 06:26:35 +05:30
nyuszika7h	52a2f994c9	[adobepass] Fix Verizon SAML login (#743 ) Original PR: https://github.com/ytdl-org/youtube-dl/pull/19136 from `64bddfe15c` Authored-by: nyuszika7h, ParadoxGBB <paradoxgbb@yahoo.com>	2021-08-23 06:08:32 +05:30
pukkandan	8b7491c8d1	Fix `add_info_extractor` when used via API Bug from: `251ae04e6a`	2021-08-23 05:31:55 +05:30
pukkandan	251ae04e6a	[lazy_extractor] Create instance only after pre-checking archive	2021-08-23 05:06:39 +05:30
pukkandan	5bc4a65eea	[lazy_extractor] Import actual class if an attribute is accessed Now all core tests pass with lazy extraction enabled	2021-08-23 04:02:06 +05:30
pukkandan	1151c4079a	[extractor] Show video id in error messages if possible	2021-08-23 02:49:07 +05:30
pukkandan	88acdbc269	[extractor] Better error message for DRM (#729 ) Closes #636	2021-08-23 01:38:38 +05:30
Tom-Oliver Heidel	9b5fa9ee7c	[youtube] Add av01 itags to known formats list (#747 ) Authored by: blackjack4494	2021-08-23 01:29:43 +05:30
mahanstreamer	aca5774e68	[bitchute] Fix test (#758 ) Authored by: mahanstreamer	2021-08-23 01:28:23 +05:30
pukkandan	3fb4e21b38	[lazy_extractors] Fix `suitable` and add flake8 test	2021-08-23 01:04:29 +05:30
pukkandan	4dfbf8696b	[utils] Add `parse_qs`	2021-08-23 00:50:43 +05:30
pukkandan	8fc54b1230	[youtube] Add `shorts` to `_VALID_URL` Normally the generic extractor will redirect the URL, but the cookies consent screen may sometimes appear instead Closes #752	2021-08-23 00:50:42 +05:30
pukkandan	da33e35b05	Don't try to merge with final extension The formats may not be directly mergable into the final extension	2021-08-23 00:50:41 +05:30
pukkandan	5ad28e7ffd	[extractor] Common function `_match_valid_url`	2021-08-23 00:50:40 +05:30
Jérôme Duval	f79ec47d71	[tv5mondeplus] Fix extractor (#739 ) Authored by: korli	2021-08-21 02:04:51 +05:30
Ashish	45b0596290	[HearThisAtIE] Fix extractor (#742 ) Closes: #740 Authored by: Ashish0804	2021-08-21 01:09:59 +05:30
Ashish	96c23f3be8	[Zee5] Fix extractor and add subtitles (#733 ) Closes #728 Authored by Ashish0804	2021-08-21 00:43:12 +05:30
CHJ85	6e7dfe4959	[BannedVideo] Add Extractor (#717 ) Closes: #669 Original PR: https://github.com/ytdl-org/youtube-dl/pull/24572 Authored by: smege1001, blackjack4494, pukkandan	2021-08-21 00:15:00 +05:30
animelover1984	c34f505b04	[bilibili] Add category extractor (#695 ) Authored by: animelover1984	2021-08-20 23:57:40 +05:30
Ashish	14183d1f80	[Hungama] Fix `HungamaSongIE` and add `HungamaAlbumPlaylistIE` (#744 ) Authored by: Ashish0804	2021-08-20 23:46:59 +05:30
pukkandan	58adec4677	Fix `extra_info` being reused across runs Closes #727	2021-08-19 03:10:58 +05:30
pukkandan	9e598870dd	Fix `playlist_index` not obeying `playlist_start` and add tests Closes #720	2021-08-17 19:06:10 +05:30
pukkandan	8f18aca871	Let `--match-filter` reject entries early Makes redundant: `--match-title`, `--reject-title`, `--min-views`, `--max-views`	2021-08-17 04:29:56 +05:30
pukkandan	3ad56b4236	Fix `-J` when there are failed videos	2021-08-17 04:29:55 +05:30
Glenn Slayden	5d62709bc7	[cleanup] Replace improper use of tab in trovo (#719 ) :ci skip Authored by: glenn-slayden	2021-08-17 04:19:31 +05:30
zootedb0t	7581d2467a	[docs] fix typo (#715 ) Authored by: zootedb0t	2021-08-16 21:59:40 +05:30
shirt	5fa206fb54	[ParamountPlus] Fix geo verification (#711 ) Closes #681 Authored by: shirt	2021-08-16 12:13:24 +05:30
mzbaulhaque	df2a5633da	[pornhub] Separate and fix playlist extractor (#700 ) Closes #680 Authored by: mzbaulhaque	2021-08-15 23:02:48 +05:30
Felix S	7a6742b5f9	[webvtt] Fix timestamp overflow adjustment (#698 ) In some streams, empty segments may appear with a bogus, non-monotone MPEG timestamp. This should not be considered as an overflow Authored by: fstirlitz	2021-08-15 21:03:06 +05:30
The Hatsune Daishi	e040bb0a41	[voicy] Add extractor (#667 ) Authored by: nao20010128nao	2021-08-15 20:49:54 +05:30
pukkandan	f8fabc9930	[kakao] Fix extractor Closes #699	2021-08-15 14:31:27 +05:30
jhwgh1968	d967c68e4c	[eroprofile] Fix page skipping in albums (#701 ) Bug from #658 Authored by: jhwgh1968	2021-08-15 11:32:11 +05:30
SsSsS	3dd39c5f9a	[instagram] Add referrer to prevent throttling (#676 ) Code from: https://github.com/ytdl-org/youtube-dl/pull/29751 Fixes: https://github.com/ytdl-org/youtube-dl/issues/29736 Authored by: u-spec-png, kikuyan	2021-08-15 00:45:01 +05:30
mzbaulhaque	be44eefd5e	[filmmodu] Add extractor (#690 ) Closes #288 Authored by: mzbaulhaque	2021-08-15 00:40:56 +05:30
pukkandan	f775c83110	Fix `--force-overwrites` when using `-k` For formats that need merge, the `.fxxx` files are not removed before downloading the corresponding `.part` files. This causes the rename to fail	2021-08-15 00:28:49 +05:30
pukkandan	b714b41f81	[soundcloud] Refetch `client_id` on 403 Closes #673	2021-08-15 00:28:49 +05:30
pukkandan	31654882e9	[options] Add `_set_from_options_callback`	2021-08-15 00:26:34 +05:30
pukkandan	86c66b2d3e	Fix `-F` for extractors that directly return url Related: #693	2021-08-15 00:26:34 +05:30
pukkandan	37242e56f2	Fix bug during subtitle conversion	2021-08-15 00:26:33 +05:30
pukkandan	6c7274ecd2	Fix resuming of single formats when using --no-part Closes #576	2021-08-15 00:26:32 +05:30
Kid	5c333d7496	[lazy_extractor] Bugfix for when plugin directory doesn't exist (#691 ) Bug introduced by: `0b2e9d2c30` Authored by: kidonng	2021-08-13 20:54:17 +05:30
coletdjnz	641ad5d813	[youtube] Extract error messages from HTTPError response (#644 ) Authored by: coletdjnz	2021-08-13 11:48:26 +05:30
Felix S	0715f7e19b	Revert erroneous use of the `Content-Length` header (#637 ) This reverts commit `6c907eb33f` The use of the Content-Length value here is erroneous and may lead to truncated downloads if a compression scheme is specified in the Content-Encoding header, as the Content-Length header refers to the size of encoded data, not of the raw bytestream. This has been noticed in the wild with WebVTT subtitle segments. Authored by: fstirlitz	2021-08-11 21:09:17 +05:30
pukkandan	a8731fcc1d	minor bugfixes bugs due to `be2fc5b212`, `e9f4ccd19e`	2021-08-11 20:27:30 +05:30
pukkandan	5a64127f94	[docs] Fix credits of `246fb276e0` It is authored by mzbaulhaque - The commit message is wrong :ci skip all	2021-08-10 22:32:23 +05:30
pukkandan	ade6dc5e9e	[version] update :ci skip all	2021-08-10 20:51:47 +05:30