Release 2021.03.01

Add option --extractor-retries to retry on known extractor errors
* Currently only used by youtube Fixes https://github.com/ytdl-org/youtube-dl/issues/28194 Possibly also fixes: https://github.com/ytdl-org/youtube-dl/issues/28289 (can not confirm since the issue isn't reliably reproducible)
2026-01-12 01:41:26 +00:00 · 2021-03-01 05:39:50 +05:30 · 2021-03-01 05:18:37 +05:30 · 2021-02-28 20:56:32 +05:30 · 2021-02-28 20:26:08 +05:30 · 2021-02-27 18:14:42 +05:30
45 changed files with 1611 additions and 1514 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1 @@
+Makefile* text whitespace=-tab-in-indent
--- a/.github/ISSUE_TEMPLATE/1_broken_site.md
+++ b/.github/ISSUE_TEMPLATE/1_broken_site.md
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.19. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running yt-dlp version **2021.02.19**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.24**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.02.19
+ [debug] yt-dlp version 2021.02.24
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/.github/ISSUE_TEMPLATE/2_site_support_request.md
+++ b/.github/ISSUE_TEMPLATE/2_site_support_request.md
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.19. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running yt-dlp version **2021.02.19**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.24**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones
--- a/.github/ISSUE_TEMPLATE/3_site_feature_request.md
+++ b/.github/ISSUE_TEMPLATE/3_site_feature_request.md
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.19. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running yt-dlp version **2021.02.19**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.24**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones


--- a/.github/ISSUE_TEMPLATE/4_bug_report.md
+++ b/.github/ISSUE_TEMPLATE/4_bug_report.md
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.19. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running yt-dlp version **2021.02.19**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.24**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -46,7 +46,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.02.19
+ [debug] yt-dlp version 2021.02.24
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/.github/ISSUE_TEMPLATE/5_feature_request.md
+++ b/.github/ISSUE_TEMPLATE/5_feature_request.md
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.19. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.02.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a feature request
- [ ] I've verified that I'm running yt-dlp version **2021.02.19**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.24**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones


--- a/.github/workflows/core.yml
+++ b/.github/workflows/core.yml
@@ -3,7 +3,7 @@ on: [push, pull_request]
 jobs:
  tests:
    name: Core Tests
-    if: "!contains(github.event.head_commit.message, 'ci skip all')"
+    if: "!contains(github.event.head_commit.message, 'ci skip')"
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: true
--- a/.github/workflows/download.yml
+++ b/.github/workflows/download.yml
@@ -3,7 +3,7 @@ on: [push, pull_request]
 jobs:
  tests:
    name: Download Tests
-    if: "!contains(github.event.head_commit.message, 'ci skip dl') && !contains(github.event.head_commit.message, 'ci skip all')"
+    if: "contains(github.event.head_commit.message, 'ci run dl')"
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: true
--- a/.gitignore
+++ b/.gitignore
@@ -8,6 +8,7 @@ dist/
 zip/
 tmp/
 venv/
+completions/

 # Misc
 *~
@@ -34,8 +35,9 @@ README.txt
 *.spec

 # Binary
-youtube-dl
-youtube-dlc
+/youtube-dl
+/youtube-dlc
+/yt-dlp
 yt-dlp.zip
 *.exe

@@ -50,12 +52,14 @@ yt-dlp.zip
 *.m4v
 *.mp3
 *.3gp
+*.webm
 *.wav
 *.ape
 *.mkv
 *.swf
 *.part
 *.ytdl
+*.dump
 *.frag
 *.frag.urls
 *.aria2
--- a/.readthedocs.yml
+++ b/.readthedocs.yml
@@ -20,4 +20,3 @@ python:
    version: 3
    install:
    - requirements: docs/requirements.txt
-    - requirements: requirements.txt
--- a/.travis.yml.disabled
+++ b/.travis.yml.disabled
@@ -1,38 +0,0 @@
-language: python
-python:
-  - "2.6"
-  - "2.7"
-  - "3.2"
-  - "3.3"
-  - "3.4"
-  - "3.5"
-  - "3.6"
-  - "pypy"
-  - "pypy3"
-dist: trusty
-env:
-  - YTDL_TEST_SET=core
-jobs:
-  include:
-    - python: 3.7
-      dist: xenial
-      env: YTDL_TEST_SET=core
-    - python: 3.8
-      dist: xenial
-      env: YTDL_TEST_SET=core
-    - python: 3.8-dev
-      dist: xenial
-      env: YTDL_TEST_SET=core
-    - env: JYTHON=true; YTDL_TEST_SET=core
-    - name: flake8
-      python: 3.8
-      dist: xenial
-      install: pip install flake8
-      script: flake8 .
-  fast_finish: true
-  allow_failures:
-    - env: YTDL_TEST_SET=download
-    - env: JYTHON=true; YTDL_TEST_SET=core
-before_install:
-  - if [ "$JYTHON" == "true" ]; then ./devscripts/install_jython.sh; export PATH="$HOME/jython/bin:$PATH"; fi
-script: ./devscripts/run_tests.sh
--- a/2
+++ b/2
@@ -23,3 +23,5 @@ tsukumi
 bbepis
 Pccode66
 Ashish
+RobinD42
+hseg
--- a/Changelog.md
+++ b/Changelog.md
@@ -17,9 +17,31 @@
 -->


+### 2021.03.01
+* Allow specifying path in `--external-downloader`
+* Add option `--sleep-requests` to sleep b/w requests
+* Add option `--extractor-retries` to retry on known extractor errors
+* Extract comments only when needed
+* `--get-comments` doesn't imply `--write-info-json` if `-J`, `-j` or `--print-json` are used
+* [youtube] Retry on more known errors than just HTTP-5xx
+* [tennistv] Fix format sorting
+* [readthedocs] Improvements by [shirt](https://github.com/shirt-dev)
+* [hls] Fix bug with m3u8 format extraction
+* [bilibiliaudio] Recognize the file as audio-only
+* [hrfensehen] Fix wrong import
+* [youtube] Fix inconsistent `webpage_url`
+* [hls] Enable `--hls-use-mpegts` by default when downloading live-streams
+* [viki] Fix viki play pass authentication by [RobinD42](https://github.com/RobinD42)
+* [embedthumbnail] Fix bug with deleting original thumbnail
+* [build] Fix completion paths, zsh pip completion install by [hseg](https://github.com/hseg)
+* [ci] Disable download tests unless specifically invoked
+* Cleanup some code and fix typos
+
+
 ### 2021.02.24
 * Moved project to an organization [yt-dlp](https://github.com/yt-dlp)
 * **Completely changed project name to yt-dlp** by [Pccode66](https://github.com/Pccode66) and [pukkandan](https://github.com/pukkandan)
+    * Also, `youtube-dlc` config files are no longer loaded
 * **Merge youtube-dl:** Upto [commit/4460329](https://github.com/ytdl-org/youtube-dl/commit/44603290e5002153f3ebad6230cc73aef42cc2cd) (except tmz, gedi)
 * [Readthedocs](https://yt-dlp.readthedocs.io) support by [shirt](https://github.com/shirt-dev)
 * [youtube] Show if video was a live stream in info (`was_live`)
@@ -28,12 +50,12 @@
 * [tennistv] Fix extractor
 * [hls] Support media initialization by [shirt](https://github.com/shirt-dev)
 * [hls] Added options `--hls-split-discontinuity` to better support media discontinuity by [shirt](https://github.com/shirt-dev)
-* [ffmpeg] Allow passing custom arguments before -i using `--ppa "ffmpeg_i1:ARGS"` synatax
+* [ffmpeg] Allow passing custom arguments before -i using `--ppa "ffmpeg_i1:ARGS"` syntax
 * Fix `--windows-filenames` removing `/` from UNIX paths
 * [hls] Show warning if pycryptodome is not found
 * [documentation] Improvements
    * Fix documentation of `Extractor Options`
-    * Document `all` in format selection (Closes #101)
+    * Document `all` in format selection
    * Document `playable_in_embed` in output templates


@@ -276,7 +298,7 @@
    * Added `--video-multistreams`, `--no-video-multistreams`, `--audio-multistreams`, `--no-audio-multistreams`
    * Added `b`,`w`,`v`,`a` as alias for `best`, `worst`, `video` and `audio` respectively
 * **Shortcut Options:** Added `--write-link`, `--write-url-link`, `--write-webloc-link`, `--write-desktop-link` by [h-h-h-h](https://github.com/h-h-h-h) - See [Internet Shortcut Options](README.md#internet-shortcut-options) for details
-* **Sponskrub integration:** Added `--sponskrub`, `--sponskrub-cut`, `--sponskrub-force`, `--sponskrub-location`, `--sponskrub-args` - See [SponSkrub Options](README.md#sponskrub-options-sponsorblock) for details
+* **Sponskrub integration:** Added `--sponskrub`, `--sponskrub-cut`, `--sponskrub-force`, `--sponskrub-location`, `--sponskrub-args` - See [SponSkrub Options](README.md#sponskrub-sponsorblock-options) for details
 * Added `--force-download-archive` (`--force-write-archive`) by [h-h-h-h](https://github.com/h-h-h-h)
 * Added `--list-formats-as-table`,  `--list-formats-old`
 * **Negative Options:** Makes it possible to negate most boolean options by adding a `no-` to the switch. Usefull when you want to reverse an option that is defined in a config file
--- a/49
+++ b/49
@@ -1,10 +1,10 @@
 all: yt-dlp doc man
 doc: README.md CONTRIBUTING.md issuetemplates supportedsites
-man: README.txt yt-dlp.1 yt-dlp.bash-completion yt-dlp.zsh yt-dlp.fish
+man: README.txt yt-dlp.1 bash-completion zsh-completion fish-completion


 clean:
-	rm -rf yt-dlp.1.temp.md yt-dlp.1 yt-dlp.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz yt-dlp.zsh yt-dlp.fish yt_dlp/extractor/lazy_extractors.py *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.spec *.frag *.frag.urls *.frag.aria2 CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe
+	rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ yt_dlp/extractor/lazy_extractors.py *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.spec *.frag *.frag.urls *.frag.aria2 CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe
 	find . -name "*.pyc" -delete
 	find . -name "*.class" -delete

@@ -21,17 +21,12 @@ SYSCONFDIR = $(shell if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then ech
 # set markdown input format to "markdown-smart" for pandoc version 2 and to "markdown" for pandoc prior to version 2
 MARKDOWN = $(shell if [ `pandoc -v | head -n1 | cut -d" " -f2 | head -c1` = "2" ]; then echo markdown-smart; else echo markdown; fi)

-install: yt-dlp yt-dlp.1 yt-dlp.bash-completion yt-dlp.zsh yt-dlp.fish
-	install -d $(DESTDIR)$(BINDIR)
-	install -m 755 yt-dlp $(DESTDIR)$(BINDIR)
-	install -d $(DESTDIR)$(MANDIR)/man1
-	install -m 644 yt-dlp.1 $(DESTDIR)$(MANDIR)/man1
-	install -d $(DESTDIR)$(SYSCONFDIR)/bash_completion.d
-	install -m 644 yt-dlp.bash-completion $(DESTDIR)$(SYSCONFDIR)/bash_completion.d/yt-dlp
-	install -d $(DESTDIR)$(SHAREDIR)/zsh/site-functions
-	install -m 644 yt-dlp.zsh $(DESTDIR)$(SHAREDIR)/zsh/site-functions/_yt-dlp
-	install -d $(DESTDIR)$(SYSCONFDIR)/fish/completions
-	install -m 644 yt-dlp.fish $(DESTDIR)$(SYSCONFDIR)/fish/completions/yt-dlp.fish
+install: yt-dlp yt-dlp.1 bash-completion zsh-completion fish-completion
+	install -Dm755 yt-dlp $(DESTDIR)$(BINDIR)
+	install -Dm644 yt-dlp.1 $(DESTDIR)$(MANDIR)/man1
+	install -Dm644 completions/bash/yt-dlp $(DESTDIR)$(SHAREDIR)/bash-completion/completions/yt-dlp
+	install -Dm644 completions/zsh/_yt-dlp $(DESTDIR)$(SHAREDIR)/zsh/site-functions/_yt-dlp
+	install -Dm644 completions/fish/yt-dlp.fish $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d/yt-dlp.fish

 codetest:
 	flake8 .
@@ -61,7 +56,7 @@ tar: yt-dlp.tar.gz

 .PHONY: all clean install test tar bash-completion pypi-files zsh-completion fish-completion ot offlinetest codetest supportedsites

-pypi-files: yt-dlp.bash-completion README.txt yt-dlp.1 yt-dlp.fish
+pypi-files: README.txt yt-dlp.1 bash-completion zsh-completion fish-completion

 yt-dlp: yt_dlp/*.py yt_dlp/*/*.py
 	mkdir -p zip
@@ -92,7 +87,7 @@ issuetemplates: devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_
 	$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/5_feature_request.md .github/ISSUE_TEMPLATE/5_feature_request.md

 supportedsites:
-	$(PYTHON) devscripts/make_supportedsites.py docs/supportedsites.md
+	$(PYTHON) devscripts/make_supportedsites.py supportedsites.md

 README.txt: README.md
 	pandoc -f $(MARKDOWN) -t plain README.md -o README.txt
@@ -102,20 +97,23 @@ yt-dlp.1: README.md
 	pandoc -s -f $(MARKDOWN) -t man yt-dlp.1.temp.md -o yt-dlp.1
 	rm -f yt-dlp.1.temp.md

-yt-dlp.bash-completion: yt_dlp/*.py yt_dlp/*/*.py devscripts/bash-completion.in
+completions/bash/yt-dlp: yt_dlp/*.py yt_dlp/*/*.py devscripts/bash-completion.in
+	mkdir -p completions/bash
 	$(PYTHON) devscripts/bash-completion.py

-bash-completion: yt-dlp.bash-completion
+bash-completion: completions/bash/yt-dlp

-yt-dlp.zsh: yt_dlp/*.py yt_dlp/*/*.py devscripts/zsh-completion.in
+completions/zsh/_yt-dlp: yt_dlp/*.py yt_dlp/*/*.py devscripts/zsh-completion.in
+	mkdir -p completions/zsh
 	$(PYTHON) devscripts/zsh-completion.py

-zsh-completion: yt-dlp.zsh
+zsh-completion: completions/zsh/_yt-dlp

-yt-dlp.fish: yt_dlp/*.py yt_dlp/*/*.py devscripts/fish-completion.in
+completions/fish/yt-dlp.fish: yt_dlp/*.py yt_dlp/*/*.py devscripts/fish-completion.in
+	mkdir -p completions/fish
 	$(PYTHON) devscripts/fish-completion.py

-fish-completion: yt-dlp.fish
+fish-completion: completions/fish/yt-dlp.fish

 lazy-extractors: yt_dlp/extractor/lazy_extractors.py

@@ -123,7 +121,7 @@ _EXTRACTOR_FILES = $(shell find yt_dlp/extractor -iname '*.py' -and -not -iname
 yt_dlp/extractor/lazy_extractors.py: devscripts/make_lazy_extractors.py devscripts/lazy_load_template.py $(_EXTRACTOR_FILES)
 	$(PYTHON) devscripts/make_lazy_extractors.py $@

-yt-dlp.tar.gz: yt-dlp README.md README.txt yt-dlp.1 yt-dlp.bash-completion yt-dlp.zsh yt-dlp.fish ChangeLog AUTHORS
+yt-dlp.tar.gz: yt-dlp README.md README.txt yt-dlp.1 bash-completion zsh-completion fish-completion ChangeLog AUTHORS
 	@tar -czf yt-dlp.tar.gz --transform "s|^|yt-dlp/|" --owner 0 --group 0 \
 		--exclude '*.DS_Store' \
 		--exclude '*.kate-swp' \
@@ -135,7 +133,6 @@ yt-dlp.tar.gz: yt-dlp README.md README.txt yt-dlp.1 yt-dlp.bash-completion yt-dl
 		--exclude 'docs/_build' \
 		-- \
 		bin devscripts test yt_dlp docs \
-		ChangeLog AUTHORS LICENSE README.md README.txt \
-		Makefile MANIFEST.in yt-dlp.1 yt-dlp.bash-completion \
-		yt-dlp.zsh yt-dlp.fish setup.py setup.cfg \
-		yt-dlp
+		ChangeLog AUTHORS LICENSE README.md supportedsites.md README.txt \
+		Makefile MANIFEST.in yt-dlp.1 completions \
+		setup.py setup.cfg yt-dlp
--- a/README.md
+++ b/README.md
@@ -11,7 +11,7 @@
 [![PyPi Downloads](https://img.shields.io/pypi/dm/yt-dlp?label=PyPi)](https://pypi.org/project/yt-dlp)
 [![Doc Status](https://readthedocs.org/projects/yt-dlp/badge/?version=latest)](https://yt-dlp.readthedocs.io)

-A command-line program to download videos from youtube.com and many other [video platforms](docs/supportedsites.md)
+A command-line program to download videos from youtube.com and many other [video platforms](supportedsites.md)

 This is a fork of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) which is inturn a fork of [youtube-dl](https://github.com/ytdl-org/youtube-dl)

@@ -57,7 +57,7 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/

 * **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))

-* **Merged with youtube-dl v2021.02.10**: You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
+* **Merged with youtube-dl v2021.02.22**: You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)

 * **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--get-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, Playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.

@@ -92,7 +92,7 @@ See [changelog](Changelog.md) or [commits](https://github.com/yt-dlp/yt-dlp/comm

 **PS**: Some of these changes are already in youtube-dlc, but are still unreleased. See [this](Changelog.md#unreleased-changes-in-blackjack4494yt-dlc) for details

-If you are coming from [youtube-dl](https://github.com/ytdl-org/youtube-dl), the amount of changes are very large. Compare [options](#options) and [supported sites](docs/supportedsites.md) with youtube-dl's to get an idea of the massive number of features/patches [youtube-dlc](https://github.com/blackjack4494/yt-dlc) has accumulated.
+If you are coming from [youtube-dl](https://github.com/ytdl-org/youtube-dl), the amount of changes are very large. Compare [options](#options) and [supported sites](supportedsites.md) with youtube-dl's to get an idea of the massive number of features/patches [youtube-dlc](https://github.com/blackjack4494/yt-dlc) has accumulated.


 # INSTALLATION
@@ -122,12 +122,12 @@ You can also build the executable without any version info or metadata by using:

 **For Unix**:
 You will need the required build tools: `python`, `make` (GNU), `pandoc`, `zip`, `nosetests`  
-Then simply run `make`. You can also run `make youtube_dlc` instead to compile only the binary without updating any of the additional files
+Then simply run `make`. You can also run `make yt-dlp` instead to compile only the binary without updating any of the additional files

 **Note**: In either platform, `devscripts\update-version.py` can be used to automatically update the version number

 # DESCRIPTION
-**yt-dlp** is a command-line program to download videos from youtube.com many other [video platforms](docs/supportedsites.md). It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific. It should work on your Unix box, on Windows or on macOS. It is released to the public domain, which means you can modify it, redistribute it or use it however you like.
+**yt-dlp** is a command-line program to download videos from youtube.com many other [video platforms](supportedsites.md). It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific. It should work on your Unix box, on Windows or on macOS. It is released to the public domain, which means you can modify it, redistribute it or use it however you like.

    yt-dlp [OPTIONS] [--] URL [URL...]

@@ -245,7 +245,7 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
                                     "OUTPUT TEMPLATE" for a list of available
                                     keys) to match if the key is present, !key
                                     to check if the key is not present,
-                                     key>NUMBER (like "comment_count > 12", also
+                                     key>NUMBER (like "view_count > 12", also
                                     works with >=, <, <=, !=, =) to compare
                                     against a number, key = 'LITERAL' (like
                                     "uploader = 'Mike Smith'", also works with
@@ -317,13 +317,19 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
                                     ffmpeg
    --hls-prefer-ffmpeg              Use ffmpeg instead of the native HLS
                                     downloader
-    --hls-use-mpegts                 Use the mpegts container for HLS videos,
-                                     allowing to play the video while
-                                     downloading (some players may not be able
-                                     to play it)
-    --external-downloader NAME       Use the specified external downloader.
-                                     Currently supports aria2c, avconv, axel,
-                                     curl, ffmpeg, httpie, wget
+    --hls-use-mpegts                 Use the mpegts container for HLS videos;
+                                     allowing some players to play the video
+                                     while downloading, and reducing the chance
+                                     of file corruption if download is
+                                     interrupted. This is enabled by default for
+                                     live streams
+    --no-hls-use-mpegts              Do not use the mpegts container for HLS
+                                     videos. This is default when not
+                                     downloading live streams
+    --external-downloader NAME       Name or path of the external downloader to
+                                     use. Currently supports aria2c, avconv,
+                                     axel, curl, ffmpeg, httpie, wget
+                                     (Recommended: aria2c)
    --downloader-args NAME:ARGS      Give these arguments to the external
                                     downloader. Specify the downloader name and
                                     the arguments separated by a colon ":". You
@@ -397,7 +403,9 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
    --no-write-playlist-metafiles    Do not write playlist metadata when using
                                     --write-info-json, --write-description etc.
    --get-comments                   Retrieve video comments to be placed in the
-                                     .info.json file
+                                     .info.json file. The comments are fetched
+                                     even without this option if the extraction
+                                     is known to be quick
    --load-info-json FILE            JSON file containing the video information
                                     (created with the "--write-info-json"
                                     option)
@@ -485,6 +493,8 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
    --bidi-workaround                Work around terminals that lack
                                     bidirectional text support. Requires bidiv
                                     or fribidi executable in PATH
+    --sleep-requests SECONDS         Number of seconds to sleep between requests
+                                     during data extraction
    --sleep-interval SECONDS         Number of seconds to sleep before each
                                     download when used alone or a lower bound
                                     of a range for randomized sleep before each
@@ -495,7 +505,8 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
                                     before each download (maximum possible
                                     number of seconds to sleep). Must only be
                                     used along with --min-sleep-interval
-    --sleep-subtitles SECONDS        Enforce sleep interval on subtitles as well
+    --sleep-subtitles SECONDS        Number of seconds to sleep before each
+                                     subtitle download

 ## Video Format Options:
    -f, --format FORMAT              Video format code, see "FORMAT SELECTION"
@@ -641,7 +652,7 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
                                     similar syntax to the output template can
                                     also be used. The parsed parameters replace
                                     any existing values and can be use in
-                                     output templateThis option can be used
+                                     output template. This option can be used
                                     multiple times. Example: --parse-metadata
                                     "title:%(artist)s - %(title)s" matches a
                                     title like "Coldplay - Paradise". Example
@@ -686,6 +697,8 @@ Then simply run `make`. You can also run `make youtube_dlc` instead to compile o
                                     directory

 ## Extractor Options:
+    --extractor-retries RETRIES      Number of retries for known extractor
+                                     errors (default is 10), or "infinite"
    --allow-dynamic-mpd              Process dynamic DASH manifests (default)
                                     (Alias: --no-ignore-dynamic-mpd)
    --ignore-dynamic-mpd             Do not process dynamic DASH manifests
@@ -805,7 +818,7 @@ The available fields are:
 - `dislike_count` (numeric): Number of negative ratings of the video
 - `repost_count` (numeric): Number of reposts of the video
 - `average_rating` (numeric): Average rating give by users, the scale used depends on the webpage
- - `comment_count` (numeric): Number of comments on the video
+ - `comment_count` (numeric): Number of comments on the video (For some extractors, comments are only downloaded at the end, and so this field cannot be used)
 - `age_limit` (numeric): Age restriction for the video (years)
 - `is_live` (boolean): Whether this video is a live stream or a fixed-length video
 - `was_live` (boolean): Whether this video was originally a live stream
--- a/devscripts/bash-completion.py
+++ b/devscripts/bash-completion.py
@@ -8,7 +8,7 @@ import sys
 sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
 import yt_dlp

-BASH_COMPLETION_FILE = "yt-dlp.bash-completion"
+BASH_COMPLETION_FILE = "completions/bash/yt-dlp"
 BASH_COMPLETION_TEMPLATE = "devscripts/bash-completion.in"


--- a/devscripts/fish-completion.py
+++ b/devscripts/fish-completion.py
@@ -10,7 +10,7 @@ sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
 import yt_dlp
 from yt_dlp.utils import shell_quote

-FISH_COMPLETION_FILE = 'yt-dlp.fish'
+FISH_COMPLETION_FILE = 'completions/fish/yt-dlp.fish'
 FISH_COMPLETION_TEMPLATE = 'devscripts/fish-completion.in'

 EXTRA_ARGS = {
--- a/devscripts/zsh-completion.py
+++ b/devscripts/zsh-completion.py
@@ -8,7 +8,7 @@ import sys
 sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
 import yt_dlp

-ZSH_COMPLETION_FILE = "yt-dlp.zsh"
+ZSH_COMPLETION_FILE = "completions/zsh/_yt-dlp"
 ZSH_COMPLETION_TEMPLATE = "devscripts/zsh-completion.in"


--- a/docs/Changelog.md
+++ b/docs/Changelog.md
@@ -0,0 +1,5 @@
+---
+orphan: true
+---
+```{include} ../Changelog.md
+```
--- a/docs/LICENSE.md
+++ b/docs/LICENSE.md
@@ -0,0 +1,6 @@
+---
+orphan: true
+---
+# LICENSE
+```{include} ../LICENSE
+```
--- a/docs/README.md
+++ b/docs/README.md
@@ -0,0 +1,2 @@
+```{include} ../README.md
+```
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -7,26 +7,21 @@ import os

 # Allows to import yt-dlp
 sys.path.insert(0, os.path.abspath('..'))
-from recommonmark.transform import AutoStructify

 # -- General configuration ------------------------------------------------

-# The suffix of source filenames.
-source_suffix = ['.rst', '.md']
-
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'sphinx.ext.autodoc',
-    'recommonmark',
+    'myst_parser',
 ]

 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

 # The master toctree document.
-master_doc = 'index'
+master_doc = 'README'

 # General information about the project.
 project = u'yt-dlp'
@@ -64,12 +59,10 @@ highlight_language = 'none'
 # so a file named "default.css" will overwrite the builtin "default.css".
 # html_static_path = ['_static']

+# Enable heading anchors
+myst_heading_anchors = 4

-def setup(app):
-    app.add_config_value('recommonmark_config', {
-        'enable_math': False,
-        'enable_inline_math': False,
-        'enable_eval_rst': True,
-        'enable_auto_toc_tree': True,
-    }, True)
-    app.add_transform(AutoStructify)
+# Suppress heading warnings
+suppress_warnings = [
+    'myst.header',
+]
--- a/docs/index.md
+++ b/docs/index.md
@@ -1 +0,0 @@
-../README.md
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,2 +1 @@
-recommonmark>=0.6.0
-m2r2
+myst-parser
--- a/docs/supportedsites.md
+++ b/docs/supportedsites.md
--- a/docs/ytdlp_plugins.md
+++ b/docs/ytdlp_plugins.md
@@ -0,0 +1,6 @@
+---
+orphan: true
+---
+# ytdlp_plugins
+
+See [https://github.com/yt-dlp/yt-dlp/tree/master/ytdlp_plugins](https://github.com/yt-dlp/yt-dlp/tree/master/ytdlp_plugins).
--- a/setup.py
+++ b/setup.py
@@ -27,8 +27,9 @@ if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
    print("inv")
 else:
    files_spec = [
-        ('etc/bash_completion.d', ['yt-dlp.bash-completion']),
-        ('etc/fish/completions', ['yt-dlp.fish']),
+        ('share/bash-completion/completions', ['completions/bash/*']),
+        ('share/zsh/site-functions', ['completions/zsh/*']),
+        ('share/fish/vendor_completions.d', ['completions/fish/*']),
        ('share/doc/yt_dlp', ['README.txt']),
        ('share/man/man1', ['yt-dlp.1'])
    ]
@@ -86,7 +87,7 @@ setup(
        #'Funding': 'https://donate.pypi.org',
    },
    classifiers=[
-	    "Topic :: Multimedia :: Video",
+        "Topic :: Multimedia :: Video",
        "Development Status :: 5 - Production/Stable",
        "Environment :: Console",
        "Programming Language :: Python",
@@ -110,7 +111,7 @@ setup(
        "Operating System :: OS Independent",
    ],
    python_requires='>=2.6',
-	
-	cmdclass={'build_lazy_extractors': build_lazy_extractors},
+
+    cmdclass={'build_lazy_extractors': build_lazy_extractors},
    **params
 )
--- a/supportedsites.md
+++ b/supportedsites.md
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -324,6 +324,8 @@ class YoutubeDL(object):
    source_address:    Client-side IP address to bind to.
    call_home:         Boolean, true iff we are allowed to contact the
                       yt-dlp servers for debugging. (BROKEN)
+    sleep_interval_requests: Number of seconds to sleep between requests
+                       during extraction
    sleep_interval:    Number of seconds to sleep before each download when
                       used alone or a lower bound of a range for randomized
                       sleep before each download (minimum possible number
@@ -334,6 +336,7 @@ class YoutubeDL(object):
                       Must only be used along with sleep_interval.
                       Actual sleep time will be a random float from range
                       [sleep_interval; max_sleep_interval].
+    sleep_interval_subtitles: Number of seconds to sleep before each subtitle download
    listformats:       Print an overview of available video formats and exit.
    list_thumbnails:   Print a table of all thumbnails and exit.
    match_filter:      A function that gets called with the info_dict of
@@ -378,17 +381,18 @@ class YoutubeDL(object):
                        Use 'default' as the name for arguments to passed to all PP

    The following options are used by the extractors:
-    dynamic_mpd:        Whether to process dynamic DASH manifests (default: True)
+    extractor_retries: Number of times to retry for known errors
+    dynamic_mpd:       Whether to process dynamic DASH manifests (default: True)
    hls_split_discontinuity: Split HLS playlists to different formats at
-                        discontinuities such as ad breaks (default: False)
+                       discontinuities such as ad breaks (default: False)
    youtube_include_dash_manifest: If True (default), DASH manifests and related
-                        data will be downloaded and processed by extractor.
-                        You can reduce network I/O by disabling it if you don't
-                        care about DASH. (only for youtube)
+                       data will be downloaded and processed by extractor.
+                       You can reduce network I/O by disabling it if you don't
+                       care about DASH. (only for youtube)
    youtube_include_hls_manifest: If True (default), HLS manifests and related
-                        data will be downloaded and processed by extractor.
-                        You can reduce network I/O by disabling it if you don't
-                        care about HLS. (only for youtube)
+                       data will be downloaded and processed by extractor.
+                       You can reduce network I/O by disabling it if you don't
+                       care about HLS. (only for youtube)
    """

    _NUMERIC_FIELDS = set((
@@ -406,6 +410,7 @@ class YoutubeDL(object):
    _ies = []
    _pps = {'beforedl': [], 'aftermove': [], 'normal': []}
    __prepare_filename_warned = False
+    _first_webpage_request = True
    _download_retcode = None
    _num_downloads = None
    _playlist_level = 0
@@ -420,6 +425,7 @@ class YoutubeDL(object):
        self._ies_instances = {}
        self._pps = {'beforedl': [], 'aftermove': [], 'normal': []}
        self.__prepare_filename_warned = False
+        self._first_webpage_request = True
        self._post_hooks = []
        self._progress_hooks = []
        self._download_retcode = 0
@@ -2036,6 +2042,7 @@ class YoutubeDL(object):
            self.to_stdout(formatSeconds(info_dict['duration']))
        print_mandatory('format')
        if self.params.get('forcejson', False):
+            self.post_extract(info_dict)
            self.to_stdout(json.dumps(info_dict))

    def process_info(self, info_dict):
@@ -2059,6 +2066,7 @@ class YoutubeDL(object):
        if self._match_entry(info_dict, incomplete=False) is not None:
            return

+        self.post_extract(info_dict)
        self._num_downloads += 1

        info_dict = self.pre_process(info_dict)
@@ -2166,15 +2174,6 @@ class YoutubeDL(object):
                    else:
                        try:
                            dl(sub_filename, sub_info, subtitle=True)
-                            '''
-                            if self.params.get('sleep_interval_subtitles', False):
-                                dl(sub_filename, sub_info)
-                            else:
-                                sub_data = ie._request_webpage(
-                                    sub_info['url'], info_dict['id'], note=False).read()
-                                with io.open(encodeFilename(sub_filename), 'wb') as subfile:
-                                    subfile.write(sub_data)
-                            '''
                            files_to_move[sub_filename] = sub_filename_final
                        except (ExtractorError, IOError, OSError, ValueError, compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error) as err:
                            self.report_warning('Unable to download subtitle for "%s": %s' %
@@ -2501,6 +2500,7 @@ class YoutubeDL(object):
                raise
            else:
                if self.params.get('dump_single_json', False):
+                    self.post_extract(res)
                    self.to_stdout(json.dumps(res))

        return self._download_retcode
@@ -2549,6 +2549,24 @@ class YoutubeDL(object):
                    del files_to_move[old_filename]
        return files_to_move, infodict

+    @staticmethod
+    def post_extract(info_dict):
+        def actual_post_extract(info_dict):
+            if info_dict.get('_type') in ('playlist', 'multi_video'):
+                for video_dict in info_dict.get('entries', {}):
+                    actual_post_extract(video_dict)
+                return
+
+            if '__post_extractor' not in info_dict:
+                return
+            post_extractor = info_dict['__post_extractor']
+            if post_extractor:
+                info_dict.update(post_extractor().items())
+            del info_dict['__post_extractor']
+            return
+
+        actual_post_extract(info_dict)
+
    def pre_process(self, ie_info):
        info = dict(ie_info)
        for pp in self._pps['beforedl']:
@@ -2940,7 +2958,7 @@ class YoutubeDL(object):
                self.to_screen('[%s] %s: Thumbnail %sis already present' %
                               (info_dict['extractor'], info_dict['id'], thumb_display_id))
            else:
-                self.to_screen('[%s] %s: Downloading thumbnail %s...' %
+                self.to_screen('[%s] %s: Downloading thumbnail %s ...' %
                               (info_dict['extractor'], info_dict['id'], thumb_display_id))
                try:
                    uf = self.urlopen(t['url'])
--- a/yt_dlp/init.py
+++ b/yt_dlp/init.py
@@ -169,25 +169,33 @@ def _real_main(argv=None):
            parser.error('max sleep interval must be greater than or equal to min sleep interval')
    else:
        opts.max_sleep_interval = opts.sleep_interval
+    if opts.sleep_interval_subtitles is not None:
+        if opts.sleep_interval_subtitles < 0:
+            parser.error('subtitles sleep interval must be positive or 0')
+    if opts.sleep_interval_requests is not None:
+        if opts.sleep_interval_requests < 0:
+            parser.error('requests sleep interval must be positive or 0')
    if opts.ap_mso and opts.ap_mso not in MSO_INFO:
        parser.error('Unsupported TV Provider, use --ap-list-mso to get a list of supported TV Providers')
    if opts.overwrites:
        # --yes-overwrites implies --no-continue
        opts.continue_dl = False

-    def parse_retries(retries):
+    def parse_retries(retries, name=''):
        if retries in ('inf', 'infinite'):
            parsed_retries = float('inf')
        else:
            try:
                parsed_retries = int(retries)
            except (TypeError, ValueError):
-                parser.error('invalid retry count specified')
+                parser.error('invalid %sretry count specified' % name)
        return parsed_retries
    if opts.retries is not None:
        opts.retries = parse_retries(opts.retries)
    if opts.fragment_retries is not None:
-        opts.fragment_retries = parse_retries(opts.fragment_retries)
+        opts.fragment_retries = parse_retries(opts.fragment_retries, 'fragment ')
+    if opts.extractor_retries is not None:
+        opts.extractor_retries = parse_retries(opts.extractor_retries, 'extractor ')
    if opts.buffersize is not None:
        numeric_buffersize = FileDownloader.parse_bytes(opts.buffersize)
        if numeric_buffersize is None:
@@ -262,6 +270,11 @@ def _real_main(argv=None):
    any_printing = opts.print_json
    download_archive_fn = expand_path(opts.download_archive) if opts.download_archive is not None else opts.download_archive

+    # If JSON is not printed anywhere, but comments are requested, save it to file
+    printing_json = opts.dumpjson or opts.print_json or opts.dump_single_json
+    if opts.getcomments and not printing_json:
+        opts.writeinfojson = True
+
    def report_conflict(arg1, arg2):
        write_string('WARNING: %s is ignored since %s was given\n' % (arg2, arg1), out=sys.stderr)
    if opts.remuxvideo and opts.recodevideo:
@@ -447,6 +460,7 @@ def _real_main(argv=None):
        'overwrites': opts.overwrites,
        'retries': opts.retries,
        'fragment_retries': opts.fragment_retries,
+        'extractor_retries': opts.extractor_retries,
        'skip_unavailable_fragments': opts.skip_unavailable_fragments,
        'keep_fragments': opts.keep_fragments,
        'buffersize': opts.buffersize,
@@ -466,7 +480,7 @@ def _real_main(argv=None):
        'updatetime': opts.updatetime,
        'writedescription': opts.writedescription,
        'writeannotations': opts.writeannotations,
-        'writeinfojson': opts.writeinfojson or opts.getcomments,
+        'writeinfojson': opts.writeinfojson,
        'allow_playlist_files': opts.allow_playlist_files,
        'getcomments': opts.getcomments,
        'writethumbnail': opts.writethumbnail,
@@ -524,6 +538,7 @@ def _real_main(argv=None):
        'fixup': opts.fixup,
        'source_address': opts.source_address,
        'call_home': opts.call_home,
+        'sleep_interval_requests': opts.sleep_interval_requests,
        'sleep_interval': opts.sleep_interval,
        'max_sleep_interval': opts.max_sleep_interval,
        'sleep_interval_subtitles': opts.sleep_interval_subtitles,
@@ -541,7 +556,6 @@ def _real_main(argv=None):
        'postprocessor_args': opts.postprocessor_args,
        'cn_verification_proxy': opts.cn_verification_proxy,
        'geo_verification_proxy': opts.geo_verification_proxy,
-        'config_location': opts.config_location,
        'geo_bypass': opts.geo_bypass,
        'geo_bypass_country': opts.geo_bypass_country,
        'geo_bypass_ip_block': opts.geo_bypass_ip_block,
--- a/yt_dlp/downloader/init.py
+++ b/yt_dlp/downloader/init.py
@@ -53,7 +53,7 @@ def get_suitable_downloader(info_dict, params={}, default=HttpFD):
    external_downloader = params.get('external_downloader')
    if external_downloader is not None:
        ed = get_external_downloader(external_downloader)
-        if ed.can_download(info_dict):
+        if ed.can_download(info_dict, external_downloader):
            return ed

    if protocol.startswith('m3u8'):
--- a/yt_dlp/downloader/common.py
+++ b/yt_dlp/downloader/common.py
@@ -312,7 +312,7 @@ class FileDownloader(object):
    def report_retry(self, err, count, retries):
        """Report retry in case of HTTP error 5xx"""
        self.to_screen(
-            '[download] Got server HTTP error: %s. Retrying (attempt %d of %s)...'
+            '[download] Got server HTTP error: %s. Retrying (attempt %d of %s) ...'
            % (error_to_compat_str(err), count, self.format_retries(retries)))

    def report_file_already_downloaded(self, file_name):
@@ -359,7 +359,7 @@ class FileDownloader(object):
                max_sleep_interval = self.params.get('max_sleep_interval', min_sleep_interval)
                sleep_interval = random.uniform(min_sleep_interval, max_sleep_interval)
                self.to_screen(
-                    '[download] Sleeping %s seconds...' % (
+                    '[download] Sleeping %s seconds ...' % (
                        int(sleep_interval) if sleep_interval.is_integer()
                        else '%.2f' % sleep_interval))
                time.sleep(sleep_interval)
@@ -369,7 +369,7 @@ class FileDownloader(object):
                sleep_interval_sub = self.params.get('sleep_interval_subtitles')
            if sleep_interval_sub > 0:
                self.to_screen(
-                    '[download] Sleeping %s seconds...' % (
+                    '[download] Sleeping %s seconds ...' % (
                        sleep_interval_sub))
                time.sleep(sleep_interval_sub)
        return self.real_download(filename, info_dict), True
--- a/yt_dlp/downloader/external.py
+++ b/yt_dlp/downloader/external.py
@@ -85,16 +85,16 @@ class ExternalFD(FileDownloader):
        return self.params.get('external_downloader')

    @classmethod
-    def available(cls):
-        return check_executable(cls.get_basename(), [cls.AVAILABLE_OPT])
+    def available(cls, path=None):
+        return check_executable(path or cls.get_basename(), [cls.AVAILABLE_OPT])

    @classmethod
    def supports(cls, info_dict):
        return info_dict['protocol'] in cls.SUPPORTED_PROTOCOLS

    @classmethod
-    def can_download(cls, info_dict):
-        return cls.available() and cls.supports(info_dict)
+    def can_download(cls, info_dict, path=None):
+        return cls.available(path) and cls.supports(info_dict)

    def _option(self, command_option, param):
        return cli_option(self.params, command_option, param)
@@ -398,7 +398,10 @@ class FFmpegFD(ExternalFD):
            args += ['-fs', compat_str(self._TEST_FILE_SIZE)]

        if protocol in ('m3u8', 'm3u8_native'):
-            if self.params.get('hls_use_mpegts', False) or tmpfilename == '-':
+            use_mpegts = (tmpfilename == '-') or self.params.get('hls_use_mpegts')
+            if use_mpegts is None:
+                use_mpegts = info_dict.get('is_live')
+            if use_mpegts:
                args += ['-f', 'mpegts']
            else:
                args += ['-f', 'mp4']
--- a/yt_dlp/downloader/fragment.py
+++ b/yt_dlp/downloader/fragment.py
@@ -55,11 +55,11 @@ class FragmentFD(FileDownloader):

    def report_retry_fragment(self, err, frag_index, count, retries):
        self.to_screen(
-            '[download] Got server HTTP error: %s. Retrying fragment %d (attempt %d of %s)...'
+            '[download] Got server HTTP error: %s. Retrying fragment %d (attempt %d of %s) ...'
            % (error_to_compat_str(err), frag_index, count, self.format_retries(retries)))

    def report_skip_fragment(self, frag_index):
-        self.to_screen('[download] Skipping fragment %d...' % frag_index)
+        self.to_screen('[download] Skipping fragment %d ...' % frag_index)

    def _prepare_url(self, info_dict, url):
        headers = info_dict.get('http_headers')
@@ -174,7 +174,7 @@ class FragmentFD(FileDownloader):
                        '.ytdl file is corrupt' if is_corrupt else
                        'Inconsistent state of incomplete fragment download')
                    self.report_warning(
-                        '%s. Restarting from the beginning...' % message)
+                        '%s. Restarting from the beginning ...' % message)
                    ctx['fragment_index'] = resume_len = 0
                    if 'ytdl_corrupt' in ctx:
                        del ctx['ytdl_corrupt']
--- a/yt_dlp/downloader/niconico.py
+++ b/yt_dlp/downloader/niconico.py
@@ -29,7 +29,7 @@ class NiconicoDmcFD(FileDownloader):
        heartbeat_url = heartbeat_info_dict['url']
        heartbeat_data = heartbeat_info_dict['data']
        heartbeat_interval = heartbeat_info_dict.get('interval', 30)
-        self.to_screen('[%s] Heartbeat with %s second interval...' % (self.FD_NAME, heartbeat_interval))
+        self.to_screen('[%s] Heartbeat with %s second interval ...' % (self.FD_NAME, heartbeat_interval))

        def heartbeat():
            try:
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -255,10 +255,6 @@ class BiliBiliIE(InfoExtractor):
            info['uploader'] = self._html_search_meta(
                'author', webpage, 'uploader', default=None)

-        comments = None
-        if self._downloader.params.get('getcomments', False):
-            comments = self._get_all_comment_pages(video_id)
-
        raw_danmaku = self._get_raw_danmaku(video_id, cid)

        raw_tags = self._get_tags(video_id)
@@ -266,11 +262,18 @@ class BiliBiliIE(InfoExtractor):

        top_level_info = {
            'raw_danmaku': raw_danmaku,
-            'comments': comments,
-            'comment_count': len(comments) if comments is not None else None,
            'tags': tags,
            'raw_tags': raw_tags,
        }
+        if self._downloader.params.get('getcomments', False):
+            def get_comments():
+                comments = self._get_all_comment_pages(video_id)
+                return {
+                    'comments': comments,
+                    'comment_count': len(comments)
+                }
+
+            top_level_info['__post_extractor'] = get_comments

        '''
        # Requires https://github.com/m13253/danmaku2ass which is licenced under GPL3
@@ -555,6 +558,7 @@ class BilibiliAudioIE(BilibiliAudioBaseIE):
        formats = [{
            'url': play_data['cdns'][0],
            'filesize': int_or_none(play_data.get('size')),
+            'vcodec': 'none'
        }]

        song = self._call_api('song/info', au_id)
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -294,6 +294,14 @@ class InfoExtractor(object):
                    players on other sites. Can be True (=always allowed),
                    False (=never allowed), None (=unknown), or a string
                    specifying the criteria for embedability (Eg: 'whitelist').
+    __post_extractor: A function to be called just before the metadata is
+                    written to either disk, logger or console. The function
+                    must return a dict which will be added to the info_dict.
+                    This is usefull for additional information that is
+                    time-consuming to extract. Note that the fields thus
+                    extracted will not be available to output template and
+                    match_filter. So, only "comments" and "comment_count" are
+                    currently allowed to be extracted via this method.

    The following fields should only be used when the video belongs to some logical
    chapter or section:
@@ -606,6 +614,14 @@ class InfoExtractor(object):

        See _download_webpage docstring for arguments specification.
        """
+        if not self._downloader._first_webpage_request:
+            sleep_interval = float_or_none(self._downloader.params.get('sleep_interval_requests')) or 0
+            if sleep_interval > 0:
+                self.to_screen('Sleeping %s seconds ...' % sleep_interval)
+                time.sleep(sleep_interval)
+        else:
+            self._downloader._first_webpage_request = False
+
        if note is None:
            self.report_download_webpage(video_id)
        elif note is not False:
@@ -1888,8 +1904,10 @@ class InfoExtractor(object):
        # media playlist and MUST NOT appear in master playlist thus we can
        # clearly detect media playlist with this criterion.

-        def _extract_m3u8_playlist_formats(format_url, m3u8_doc=None):
+        def _extract_m3u8_playlist_formats(format_url=None, m3u8_doc=None):
            if not m3u8_doc:
+                if not format_url:
+                    return []
                res = self._download_webpage_handle(
                    format_url, video_id,
                    note=False,
@@ -1928,7 +1946,7 @@ class InfoExtractor(object):

        if '#EXT-X-TARGETDURATION' in m3u8_doc:  # media playlist, return as is

-            playlist_formats = _extract_m3u8_playlist_formats(m3u8_doc, True)
+            playlist_formats = _extract_m3u8_playlist_formats(m3u8_doc=m3u8_doc)

            for format in playlist_formats:
                format_id = []
--- a/yt_dlp/extractor/hrfensehen.py
+++ b/yt_dlp/extractor/hrfensehen.py
@@ -4,7 +4,7 @@ from __future__ import unicode_literals
 import json
 import re

-from yt_dlp.utils import int_or_none, unified_timestamp, unescapeHTML
+from ..utils import int_or_none, unified_timestamp, unescapeHTML
 from .common import InfoExtractor


--- a/yt_dlp/extractor/tennistv.py
+++ b/yt_dlp/extractor/tennistv.py
@@ -86,6 +86,7 @@ class TennisTVIE(InfoExtractor):
            'https://www.tennistv.com/api/users/v1/entitlementchecknondiva',
            video_id, note='Checking video authorization', headers=headers, data=check_json)
        formats = self._extract_m3u8_formats(check_result['contentUrl'], video_id, ext='mp4')
+        self._sort_formats(formats)

        vdata = self._download_json(
            'https://www.tennistv.com/api/en/v2/none/common/video/%s' % video_id,
--- a/yt_dlp/extractor/viki.py
+++ b/yt_dlp/extractor/viki.py
@@ -255,15 +255,8 @@ class VikiIE(VikiBaseIE):
    def _real_extract(self, url):
        video_id = self._match_id(url)

-        resp = self._download_json(
-            'https://www.viki.com/api/videos/' + video_id,
-            video_id, 'Downloading video JSON', headers={
-                'x-client-user-agent': std_headers['User-Agent'],
-                'x-viki-as-id': self._APP,
-                'x-viki-app-ver': self._APP_VERSION,
-            })
-        video = resp['video']
-
+        video = self._call_api(
+            'videos/%s.json' % video_id, video_id, 'Downloading video JSON')
        self._check_errors(video)

        title = self.dict_selection(video.get('titles', {}), 'en', allow_fallback=False)
@@ -286,30 +279,12 @@ class VikiIE(VikiBaseIE):
            })

        subtitles = {}
-        try:
-            # New way to fetch subtitles
-            new_video = self._download_json(
-                'https://www.viki.com/api/videos/%s' % video_id, video_id,
-                'Downloading new video JSON to get subtitles', fatal=False,
-                headers={
-                    'x-client-user-agent': std_headers['User-Agent'],
-                    'x-viki-as-id': self._APP,
-                    'x-viki-app-ver': self._APP_VERSION,
-                })
-            for sub in new_video.get('streamSubtitles').get('dash'):
-                subtitles[sub.get('srclang')] = [{
-                    'ext': 'vtt',
-                    'url': sub.get('src'),
-                    'completion': sub.get('percentage'),
-                }]
-        except AttributeError:
-            # fall-back to the old way if there isn't a streamSubtitles attribute
-            for subtitle_lang, _ in (video.get('subtitle_completions') or {}).items():
-                subtitles[subtitle_lang] = [{
-                    'ext': subtitles_format,
-                    'url': self._prepare_call(
-                        'videos/%s/subtitles/%s.%s' % (video_id, subtitle_lang, subtitles_format)),
-                } for subtitles_format in ('srt', 'vtt')]
+        for subtitle_lang, _ in (video.get('subtitle_completions') or {}).items():
+            subtitles[subtitle_lang] = [{
+                'ext': subtitles_format,
+                'url': self._prepare_call(
+                    'videos/%s/subtitles/%s.%s' % (video_id, subtitle_lang, subtitles_format)),
+            } for subtitles_format in ('srt', 'vtt')]

        result = {
            'id': video_id,
@@ -386,23 +361,20 @@ class VikiIE(VikiBaseIE):
                    'filesize': int_or_none(urlh.headers.get('Content-Length')),
                })

-        for format_id, format_dict in (resp.get('streams') or {}).items():
-            add_format(format_id, format_dict)
-        if not formats:
-            streams = self._call_api(
-                'videos/%s/streams.json' % video_id, video_id,
-                'Downloading video streams JSON')
+        streams = self._call_api(
+            'videos/%s/streams.json' % video_id, video_id,
+            'Downloading video streams JSON')

-            if 'external' in streams:
-                result.update({
-                    '_type': 'url_transparent',
-                    'url': streams['external']['url'],
-                })
-                return result
+        if 'external' in streams:
+            result.update({
+                '_type': 'url_transparent',
+                'url': streams['external']['url'],
+            })
+            return result

-            for format_id, stream_dict in streams.items():
-                for protocol, format_dict in stream_dict.items():
-                    add_format(format_id, format_dict, protocol)
+        for format_id, stream_dict in streams.items():
+            for protocol, format_dict in stream_dict.items():
+                add_format(format_id, format_dict, protocol)
        self._sort_formats(formats)

        result['formats'] = formats
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -1452,8 +1452,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
        url, smuggled_data = unsmuggle_url(url, {})
        video_id = self._match_id(url)
        base_url = self.http_scheme() + '//www.youtube.com/'
-        webpage_url = base_url + 'watch?v=' + video_id + '&has_verified=1&bpctr=9999999999'
-        webpage = self._download_webpage(webpage_url, video_id, fatal=False)
+        webpage_url = base_url + 'watch?v=' + video_id
+        webpage = self._download_webpage(
+            webpage_url + '&has_verified=1&bpctr=9999999999',
+            video_id, fatal=False)

        player_response = None
        if webpage:
@@ -2010,9 +2012,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):

        # Get comments
        # TODO: Refactor and move to seperate function
-        if get_comments:
+        def extract_comments():
            expected_video_comment_count = 0
            video_comments = []
+            comment_xsrf = xsrf_token

            def find_value(html, key, num_chars=2, separator='"'):
                pos_begin = html.find(key) + len(key) + num_chars
@@ -2081,7 +2084,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
            self.to_screen('Downloading comments')
            while continuations:
                continuation = continuations.pop()
-                comment_response = get_continuation(continuation, xsrf_token)
+                comment_response = get_continuation(continuation, comment_xsrf)
                if not comment_response:
                    continue
                if list(search_dict(comment_response, 'externalErrorMessage')):
@@ -2092,7 +2095,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                    continue
                # not sure if this actually helps
                if 'xsrf_token' in comment_response:
-                    xsrf_token = comment_response['xsrf_token']
+                    comment_xsrf = comment_response['xsrf_token']

                item_section = comment_response['response']['continuationContents']['itemSectionContinuation']
                if first_continuation:
@@ -2121,7 +2124,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                    while reply_continuations:
                        time.sleep(1)
                        continuation = reply_continuations.pop()
-                        replies_data = get_continuation(continuation, xsrf_token, True)
+                        replies_data = get_continuation(continuation, comment_xsrf, True)
                        if not replies_data or 'continuationContents' not in replies_data[1]['response']:
                            continue

@@ -2150,10 +2153,13 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                time.sleep(1)

            self.to_screen('Total comments downloaded: %d of ~%d' % (len(video_comments), expected_video_comment_count))
-            info.update({
+            return {
                'comments': video_comments,
                'comment_count': expected_video_comment_count
-            })
+            }
+
+        if get_comments:
+            info['__post_extractor'] = extract_comments

        self.mark_watched(video_id, player_response)

@@ -2756,28 +2762,36 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
        for page_num in itertools.count(1):
            if not continuation:
                break
-            count = 0
-            retries = 3
-            while count <= retries:
+            retries = self._downloader.params.get('extractor_retries', 3)
+            count = -1
+            last_error = None
+            while count < retries:
+                count += 1
+                if last_error:
+                    self.report_warning('%s. Retrying ...' % last_error)
                try:
-                    # Downloading page may result in intermittent 5xx HTTP error
-                    # that is usually worked around with a retry
                    browse = self._download_json(
                        'https://www.youtube.com/browse_ajax', None,
                        'Downloading page %d%s'
                        % (page_num, ' (retry #%d)' % count if count else ''),
                        headers=headers, query=continuation)
-                    break
                except ExtractorError as e:
-                    if isinstance(e.cause, compat_HTTPError) and e.cause.code in (500, 503):
-                        count += 1
-                        if count <= retries:
+                    if isinstance(e.cause, compat_HTTPError) and e.cause.code in (500, 503, 404):
+                        # Downloading page may result in intermittent 5xx HTTP error
+                        # Sometimes a 404 is also recieved. See: https://github.com/ytdl-org/youtube-dl/issues/28289
+                        last_error = 'HTTP Error %s' % e.cause.code
+                        if count < retries:
                            continue
                    raise
-            if not browse:
-                break
-            response = try_get(browse, lambda x: x[1]['response'], dict)
-            if not response:
+                else:
+                    response = try_get(browse, lambda x: x[1]['response'], dict)
+
+                    # Youtube sometimes sends incomplete data
+                    # See: https://github.com/ytdl-org/youtube-dl/issues/28194
+                    if response.get('continuationContents') or response.get('onResponseReceivedActions'):
+                        break
+                    last_error = 'Incomplete data recieved'
+            if not browse or not response:
                break

            known_continuation_renderers = {
@@ -2998,19 +3012,32 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                return self.url_result(video_id, ie=YoutubeIE.ie_key(), video_id=video_id)
            self.to_screen('Downloading playlist %s - add --no-playlist to just download video %s' % (playlist_id, video_id))

-        webpage = self._download_webpage(url, item_id)
-        identity_token = self._extract_identity_token(webpage, item_id)
-        data = self._extract_yt_initial_data(item_id, webpage)
-        err_msg = None
-        for alert_type, alert_message in self._extract_alerts(data):
-            if alert_type.lower() == 'error':
-                if err_msg:
-                    self._downloader.report_warning('YouTube said: %s - %s' % ('ERROR', err_msg))
-                err_msg = alert_message
-            else:
-                self._downloader.report_warning('YouTube said: %s - %s' % (alert_type, alert_message))
-        if err_msg:
-            raise ExtractorError('YouTube said: %s' % err_msg, expected=True)
+        retries = self._downloader.params.get('extractor_retries', 3)
+        count = -1
+        while count < retries:
+            count += 1
+            # Sometimes youtube returns a webpage with incomplete ytInitialData
+            # See: https://github.com/yt-dlp/yt-dlp/issues/116
+            if count:
+                self.report_warning('Incomplete yt initial data recieved. Retrying ...')
+            webpage = self._download_webpage(
+                url, item_id,
+                'Downloading webpage%s' % ' (retry #%d)' % count if count else '')
+            identity_token = self._extract_identity_token(webpage, item_id)
+            data = self._extract_yt_initial_data(item_id, webpage)
+            err_msg = None
+            for alert_type, alert_message in self._extract_alerts(data):
+                if alert_type.lower() == 'error':
+                    if err_msg:
+                        self._downloader.report_warning('YouTube said: %s - %s' % ('ERROR', err_msg))
+                    err_msg = alert_message
+                else:
+                    self._downloader.report_warning('YouTube said: %s - %s' % (alert_type, alert_message))
+            if err_msg:
+                raise ExtractorError('YouTube said: %s' % err_msg, expected=True)
+            if data.get('contents') or data.get('currentVideoEndpoint'):
+                break
+
        tabs = try_get(
            data, lambda x: x['contents']['twoColumnBrowseResultsRenderer']['tabs'], list)
        if tabs:
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -347,7 +347,7 @@ def parseOpts(overrideArguments=None):
            'Specify any key (see "OUTPUT TEMPLATE" for a list of available keys) to '
            'match if the key is present, '
            '!key to check if the key is not present, '
-            'key>NUMBER (like "comment_count > 12", also works with '
+            'key>NUMBER (like "view_count > 12", also works with '
            '>=, <, <=, !=, =) to compare against a number, '
            'key = \'LITERAL\' (like "uploader = \'Mike Smith\'", also works with !=) '
            'to match against a string literal '
@@ -369,7 +369,7 @@ def parseOpts(overrideArguments=None):
        help='Download only the video, if the URL refers to a video and a playlist')
    selection.add_option(
        '--yes-playlist',
-        action='store_false', dest='noplaylist', default=False,
+        action='store_false', dest='noplaylist',
        help='Download the playlist, if the URL refers to a video and a playlist')
    selection.add_option(
        '--age-limit',
@@ -634,16 +634,24 @@ def parseOpts(overrideArguments=None):
        help='Use ffmpeg instead of the native HLS downloader')
    downloader.add_option(
        '--hls-use-mpegts',
-        dest='hls_use_mpegts', action='store_true',
+        dest='hls_use_mpegts', action='store_true', default=None,
        help=(
-            'Use the mpegts container for HLS videos, allowing to play the '
-            'video while downloading (some players may not be able to play it)'))
+            'Use the mpegts container for HLS videos; '
+            'allowing some players to play the video while downloading, '
+            'and reducing the chance of file corruption if download is interrupted. '
+            'This is enabled by default for live streams'))
+    downloader.add_option(
+        '--no-hls-use-mpegts',
+        dest='hls_use_mpegts', action='store_false',
+        help=(
+            'Do not use the mpegts container for HLS videos. '
+            'This is default when not downloading live streams'))
    downloader.add_option(
        '--external-downloader',
        dest='external_downloader', metavar='NAME',
        help=(
-            'Use the specified external downloader. '
-            'Currently supports %s' % ', '.join(list_external_downloaders())))
+            'Name or path of the external downloader to use. '
+            'Currently supports %s (Recommended: aria2c)' % ', '.join(list_external_downloaders())))
    downloader.add_option(
        '--downloader-args', '--external-downloader-args',
        metavar='NAME:ARGS', dest='external_downloader_args', default={}, type='str',
@@ -688,6 +696,10 @@ def parseOpts(overrideArguments=None):
        '--bidi-workaround',
        dest='bidi_workaround', action='store_true',
        help='Work around terminals that lack bidirectional text support. Requires bidiv or fribidi executable in PATH')
+    workarounds.add_option(
+        '--sleep-requests', metavar='SECONDS',
+        dest='sleep_interval_requests', type=float,
+        help='Number of seconds to sleep between requests during data extraction')
    workarounds.add_option(
        '--sleep-interval', '--min-sleep-interval', metavar='SECONDS',
        dest='sleep_interval', type=float,
@@ -706,7 +718,7 @@ def parseOpts(overrideArguments=None):
    workarounds.add_option(
        '--sleep-subtitles', metavar='SECONDS',
        dest='sleep_interval_subtitles', default=0, type=int,
-        help='Enforce sleep interval on subtitles as well')
+        help='Number of seconds to sleep before each subtitle download')

    verbosity = optparse.OptionGroup(parser, 'Verbosity and Simulation Options')
    verbosity.add_option(
@@ -973,7 +985,9 @@ def parseOpts(overrideArguments=None):
    filesystem.add_option(
        '--get-comments',
        action='store_true', dest='getcomments', default=False,
-        help='Retrieve video comments to be placed in the .info.json file')
+        help=(
+            'Retrieve video comments to be placed in the .info.json file. '
+            'The comments are fetched even without this option if the extraction is known to be quick'))
    filesystem.add_option(
        '--load-info-json', '--load-info',
        dest='load_info_filename', metavar='FILE',
@@ -1129,7 +1143,7 @@ def parseOpts(overrideArguments=None):
            'Give field name to extract data from, and format of the field seperated by a ":". '
            'Either regular expression with named capture groups or a '
            'similar syntax to the output template can also be used. '
-            'The parsed parameters replace any existing values and can be use in output template'
+            'The parsed parameters replace any existing values and can be use in output template. '
            'This option can be used multiple times. '
            'Example: --parse-metadata "title:%(artist)s - %(title)s" matches a title like '
            '"Coldplay - Paradise". '
@@ -1204,6 +1218,10 @@ def parseOpts(overrideArguments=None):
        help=optparse.SUPPRESS_HELP)

    extractor = optparse.OptionGroup(parser, 'Extractor Options')
+    extractor.add_option(
+        '--extractor-retries',
+        dest='extractor_retries', metavar='RETRIES', default=10,
+        help='Number of retries for known extractor errors (default is %default), or "infinite"')
    extractor.add_option(
        '--allow-dynamic-mpd', '--no-ignore-dynamic-mpd',
        action='store_true', dest='dynamic_mpd', default=True,
--- a/yt_dlp/postprocessor/embedthumbnail.py
+++ b/yt_dlp/postprocessor/embedthumbnail.py
@@ -193,4 +193,6 @@ class EmbedThumbnailPP(FFmpegPostProcessor):
                info['__thumbnail_filename'], os.path.splitext(original_thumbnail)[1][1:])
            if original_thumbnail == thumbnail_filename:
                files_to_delete = []
+        elif original_thumbnail != thumbnail_filename:
+            files_to_delete.append(original_thumbnail)
        return files_to_delete, info
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -5945,9 +5945,13 @@ def make_dir(path, to_screen=None):


 def get_executable_path():
-    path = os.path.dirname(sys.argv[0])
-    if os.path.basename(sys.argv[0]) == '__main__':  # Running from source
-        path = os.path.join(path, '..')
+    from zipimport import zipimporter
+    if hasattr(sys, 'frozen'):  # Running from PyInstaller
+        path = os.path.dirname(sys.executable)
+    elif isinstance(globals().get('__loader__'), zipimporter):  # Running from ZIP
+        path = os.path.join(os.path.dirname(__file__), '../..')
+    else:
+        path = os.path.join(os.path.dirname(__file__), '..')
    return os.path.abspath(path)


--- a/yt_dlp/version.py
+++ b/yt_dlp/version.py
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals

-__version__ = '2021.02.19'
+__version__ = '2021.02.24'
Author	SHA1	Message	Date
pukkandan	5ef7d9bdd8	Release 2021.03.01	2021-03-01 05:39:50 +05:30
pukkandan	62bff2c170	Add option `--extractor-retries` to retry on known extractor errors * Currently only used by youtube Fixes https://github.com/ytdl-org/youtube-dl/issues/28194 Possibly also fixes: https://github.com/ytdl-org/youtube-dl/issues/28289 (can not confirm since the issue isn't reliably reproducible)	2021-03-01 05:18:37 +05:30
pukkandan	f0884c8b3f	Cleanup some code (see desc) * `--get-comments` doesn't imply `--write-info-json` if `-J`, `-j` or `--print-json` are used * Don't pass `config_location` to `YoutubeDL` (it is unused) * [bilibiliaudio] Recognize the file as audio-only * Update gitignore * Fix typos	2021-02-28 20:56:32 +05:30
pukkandan	277d6ff5f2	Extract comments only when needed #95 (Closes #94 )	2021-02-28 20:26:08 +05:30
pukkandan	1cf376f55a	Add option `--sleep-requests` to sleep b/w requests (Closes #106 ) * Also fix documentation of `sleep_interval_subtitles` Related issues: https://github.com/blackjack4494/yt-dlc/issues/158 https://github.com/blackjack4494/youtube-dlc/issues/195 https://github.com/ytdl-org/youtube-dl/pull/28270 https://github.com/ytdl-org/youtube-dl/pull/28144 https://github.com/ytdl-org/youtube-dl/issues/27767 https://github.com/ytdl-org/youtube-dl/issues/23638 https://github.com/ytdl-org/youtube-dl/issues/26287 https://github.com/ytdl-org/youtube-dl/issues/26319	2021-02-27 18:14:42 +05:30
pukkandan	7f7de7f94d	Allow specifying path in `--external-downloader`	2021-02-27 16:52:27 +05:30
pukkandan	86878b6cd9	[hrfensehen] Fix wrong import	2021-02-27 15:35:41 +05:30
pukkandan	b3d1242534	[youtube] Fix inconsistent `webpage_url` (closes #119 )	2021-02-27 14:45:56 +05:30
pukkandan	9bd2020476	[hls] Enable `--hls-use-mpegts` by default when downloading live-streams * Also added option `--no-hls-use-mpegts` to disable this Related: #96	2021-02-26 21:52:16 +05:30
pukkandan	ed9b7e3dd3	Fix bug with m3u8 format extraction	2021-02-26 18:32:28 +05:30
shirt-dev	c552ae8838	Fix `get_executable_path` (#117 ) Authored-by: shirtjs <2660574+shirtjs@users.noreply.github.com>	2021-02-26 04:28:02 +05:30
Robin Dunn	31a5e037a7	[viki] Fix viki play pass authentication (#111 ) Authored by: RobinD42	2021-02-26 03:33:00 +05:30
pukkandan	3638226215	[ci] Disable download tests unless specifically invoked Tests can be enabled/disabled using the following in the commit message * Run Download: `ci-run-dl` * Skip Core: `ci-skip` * Skip Quick & Core: `ci-skip-all` (replace "-" by a space " ")	2021-02-26 03:28:18 +05:30
pukkandan	14fdfea973	[youtube] Retry on incomplete `ytInitialData` Related: #116	2021-02-26 03:23:08 +05:30
shirt-dev	b45d4e4a8e	Fix completion paths, zsh pip completion install (#114 )	2021-02-25 11:00:29 -05:00
pukkandan	3e39273418	Merge branch 'master' into fix-paths	2021-02-25 21:17:46 +05:30
shirt-dev	b965087396	Readthedocs improvements (#115 ) Authored-by: shirtjs <2660574+shirtjs@users.noreply.github.com> :ci skip dl	2021-02-25 21:16:08 +05:30
hseg	359d6d8650	Fix completion paths, zsh pip completion install Closes: #108, #110	2021-02-25 16:49:57 +02:00
pukkandan	0e0040519b	[embedthumbnail] Fix bug with deleting original thumbnail (Closes #113 ) :ci skip dl	2021-02-25 18:35:04 +05:30
pukkandan	127d075955	[documentation] Fix typos (Closes #112 ) :ci skip all	2021-02-25 16:08:25 +05:30
pukkandan	bce8cbb089	[tennistv] Fix format sorting	2021-02-25 16:07:38 +05:30
pukkandan	aae273ded8	[version] update :ci skip dl	2021-02-25 02:44:10 +05:30