1
0
mirror of https://github.com/yt-dlp/yt-dlp.git synced 2026-02-22 08:26:00 +00:00

Compare commits

...

140 Commits

Author SHA1 Message Date
github-actions[bot]
e2a9cc7d13 Release 2026.02.21
Created by: bashonly

:ci skip all
2026-02-21 20:22:26 +00:00
Simon Sawicki
646bb31f39 [cleanup] Misc
Authored by: Grub4K
2026-02-21 21:07:56 +01:00
Simon Sawicki
1fbbe29b99 [ie] Limit netrc_machine parameter to shell-safe characters
Also adapts some extractor regexes to adhere to this limitation

See: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-g3gw-q23r-pgqm

Authored by: Grub4K
2026-02-21 21:07:36 +01:00
bashonly
c105461647 [ie/youtube] Update ejs to 0.5.0 (#16031)
Authored by: bashonly
2026-02-21 20:05:38 +00:00
bashonly
1d1358d09f [ie] Add browser impersonation support to more extractors (#16029)
Closes #7001, Closes #7444, Closes #16004
Authored by: bashonly
2026-02-21 19:24:05 +00:00
blauerdorf
1fe0bf23aa [ie/spankbang] Fix playlist title extraction (#14132)
Closes #14131
Authored by: blauerdorf
2026-02-21 18:57:20 +00:00
blauerdorf
f05e1cd1f1 [ie/spankbang] Support browser impersonation (#14130)
Closes #14129
Authored by: blauerdorf
2026-02-21 18:51:52 +00:00
bashonly
46d5b6f2b7 [ie/learningonscreen] Fix extractor (#16028)
Closes #15934
Authored by: bashonly, 0xvd
2026-02-21 18:27:33 +00:00
LordMZTE
166356d1a1 [ie/opencast] Support oc-p.uni-jena.de URLs (#16026)
Closes #16023
Authored by: LordMZTE
2026-02-21 18:01:34 +00:00
Sipherdrakon
2485653859 [ie/aenetworks] Fix extractor (#14959)
Closes #14578
Authored by: Sipherdrakon
2026-02-21 17:46:59 +00:00
bashonly
f532a91cef [ie/soundcloud] Support browser impersonation (#16020)
Closes #15660
Authored by: bashonly
2026-02-21 14:50:22 +00:00
bashonly
81bdea03f3 [ie/soundcloud] Fix client ID extraction (#16019)
Authored by: bashonly
2026-02-21 00:21:29 +00:00
bashonly
e74076141d [rh:curl_cffi] Deprioritize unreliable impersonate targets (#16018)
Closes #16012
Authored by: bashonly
2026-02-20 23:48:16 +00:00
Parker Wahle
97f03660f5 [ie/SaucePlusChannel] Add extractor (#15830)
Closes #14985
Authored by: regulad
2026-02-20 00:07:48 +00:00
bashonly
772559e3db [ie/tele5] Fix extractor (#16005)
Closes #16003
Authored by: bashonly
2026-02-19 23:53:53 +00:00
Achraf
c7945800e4 [ie/youtube:search:date] Remove broken ytsearchdate support (#15959)
Closes #15898
Authored by: stastix
2026-02-19 23:18:02 +00:00
bashonly
e2444584a3 [ie/facebook:ads] Fix extractor (#16002)
Closes #16000
Authored by: bashonly
2026-02-19 23:08:08 +00:00
bashonly
acfc00a955 [ie/vk] Solve JS challenges using native JS interpreter (#15992)
Closes #12970
Authored by: bashonly, 0xvd
2026-02-19 15:14:37 +00:00
bashonly
224fe478b0 [ie/dailymotion] Fix extraction (#15995)
Fix 2b61a2a4b2

Authored by: bashonly
2026-02-19 15:11:23 +00:00
bashonly
77221098fc [ie/twitter] Fix error handling again (#15999)
Fix 0d8898c3f4

Closes #15998
Authored by: bashonly
2026-02-19 15:03:07 +00:00
CanOfSocks
319a2bda83 [ie/youtube] Extract live adaptive incomplete formats (#15937)
Closes #10148
Authored by: CanOfSocks, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2026-02-18 23:52:13 +00:00
bashonly
2204cee6d8 [ie/youtube] Add more known player JS variants (#15975)
Authored by: bashonly
2026-02-18 20:23:00 +00:00
bashonly
071ad7dfa0 [ie/odnoklassniki] Fix inefficient regular expression (#15974)
Closes #15958
Authored by: bashonly
2026-02-18 20:03:24 +00:00
bashonly
0d8898c3f4 [ie/twitter] Fix error handling (#15993)
Closes #15963
Authored by: bashonly
2026-02-18 19:55:48 +00:00
bashonly
d108ca10b9 [jsinterp] Support string concatenation with + and += (#15990)
Authored by: bashonly
2026-02-17 23:46:20 +00:00
bashonly
c9c8651975 [jsinterp] Stringify bracket notation keys in object access (#15989)
Authored by: bashonly
2026-02-17 23:20:54 +00:00
bashonly
62574f5763 [jsinterp] Fix bitwise operations (#15985)
Authored by: bashonly
2026-02-17 23:10:18 +00:00
Simon Sawicki
abade83f8d [cleanup] Bump ruff to 0.15.x (#15951)
Authored by: Grub4K
2026-02-16 20:11:02 +00:00
bashonly
43229d1d5f [cookies] Ignore cookies with control characters (#15862)
http.cookies.Morsel was patched in Python 3.14.3 and 3.13.12
to raise a CookieError if the cookie name, value or any attribute
of its input contains a control character.

yt_dlp.cookies.LenientSimpleCookie now preemptively discards
any cookies containing control characters, which is consistent
with its more lenient parsing.

Ref: https://github.com/python/cpython/issues/143919

Closes #15849
Authored by: bashonly, syphyr

Co-authored-by: syphyr <syphyr@gmail.com>
2026-02-16 19:59:34 +00:00
Gareth Seddon
8d6e0b29bf [ie/MatchiTV] Add extractor (#15204)
Authored by: gseddon
2026-02-12 08:14:56 +00:00
Corey Wright
1ea7329cc9 [ie/ApplePodcasts] Fix extractor (#15901)
Closes #15900
Authored by: coreywright
2026-02-12 08:09:37 +00:00
doe1080
a13f281012 [ie/tvo] Add extractor (#15903)
Authored by: doe1080
2026-02-09 20:57:54 +00:00
doe1080
02ce3efbfe [ie/tver:olympic] Add extractor (#15885)
Authored by: doe1080
2026-02-09 20:56:39 +00:00
doe1080
1a9c4b8238 [ie/steam] Fix extractor (#15028)
Closes #15014
Authored by: doe1080
2026-02-09 20:33:36 +00:00
bashonly
637ae202ac [ie/gem.cbc.ca] Support standalone, series & Olympics URLs (#15878)
Closes #8382, Closes #8790, Closes #15850
Authored by: bashonly, makew0rld, 0xvd

Co-authored-by: makeworld <makeworld@protonmail.com>
Co-authored-by: 0xvd <0xvd12@gmail.com>
2026-02-07 23:12:45 +00:00
hunter-gatherer8
23c059a455 [ie/1tv] Extract chapters (#15848)
Authored by: hunter-gatherer8
2026-02-06 20:45:47 +00:00
beacdeac
6f38df31b4 [ie/pornhub] Fix extractor (#15858)
Closes #15827
Authored by: beacdeac
2026-02-06 20:41:56 +00:00
doe1080
442c90da3e [ie/locipo] Add extractors (#15486)
Closes #13656
Authored by: doe1080, gravesducking

Co-authored-by: gravesducking <219445875+gravesducking@users.noreply.github.com>
2026-02-04 21:06:39 +00:00
0x∅
133cb959be [ie/xhamster] Fix extractor (#15831)
Closes #15802
Authored by: 0xvd
2026-02-04 20:49:07 +00:00
doe1080
c7c45f5289 [ie/visir] Add extractor (#15811)
Closes #11901
Authored by: doe1080
2026-02-04 15:33:00 +00:00
github-actions[bot]
bb3af7e6d5 Release 2026.02.04
Created by: bashonly

:ci skip all
2026-02-04 00:31:48 +00:00
doe1080
c677d866d4 [ie/unsupported] Update unsupported URLs (#15812)
Closes #8821, Closes #9851, Closes #13220, Closes #14564, Closes #14620
Authored by: doe1080
2026-02-03 23:30:59 +00:00
bashonly
1a895c18aa [ie/youtube] Default to tv player JS variant (#15818)
Closes #15814
Authored by: bashonly
2026-02-03 23:26:30 +00:00
github-actions[bot]
891613b098 Release 2026.01.31
Created by: bashonly

:ci skip all
2026-01-31 23:52:50 +00:00
bashonly
9a9a6b6fe4 [cleanup] Misc (#15788)
Authored by: bashonly
2026-01-31 23:50:20 +00:00
bashonly
8eb794366e [ie/youtube] Remove broken tv_embedded player client (#15787)
Authored by: bashonly
2026-01-31 23:25:21 +00:00
bashonly
c3674575fa [ie/youtube] Remove broken ios_downgraded player client (#15786)
Closes #15782
Authored by: bashonly
2026-01-31 22:55:35 +00:00
bashonly
bb1c05752c [ie/youtube] Add web_embedded fallback for android_vr client (#15785)
Closes #15751, Closes #15780
Authored by: bashonly
2026-01-31 22:48:41 +00:00
bashonly
bf5d8c2a66 [ie/unsupported] Update unsupported URLs (#15410)
Closes #14743
Authored by: bashonly
2026-01-30 10:50:43 +00:00
thematuu
d0bf3d0fc3 [ie/soop] Support subscription-only VODs (#15523)
* Add custom downloader SoopVodFD

Closes #13636
Authored by: thematuu
2026-01-30 00:25:58 +00:00
azdlonky
0d8ee637e8 [ie/whyp] Extract more metadata (#15757)
Authored by: azdlonky
2026-01-30 00:03:52 +00:00
github-actions[bot]
e4c120f315 Release 2026.01.29
Created by: bashonly

:ci skip all
2026-01-29 17:00:19 +00:00
bashonly
8b275536d9 [cleanup] Misc (#15749)
* Documentation fixes
* Bump PyInstaller to 6.18.0 for Windows builds

Authored by: bashonly
2026-01-29 16:55:27 +00:00
bashonly
88b35ff911 [ie/youtube] Update ejs to 0.4.0 (#15747)
Authored by: bashonly
2026-01-29 16:47:00 +00:00
bashonly
a65349443b [cleanup] Misc (#15430)
Authored by: bashonly, Grub4K, seproDev

Co-authored-by: sepro <sepro@sepr0.com>
Co-authored-by: Simon Sawicki <contact@grub4k.dev>
2026-01-29 16:22:35 +00:00
bashonly
ba5e2227c8 [ie/vimeo] Add macos client (#15746)
Authored by: bashonly, gamer191
2026-01-29 16:19:59 +00:00
bashonly
309b03f2ad [ie/youtube] Fix default player clients (#15726)
* Add `ios_downgraded` player client
* Remove `android_sdkless` player client

Closes #15712
Authored by: bashonly
2026-01-29 06:57:13 +00:00
bashonly
f70ebf97ea [ie/whyp] Fix extractor (#15721)
Closes #15719
Authored by: bashonly
2026-01-29 00:28:55 +00:00
N/Ame
5bf91072bc Fix concurrent formats downloading to stdout (#15617)
Authored by: grqz
2026-01-28 03:57:09 +00:00
N/Ame
1829a53a54 Fix interactive format/video selection when downloading to stdout (#15626)
Authored by: grqz
2026-01-28 01:11:19 +00:00
rdamas
1c739bf53e [ie/ERRArhiiv] Add extractor (#15667)
Closes #15663
Authored by: rdamas
2026-01-27 16:53:38 +00:00
bashonly
e08fdaaec2 [ie/franceinfo] Fix extraction (#15704)
Closes #15701
Authored by: bashonly
2026-01-27 15:40:47 +00:00
Romain Reignier
ac3a566434 [ie/franceinfo] Support new domain URLs (#15669)
Closes #13173
Authored by: romainreignier
2026-01-27 14:09:16 +00:00
Alexander Bocken
1f4b26c39f [ie/TheChosen] Support new URL format (#15687)
Closes #15686
Authored by: AlexBocken
2026-01-27 14:08:22 +00:00
bashonly
14998eef63 [ie/patreon] Extract inlined media (#15498)
Closes #15473
Authored by: bashonly
2026-01-27 12:52:49 +00:00
bashonly
a893774096 [ie/dailymotion] Support browser impersonation (#15697)
Fix 2b61a2a4b2

Closes #15526
Authored by: bashonly
2026-01-27 12:47:19 +00:00
nlurker
a810871608 [ie/pbs] Fix extraction (#15083)
Closes #13299
Authored by: nlurker
2026-01-27 12:45:19 +00:00
Md5Lukas
f9a06197f5 [ie/boosty] Improve metadata extraction (#15543)
Authored by: Sytm
2026-01-27 12:39:10 +00:00
Mivik
a421eb06d1 [ie/neteasemusic] Fix merged lyrics extraction (#15052)
Authored by: Mivik
2026-01-27 12:30:11 +00:00
wesson09
bc6ff877dd [ie/wat.tv] Improve DRM detection (#15659)
Closes #15647
Authored by: wesson09
2026-01-27 12:29:09 +00:00
Subrat Lima
1effa06dbf [ie/volejtv] Fix and add extractors (#13226)
Closes #13203
Authored by: subrat-lima
2026-01-27 12:22:55 +00:00
Ștefan-Gabriel Muscalu
f8b3fe33f6 [ie/facebook:ads] Fix extractor (#15582)
Closes #15577
Authored by: legraphista
2026-01-27 11:59:50 +00:00
christoph-heinrich
0e4d1e9de6 [ie/lbry] Support filtering of flat playlist results (#15695)
Closes #15683
Authored by: christoph-heinrich, dirkf

Co-authored-by: dirkf <1222880+dirkf@users.noreply.github.com>
2026-01-27 02:06:38 +00:00
christoph-heinrich
0dec80c02a [ie/RumbleChannel] Support filtering of flat playlist results (#15694)
Authored by: christoph-heinrich
2026-01-27 02:05:39 +00:00
bashonly
e3f0d8b731 [ie/tiktok] Solve JS challenges with native Python implementation (#15672)
Closes #15418
Authored by: bashonly, DTrombett

Co-authored-by: DTrombett <d@trombett.org>
2026-01-25 23:16:01 +00:00
bashonly
2b61a2a4b2 [ie/dailymotion] Fix extractor (#15682)
Closes #15526
Authored by: bashonly
2026-01-25 23:03:55 +00:00
rdamas
c8680b65f7 [ie/media.ccc.de] Fix extractor (#15608)
Closes #15607
Authored by: rdamas
2026-01-19 23:16:08 +00:00
Subrat Lima
457dd036af [ie/cbc] Fix extractors (#15631)
Closes #15584
Authored by: subrat-lima
2026-01-19 22:39:27 +00:00
bashonly
5382c6c81b Add --compat-options 2025 (#15499)
Authored by: bashonly
2026-01-19 20:16:33 +00:00
Nil Admirari
b16b06378a Add --format-sort-reset option (#13809)
Authored by: nihil-admirari
2026-01-19 17:40:08 +00:00
bashonly
0b08b833bf [build] Fix manually triggered nightly releases (#15615)
Fix 3763d0d4ab

Authored by: bashonly
2026-01-19 09:25:37 +00:00
bashonly
9ab4777b97 [rh:curl_cffi] Support curl_cffi 0.14.x (#15613)
Closes #11860
Authored by: bashonly
2026-01-18 23:40:37 +00:00
Karl Knechtel
dde5eab3b3 Support Deno installed via Python package (#15614)
* Add `deno` extra
* Check Python "scripts" path for runtime executables

Closes #15530
Authored by: zahlman, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2026-01-18 23:31:54 +00:00
bashonly
23b8465063 [ie/youtube] Adjust default clients (#15601)
* Remove `tv` client from logged-out defaults due to #15583
* Remove all HTML5 clients from "JS-less" defaults due to #15569
* Prioritize `web` over `web_safari` until we request latter's config
* Bump all player client versions
* Do not warn for expected SABR-only responses (`web`/`web_safari`)
* Improve PO Token binding experiment debug output

Authored by: bashonly
2026-01-18 19:26:16 +00:00
bashonly
d20f58d721 [ie/youtube] Solve n challenges for manifest formats (#15602)
* Solve n challenges in HLS/DASH manifest URL path parameters
* Collect all challenges in advance to solve in bulk once per video
* Improve & always use the load/store helper methods for player cache

Closes #15569, Closes #15586, Closes #15587, Closes #15600
Authored by: bashonly
2026-01-18 16:34:13 +00:00
Simon Sawicki
e2ea6bd6ab [ie/youtube] Fix priorization of youtube URL matching (#15596)
Authored by: Grub4K
2026-01-18 16:11:29 +01:00
Simon Sawicki
ede54330fb [utils] devalue: Fix calling reviver on cached value (#15568)
Authored by: Grub4K
2026-01-16 15:53:32 +01:00
bashonly
27afb31edc [ie/tarangplus] Sanitize m3u8 URLs (#15502)
Fix 260ba3abba

Closes #15501
Authored by: bashonly
2026-01-06 05:44:30 +00:00
InvalidUsernameException
48b845a296 [ie/zdf] Support sister sites URLs (#15370)
Closes #13319
Authored by: InvalidUsernameException
2026-01-06 04:56:18 +00:00
clayote
cec1f1df79 Fix --parse-metadata when TO is a single field name (#14577)
Closes #14576
Authored by: clayote, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2026-01-05 03:19:30 +00:00
0x∅
ba499ab0dc [ie/croatian.film] Add extractor (#15468)
Closes #15464
Authored by: 0xvd
2026-01-04 17:43:47 +00:00
0x∅
5a481d65fa [ie/hotstar] Extract from new API (#15480)
Closes #15479
Authored by: 0xvd
2026-01-04 04:52:37 +00:00
Cédric Luthi
6ae9e95687 [ie/tv5unis] Fix extractors (#15477)
Closes #12662
Authored by: 0xced
2026-01-04 01:02:29 +00:00
pomtnp
9c393e3f62 [ie/tiktok] Extract save_count (#15054)
Closes #15053
Authored by: pomtnp
2026-01-03 21:48:42 +00:00
Emi
87a265d820 [ie/tumblr] Extract timestamp (#15462)
Authored by: alch-emi
2026-01-03 20:54:29 +00:00
doe1080
4d4c7e1c69 [utils] js_to_json: Prevent false positives for octals (#15474)
Authored by: doe1080
2026-01-03 20:53:16 +00:00
João Victor Fernandes Oliveira
0066de5b7e [ie/zoom] Extract recordings with start times (#15475)
Authored by: JV-Fernandes
2026-01-03 20:30:38 +00:00
Oliver Pfeiffer
5026548d65 [ie/bigo] Support --wait-for-video (#15463)
Authored by: olipfei
2026-01-03 00:20:59 +00:00
0x∅
e15ca65874 [ie/twitch:videos] Raise error when channel is not found (#15458)
Closes #15450
Authored by: 0xvd
2026-01-03 00:17:38 +00:00
bashonly
3763d0d4ab [build] Improve nightly release check (#15455)
Authored by: bashonly
2026-01-02 16:02:58 +00:00
Subrat Lima
260ba3abba [ie/tarangplus] Add extractors (#13060)
Closes #13020
Authored by: subrat-lima
2026-01-02 00:15:25 +00:00
ptlydpr
878a41e283 [ie/pandatv] Add extractor (#13210)
Authored by: ptlydpr
2026-01-01 01:24:14 +01:00
bashonly
76c31a7a21 [ie/youtube] Fix comment subthreads extraction (#15448)
Fix d22436e5dc

Closes #15444
Authored by: bashonly
2025-12-31 09:56:26 +00:00
bashonly
ab3ff2d5dd [build] Harden CI/CD pipeline (#15387)
* NOTE: the release workflows' new handling of secrets
  may be a breaking change for forks that are using any secrets
  other than GPG_SIGNING_KEY or ARCHIVE_REPO_TOKEN.

  Previously, the release workflow would try to resolve a token
  secret name based on the `target` or `source` input,
  e.g. NIGHTLY_ARCHIVE_REPO_TOKEN or CUSTOM_ARCHIVE_REPO_TOKEN,
  and then fall back to using the ARCHIVE_REPO_TOKEN secret if the
  resolved token secret name was not found in the repository.

  This behavior has been replaced by the release workflow
  always using the ARCHIVE_REPO_TOKEN secret as the token
  for publishing releases to any external archive repository.

* Add zizmor CI job for auditing workflows

* Pin all actions to commit hashes instead of symbolic references

* Explicitly set GITHUB_TOKEN permissions at the job level

* Use actions/checkout with `persist-credentials: false` whenever possible

* Remove/replace template expansions in workflow scripts

* Remove all usage of actions/cache from build/release workflows

* Remove the cache-warmer.yml workflow

* Remove the unused download.yml workflow

* Set concurrency limits for any workflows that are triggered by PRs

* Avoid loading the entire secrets context

* Replace usage of `secrets: inherit` with explicit `secrets:` blocks

* Pin all external docker images to hash that are used by the build workflow

* Explicitly set `shell: bash` for some steps to avoid pwsh or set pipefail

* Ensure any pwsh steps will fail on non-zero exit codes

Authored by: bashonly
2025-12-30 21:05:10 +00:00
bashonly
468aa6a9b4 [ie/youtube] Fix tracking of parent comment among replies (#15439)
Fix d22436e5dc

Closes #15438
Authored by: bashonly
2025-12-30 20:53:33 +00:00
prettysunflower
6c918c5071 [ie/nebula:season] Support more URLs (#15436)
Authored by: prettysunflower
2025-12-30 21:41:19 +01:00
sepro
09078190b0 [ie/iqiyi] Remove broken login support (#15441)
Authored by: seproDev
2025-12-30 15:02:35 +01:00
sepro
4a772e5289 [ie/scte] Remove extractors (#15442)
Authored by: seproDev
2025-12-30 15:01:24 +01:00
cesbar
f24b9ac0c9 [utils] decode_packed_codes: Fix missing key handling (#15440)
Authored by: cesbar
2025-12-30 14:57:42 +01:00
bashonly
2a7e048a60 [ie/facebook] Remove broken login support (#15434)
Authored by: bashonly
2025-12-30 00:48:11 +00:00
bashonly
a6ba714005 [ie/twitter] Remove broken login support (#15432)
Closes #12616
Authored by: bashonly
2025-12-30 00:22:33 +00:00
bashonly
ce9a3591f8 [ie/twitter] Do not extract non-video posts from unified_cards (#15431)
Closes #15402
Authored by: bashonly
2025-12-30 00:20:44 +00:00
bashonly
d22436e5dc [ie/youtube] Support comment subthreads (#15419)
* Support newly rolled out comment "subthreads"
* Fix comments extraction: all replies were being missed
* Add a `max-depth` element to the `max_comments` extractor-arg
* Fully remove the deprecated `max_comment_depth` extractor-arg

Closes #15303
Authored by: bashonly
2025-12-29 21:46:29 +00:00
bashonly
abf29e3e72 [ie/youtube] Fix skip_player=js extractor-arg (#15428)
Authored by: bashonly
2025-12-29 21:41:48 +00:00
Mike Fährmann
fcd47d2db3 [ie/picarto] Fix extraction when stream has no title (#15407)
Closes #14540
Authored by: mikf
2025-12-29 02:50:03 +00:00
bashonly
cea825e7e0 [ie/generic] Improve detection of blockage due to TLS fingerprint (#15426)
Authored by: bashonly
2025-12-29 01:02:09 +00:00
sepro
c0a7c594a9 [utils] mimetype2ext: Recognize more srt types (#15411)
Authored by: seproDev
2025-12-26 19:00:45 +01:00
sepro
6b23305822 [ie/manoto] Remove extractor (#15414)
Authored by: seproDev
2025-12-26 18:57:08 +01:00
sepro
6d92f87ddc [ie/cda] Support mobile URLs (#15398)
Closes #15397
Authored by: seproDev
2025-12-25 02:25:03 +01:00
sepro
9bf040dc6f [utils] random_user_agent: Bump versions (#15396)
Authored by: seproDev
2025-12-24 21:47:50 +01:00
doe1080
15263d049c [utils] unified_timestamp: Add tz_offset parameter (#15357)
Allows datetime strings without a timezone to be parsed with the correct offset

Authored by: doe1080
2025-12-20 19:52:53 +00:00
0x∅
0ea6cc6d82 [ie/netease:program] Support DJ URLs (#15365)
Closes #15364
Authored by: 0xvd
2025-12-20 10:09:22 +00:00
0x∅
e9d4b22b9b [ie/bandcamp:weekly] Fix extractor (#15208)
Closes #13963
Authored by: 0xvd, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-12-20 03:54:08 +00:00
0x∅
97fb78a5b9 [ie/yahoo] Fix extractor (#15314)
Closes #15211
Authored by: 0xvd, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-12-20 02:58:47 +00:00
0x∅
f5270705e8 [ie/nebula:season] Add extractor (#15347)
Closes #15343
Authored by: 0xvd, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-12-20 01:51:09 +00:00
bashonly
a6a8f6b6d6 [ci] Explicitly declare permissions and limit credentials (#15324)
Authored by: bashonly
2025-12-19 19:22:23 +00:00
bashonly
825648a740 [build] Bump official actions to latest versions (#15305)
* Bump actions/cache → v5
* Bump actions/upload-artifact → v6
* Bump actions/download-artifact → v7

Authored by: bashonly
2025-12-19 19:04:52 +00:00
bashonly
e0bb477732 Bypass interactive format selection if no formats are found (#15278)
Authored by: bashonly
2025-12-19 18:57:55 +00:00
delta
c0c9cac554 [ie/filmarchiv] Add extractor (#13490)
Closes #14821
Authored by: 4elta
2025-12-19 00:44:58 +00:00
0x∅
f0bc71abf6 [ie/tubitv] Support URLs with locales (#15205)
Closes #15176
Authored by: 0xvd
2025-12-19 00:26:53 +00:00
0x∅
8a4b626daf [ie/dropbox] Support videos in folders (#15313)
Closes #15312
Authored by: 0xvd
2025-12-19 00:24:13 +00:00
0x∅
f6dc7d5279 Accept float values for --sleep-subtitles (#15282)
Closes #15269
Authored by: 0xvd
2025-12-18 23:42:50 +00:00
quietvoid
c5e55e0479 [ie/gofile] Fix extractor (#15296)
Authored by: quietvoid
2025-12-18 23:42:13 +00:00
doe1080
6d4984e64e [ie/nextmedia] Remove extractors (#15354)
Authored by: doe1080
2025-12-18 21:36:15 +00:00
doe1080
a27ec9efc6 [ie/netzkino] Rework extractor (#15351)
Authored by: doe1080
2025-12-18 21:32:54 +00:00
bashonly
ff61bef041 [ie/youtube:tab] Fix flat thumbnails extraction for shorts (#15331)
Closes #15329
Authored by: bashonly
2025-12-15 22:37:25 +00:00
sepro
04f2ec4b97 [ie/parti] Fix extractors (#15319)
Authored by: seproDev
2025-12-13 20:00:56 +01:00
0x∅
b6f24745bf [ie/telecinco] Fix extractor (#15311)
Closes #15240
Authored by: 0xvd, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2025-12-12 22:25:45 +00:00
norepro
f2ee2a46fc [ie/pornhub] Optimize metadata extraction (#15231)
Closes #14621
Authored by: norepro
2025-12-12 20:52:09 +00:00
bashonly
5f37f67d37 [ie/archive.org] Fix metadata extraction (#15286)
Closes #15280
Authored by: bashonly
2025-12-09 19:05:12 +00:00
138 changed files with 4925 additions and 2753 deletions

View File

@@ -1,5 +1,4 @@
config-variables: config-variables:
- KEEP_CACHE_WARM
- PUSH_VERSION_COMMIT - PUSH_VERSION_COMMIT
- UPDATE_TO_VERIFICATION - UPDATE_TO_VERIFICATION
- PYPI_PROJECT - PYPI_PROJECT

View File

@@ -74,11 +74,11 @@ on:
default: true default: true
type: boolean type: boolean
permissions: permissions: {}
contents: read
jobs: jobs:
process: process:
name: Process
runs-on: ubuntu-latest runs-on: ubuntu-latest
outputs: outputs:
origin: ${{ steps.process_inputs.outputs.origin }} origin: ${{ steps.process_inputs.outputs.origin }}
@@ -146,7 +146,6 @@ jobs:
'runner': 'ubuntu-24.04-arm', 'runner': 'ubuntu-24.04-arm',
'qemu_platform': 'linux/arm/v7', 'qemu_platform': 'linux/arm/v7',
'onefile': False, 'onefile': False,
'cache_requirements': True,
'update_to': 'yt-dlp/yt-dlp@2023.03.04', 'update_to': 'yt-dlp/yt-dlp@2023.03.04',
}], }],
'musllinux': [{ 'musllinux': [{
@@ -175,7 +174,6 @@ jobs:
exe.setdefault('qemu_platform', None) exe.setdefault('qemu_platform', None)
exe.setdefault('onefile', True) exe.setdefault('onefile', True)
exe.setdefault('onedir', True) exe.setdefault('onedir', True)
exe.setdefault('cache_requirements', False)
exe.setdefault('python_version', os.environ['PYTHON_VERSION']) exe.setdefault('python_version', os.environ['PYTHON_VERSION'])
exe.setdefault('update_to', os.environ['UPDATE_TO']) exe.setdefault('update_to', os.environ['UPDATE_TO'])
if not any(INPUTS.get(key) for key in EXE_MAP): if not any(INPUTS.get(key) for key in EXE_MAP):
@@ -186,8 +184,11 @@ jobs:
f.write(f'matrix={json.dumps(matrix)}') f.write(f'matrix={json.dumps(matrix)}')
unix: unix:
needs: process name: unix
needs: [process]
if: inputs.unix if: inputs.unix
permissions:
contents: read
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
CHANNEL: ${{ inputs.channel }} CHANNEL: ${{ inputs.channel }}
@@ -196,11 +197,12 @@ jobs:
UPDATE_TO: yt-dlp/yt-dlp@2025.09.05 UPDATE_TO: yt-dlp/yt-dlp@2025.09.05
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: with:
fetch-depth: 0 # Needed for changelog fetch-depth: 0 # Needed for changelog
persist-credentials: false
- uses: actions/setup-python@v6 - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: "3.10" python-version: "3.10"
@@ -229,7 +231,7 @@ jobs:
[[ "${version}" != "${downgraded_version}" ]] [[ "${version}" != "${downgraded_version}" ]]
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with: with:
name: build-bin-${{ github.job }} name: build-bin-${{ github.job }}
path: | path: |
@@ -239,8 +241,10 @@ jobs:
linux: linux:
name: ${{ matrix.os }} (${{ matrix.arch }}) name: ${{ matrix.os }} (${{ matrix.arch }})
needs: [process]
if: inputs.linux || inputs.linux_armv7l || inputs.musllinux if: inputs.linux || inputs.linux_armv7l || inputs.musllinux
needs: process permissions:
contents: read
runs-on: ${{ matrix.runner }} runs-on: ${{ matrix.runner }}
strategy: strategy:
fail-fast: false fail-fast: false
@@ -257,26 +261,16 @@ jobs:
SKIP_ONEFILE_BUILD: ${{ (!matrix.onefile && '1') || '' }} SKIP_ONEFILE_BUILD: ${{ (!matrix.onefile && '1') || '' }}
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Cache requirements
if: matrix.cache_requirements
id: cache-venv
uses: actions/cache@v4
env:
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
with: with:
path: | persist-credentials: false
venv
key: cache-reqs-${{ matrix.os }}_${{ matrix.arch }}-${{ github.ref }}-${{ needs.process.outputs.timestamp }}
restore-keys: |
cache-reqs-${{ matrix.os }}_${{ matrix.arch }}-${{ github.ref }}-
cache-reqs-${{ matrix.os }}_${{ matrix.arch }}-
- name: Set up QEMU - name: Set up QEMU
if: matrix.qemu_platform if: matrix.qemu_platform
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0
with: with:
image: tonistiigi/binfmt:qemu-v10.0.4-56@sha256:30cc9a4d03765acac9be2ed0afc23af1ad018aed2c28ea4be8c2eb9afe03fbd1
cache-image: false
platforms: ${{ matrix.qemu_platform }} platforms: ${{ matrix.qemu_platform }}
- name: Build executable - name: Build executable
@@ -300,7 +294,7 @@ jobs:
docker compose up --build --exit-code-from "${SERVICE}" "${SERVICE}" docker compose up --build --exit-code-from "${SERVICE}" "${SERVICE}"
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with: with:
name: build-bin-${{ matrix.os }}_${{ matrix.arch }} name: build-bin-${{ matrix.os }}_${{ matrix.arch }}
path: | path: |
@@ -308,7 +302,8 @@ jobs:
compression-level: 0 compression-level: 0
macos: macos:
needs: process name: macos
needs: [process]
if: inputs.macos if: inputs.macos
permissions: permissions:
contents: read contents: read
@@ -320,21 +315,11 @@ jobs:
UPDATE_TO: yt-dlp/yt-dlp@2025.09.05 UPDATE_TO: yt-dlp/yt-dlp@2025.09.05
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# NB: Building universal2 does not work with python from actions/setup-python
- name: Cache requirements
id: cache-venv
uses: actions/cache@v4
env:
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
with: with:
path: | persist-credentials: false
~/yt-dlp-build-venv
key: cache-reqs-${{ github.job }}-${{ github.ref }}-${{ needs.process.outputs.timestamp }} # NB: Building universal2 does not work with python from actions/setup-python
restore-keys: |
cache-reqs-${{ github.job }}-${{ github.ref }}-
cache-reqs-${{ github.job }}-
- name: Install Requirements - name: Install Requirements
run: | run: |
@@ -350,7 +335,7 @@ jobs:
# We need to fuse our own universal2 wheels for curl_cffi # We need to fuse our own universal2 wheels for curl_cffi
python3 -m pip install -U 'delocate==0.11.0' python3 -m pip install -U 'delocate==0.11.0'
mkdir curl_cffi_whls curl_cffi_universal2 mkdir curl_cffi_whls curl_cffi_universal2
python3 devscripts/install_deps.py --print --omit-default --include-extra curl-cffi > requirements.txt python3 devscripts/install_deps.py --print --omit-default --include-extra build-curl-cffi > requirements.txt
for platform in "macosx_11_0_arm64" "macosx_11_0_x86_64"; do for platform in "macosx_11_0_arm64" "macosx_11_0_x86_64"; do
python3 -m pip download \ python3 -m pip download \
--only-binary=:all: \ --only-binary=:all: \
@@ -399,7 +384,7 @@ jobs:
[[ "$version" != "$downgraded_version" ]] [[ "$version" != "$downgraded_version" ]]
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with: with:
name: build-bin-${{ github.job }} name: build-bin-${{ github.job }}
path: | path: |
@@ -409,7 +394,7 @@ jobs:
windows: windows:
name: windows (${{ matrix.arch }}) name: windows (${{ matrix.arch }})
needs: process needs: [process]
if: inputs.windows if: inputs.windows
permissions: permissions:
contents: read contents: read
@@ -422,23 +407,23 @@ jobs:
runner: windows-2025 runner: windows-2025
python_version: '3.10' python_version: '3.10'
platform_tag: win_amd64 platform_tag: win_amd64
pyi_version: '6.17.0' pyi_version: '6.18.0'
pyi_tag: '2025.11.29.054325' pyi_tag: '2026.01.29.160356'
pyi_hash: e28cc13e4ad0cc74330d832202806d0c1976e9165da6047309348ca663c0ed3d pyi_hash: bb9cd0b0b233e4d031a295211cb8aa7c7f8b3c12ff33f1d57a40849ab4d3cf42
- arch: 'x86' - arch: 'x86'
runner: windows-2025 runner: windows-2025
python_version: '3.10' python_version: '3.10'
platform_tag: win32 platform_tag: win32
pyi_version: '6.17.0' pyi_version: '6.18.0'
pyi_tag: '2025.11.29.054325' pyi_tag: '2026.01.29.160356'
pyi_hash: c00f600c17de3bdd589f043f60ab64fc34fcba6dd902ad973af9c8afc74f80d1 pyi_hash: aa8f260e735d94f1e2e1aac42e322f508eb54d0433de803c2998c337f72045e4
- arch: 'arm64' - arch: 'arm64'
runner: windows-11-arm runner: windows-11-arm
python_version: '3.13' # arm64 only has Python >= 3.11 available python_version: '3.13' # arm64 only has Python >= 3.11 available
platform_tag: win_arm64 platform_tag: win_arm64
pyi_version: '6.17.0' pyi_version: '6.18.0'
pyi_tag: '2025.11.29.054325' pyi_tag: '2026.01.29.160356'
pyi_hash: a2033b18b4f7bc6108b5fd76a92c6c1de0a12ec4fe98a23396a9f978cb4b7d7b pyi_hash: 4bbca67d0cdfa860d92ac9cc7e4c2586fd393d1e814e3f1375b8c62d5cfb6771
env: env:
CHANNEL: ${{ inputs.channel }} CHANNEL: ${{ inputs.channel }}
ORIGIN: ${{ needs.process.outputs.origin }} ORIGIN: ${{ needs.process.outputs.origin }}
@@ -450,26 +435,15 @@ jobs:
PYI_WHEEL: pyinstaller-${{ matrix.pyi_version }}-py3-none-${{ matrix.platform_tag }}.whl PYI_WHEEL: pyinstaller-${{ matrix.pyi_version }}-py3-none-${{ matrix.platform_tag }}.whl
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@v6 with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: ${{ matrix.python_version }} python-version: ${{ matrix.python_version }}
architecture: ${{ matrix.arch }} architecture: ${{ matrix.arch }}
- name: Cache requirements
id: cache-venv
if: matrix.arch == 'arm64'
uses: actions/cache@v4
env:
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
with:
path: |
/yt-dlp-build-venv
key: ${{ env.BASE_CACHE_KEY }}-${{ github.ref }}-${{ needs.process.outputs.timestamp }}
restore-keys: |
${{ env.BASE_CACHE_KEY }}-${{ github.ref }}-
${{ env.BASE_CACHE_KEY }}-
- name: Install Requirements - name: Install Requirements
env: env:
ARCH: ${{ matrix.arch }} ARCH: ${{ matrix.arch }}
@@ -477,6 +451,8 @@ jobs:
PYI_HASH: ${{ matrix.pyi_hash }} PYI_HASH: ${{ matrix.pyi_hash }}
shell: pwsh shell: pwsh
run: | run: |
$ErrorActionPreference = "Stop"
$PSNativeCommandUseErrorActionPreference = $true
python -m venv /yt-dlp-build-venv python -m venv /yt-dlp-build-venv
/yt-dlp-build-venv/Scripts/Activate.ps1 /yt-dlp-build-venv/Scripts/Activate.ps1
python -m pip install -U pip python -m pip install -U pip
@@ -488,18 +464,22 @@ jobs:
if ("${Env:ARCH}" -eq "x86") { if ("${Env:ARCH}" -eq "x86") {
python devscripts/install_deps.py python devscripts/install_deps.py
} else { } else {
python devscripts/install_deps.py --include-extra curl-cffi python devscripts/install_deps.py --include-extra build-curl-cffi
} }
- name: Prepare - name: Prepare
shell: pwsh shell: pwsh
run: | run: |
$ErrorActionPreference = "Stop"
$PSNativeCommandUseErrorActionPreference = $true
python devscripts/update-version.py -c "${Env:CHANNEL}" -r "${Env:ORIGIN}" "${Env:VERSION}" python devscripts/update-version.py -c "${Env:CHANNEL}" -r "${Env:ORIGIN}" "${Env:VERSION}"
python devscripts/make_lazy_extractors.py python devscripts/make_lazy_extractors.py
- name: Build - name: Build
shell: pwsh shell: pwsh
run: | run: |
$ErrorActionPreference = "Stop"
$PSNativeCommandUseErrorActionPreference = $true
/yt-dlp-build-venv/Scripts/Activate.ps1 /yt-dlp-build-venv/Scripts/Activate.ps1
python -m bundle.pyinstaller python -m bundle.pyinstaller
python -m bundle.pyinstaller --onedir python -m bundle.pyinstaller --onedir
@@ -509,6 +489,8 @@ jobs:
if: vars.UPDATE_TO_VERIFICATION if: vars.UPDATE_TO_VERIFICATION
shell: pwsh shell: pwsh
run: | run: |
$ErrorActionPreference = "Stop"
$PSNativeCommandUseErrorActionPreference = $true
$name = "yt-dlp${Env:SUFFIX}" $name = "yt-dlp${Env:SUFFIX}"
Copy-Item "./dist/${name}.exe" "./dist/${name}_downgraded.exe" Copy-Item "./dist/${name}.exe" "./dist/${name}_downgraded.exe"
$version = & "./dist/${name}.exe" --version $version = & "./dist/${name}.exe" --version
@@ -519,7 +501,7 @@ jobs:
} }
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with: with:
name: build-bin-${{ github.job }}-${{ matrix.arch }} name: build-bin-${{ github.job }}-${{ matrix.arch }}
path: | path: |
@@ -528,23 +510,25 @@ jobs:
compression-level: 0 compression-level: 0
meta_files: meta_files:
if: always() && !cancelled() name: Metadata files
needs: needs:
- process - process
- unix - unix
- linux - linux
- macos - macos
- windows - windows
if: always() && !failure() && !cancelled()
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v5 uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with: with:
path: artifact path: artifact
pattern: build-bin-* pattern: build-bin-*
merge-multiple: true merge-multiple: true
- name: Make SHA2-SUMS files - name: Make SHA2-SUMS files
shell: bash
run: | run: |
cd ./artifact/ cd ./artifact/
# make sure SHA sums are also printed to stdout # make sure SHA sums are also printed to stdout
@@ -600,13 +584,13 @@ jobs:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }} GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
if: env.GPG_SIGNING_KEY if: env.GPG_SIGNING_KEY
run: | run: |
gpg --batch --import <<< "${{ secrets.GPG_SIGNING_KEY }}" gpg --batch --import <<< "${GPG_SIGNING_KEY}"
for signfile in ./SHA*SUMS; do for signfile in ./SHA*SUMS; do
gpg --batch --detach-sign "$signfile" gpg --batch --detach-sign "$signfile"
done done
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with: with:
name: build-${{ github.job }} name: build-${{ github.job }}
path: | path: |

View File

@@ -1,23 +0,0 @@
name: Keep cache warm
on:
workflow_dispatch:
schedule:
- cron: '0 22 1,6,11,16,21,27 * *'
jobs:
build:
if: |
vars.KEEP_CACHE_WARM || github.event_name == 'workflow_dispatch'
uses: ./.github/workflows/build.yml
with:
version: '999999'
channel: stable
origin: ${{ github.repository }}
unix: false
linux: false
linux_armv7l: true
musllinux: false
macos: true
windows: true
permissions:
contents: read

View File

@@ -16,8 +16,8 @@ on:
- yt_dlp/extractor/youtube/jsc/**.py - yt_dlp/extractor/youtube/jsc/**.py
- yt_dlp/extractor/youtube/pot/**.py - yt_dlp/extractor/youtube/pot/**.py
- yt_dlp/utils/_jsruntime.py - yt_dlp/utils/_jsruntime.py
permissions:
contents: read permissions: {}
concurrency: concurrency:
group: challenge-tests-${{ github.event.pull_request.number || github.ref }} group: challenge-tests-${{ github.event.pull_request.number || github.ref }}
@@ -26,6 +26,9 @@ concurrency:
jobs: jobs:
tests: tests:
name: Challenge Tests name: Challenge Tests
if: ${{ !contains(github.event.head_commit.message, ':ci skip') }}
permissions:
contents: read
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
fail-fast: false fail-fast: false
@@ -35,26 +38,30 @@ jobs:
env: env:
QJS_VERSION: '2025-04-26' # Earliest version with rope strings QJS_VERSION: '2025-04-26' # Earliest version with rope strings
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6 uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Deno - name: Install Deno
uses: denoland/setup-deno@v2 uses: denoland/setup-deno@e95548e56dfa95d4e1a28d6f422fafe75c4c26fb # v2.0.3
with: with:
deno-version: '2.0.0' # minimum supported version deno-version: '2.0.0' # minimum supported version
- name: Install Bun - name: Install Bun
uses: oven-sh/setup-bun@v2 uses: oven-sh/setup-bun@3d267786b128fe76c2f16a390aa2448b815359f3 # v2.1.2
with: with:
# minimum supported version is 1.0.31 but earliest available Windows version is 1.1.0 # minimum supported version is 1.0.31 but earliest available Windows version is 1.1.0
bun-version: ${{ (matrix.os == 'windows-latest' && '1.1.0') || '1.0.31' }} bun-version: ${{ (matrix.os == 'windows-latest' && '1.1.0') || '1.0.31' }}
no-cache: true
- name: Install Node - name: Install Node
uses: actions/setup-node@v6 uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with: with:
node-version: '20.0' # minimum supported version node-version: '20.0' # minimum supported version
- name: Install QuickJS (Linux) - name: Install QuickJS (Linux)
if: matrix.os == 'ubuntu-latest' if: matrix.os == 'ubuntu-latest'
shell: bash
run: | run: |
wget "https://bellard.org/quickjs/binary_releases/quickjs-linux-x86_64-${QJS_VERSION}.zip" -O quickjs.zip wget "https://bellard.org/quickjs/binary_releases/quickjs-linux-x86_64-${QJS_VERSION}.zip" -O quickjs.zip
unzip quickjs.zip qjs unzip quickjs.zip qjs
@@ -63,15 +70,19 @@ jobs:
if: matrix.os == 'windows-latest' if: matrix.os == 'windows-latest'
shell: pwsh shell: pwsh
run: | run: |
$ErrorActionPreference = "Stop"
$PSNativeCommandUseErrorActionPreference = $true
Invoke-WebRequest "https://bellard.org/quickjs/binary_releases/quickjs-win-x86_64-${Env:QJS_VERSION}.zip" -OutFile quickjs.zip Invoke-WebRequest "https://bellard.org/quickjs/binary_releases/quickjs-win-x86_64-${Env:QJS_VERSION}.zip" -OutFile quickjs.zip
unzip quickjs.zip unzip quickjs.zip
- name: Install test requirements - name: Install test requirements
shell: bash
run: | run: |
python ./devscripts/install_deps.py --print --omit-default --include-extra test > requirements.txt python ./devscripts/install_deps.py --print --omit-default --include-extra test > requirements.txt
python ./devscripts/install_deps.py --print -c certifi -c requests -c urllib3 -c yt-dlp-ejs >> requirements.txt python ./devscripts/install_deps.py --print -c certifi -c requests -c urllib3 -c yt-dlp-ejs >> requirements.txt
python -m pip install -U -r requirements.txt python -m pip install -U -r requirements.txt
- name: Run tests - name: Run tests
timeout-minutes: 15 timeout-minutes: 15
shell: bash
run: | run: |
python -m yt_dlp -v --js-runtimes node --js-runtimes bun --js-runtimes quickjs || true python -m yt_dlp -v --js-runtimes node --js-runtimes bun --js-runtimes quickjs || true
python ./devscripts/run_tests.py test/test_jsc -k download python ./devscripts/run_tests.py test/test_jsc -k download

View File

@@ -9,14 +9,20 @@ on:
schedule: schedule:
- cron: '59 11 * * 5' - cron: '59 11 * * 5'
permissions: {}
concurrency:
group: codeql-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs: jobs:
analyze: analyze:
name: Analyze (${{ matrix.language }}) name: Analyze (${{ matrix.language }})
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
actions: read actions: read # Needed by github/codeql-action if repository is private
contents: read contents: read
security-events: write security-events: write # Needed to use github/codeql-action with Github Advanced Security
strategy: strategy:
fail-fast: false fail-fast: false
@@ -25,15 +31,17 @@ jobs:
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v6 uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@v4 uses: github/codeql-action/init@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
build-mode: none build-mode: none
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4 uses: github/codeql-action/analyze@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
with: with:
category: "/language:${{matrix.language}}" category: "/language:${{matrix.language}}"

View File

@@ -22,8 +22,8 @@ on:
- yt_dlp/extractor/__init__.py - yt_dlp/extractor/__init__.py
- yt_dlp/extractor/common.py - yt_dlp/extractor/common.py
- yt_dlp/extractor/extractors.py - yt_dlp/extractor/extractors.py
permissions:
contents: read permissions: {}
concurrency: concurrency:
group: core-${{ github.event.pull_request.number || github.ref }} group: core-${{ github.event.pull_request.number || github.ref }}
@@ -32,7 +32,9 @@ concurrency:
jobs: jobs:
tests: tests:
name: Core Tests name: Core Tests
if: "!contains(github.event.head_commit.message, 'ci skip')" if: ${{ !contains(github.event.head_commit.message, ':ci skip') }}
permissions:
contents: read
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
fail-fast: false fail-fast: false
@@ -55,11 +57,12 @@ jobs:
- os: windows-latest - os: windows-latest
python-version: pypy-3.11 python-version: pypy-3.11
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6 uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install test requirements - name: Install test requirements

View File

@@ -1,48 +0,0 @@
name: Download Tests
on: [push, pull_request]
permissions:
contents: read
jobs:
quick:
name: Quick Download Tests
if: "contains(github.event.head_commit.message, 'ci run dl')"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.10'
- name: Install test requirements
run: python ./devscripts/install_deps.py --include-extra dev
- name: Run tests
continue-on-error: true
run: python ./devscripts/run_tests.py download
full:
name: Full Download Tests
if: "contains(github.event.head_commit.message, 'ci run dl all')"
runs-on: ${{ matrix.os }}
strategy:
fail-fast: true
matrix:
os: [ubuntu-latest]
python-version: ['3.11', '3.12', '3.13', '3.14', pypy-3.11]
include:
# atleast one of each CPython/PyPy tests must be in windows
- os: windows-latest
python-version: '3.10'
- os: windows-latest
python-version: pypy-3.11
steps:
- uses: actions/checkout@v6
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install test requirements
run: python ./devscripts/install_deps.py --include-extra dev
- name: Run tests
continue-on-error: true
run: python ./devscripts/run_tests.py download

View File

@@ -3,13 +3,14 @@ on:
issues: issues:
types: [opened] types: [opened]
permissions: permissions: {}
issues: write
jobs: jobs:
lockdown: lockdown:
name: Issue Lockdown name: Issue Lockdown
if: vars.ISSUE_LOCKDOWN if: vars.ISSUE_LOCKDOWN
permissions:
issues: write # Needed to lock issues
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: "Lock new issue" - name: "Lock new issue"

View File

@@ -1,33 +1,47 @@
name: Quick Test name: Quick Test
on: [push, pull_request] on: [push, pull_request]
permissions:
contents: read permissions: {}
concurrency:
group: quick-test-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs: jobs:
tests: tests:
name: Core Test name: Core Test
if: "!contains(github.event.head_commit.message, 'ci skip all')" if: ${{ !contains(github.event.head_commit.message, ':ci skip all') }}
permissions:
contents: read
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Python 3.10 - name: Set up Python 3.10
uses: actions/setup-python@v6 uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: '3.10' python-version: '3.10'
- name: Install test requirements - name: Install test requirements
shell: bash
run: python ./devscripts/install_deps.py --omit-default --include-extra test run: python ./devscripts/install_deps.py --omit-default --include-extra test
- name: Run tests - name: Run tests
timeout-minutes: 15 timeout-minutes: 15
shell: bash
run: | run: |
python3 -m yt_dlp -v || true python3 -m yt_dlp -v || true
python3 ./devscripts/run_tests.py --pytest-args '--reruns 2 --reruns-delay 3.0' core python3 ./devscripts/run_tests.py --pytest-args '--reruns 2 --reruns-delay 3.0' core
check: check:
name: Code check name: Code check
if: "!contains(github.event.head_commit.message, 'ci skip all')" if: ${{ !contains(github.event.head_commit.message, ':ci skip all') }}
permissions:
contents: read
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@v6 with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: '3.10' python-version: '3.10'
- name: Install dev dependencies - name: Install dev dependencies
@@ -39,4 +53,5 @@ jobs:
- name: Run autopep8 - name: Run autopep8
run: autopep8 --diff . run: autopep8 --diff .
- name: Check file mode - name: Check file mode
shell: bash
run: git ls-files --format="%(objectmode) %(path)" yt_dlp/ | ( ! grep -v "^100644" ) run: git ls-files --format="%(objectmode) %(path)" yt_dlp/ | ( ! grep -v "^100644" )

View File

@@ -14,35 +14,39 @@ on:
- ".github/workflows/release-master.yml" - ".github/workflows/release-master.yml"
concurrency: concurrency:
group: release-master group: release-master
permissions:
contents: read permissions: {}
jobs: jobs:
release: release:
name: Publish Github release
if: vars.BUILD_MASTER if: vars.BUILD_MASTER
permissions:
contents: write # May be needed to publish release
id-token: write # Needed for trusted publishing
uses: ./.github/workflows/release.yml uses: ./.github/workflows/release.yml
with: with:
prerelease: true prerelease: true
source: ${{ (github.repository != 'yt-dlp/yt-dlp' && vars.MASTER_ARCHIVE_REPO) || 'master' }} source: ${{ (github.repository != 'yt-dlp/yt-dlp' && vars.MASTER_ARCHIVE_REPO) || 'master' }}
target: 'master' target: 'master'
permissions: secrets:
contents: write ARCHIVE_REPO_TOKEN: ${{ secrets.ARCHIVE_REPO_TOKEN }}
id-token: write # mandatory for trusted publishing GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
secrets: inherit
publish_pypi: publish_pypi:
name: Publish to PyPI
needs: [release] needs: [release]
if: vars.MASTER_PYPI_PROJECT if: vars.MASTER_PYPI_PROJECT
runs-on: ubuntu-latest
permissions: permissions:
id-token: write # mandatory for trusted publishing id-token: write # Needed for trusted publishing
runs-on: ubuntu-latest
steps: steps:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v5 uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with: with:
path: dist path: dist
name: build-pypi name: build-pypi
- name: Publish to PyPI - name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1 uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with: with:
verbose: true verbose: true

View File

@@ -2,21 +2,43 @@ name: Release (nightly)
on: on:
schedule: schedule:
- cron: '23 23 * * *' - cron: '23 23 * * *'
permissions: workflow_dispatch:
contents: read
permissions: {}
jobs: jobs:
check_nightly: check_nightly:
if: vars.BUILD_NIGHTLY name: Check for new commits
if: github.event_name == 'workflow_dispatch' || vars.BUILD_NIGHTLY
permissions:
contents: read
runs-on: ubuntu-latest runs-on: ubuntu-latest
outputs: outputs:
commit: ${{ steps.check_for_new_commits.outputs.commit }} commit: ${{ steps.check_for_new_commits.outputs.commit }}
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false
- name: Retrieve HEAD commit hash
id: head
shell: bash
run: echo "head=$(git rev-parse HEAD)" | tee -a "${GITHUB_OUTPUT}"
- name: Cache nightly commit hash
uses: actions/cache@8b402f58fbc84540c8b491a91e594a4576fec3d7 # v5.0.2
env:
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1
with:
path: .nightly_commit_hash
key: release-nightly-${{ steps.head.outputs.head }}
restore-keys: |
release-nightly-
- name: Check for new commits - name: Check for new commits
id: check_for_new_commits id: check_for_new_commits
shell: bash
run: | run: |
relevant_files=( relevant_files=(
"yt_dlp/*.py" "yt_dlp/*.py"
@@ -30,34 +52,54 @@ jobs:
".github/workflows/release.yml" ".github/workflows/release.yml"
".github/workflows/release-nightly.yml" ".github/workflows/release-nightly.yml"
) )
echo "commit=$(git log --format=%H -1 --since="24 hours ago" -- "${relevant_files[@]}")" | tee "$GITHUB_OUTPUT" if [[ -f .nightly_commit_hash ]]; then
limit_args=(
"$(cat .nightly_commit_hash)..HEAD"
)
else
limit_args=(
--since="24 hours ago"
)
fi
echo "commit=$(git log --format=%H -1 "${limit_args[@]}" -- "${relevant_files[@]}")" | tee -a "${GITHUB_OUTPUT}"
- name: Record new nightly commit hash
env:
HEAD: ${{ steps.head.outputs.head }}
shell: bash
run: echo "${HEAD}" | tee .nightly_commit_hash
release: release:
name: Publish Github release
needs: [check_nightly] needs: [check_nightly]
if: ${{ needs.check_nightly.outputs.commit }} if: needs.check_nightly.outputs.commit
permissions:
contents: write # May be needed to publish release
id-token: write # Needed for trusted publishing
uses: ./.github/workflows/release.yml uses: ./.github/workflows/release.yml
with: with:
prerelease: true prerelease: true
source: ${{ (github.repository != 'yt-dlp/yt-dlp' && vars.NIGHTLY_ARCHIVE_REPO) || 'nightly' }} source: ${{ (github.repository != 'yt-dlp/yt-dlp' && vars.NIGHTLY_ARCHIVE_REPO) || 'nightly' }}
target: 'nightly' target: 'nightly'
permissions: secrets:
contents: write ARCHIVE_REPO_TOKEN: ${{ secrets.ARCHIVE_REPO_TOKEN }}
id-token: write # mandatory for trusted publishing GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
secrets: inherit
publish_pypi: publish_pypi:
name: Publish to PyPI
needs: [release] needs: [release]
if: vars.NIGHTLY_PYPI_PROJECT if: vars.NIGHTLY_PYPI_PROJECT
runs-on: ubuntu-latest
permissions: permissions:
id-token: write # mandatory for trusted publishing id-token: write # Needed for trusted publishing
runs-on: ubuntu-latest
steps: steps:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v5 uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with: with:
path: dist path: dist
name: build-pypi name: build-pypi
- name: Publish to PyPI - name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1 uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with: with:
verbose: true verbose: true

View File

@@ -22,6 +22,11 @@ on:
required: false required: false
default: true default: true
type: boolean type: boolean
secrets:
ARCHIVE_REPO_TOKEN:
required: false
GPG_SIGNING_KEY:
required: false
workflow_dispatch: workflow_dispatch:
inputs: inputs:
source: source:
@@ -56,30 +61,30 @@ on:
default: false default: false
type: boolean type: boolean
permissions: permissions: {}
contents: read
jobs: jobs:
prepare: prepare:
name: Prepare
permissions: permissions:
contents: write contents: write # Needed to git-push the release commit
runs-on: ubuntu-latest runs-on: ubuntu-latest
outputs: outputs:
channel: ${{ steps.setup_variables.outputs.channel }} channel: ${{ steps.setup_variables.outputs.channel }}
version: ${{ steps.setup_variables.outputs.version }} version: ${{ steps.setup_variables.outputs.version }}
target_repo: ${{ steps.setup_variables.outputs.target_repo }} target_repo: ${{ steps.setup_variables.outputs.target_repo }}
target_repo_token: ${{ steps.setup_variables.outputs.target_repo_token }}
target_tag: ${{ steps.setup_variables.outputs.target_tag }} target_tag: ${{ steps.setup_variables.outputs.target_tag }}
pypi_project: ${{ steps.setup_variables.outputs.pypi_project }} pypi_project: ${{ steps.setup_variables.outputs.pypi_project }}
pypi_suffix: ${{ steps.setup_variables.outputs.pypi_suffix }} pypi_suffix: ${{ steps.setup_variables.outputs.pypi_suffix }}
head_sha: ${{ steps.get_target.outputs.head_sha }} head_sha: ${{ steps.get_target.outputs.head_sha }}
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: true # Needed to git-push the release commit
- uses: actions/setup-python@v6 - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: "3.10" # Keep this in sync with test-workflows.yml python-version: "3.10" # Keep this in sync with test-workflows.yml
@@ -104,8 +109,6 @@ jobs:
TARGET_PYPI_SUFFIX: ${{ vars[format('{0}_pypi_suffix', steps.process_inputs.outputs.target_repo)] }} TARGET_PYPI_SUFFIX: ${{ vars[format('{0}_pypi_suffix', steps.process_inputs.outputs.target_repo)] }}
SOURCE_ARCHIVE_REPO: ${{ vars[format('{0}_archive_repo', steps.process_inputs.outputs.source_repo)] }} SOURCE_ARCHIVE_REPO: ${{ vars[format('{0}_archive_repo', steps.process_inputs.outputs.source_repo)] }}
TARGET_ARCHIVE_REPO: ${{ vars[format('{0}_archive_repo', steps.process_inputs.outputs.target_repo)] }} TARGET_ARCHIVE_REPO: ${{ vars[format('{0}_archive_repo', steps.process_inputs.outputs.target_repo)] }}
HAS_SOURCE_ARCHIVE_REPO_TOKEN: ${{ !!secrets[format('{0}_archive_repo_token', steps.process_inputs.outputs.source_repo)] }}
HAS_TARGET_ARCHIVE_REPO_TOKEN: ${{ !!secrets[format('{0}_archive_repo_token', steps.process_inputs.outputs.target_repo)] }}
HAS_ARCHIVE_REPO_TOKEN: ${{ !!secrets.ARCHIVE_REPO_TOKEN }} HAS_ARCHIVE_REPO_TOKEN: ${{ !!secrets.ARCHIVE_REPO_TOKEN }}
run: | run: |
python -m devscripts.setup_variables python -m devscripts.setup_variables
@@ -127,8 +130,7 @@ jobs:
VERSION: ${{ steps.setup_variables.outputs.version }} VERSION: ${{ steps.setup_variables.outputs.version }}
GITHUB_EVENT_SENDER_LOGIN: ${{ github.event.sender.login }} GITHUB_EVENT_SENDER_LOGIN: ${{ github.event.sender.login }}
GITHUB_EVENT_REF: ${{ github.event.ref }} GITHUB_EVENT_REF: ${{ github.event.ref }}
if: | if: steps.setup_variables.outputs.target_repo == github.repository && !inputs.prerelease
!inputs.prerelease && steps.setup_variables.outputs.target_repo == github.repository
run: | run: |
git config --global user.name "github-actions[bot]" git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
@@ -145,35 +147,38 @@ jobs:
- name: Update master - name: Update master
env: env:
GITHUB_EVENT_REF: ${{ github.event.ref }} GITHUB_EVENT_REF: ${{ github.event.ref }}
if: | if: vars.PUSH_VERSION_COMMIT && !inputs.prerelease && steps.setup_variables.outputs.target_repo == github.repository
vars.PUSH_VERSION_COMMIT && !inputs.prerelease && steps.setup_variables.outputs.target_repo == github.repository
run: git push origin "${GITHUB_EVENT_REF}" run: git push origin "${GITHUB_EVENT_REF}"
build: build:
needs: prepare name: Build
needs: [prepare]
permissions:
contents: read
uses: ./.github/workflows/build.yml uses: ./.github/workflows/build.yml
with: with:
version: ${{ needs.prepare.outputs.version }} version: ${{ needs.prepare.outputs.version }}
channel: ${{ needs.prepare.outputs.channel }} channel: ${{ needs.prepare.outputs.channel }}
origin: ${{ needs.prepare.outputs.target_repo }} origin: ${{ needs.prepare.outputs.target_repo }}
linux_armv7l: ${{ inputs.linux_armv7l }} linux_armv7l: ${{ inputs.linux_armv7l }}
permissions:
contents: read
secrets: secrets:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }} GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
publish_pypi: publish_pypi:
name: Publish to PyPI
needs: [prepare, build] needs: [prepare, build]
if: ${{ needs.prepare.outputs.pypi_project }} if: needs.prepare.outputs.pypi_project
runs-on: ubuntu-latest
permissions: permissions:
id-token: write # mandatory for trusted publishing contents: read
id-token: write # Needed for trusted publishing
runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: with:
fetch-depth: 0 fetch-depth: 0 # Needed for changelog
- uses: actions/setup-python@v6 persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: "3.10" python-version: "3.10"
@@ -208,8 +213,8 @@ jobs:
python -m build --no-isolation . python -m build --no-isolation .
- name: Upload artifacts - name: Upload artifacts
if: github.event_name != 'workflow_dispatch' if: github.event.workflow != '.github/workflows/release.yml' # Reusable workflow_call
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with: with:
name: build-pypi name: build-pypi
path: | path: |
@@ -217,15 +222,16 @@ jobs:
compression-level: 0 compression-level: 0
- name: Publish to PyPI - name: Publish to PyPI
if: github.event_name == 'workflow_dispatch' if: github.event.workflow == '.github/workflows/release.yml' # Direct workflow_dispatch
uses: pypa/gh-action-pypi-publish@release/v1 uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with: with:
verbose: true verbose: true
publish: publish:
name: Publish Github release
needs: [prepare, build] needs: [prepare, build]
permissions: permissions:
contents: write contents: write # Needed by gh to publish release to Github
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
TARGET_REPO: ${{ needs.prepare.outputs.target_repo }} TARGET_REPO: ${{ needs.prepare.outputs.target_repo }}
@@ -233,15 +239,16 @@ jobs:
VERSION: ${{ needs.prepare.outputs.version }} VERSION: ${{ needs.prepare.outputs.version }}
HEAD_SHA: ${{ needs.prepare.outputs.head_sha }} HEAD_SHA: ${{ needs.prepare.outputs.head_sha }}
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: with:
fetch-depth: 0 fetch-depth: 0
- uses: actions/download-artifact@v5 persist-credentials: false
- uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with: with:
path: artifact path: artifact
pattern: build-* pattern: build-*
merge-multiple: true merge-multiple: true
- uses: actions/setup-python@v6 - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: "3.10" python-version: "3.10"
@@ -282,12 +289,11 @@ jobs:
- name: Publish to archive repo - name: Publish to archive repo
env: env:
GH_TOKEN: ${{ secrets[needs.prepare.outputs.target_repo_token] }} GH_TOKEN: ${{ secrets.ARCHIVE_REPO_TOKEN }}
GH_REPO: ${{ needs.prepare.outputs.target_repo }} GH_REPO: ${{ needs.prepare.outputs.target_repo }}
TITLE_PREFIX: ${{ startswith(env.TARGET_REPO, 'yt-dlp/') && 'yt-dlp ' || '' }} TITLE_PREFIX: ${{ startswith(env.TARGET_REPO, 'yt-dlp/') && 'yt-dlp ' || '' }}
TITLE: ${{ inputs.target != env.TARGET_REPO && inputs.target || needs.prepare.outputs.channel }} TITLE: ${{ inputs.target != env.TARGET_REPO && inputs.target || needs.prepare.outputs.channel }}
if: | if: inputs.prerelease && env.GH_TOKEN && env.GH_REPO && env.GH_REPO != github.repository
inputs.prerelease && env.GH_TOKEN && env.GH_REPO && env.GH_REPO != github.repository
run: | run: |
gh release create \ gh release create \
--notes-file ARCHIVE_NOTES \ --notes-file ARCHIVE_NOTES \
@@ -298,8 +304,7 @@ jobs:
- name: Prune old release - name: Prune old release
env: env:
GH_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }}
if: | if: env.TARGET_REPO == github.repository && env.TARGET_TAG != env.VERSION
env.TARGET_REPO == github.repository && env.TARGET_TAG != env.VERSION
run: | run: |
gh release delete --yes --cleanup-tag "${TARGET_TAG}" || true gh release delete --yes --cleanup-tag "${TARGET_TAG}" || true
git tag --delete "${TARGET_TAG}" || true git tag --delete "${TARGET_TAG}" || true
@@ -312,8 +317,7 @@ jobs:
TITLE_PREFIX: ${{ github.repository == 'yt-dlp/yt-dlp' && 'yt-dlp ' || '' }} TITLE_PREFIX: ${{ github.repository == 'yt-dlp/yt-dlp' && 'yt-dlp ' || '' }}
TITLE: ${{ env.TARGET_TAG != env.VERSION && format('{0} ', env.TARGET_TAG) || '' }} TITLE: ${{ env.TARGET_TAG != env.VERSION && format('{0} ', env.TARGET_TAG) || '' }}
PRERELEASE: ${{ inputs.prerelease && '1' || '0' }} PRERELEASE: ${{ inputs.prerelease && '1' || '0' }}
if: | if: env.TARGET_REPO == github.repository
env.TARGET_REPO == github.repository
run: | run: |
gh_options=( gh_options=(
--notes-file "${NOTES_FILE}" --notes-file "${NOTES_FILE}"

View File

@@ -4,14 +4,15 @@ on:
issue_comment: issue_comment:
types: [created, edited] types: [created, edited]
permissions: permissions: {}
issues: write
jobs: jobs:
sanitize-comment: sanitize-comment:
name: Sanitize comment name: Sanitize comment
if: vars.SANITIZE_COMMENT && !github.event.issue.pull_request if: vars.SANITIZE_COMMENT && !github.event.issue.pull_request
permissions:
issues: write # Needed by yt-dlp/sanitize-comment to edit comments
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Sanitize comment - name: Sanitize comment
uses: yt-dlp/sanitize-comment@v1 uses: yt-dlp/sanitize-comment@4536c691101b89f5373d50fe8a7980cae146346b # v1.0.0

View File

@@ -1,21 +1,30 @@
name: Test and lint workflows name: Test and lint workflows
on: on:
push: push:
branches: [master]
paths: paths:
- .github/*.yml
- .github/workflows/* - .github/workflows/*
- bundle/docker/linux/*.sh - bundle/docker/linux/*.sh
- devscripts/setup_variables.py - devscripts/setup_variables.py
- devscripts/setup_variables_tests.py - devscripts/setup_variables_tests.py
- devscripts/utils.py - devscripts/utils.py
pull_request: pull_request:
branches: [master]
paths: paths:
- .github/*.yml
- .github/workflows/* - .github/workflows/*
- bundle/docker/linux/*.sh - bundle/docker/linux/*.sh
- devscripts/setup_variables.py - devscripts/setup_variables.py
- devscripts/setup_variables_tests.py - devscripts/setup_variables_tests.py
- devscripts/utils.py - devscripts/utils.py
permissions:
contents: read permissions: {}
concurrency:
group: test-workflows-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env: env:
ACTIONLINT_VERSION: "1.7.9" ACTIONLINT_VERSION: "1.7.9"
ACTIONLINT_SHA256SUM: 233b280d05e100837f4af1433c7b40a5dcb306e3aa68fb4f17f8a7f45a7df7b4 ACTIONLINT_SHA256SUM: 233b280d05e100837f4af1433c7b40a5dcb306e3aa68fb4f17f8a7f45a7df7b4
@@ -24,15 +33,20 @@ env:
jobs: jobs:
check: check:
name: Check workflows name: Check workflows
permissions:
contents: read
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@v6 with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: "3.10" # Keep this in sync with release.yml's prepare job python-version: "3.10" # Keep this in sync with release.yml's prepare job
- name: Install requirements - name: Install requirements
env: env:
ACTIONLINT_TARBALL: ${{ format('actionlint_{0}_linux_amd64.tar.gz', env.ACTIONLINT_VERSION) }} ACTIONLINT_TARBALL: ${{ format('actionlint_{0}_linux_amd64.tar.gz', env.ACTIONLINT_VERSION) }}
shell: bash
run: | run: |
python -m devscripts.install_deps --omit-default --include-extra test python -m devscripts.install_deps --omit-default --include-extra test
sudo apt -y install shellcheck sudo apt -y install shellcheck
@@ -50,3 +64,20 @@ jobs:
- name: Test GHA devscripts - name: Test GHA devscripts
run: | run: |
pytest -Werror --tb=short --color=yes devscripts/setup_variables_tests.py pytest -Werror --tb=short --color=yes devscripts/setup_variables_tests.py
zizmor:
name: Run zizmor
permissions:
contents: read
actions: read # Needed by zizmorcore/zizmor-action if repository is private
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Run zizmor
uses: zizmorcore/zizmor-action@135698455da5c3b3e55f73f4419e481ab68cdd95 # v0.4.1
with:
advanced-security: false
persona: pedantic
version: v1.22.0

15
.github/zizmor.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
rules:
concurrency-limits:
ignore:
- build.yml # Can only be triggered by maintainers or cronjob
- issue-lockdown.yml # It *should* run for *every* new issue
- release-nightly.yml # Can only be triggered by once-daily cronjob
- release.yml # Can only be triggered by maintainers or cronjob
- sanitize-comment.yml # It *should* run for *every* new comment/edit
obfuscation:
ignore:
- release.yml # Not actual obfuscation
unpinned-uses:
config:
policies:
"*": hash-pin

View File

@@ -843,3 +843,34 @@ oxyzenQ
putridambassador121 putridambassador121
RezSat RezSat
WhatAmISupposedToPutHere WhatAmISupposedToPutHere
0xced
4elta
alch-emi
AlexBocken
cesbar
clayote
JV-Fernandes
legraphista
Mivik
nlurker
norepro
olipfei
pomtnp
prettysunflower
ptlydpr
quietvoid
romainreignier
Sytm
zahlman
azdlonky
thematuu
beacdeac
blauerdorf
CanOfSocks
gravesducking
gseddon
hunter-gatherer8
LordMZTE
regulad
stastix
syphyr

View File

@@ -4,6 +4,187 @@
# To create a release, dispatch the https://github.com/yt-dlp/yt-dlp/actions/workflows/release.yml workflow on master # To create a release, dispatch the https://github.com/yt-dlp/yt-dlp/actions/workflows/release.yml workflow on master
--> -->
### 2026.02.21
#### Important changes
- Security: [[CVE-2026-26331](https://nvd.nist.gov/vuln/detail/CVE-2026-26331)] [Arbitrary command injection with the `--netrc-cmd` option](https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-g3gw-q23r-pgqm)
- The argument passed to the command in `--netrc-cmd` is now limited to a safe subset of characters
#### Core changes
- **cookies**: [Ignore cookies with control characters](https://github.com/yt-dlp/yt-dlp/commit/43229d1d5f47b313e1958d719faff6321d853ed3) ([#15862](https://github.com/yt-dlp/yt-dlp/issues/15862)) by [bashonly](https://github.com/bashonly), [syphyr](https://github.com/syphyr)
- **jsinterp**
- [Fix bitwise operations](https://github.com/yt-dlp/yt-dlp/commit/62574f5763755a8637880044630b12582e4a55a5) ([#15985](https://github.com/yt-dlp/yt-dlp/issues/15985)) by [bashonly](https://github.com/bashonly)
- [Stringify bracket notation keys in object access](https://github.com/yt-dlp/yt-dlp/commit/c9c86519753d6cdafa052945d2de0d3fcd448927) ([#15989](https://github.com/yt-dlp/yt-dlp/issues/15989)) by [bashonly](https://github.com/bashonly)
- [Support string concatenation with `+` and `+=`](https://github.com/yt-dlp/yt-dlp/commit/d108ca10b926410ed99031fec86894bfdea8f8eb) ([#15990](https://github.com/yt-dlp/yt-dlp/issues/15990)) by [bashonly](https://github.com/bashonly)
#### Extractor changes
- [Add browser impersonation support to more extractors](https://github.com/yt-dlp/yt-dlp/commit/1d1358d09fedcdc6b3e83538a29b0b539cb9be3f) ([#16029](https://github.com/yt-dlp/yt-dlp/issues/16029)) by [bashonly](https://github.com/bashonly)
- [Limit `netrc_machine` parameter to shell-safe characters](https://github.com/yt-dlp/yt-dlp/commit/1fbbe29b99dc61375bf6d786f824d9fcf6ea9c1a) by [Grub4K](https://github.com/Grub4K)
- **1tv**: [Extract chapters](https://github.com/yt-dlp/yt-dlp/commit/23c059a455acbb317b2bbe657efd59113bf4d5ac) ([#15848](https://github.com/yt-dlp/yt-dlp/issues/15848)) by [hunter-gatherer8](https://github.com/hunter-gatherer8)
- **aenetworks**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/24856538595a3b25c75e1199146fcc82ea812d97) ([#14959](https://github.com/yt-dlp/yt-dlp/issues/14959)) by [Sipherdrakon](https://github.com/Sipherdrakon)
- **applepodcasts**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/1ea7329cc91da38a790174e831fffafcb3ea3c3d) ([#15901](https://github.com/yt-dlp/yt-dlp/issues/15901)) by [coreywright](https://github.com/coreywright)
- **dailymotion**: [Fix extraction](https://github.com/yt-dlp/yt-dlp/commit/224fe478b0ef83d13b36924befa53686290cb000) ([#15995](https://github.com/yt-dlp/yt-dlp/issues/15995)) by [bashonly](https://github.com/bashonly)
- **facebook**: ads: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/e2444584a3e590077b81828ad8a12fc4c3b1aa6d) ([#16002](https://github.com/yt-dlp/yt-dlp/issues/16002)) by [bashonly](https://github.com/bashonly)
- **gem.cbc.ca**: [Support standalone, series & Olympics URLs](https://github.com/yt-dlp/yt-dlp/commit/637ae202aca7a990b3b61bc33d692870dc16c3ad) ([#15878](https://github.com/yt-dlp/yt-dlp/issues/15878)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly), [makew0rld](https://github.com/makew0rld)
- **learningonscreen**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/46d5b6f2b7989d8991a59215d434fb8b5a8ec7bb) ([#16028](https://github.com/yt-dlp/yt-dlp/issues/16028)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly)
- **locipo**: [Add extractors](https://github.com/yt-dlp/yt-dlp/commit/442c90da3ec680037b7d94abf91ec63b2e5a9ade) ([#15486](https://github.com/yt-dlp/yt-dlp/issues/15486)) by [doe1080](https://github.com/doe1080), [gravesducking](https://github.com/gravesducking)
- **matchitv**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/8d6e0b29bf15365638e0ceeb803a274e4db6157d) ([#15204](https://github.com/yt-dlp/yt-dlp/issues/15204)) by [gseddon](https://github.com/gseddon)
- **odnoklassniki**: [Fix inefficient regular expression](https://github.com/yt-dlp/yt-dlp/commit/071ad7dfa012f5b71572d29ef96fc154cb2dc9cc) ([#15974](https://github.com/yt-dlp/yt-dlp/issues/15974)) by [bashonly](https://github.com/bashonly)
- **opencast**: [Support `oc-p.uni-jena.de` URLs](https://github.com/yt-dlp/yt-dlp/commit/166356d1a1cac19cac14298e735eeae44b52c70e) ([#16026](https://github.com/yt-dlp/yt-dlp/issues/16026)) by [LordMZTE](https://github.com/LordMZTE)
- **pornhub**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/6f38df31b477cf5ea3c8f91207452e3a4e8d5aa6) ([#15858](https://github.com/yt-dlp/yt-dlp/issues/15858)) by [beacdeac](https://github.com/beacdeac)
- **saucepluschannel**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/97f03660f55696dc9fce56e7ee43fbe3324a9867) ([#15830](https://github.com/yt-dlp/yt-dlp/issues/15830)) by [regulad](https://github.com/regulad)
- **soundcloud**
- [Fix client ID extraction](https://github.com/yt-dlp/yt-dlp/commit/81bdea03f3414dd4d086610c970ec14e15bd3d36) ([#16019](https://github.com/yt-dlp/yt-dlp/issues/16019)) by [bashonly](https://github.com/bashonly)
- [Support browser impersonation](https://github.com/yt-dlp/yt-dlp/commit/f532a91cef11075eb5a7809255259b32d2bca8ca) ([#16020](https://github.com/yt-dlp/yt-dlp/issues/16020)) by [bashonly](https://github.com/bashonly)
- **spankbang**
- [Fix playlist title extraction](https://github.com/yt-dlp/yt-dlp/commit/1fe0bf23aa2249858c08408b7cc6287aaf528690) ([#14132](https://github.com/yt-dlp/yt-dlp/issues/14132)) by [blauerdorf](https://github.com/blauerdorf)
- [Support browser impersonation](https://github.com/yt-dlp/yt-dlp/commit/f05e1cd1f1052cb40fc966d2fc175571986da863) ([#14130](https://github.com/yt-dlp/yt-dlp/issues/14130)) by [blauerdorf](https://github.com/blauerdorf)
- **steam**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/1a9c4b8238434c760b3e27d0c9df6a4a2482d918) ([#15028](https://github.com/yt-dlp/yt-dlp/issues/15028)) by [doe1080](https://github.com/doe1080)
- **tele5**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/772559e3db2eb82e5d862d6d779588ca4b0b048d) ([#16005](https://github.com/yt-dlp/yt-dlp/issues/16005)) by [bashonly](https://github.com/bashonly)
- **tver**: olympic: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/02ce3efbfe51d54cb0866953af423fc6d1f38933) ([#15885](https://github.com/yt-dlp/yt-dlp/issues/15885)) by [doe1080](https://github.com/doe1080)
- **tvo**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/a13f281012a21c85f76cf3e320fc3b00d480d6c6) ([#15903](https://github.com/yt-dlp/yt-dlp/issues/15903)) by [doe1080](https://github.com/doe1080)
- **twitter**: [Fix error handling](https://github.com/yt-dlp/yt-dlp/commit/0d8898c3f4e76742afb2b877f817fdee89fa1258) ([#15993](https://github.com/yt-dlp/yt-dlp/issues/15993)) by [bashonly](https://github.com/bashonly) (With fixes in [7722109](https://github.com/yt-dlp/yt-dlp/commit/77221098fc5016f12118421982f02b662021972c))
- **visir**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/c7c45f52890eee40565188aee874ff4e58e95c4f) ([#15811](https://github.com/yt-dlp/yt-dlp/issues/15811)) by [doe1080](https://github.com/doe1080)
- **vk**: [Solve JS challenges using native JS interpreter](https://github.com/yt-dlp/yt-dlp/commit/acfc00a955208ee780b4cb18ae26de7b62444153) ([#15992](https://github.com/yt-dlp/yt-dlp/issues/15992)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly)
- **xhamster**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/133cb959be4d268e2cd6b3f1d9bf87fba4c3743e) ([#15831](https://github.com/yt-dlp/yt-dlp/issues/15831)) by [0xvd](https://github.com/0xvd)
- **youtube**
- [Add more known player JS variants](https://github.com/yt-dlp/yt-dlp/commit/2204cee6d8301e491d8455a2c54fd0e1b23468f5) ([#15975](https://github.com/yt-dlp/yt-dlp/issues/15975)) by [bashonly](https://github.com/bashonly)
- [Extract live adaptive `incomplete` formats](https://github.com/yt-dlp/yt-dlp/commit/319a2bda83f5e54054661c56c1391533f82473c2) ([#15937](https://github.com/yt-dlp/yt-dlp/issues/15937)) by [bashonly](https://github.com/bashonly), [CanOfSocks](https://github.com/CanOfSocks)
- [Update ejs to 0.5.0](https://github.com/yt-dlp/yt-dlp/commit/c105461647315f7f479091194944713b392ca729) ([#16031](https://github.com/yt-dlp/yt-dlp/issues/16031)) by [bashonly](https://github.com/bashonly)
- date, search: [Remove broken `ytsearchdate` support](https://github.com/yt-dlp/yt-dlp/commit/c7945800e4ccd8cad2d5ee7806a872963c0c6d44) ([#15959](https://github.com/yt-dlp/yt-dlp/issues/15959)) by [stastix](https://github.com/stastix)
#### Networking changes
- **Request Handler**: curl_cffi: [Deprioritize unreliable impersonate targets](https://github.com/yt-dlp/yt-dlp/commit/e74076141dc86d5603680ea641d7cec86a821ac8) ([#16018](https://github.com/yt-dlp/yt-dlp/issues/16018)) by [bashonly](https://github.com/bashonly)
#### Misc. changes
- **cleanup**
- [Bump ruff to 0.15.x](https://github.com/yt-dlp/yt-dlp/commit/abade83f8ddb63a11746b69038ebcd9c1405a00a) ([#15951](https://github.com/yt-dlp/yt-dlp/issues/15951)) by [Grub4K](https://github.com/Grub4K)
- Miscellaneous: [646bb31](https://github.com/yt-dlp/yt-dlp/commit/646bb31f39614e6c2f7ba687c53e7496394cbadb) by [Grub4K](https://github.com/Grub4K)
### 2026.02.04
#### Extractor changes
- **unsupported**: [Update unsupported URLs](https://github.com/yt-dlp/yt-dlp/commit/c677d866d41eb4075b0a5e0c944a6543fc13f15d) ([#15812](https://github.com/yt-dlp/yt-dlp/issues/15812)) by [doe1080](https://github.com/doe1080)
- **youtube**: [Default to `tv` player JS variant](https://github.com/yt-dlp/yt-dlp/commit/1a895c18aaaf00f557aa8cbacb21faa638842431) ([#15818](https://github.com/yt-dlp/yt-dlp/issues/15818)) by [bashonly](https://github.com/bashonly)
### 2026.01.31
#### Extractor changes
- **soop**: [Support subscription-only VODs](https://github.com/yt-dlp/yt-dlp/commit/d0bf3d0fc3455d411ae44c0a5dc974dd1481e3aa) ([#15523](https://github.com/yt-dlp/yt-dlp/issues/15523)) by [thematuu](https://github.com/thematuu)
- **unsupported**: [Update unsupported URLs](https://github.com/yt-dlp/yt-dlp/commit/bf5d8c2a663ac690711262aebc733c1b06a54b26) ([#15410](https://github.com/yt-dlp/yt-dlp/issues/15410)) by [bashonly](https://github.com/bashonly)
- **whyp**: [Extract more metadata](https://github.com/yt-dlp/yt-dlp/commit/0d8ee637e83d62edaf22aa85833a51c70d560389) ([#15757](https://github.com/yt-dlp/yt-dlp/issues/15757)) by [azdlonky](https://github.com/azdlonky)
- **youtube**
- [Add `web_embedded` fallback for `android_vr` client](https://github.com/yt-dlp/yt-dlp/commit/bb1c05752c288a81e0e281f1caf5395411936376) ([#15785](https://github.com/yt-dlp/yt-dlp/issues/15785)) by [bashonly](https://github.com/bashonly)
- [Remove broken `ios_downgraded` player client](https://github.com/yt-dlp/yt-dlp/commit/c3674575faa23b20e97be8b73f68b9f7b4cea9ab) ([#15786](https://github.com/yt-dlp/yt-dlp/issues/15786)) by [bashonly](https://github.com/bashonly)
- [Remove broken `tv_embedded` player client](https://github.com/yt-dlp/yt-dlp/commit/8eb794366eb69e7377ff88eed7929c00195c8d74) ([#15787](https://github.com/yt-dlp/yt-dlp/issues/15787)) by [bashonly](https://github.com/bashonly)
#### Misc. changes
- **cleanup**: Miscellaneous: [9a9a6b6](https://github.com/yt-dlp/yt-dlp/commit/9a9a6b6fe44a30458c1754ef064f354f04a84004) by [bashonly](https://github.com/bashonly)
### 2026.01.29
#### Core changes
- [Accept float values for `--sleep-subtitles`](https://github.com/yt-dlp/yt-dlp/commit/f6dc7d5279bcb7f29839c700d54ac148b332d208) ([#15282](https://github.com/yt-dlp/yt-dlp/issues/15282)) by [0xvd](https://github.com/0xvd)
- [Add `--compat-options 2025`](https://github.com/yt-dlp/yt-dlp/commit/5382c6c81bb22a382e46adb646e1379ccfc462b6) ([#15499](https://github.com/yt-dlp/yt-dlp/issues/15499)) by [bashonly](https://github.com/bashonly)
- [Add `--format-sort-reset` option](https://github.com/yt-dlp/yt-dlp/commit/b16b06378a0805430699131ca6b786f971ae05b5) ([#13809](https://github.com/yt-dlp/yt-dlp/issues/13809)) by [nihil-admirari](https://github.com/nihil-admirari)
- [Bypass interactive format selection if no formats are found](https://github.com/yt-dlp/yt-dlp/commit/e0bb4777328a7d1eb96f2d0256fa33ae06b5930d) ([#15278](https://github.com/yt-dlp/yt-dlp/issues/15278)) by [bashonly](https://github.com/bashonly)
- [Fix `--parse-metadata` when `TO` is a single field name](https://github.com/yt-dlp/yt-dlp/commit/cec1f1df792fe521fff2d5ca54b5c70094b3d96a) ([#14577](https://github.com/yt-dlp/yt-dlp/issues/14577)) by [bashonly](https://github.com/bashonly), [clayote](https://github.com/clayote)
- [Fix concurrent formats downloading to stdout](https://github.com/yt-dlp/yt-dlp/commit/5bf91072bcfbb26e6618d668a0b3379a3a862f8c) ([#15617](https://github.com/yt-dlp/yt-dlp/issues/15617)) by [grqz](https://github.com/grqz)
- [Fix interactive format/video selection when downloading to stdout](https://github.com/yt-dlp/yt-dlp/commit/1829a53a543e63bf0391da572cefcd2526c0a806) ([#15626](https://github.com/yt-dlp/yt-dlp/issues/15626)) by [grqz](https://github.com/grqz)
- [Support Deno installed via Python package](https://github.com/yt-dlp/yt-dlp/commit/dde5eab3b3a356449b5c8c09506553b1c2842953) ([#15614](https://github.com/yt-dlp/yt-dlp/issues/15614)) by [bashonly](https://github.com/bashonly), [zahlman](https://github.com/zahlman)
- **utils**
- `decode_packed_codes`: [Fix missing key handling](https://github.com/yt-dlp/yt-dlp/commit/f24b9ac0c94aff3311ab0b935ce8103b5a3faeb1) ([#15440](https://github.com/yt-dlp/yt-dlp/issues/15440)) by [cesbar](https://github.com/cesbar)
- `devalue`: [Fix calling reviver on cached value](https://github.com/yt-dlp/yt-dlp/commit/ede54330fb38866936c63ebb96c490a2d4b1b58c) ([#15568](https://github.com/yt-dlp/yt-dlp/issues/15568)) by [Grub4K](https://github.com/Grub4K)
- `js_to_json`: [Prevent false positives for octals](https://github.com/yt-dlp/yt-dlp/commit/4d4c7e1c6930861f8388ce3cdd7a5335bf860e7d) ([#15474](https://github.com/yt-dlp/yt-dlp/issues/15474)) by [doe1080](https://github.com/doe1080)
- `mimetype2ext`: [Recognize more srt types](https://github.com/yt-dlp/yt-dlp/commit/c0a7c594a9e67ac2ee4cde38fa4842a0b2d675e8) ([#15411](https://github.com/yt-dlp/yt-dlp/issues/15411)) by [seproDev](https://github.com/seproDev)
- `random_user_agent`: [Bump versions](https://github.com/yt-dlp/yt-dlp/commit/9bf040dc6f348bf22abc71233446a0a5017e613c) ([#15396](https://github.com/yt-dlp/yt-dlp/issues/15396)) by [seproDev](https://github.com/seproDev)
- `unified_timestamp`: [Add `tz_offset` parameter](https://github.com/yt-dlp/yt-dlp/commit/15263d049cb3f47e921b414782490052feca3def) ([#15357](https://github.com/yt-dlp/yt-dlp/issues/15357)) by [doe1080](https://github.com/doe1080)
#### Extractor changes
- [Fix prioritization of Youtube URL matching](https://github.com/yt-dlp/yt-dlp/commit/e2ea6bd6ab639f910b99e55add18856974ff4c3a) ([#15596](https://github.com/yt-dlp/yt-dlp/issues/15596)) by [Grub4K](https://github.com/Grub4K)
- **archive.org**: [Fix metadata extraction](https://github.com/yt-dlp/yt-dlp/commit/5f37f67d37b54bf9bd6fe7fa3083492d42f7a20a) ([#15286](https://github.com/yt-dlp/yt-dlp/issues/15286)) by [bashonly](https://github.com/bashonly)
- **bandcamp**: weekly: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/e9d4b22b9b09a30f31b557df740b01b09a8aefe8) ([#15208](https://github.com/yt-dlp/yt-dlp/issues/15208)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly)
- **bigo**: [Support `--wait-for-video`](https://github.com/yt-dlp/yt-dlp/commit/5026548d65276732ec290751d97994e23bdecc20) ([#15463](https://github.com/yt-dlp/yt-dlp/issues/15463)) by [olipfei](https://github.com/olipfei)
- **boosty**: [Improve metadata extraction](https://github.com/yt-dlp/yt-dlp/commit/f9a06197f563a2ccadce2603e91ceec523e88d91) ([#15543](https://github.com/yt-dlp/yt-dlp/issues/15543)) by [Sytm](https://github.com/Sytm)
- **cbc**: [Fix extractors](https://github.com/yt-dlp/yt-dlp/commit/457dd036af907aa8b1b544b95311847abe470bf1) ([#15631](https://github.com/yt-dlp/yt-dlp/issues/15631)) by [subrat-lima](https://github.com/subrat-lima)
- **cda**: [Support mobile URLs](https://github.com/yt-dlp/yt-dlp/commit/6d92f87ddc40a31959097622ff01d4a7ca833a13) ([#15398](https://github.com/yt-dlp/yt-dlp/issues/15398)) by [seproDev](https://github.com/seproDev)
- **croatian.film**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/ba499ab0dcf2486d97f739e155264b305e0abd26) ([#15468](https://github.com/yt-dlp/yt-dlp/issues/15468)) by [0xvd](https://github.com/0xvd)
- **dailymotion**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/2b61a2a4b20b499d6497c9212207f72a52b922a6) ([#15682](https://github.com/yt-dlp/yt-dlp/issues/15682)) by [bashonly](https://github.com/bashonly) (With fixes in [a893774](https://github.com/yt-dlp/yt-dlp/commit/a8937740969b60df1c2a634e58ab959352c9504c))
- **dropbox**: [Support videos in folders](https://github.com/yt-dlp/yt-dlp/commit/8a4b626daf59d0ecb6117ed275cb43dd68768b85) ([#15313](https://github.com/yt-dlp/yt-dlp/issues/15313)) by [0xvd](https://github.com/0xvd)
- **errarhiiv**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/1c739bf53e673e06d2a43feddb5a31ee8496fa6e) ([#15667](https://github.com/yt-dlp/yt-dlp/issues/15667)) by [rdamas](https://github.com/rdamas)
- **facebook**
- [Remove broken login support](https://github.com/yt-dlp/yt-dlp/commit/2a7e048a60b76a245deeea734885bdce5e6571ae) ([#15434](https://github.com/yt-dlp/yt-dlp/issues/15434)) by [bashonly](https://github.com/bashonly)
- ads: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/f8b3fe33f68495ade453602a201b33e3aa69ed1f) ([#15582](https://github.com/yt-dlp/yt-dlp/issues/15582)) by [legraphista](https://github.com/legraphista)
- **filmarchiv**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/c0c9cac55446f7bf48370ba60c06f9cf5bc48d15) ([#13490](https://github.com/yt-dlp/yt-dlp/issues/13490)) by [4elta](https://github.com/4elta)
- **franceinfo**
- [Fix extraction](https://github.com/yt-dlp/yt-dlp/commit/e08fdaaec2b253abb1e08899d1d13ec5072d76f2) ([#15704](https://github.com/yt-dlp/yt-dlp/issues/15704)) by [bashonly](https://github.com/bashonly)
- [Support new domain URLs](https://github.com/yt-dlp/yt-dlp/commit/ac3a566434c68cbf960dfb357c6c8a275e8bf8eb) ([#15669](https://github.com/yt-dlp/yt-dlp/issues/15669)) by [romainreignier](https://github.com/romainreignier)
- **generic**: [Improve detection of blockage due to TLS fingerprint](https://github.com/yt-dlp/yt-dlp/commit/cea825e7e0a1a93a1a355a86bbb2b9e77594f569) ([#15426](https://github.com/yt-dlp/yt-dlp/issues/15426)) by [bashonly](https://github.com/bashonly)
- **gofile**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/c5e55e04795636a2855a1be80cea0f6b2d0f0cc6) ([#15296](https://github.com/yt-dlp/yt-dlp/issues/15296)) by [quietvoid](https://github.com/quietvoid)
- **hotstar**: [Extract from new API](https://github.com/yt-dlp/yt-dlp/commit/5a481d65fa99862110bb84d10a2f15f0cb47cab3) ([#15480](https://github.com/yt-dlp/yt-dlp/issues/15480)) by [0xvd](https://github.com/0xvd)
- **iqiyi**: [Remove broken login support](https://github.com/yt-dlp/yt-dlp/commit/09078190b0f33d14ae2b402913c64b724acf4bcb) ([#15441](https://github.com/yt-dlp/yt-dlp/issues/15441)) by [seproDev](https://github.com/seproDev)
- **lbry**: [Support filtering of flat playlist results](https://github.com/yt-dlp/yt-dlp/commit/0e4d1e9de6250a80453d46f94b9fade5f10197a0) ([#15695](https://github.com/yt-dlp/yt-dlp/issues/15695)) by [christoph-heinrich](https://github.com/christoph-heinrich), [dirkf](https://github.com/dirkf)
- **manoto**: [Remove extractor](https://github.com/yt-dlp/yt-dlp/commit/6b23305822d406eff8e813244d95f328c22e821e) ([#15414](https://github.com/yt-dlp/yt-dlp/issues/15414)) by [seproDev](https://github.com/seproDev)
- **media.ccc.de**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/c8680b65f79cfeb23b342b70ffe1e233902f7933) ([#15608](https://github.com/yt-dlp/yt-dlp/issues/15608)) by [rdamas](https://github.com/rdamas)
- **nebula**
- season
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/f5270705e816a24caef7357a7ce8e17471899d73) ([#15347](https://github.com/yt-dlp/yt-dlp/issues/15347)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly)
- [Support more URLs](https://github.com/yt-dlp/yt-dlp/commit/6c918c5071dec8290686a4d030a1f74da3d9debf) ([#15436](https://github.com/yt-dlp/yt-dlp/issues/15436)) by [prettysunflower](https://github.com/prettysunflower)
- **netease**: program: [Support DJ URLs](https://github.com/yt-dlp/yt-dlp/commit/0ea6cc6d82318e554ffa0b5eaf9da4f4379ccbe9) ([#15365](https://github.com/yt-dlp/yt-dlp/issues/15365)) by [0xvd](https://github.com/0xvd)
- **neteasemusic**: [Fix merged lyrics extraction](https://github.com/yt-dlp/yt-dlp/commit/a421eb06d111cfa75e42569dc42331e9f3d8f27b) ([#15052](https://github.com/yt-dlp/yt-dlp/issues/15052)) by [Mivik](https://github.com/Mivik)
- **netzkino**: [Rework extractor](https://github.com/yt-dlp/yt-dlp/commit/a27ec9efc63da1cfb2a390eb028549585dbb2f41) ([#15351](https://github.com/yt-dlp/yt-dlp/issues/15351)) by [doe1080](https://github.com/doe1080)
- **nextmedia**: [Remove extractors](https://github.com/yt-dlp/yt-dlp/commit/6d4984e64e893dd954e781046a3532eb7abbfa16) ([#15354](https://github.com/yt-dlp/yt-dlp/issues/15354)) by [doe1080](https://github.com/doe1080)
- **pandatv**: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/878a41e283878ee34b052a395b1f9499f2b9ef81) ([#13210](https://github.com/yt-dlp/yt-dlp/issues/13210)) by [ptlydpr](https://github.com/ptlydpr)
- **parti**: [Fix extractors](https://github.com/yt-dlp/yt-dlp/commit/04f2ec4b976271e1e7ad3e650a0be2c4fd796ee0) ([#15319](https://github.com/yt-dlp/yt-dlp/issues/15319)) by [seproDev](https://github.com/seproDev)
- **patreon**: [Extract inlined media](https://github.com/yt-dlp/yt-dlp/commit/14998eef63a1462961a666d71318f804aca12220) ([#15498](https://github.com/yt-dlp/yt-dlp/issues/15498)) by [bashonly](https://github.com/bashonly)
- **pbs**: [Fix extraction](https://github.com/yt-dlp/yt-dlp/commit/a81087160812ec7a2059e1641a9785bfa4629023) ([#15083](https://github.com/yt-dlp/yt-dlp/issues/15083)) by [nlurker](https://github.com/nlurker)
- **picarto**: [Fix extraction when stream has no title](https://github.com/yt-dlp/yt-dlp/commit/fcd47d2db3f871c7b7d638773c36cc503119742d) ([#15407](https://github.com/yt-dlp/yt-dlp/issues/15407)) by [mikf](https://github.com/mikf)
- **pornhub**: [Optimize metadata extraction](https://github.com/yt-dlp/yt-dlp/commit/f2ee2a46fc2a4efb6ed58ee9e67c506c6b72b843) ([#15231](https://github.com/yt-dlp/yt-dlp/issues/15231)) by [norepro](https://github.com/norepro)
- **rumblechannel**: [Support filtering of flat playlist results](https://github.com/yt-dlp/yt-dlp/commit/0dec80c02a0c9edcc52d33d3ac83435dd8bcaa08) ([#15694](https://github.com/yt-dlp/yt-dlp/issues/15694)) by [christoph-heinrich](https://github.com/christoph-heinrich)
- **scte**: [Remove extractors](https://github.com/yt-dlp/yt-dlp/commit/4a772e5289b939013202ad7707d5b989794ed287) ([#15442](https://github.com/yt-dlp/yt-dlp/issues/15442)) by [seproDev](https://github.com/seproDev)
- **tarangplus**: [Add extractors](https://github.com/yt-dlp/yt-dlp/commit/260ba3abba2849aa175dd0bcfec308fc6ba6a678) ([#13060](https://github.com/yt-dlp/yt-dlp/issues/13060)) by [subrat-lima](https://github.com/subrat-lima) (With fixes in [27afb31](https://github.com/yt-dlp/yt-dlp/commit/27afb31edc492cb079f9bce9773498d08e568ff3) by [bashonly](https://github.com/bashonly))
- **telecinco**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/b6f24745bfb89ec0eaaa181a68203c2e81e58802) ([#15311](https://github.com/yt-dlp/yt-dlp/issues/15311)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly)
- **thechosen**: [Support new URL format](https://github.com/yt-dlp/yt-dlp/commit/1f4b26c39fb09782cf03615d089e712975395d6d) ([#15687](https://github.com/yt-dlp/yt-dlp/issues/15687)) by [AlexBocken](https://github.com/AlexBocken)
- **tiktok**
- [Extract `save_count`](https://github.com/yt-dlp/yt-dlp/commit/9c393e3f6220d34d534bef7d9d345782003b58ad) ([#15054](https://github.com/yt-dlp/yt-dlp/issues/15054)) by [pomtnp](https://github.com/pomtnp)
- [Solve JS challenges with native Python implementation](https://github.com/yt-dlp/yt-dlp/commit/e3f0d8b731b40176bcc632bf92cfe5149402b202) ([#15672](https://github.com/yt-dlp/yt-dlp/issues/15672)) by [bashonly](https://github.com/bashonly), [DTrombett](https://github.com/DTrombett)
- **tubitv**: [Support URLs with locales](https://github.com/yt-dlp/yt-dlp/commit/f0bc71abf68480b3b65b27c2a60319bc88e5eea2) ([#15205](https://github.com/yt-dlp/yt-dlp/issues/15205)) by [0xvd](https://github.com/0xvd)
- **tumblr**: [Extract timestamp](https://github.com/yt-dlp/yt-dlp/commit/87a265d820fbf9e3ce47c149609100fc8e9e13c5) ([#15462](https://github.com/yt-dlp/yt-dlp/issues/15462)) by [alch-emi](https://github.com/alch-emi)
- **tv5unis**: [Fix extractors](https://github.com/yt-dlp/yt-dlp/commit/6ae9e9568701b9c960e817c6dc35bcd824719a80) ([#15477](https://github.com/yt-dlp/yt-dlp/issues/15477)) by [0xced](https://github.com/0xced)
- **twitch**: videos: [Raise error when channel is not found](https://github.com/yt-dlp/yt-dlp/commit/e15ca65874b2a8bcd7435696b8f01252c39512ba) ([#15458](https://github.com/yt-dlp/yt-dlp/issues/15458)) by [0xvd](https://github.com/0xvd)
- **twitter**
- [Do not extract non-video posts from `unified_card`s](https://github.com/yt-dlp/yt-dlp/commit/ce9a3591f8292aeb93ffdad10028bfcddda3976b) ([#15431](https://github.com/yt-dlp/yt-dlp/issues/15431)) by [bashonly](https://github.com/bashonly)
- [Remove broken login support](https://github.com/yt-dlp/yt-dlp/commit/a6ba7140051dbe1d63a1da4de263bb9c886c0a32) ([#15432](https://github.com/yt-dlp/yt-dlp/issues/15432)) by [bashonly](https://github.com/bashonly)
- **vimeo**: [Add `macos` client](https://github.com/yt-dlp/yt-dlp/commit/ba5e2227c8c49fa76d9d30332aad2416774ddb31) ([#15746](https://github.com/yt-dlp/yt-dlp/issues/15746)) by [bashonly](https://github.com/bashonly), [gamer191](https://github.com/gamer191)
- **volejtv**: [Fix and add extractors](https://github.com/yt-dlp/yt-dlp/commit/1effa06dbf4dfd2e307b445a55a465d897205213) ([#13226](https://github.com/yt-dlp/yt-dlp/issues/13226)) by [subrat-lima](https://github.com/subrat-lima)
- **wat.tv**: [Improve DRM detection](https://github.com/yt-dlp/yt-dlp/commit/bc6ff877dd371d405b11f0ab16634c4d4b5d645e) ([#15659](https://github.com/yt-dlp/yt-dlp/issues/15659)) by [wesson09](https://github.com/wesson09)
- **whyp**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/f70ebf97ea7ef3b00c3e9213acf40d1b004c31d9) ([#15721](https://github.com/yt-dlp/yt-dlp/issues/15721)) by [bashonly](https://github.com/bashonly)
- **yahoo**: [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/97fb78a5b95a98a698f77281ea0c101bf090ed4c) ([#15314](https://github.com/yt-dlp/yt-dlp/issues/15314)) by [0xvd](https://github.com/0xvd), [bashonly](https://github.com/bashonly)
- **youtube**
- [Adjust default clients](https://github.com/yt-dlp/yt-dlp/commit/23b846506378a6a9c9a0958382d37f943f7cfa51) ([#15601](https://github.com/yt-dlp/yt-dlp/issues/15601)) by [bashonly](https://github.com/bashonly)
- [Fix `player_skip=js` extractor-arg](https://github.com/yt-dlp/yt-dlp/commit/abf29e3e72e8a4dcae61e2ceaf37ce8405af61ab) ([#15428](https://github.com/yt-dlp/yt-dlp/issues/15428)) by [bashonly](https://github.com/bashonly)
- [Fix default player clients](https://github.com/yt-dlp/yt-dlp/commit/309b03f2ad09fcfcf4ce81e757f8d3796bb56add) ([#15726](https://github.com/yt-dlp/yt-dlp/issues/15726)) by [bashonly](https://github.com/bashonly)
- [Solve n challenges for manifest formats](https://github.com/yt-dlp/yt-dlp/commit/d20f58d721fe45fe873e3389a0d17a72352aecec) ([#15602](https://github.com/yt-dlp/yt-dlp/issues/15602)) by [bashonly](https://github.com/bashonly)
- [Support comment subthreads](https://github.com/yt-dlp/yt-dlp/commit/d22436e5dc7c6808d931e27cbb967b1b2a33c17c) ([#15419](https://github.com/yt-dlp/yt-dlp/issues/15419)) by [bashonly](https://github.com/bashonly) (With fixes in [76c31a7](https://github.com/yt-dlp/yt-dlp/commit/76c31a7a216a3894884381c7775f838b811fde06), [468aa6a](https://github.com/yt-dlp/yt-dlp/commit/468aa6a9b431194949ede9eaad8f33e314a288d6))
- [Update ejs to 0.4.0](https://github.com/yt-dlp/yt-dlp/commit/88b35ff911a999e0b479417237010c305114ba08) ([#15747](https://github.com/yt-dlp/yt-dlp/issues/15747)) by [bashonly](https://github.com/bashonly)
- tab: [Fix flat thumbnails extraction for shorts](https://github.com/yt-dlp/yt-dlp/commit/ff61bef041d1f69fec1044f783fb938c005128af) ([#15331](https://github.com/yt-dlp/yt-dlp/issues/15331)) by [bashonly](https://github.com/bashonly)
- **zdf**: [Support sister sites URLs](https://github.com/yt-dlp/yt-dlp/commit/48b845a29623cbc814ad6c6b2ef285e3f3c0fe91) ([#15370](https://github.com/yt-dlp/yt-dlp/issues/15370)) by [InvalidUsernameException](https://github.com/InvalidUsernameException)
- **zoom**: [Extract recordings with start times](https://github.com/yt-dlp/yt-dlp/commit/0066de5b7e146a96e4cb4352f65dc3f1e283af4a) ([#15475](https://github.com/yt-dlp/yt-dlp/issues/15475)) by [JV-Fernandes](https://github.com/JV-Fernandes)
#### Networking changes
- **Request Handler**: curl_cffi: [Support `curl_cffi` 0.14.x](https://github.com/yt-dlp/yt-dlp/commit/9ab4777b97b5280ae1f53d1fe1b8ac542727238b) ([#15613](https://github.com/yt-dlp/yt-dlp/issues/15613)) by [bashonly](https://github.com/bashonly)
#### Misc. changes
- **build**
- [Bump official actions to latest versions](https://github.com/yt-dlp/yt-dlp/commit/825648a740867cbecd2e593963d7aaf3d568db84) ([#15305](https://github.com/yt-dlp/yt-dlp/issues/15305)) by [bashonly](https://github.com/bashonly)
- [Harden CI/CD pipeline](https://github.com/yt-dlp/yt-dlp/commit/ab3ff2d5dd220aa35805dadb6fae66ae9a0e2553) ([#15387](https://github.com/yt-dlp/yt-dlp/issues/15387)) by [bashonly](https://github.com/bashonly)
- [Improve nightly release check](https://github.com/yt-dlp/yt-dlp/commit/3763d0d4ab8bdbe433ce08e45e21f36ebdeb5db3) ([#15455](https://github.com/yt-dlp/yt-dlp/issues/15455)) by [bashonly](https://github.com/bashonly) (With fixes in [0b08b83](https://github.com/yt-dlp/yt-dlp/commit/0b08b833bfca6a0882f4741bb8fa46c1698c77e5))
- **ci**: [Explicitly declare permissions and limit credentials](https://github.com/yt-dlp/yt-dlp/commit/a6a8f6b6d6775caa031e5016b79db28c6aaadfcb) ([#15324](https://github.com/yt-dlp/yt-dlp/issues/15324)) by [bashonly](https://github.com/bashonly)
- **cleanup**
- Miscellaneous
- [a653494](https://github.com/yt-dlp/yt-dlp/commit/a65349443b959b8ab6bdec8e573777006d29b827) by [bashonly](https://github.com/bashonly), [Grub4K](https://github.com/Grub4K), [seproDev](https://github.com/seproDev)
- [8b27553](https://github.com/yt-dlp/yt-dlp/commit/8b275536d945c4b3d07b6c520677922c67a7c10f) by [bashonly](https://github.com/bashonly)
### 2025.12.08 ### 2025.12.08
#### Core changes #### Core changes

View File

@@ -41,14 +41,6 @@ Core Maintainers are responsible for reviewing and merging contributions, publis
* Improved/fixed/added Bundestag, crunchyroll, pr0gramm, Twitter, WrestleUniverse etc * Improved/fixed/added Bundestag, crunchyroll, pr0gramm, Twitter, WrestleUniverse etc
### [sepro](https://github.com/seproDev)
* UX improvements: Warn when ffmpeg is missing, warn when double-clicking exe
* Helped in implementing support for external JavaScript runtimes/engines
* Code cleanup: Remove dead extractors, mark extractors as broken, enable/apply ruff rules
* Improved/fixed/added ArdMediathek, DRTV, Floatplane, MagentaMusik, Naver, Nebula, OnDemandKorea, Vbox7 etc
## Inactive Core Maintainers ## Inactive Core Maintainers
### [pukkandan](https://github.com/pukkandan) ### [pukkandan](https://github.com/pukkandan)
@@ -77,6 +69,15 @@ Core Maintainers are responsible for reviewing and merging contributions, publis
* Added playlist/series downloads for Hotstar, ParamountPlus, Rumble, SonyLIV, Trovo, TubiTv, Voot etc * Added playlist/series downloads for Hotstar, ParamountPlus, Rumble, SonyLIV, Trovo, TubiTv, Voot etc
* Improved/fixed support for HiDive, HotStar, Hungama, LBRY, LinkedInLearning, Mxplayer, SonyLiv, TV2, Vimeo, VLive etc * Improved/fixed support for HiDive, HotStar, Hungama, LBRY, LinkedInLearning, Mxplayer, SonyLiv, TV2, Vimeo, VLive etc
### [sepro](https://github.com/seproDev)
* UX improvements: Warn when ffmpeg is missing, warn when double-clicking exe
* Helped in implementing support for external JavaScript runtimes/engines
* Code cleanup: Remove dead extractors, mark extractors as broken, enable/apply ruff rules
* Improved/fixed/added ArdMediathek, DRTV, Floatplane, MagentaMusik, Naver, Nebula, OnDemandKorea, Vbox7 etc
## Triage Maintainers ## Triage Maintainers
Triage Maintainers are frequent contributors who can manage issues and pull requests. Triage Maintainers are frequent contributors who can manage issues and pull requests.

View File

@@ -202,9 +202,9 @@ CONTRIBUTORS: Changelog.md
# The following EJS_-prefixed variables are auto-generated by devscripts/update_ejs.py # The following EJS_-prefixed variables are auto-generated by devscripts/update_ejs.py
# DO NOT EDIT! # DO NOT EDIT!
EJS_VERSION = 0.3.2 EJS_VERSION = 0.5.0
EJS_WHEEL_NAME = yt_dlp_ejs-0.3.2-py3-none-any.whl EJS_WHEEL_NAME = yt_dlp_ejs-0.5.0-py3-none-any.whl
EJS_WHEEL_HASH = sha256:f2dc6b3d1b909af1f13e021621b0af048056fca5fb07c4db6aa9bbb37a4f66a9 EJS_WHEEL_HASH = sha256:674fc0efea741d3100cdf3f0f9e123150715ee41edf47ea7a62fbdeda204bdec
EJS_PY_FOLDERS = yt_dlp_ejs yt_dlp_ejs/yt yt_dlp_ejs/yt/solver EJS_PY_FOLDERS = yt_dlp_ejs yt_dlp_ejs/yt yt_dlp_ejs/yt/solver
EJS_PY_FILES = yt_dlp_ejs/__init__.py yt_dlp_ejs/_version.py yt_dlp_ejs/yt/__init__.py yt_dlp_ejs/yt/solver/__init__.py EJS_PY_FILES = yt_dlp_ejs/__init__.py yt_dlp_ejs/_version.py yt_dlp_ejs/yt/__init__.py yt_dlp_ejs/yt/solver/__init__.py
EJS_JS_FOLDERS = yt_dlp_ejs/yt/solver EJS_JS_FOLDERS = yt_dlp_ejs/yt/solver

View File

@@ -213,7 +213,7 @@ While all the other dependencies are optional, `ffmpeg`, `ffprobe`, `yt-dlp-ejs`
**Important**: What you need is ffmpeg *binary*, **NOT** [the Python package of the same name](https://pypi.org/project/ffmpeg) **Important**: What you need is ffmpeg *binary*, **NOT** [the Python package of the same name](https://pypi.org/project/ffmpeg)
* [**yt-dlp-ejs**](https://github.com/yt-dlp/ejs) - Required for deciphering YouTube n/sig values. Licensed under [Unlicense](https://github.com/yt-dlp/ejs/blob/main/LICENSE), bundles [MIT](https://github.com/davidbonnet/astring/blob/main/LICENSE) and [ISC](https://github.com/meriyah/meriyah/blob/main/LICENSE.md) components. * [**yt-dlp-ejs**](https://github.com/yt-dlp/ejs) - Required for full YouTube support. Licensed under [Unlicense](https://github.com/yt-dlp/ejs/blob/main/LICENSE), bundles [MIT](https://github.com/davidbonnet/astring/blob/main/LICENSE) and [ISC](https://github.com/meriyah/meriyah/blob/main/LICENSE.md) components.
A JavaScript runtime/engine like [**deno**](https://deno.land) (recommended), [**node.js**](https://nodejs.org), [**bun**](https://bun.sh), or [**QuickJS**](https://bellard.org/quickjs/) is also required to run yt-dlp-ejs. See [the wiki](https://github.com/yt-dlp/yt-dlp/wiki/EJS). A JavaScript runtime/engine like [**deno**](https://deno.land) (recommended), [**node.js**](https://nodejs.org), [**bun**](https://bun.sh), or [**QuickJS**](https://bellard.org/quickjs/) is also required to run yt-dlp-ejs. See [the wiki](https://github.com/yt-dlp/yt-dlp/wiki/EJS).
@@ -406,7 +406,7 @@ Tip: Use `CTRL`+`F` (or `Command`+`F`) to search by keywords
(default) (default)
--live-from-start Download livestreams from the start. --live-from-start Download livestreams from the start.
Currently experimental and only supported Currently experimental and only supported
for YouTube and Twitch for YouTube, Twitch, and TVer
--no-live-from-start Download livestreams from the current time --no-live-from-start Download livestreams from the current time
(default) (default)
--wait-for-video MIN[-MAX] Wait for scheduled streams to become --wait-for-video MIN[-MAX] Wait for scheduled streams to become
@@ -858,6 +858,8 @@ Tip: Use `CTRL`+`F` (or `Command`+`F`) to search by keywords
for more details for more details
-S, --format-sort SORTORDER Sort the formats by the fields given, see -S, --format-sort SORTORDER Sort the formats by the fields given, see
"Sorting Formats" for more details "Sorting Formats" for more details
--format-sort-reset Disregard previous user specified sort order
and reset to the default
--format-sort-force Force user specified sort order to have --format-sort-force Force user specified sort order to have
precedence over all fields, see "Sorting precedence over all fields, see "Sorting
Formats" for more details (Alias: --S-force) Formats" for more details (Alias: --S-force)
@@ -1351,6 +1353,7 @@ The available fields are:
- `repost_count` (numeric): Number of reposts of the video - `repost_count` (numeric): Number of reposts of the video
- `average_rating` (numeric): Average rating given by users, the scale used depends on the webpage - `average_rating` (numeric): Average rating given by users, the scale used depends on the webpage
- `comment_count` (numeric): Number of comments on the video (For some extractors, comments are only downloaded at the end, and so this field cannot be used) - `comment_count` (numeric): Number of comments on the video (For some extractors, comments are only downloaded at the end, and so this field cannot be used)
- `save_count` (numeric): Number of times the video has been saved or bookmarked
- `age_limit` (numeric): Age restriction for the video (years) - `age_limit` (numeric): Age restriction for the video (years)
- `live_status` (string): One of "not_live", "is_live", "is_upcoming", "was_live", "post_live" (was live, but VOD is not yet processed) - `live_status` (string): One of "not_live", "is_live", "is_upcoming", "was_live", "post_live" (was live, but VOD is not yet processed)
- `is_live` (boolean): Whether this video is a live stream or a fixed-length video - `is_live` (boolean): Whether this video is a live stream or a fixed-length video
@@ -1644,6 +1647,8 @@ Note that the default for hdr is `hdr:12`; i.e. Dolby Vision is not preferred. T
If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all respects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`. If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all respects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`.
If you use the `-S`/`--format-sort` option multiple times, each subsequent sorting argument will be prepended to the previous one, and only the highest priority entry of any duplicated field will be preserved. E.g. `-S proto -S res` is equivalent to `-S res,proto`, and `-S res:720,fps -S vcodec,res:1080` is equivalent to `-S vcodec,res:1080,fps`. You can use `--format-sort-reset` to disregard any previously passed `-S`/`--format-sort` arguments and reset to the default order.
**Tip**: You can use the `-v -F` to see how the formats have been sorted (worst to best). **Tip**: You can use the `-v -F` to see how the formats have been sorted (worst to best).
## Format Selection examples ## Format Selection examples
@@ -1820,6 +1825,9 @@ $ yt-dlp --parse-metadata "title:%(artist)s - %(title)s"
# Regex example # Regex example
$ yt-dlp --parse-metadata "description:Artist - (?P<artist>.+)" $ yt-dlp --parse-metadata "description:Artist - (?P<artist>.+)"
# Copy the episode field to the title field (with FROM and TO as single fields)
$ yt-dlp --parse-metadata "episode:title"
# Set title as "Series name S01E05" # Set title as "Series name S01E05"
$ yt-dlp --parse-metadata "%(series)s S%(season_number)02dE%(episode_number)02d:%(title)s" $ yt-dlp --parse-metadata "%(series)s S%(season_number)02dE%(episode_number)02d:%(title)s"
@@ -1852,16 +1860,17 @@ The following extractors use this feature:
#### youtube #### youtube
* `lang`: Prefer translated metadata (`title`, `description` etc) of this language code (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube/_base.py](https://github.com/yt-dlp/yt-dlp/blob/415b4c9f955b1a0391204bd24a7132590e7b3bdb/yt_dlp/extractor/youtube/_base.py#L402-L409) for the list of supported content language codes * `lang`: Prefer translated metadata (`title`, `description` etc) of this language code (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube/_base.py](https://github.com/yt-dlp/yt-dlp/blob/415b4c9f955b1a0391204bd24a7132590e7b3bdb/yt_dlp/extractor/youtube/_base.py#L402-L409) for the list of supported content language codes
* `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively * `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively
* `player_client`: Clients to extract video data from. The currently available clients are `web`, `web_safari`, `web_embedded`, `web_music`, `web_creator`, `mweb`, `ios`, `android`, `android_sdkless`, `android_vr`, `tv`, `tv_simply`, `tv_downgraded`, and `tv_embedded`. By default, `tv,android_sdkless,web` is used. If no JavaScript runtime/engine is available, then `android_sdkless,web_safari,web` is used. If logged-in cookies are passed to yt-dlp, then `tv_downgraded,web_safari,web` is used for free accounts and `tv_downgraded,web_creator,web` is used for premium accounts. The `web_music` client is added for `music.youtube.com` URLs when logged-in cookies are used. The `web_embedded` client is added for age-restricted videos but only works if the video is embeddable. The `tv_embedded` and `web_creator` clients are added for age-restricted videos if account age-verification is required. Some clients, such as `web` and `web_music`, require a `po_token` for their formats to be downloadable. Some clients, such as `web_creator`, will only work with authentication. Not all clients support authentication via cookies. You can use `default` for the default clients, or you can use `all` for all clients (not recommended). You can prefix a client with `-` to exclude it, e.g. `youtube:player_client=default,-ios` * `player_client`: Clients to extract video data from. The currently available clients are `web`, `web_safari`, `web_embedded`, `web_music`, `web_creator`, `mweb`, `ios`, `android`, `android_vr`, `tv`, `tv_downgraded`, and `tv_simply`. By default, `android_vr,web,web_safari` is used. If no JavaScript runtime/engine is available, then only `android_vr` is used. If logged-in cookies are passed to yt-dlp, then `tv_downgraded,web,web_safari` is used for free accounts and `tv_downgraded,web_creator,web` is used for premium accounts. The `web_music` client is added for `music.youtube.com` URLs when logged-in cookies are used. The `web_embedded` client is added for age-restricted videos but only successfully works around the age-restriction sometimes (e.g. if the video is embeddable), and may be added as a fallback if `android_vr` is unable to access a video. The `web_creator` client is added for age-restricted videos if account age-verification is required. Some clients, such as `web_creator` and `web_music`, require a `po_token` for their formats to be downloadable. Some clients, such as `web_creator`, will only work with authentication. Not all clients support authentication via cookies. You can use `default` for the default clients, or you can use `all` for all clients (not recommended). You can prefix a client with `-` to exclude it, e.g. `youtube:player_client=default,-web`
* `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player), `initial_data` (skip initial data/next ep request). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause issues such as missing formats or metadata. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) and [#12826](https://github.com/yt-dlp/yt-dlp/issues/12826) for more details * `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player), `initial_data` (skip initial data/next ep request). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause issues such as missing formats or metadata. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) and [#12826](https://github.com/yt-dlp/yt-dlp/issues/12826) for more details
* `webpage_skip`: Skip extraction of embedded webpage data. One or both of `player_response`, `initial_data`. These options are for testing purposes and don't skip any network requests * `webpage_skip`: Skip extraction of embedded webpage data. One or both of `player_response`, `initial_data`. These options are for testing purposes and don't skip any network requests
* `player_params`: YouTube player parameters to use for player requests. Will overwrite any default ones set by yt-dlp. * `player_params`: YouTube player parameters to use for player requests. Will overwrite any default ones set by yt-dlp.
* `player_js_variant`: The player javascript variant to use for n/sig deciphering. The known variants are: `main`, `tcc`, `tce`, `es5`, `es6`, `tv`, `tv_es6`, `phone`, `tablet`. The default is `main`, and the others are for debugging purposes. You can use `actual` to go with what is prescribed by the site * `player_js_variant`: The player javascript variant to use for n/sig deciphering. The known variants are: `main`, `tcc`, `tce`, `es5`, `es6`, `es6_tcc`, `es6_tce`, `tv`, `tv_es6`, `phone`, `house`. The default is `tv`, and the others are for debugging purposes. You can use `actual` to go with what is prescribed by the site
* `player_js_version`: The player javascript version to use for n/sig deciphering, in the format of `signature_timestamp@hash` (e.g. `20348@0004de42`). The default is to use what is prescribed by the site, and can be selected with `actual` * `player_js_version`: The player javascript version to use for n/sig deciphering, in the format of `signature_timestamp@hash` (e.g. `20348@0004de42`). The default is to use what is prescribed by the site, and can be selected with `actual`
* `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side) * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side)
* `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread`. Default is `all,all,all,all` * `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread,max-depth`. Default is `all,all,all,all,all`
* E.g. `all,all,1000,10` will get a maximum of 1000 replies total, with up to 10 replies per thread. `1000,all,100` will get a maximum of 1000 comments, with a maximum of 100 replies total * A `max-depth` value of `1` will discard all replies, regardless of the `max-replies` or `max-replies-per-thread` values given
* `formats`: Change the types of formats to return. `dashy` (convert HTTP to DASH), `duplicate` (identical content but different URLs or protocol; includes `dashy`), `incomplete` (cannot be downloaded completely - live dash and post-live m3u8), `missing_pot` (include formats that require a PO Token but are missing one) * E.g. `all,all,1000,10,2` will get a maximum of 1000 replies total, with up to 10 replies per thread, and only 2 levels of depth (i.e. top-level comments plus their immediate replies). `1000,all,100` will get a maximum of 1000 comments, with a maximum of 100 replies total
* `formats`: Change the types of formats to return. `dashy` (convert HTTP to DASH), `duplicate` (identical content but different URLs or protocol; includes `dashy`), `incomplete` (cannot be downloaded completely - live dash, live adaptive https, and post-live m3u8), `missing_pot` (include formats that require a PO Token but are missing one)
* `innertube_host`: Innertube API host to use for all API requests; e.g. `studio.youtube.com`, `youtubei.googleapis.com`. Note that cookies exported from one subdomain will not work on others * `innertube_host`: Innertube API host to use for all API requests; e.g. `studio.youtube.com`, `youtubei.googleapis.com`. Note that cookies exported from one subdomain will not work on others
* `innertube_key`: Innertube API key to use for all API requests. By default, no API key is used * `innertube_key`: Innertube API key to use for all API requests. By default, no API key is used
* `raise_incomplete_data`: `Incomplete Data Received` raises an error instead of reporting a warning * `raise_incomplete_data`: `Incomplete Data Received` raises an error instead of reporting a warning
@@ -1963,7 +1972,7 @@ The following extractors use this feature:
* `backend`: Backend API to use for extraction - one of `streaks` (default) or `brightcove` (deprecated) * `backend`: Backend API to use for extraction - one of `streaks` (default) or `brightcove` (deprecated)
#### vimeo #### vimeo
* `client`: Client to extract video data from. The currently available clients are `android`, `ios`, and `web`. Only one client can be used. The `web` client is used by default. The `web` client only works with account cookies or login credentials. The `android` and `ios` clients only work with previously cached OAuth tokens * `client`: Client to extract video data from. The currently available clients are `android`, `ios`, `macos` and `web`. Only one client can be used. The `macos` client is used by default, but the `web` client is used when logged-in. The `web` client only works with account cookies or login credentials. The `android` and `ios` clients only work with previously cached OAuth tokens
* `original_format_policy`: Policy for when to try extracting original formats. One of `always`, `never`, or `auto`. The default `auto` policy tries to avoid exceeding the web client's API rate-limit by only making an extra request when Vimeo publicizes the video's downloadability * `original_format_policy`: Policy for when to try extracting original formats. One of `always`, `never`, or `auto`. The default `auto` policy tries to avoid exceeding the web client's API rate-limit by only making an extra request when Vimeo publicizes the video's downloadability
**Note**: These options may be changed/removed in the future without concern for backward compatibility **Note**: These options may be changed/removed in the future without concern for backward compatibility
@@ -2252,7 +2261,7 @@ with yt_dlp.YoutubeDL(ydl_opts) as ydl:
* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details. * **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
* **YouTube improvements**: * **YouTube improvements**:
* Supports Clips, Stories (`ytstories:<channel UCID>`), Search (including filters)**\***, YouTube Music Search, Channel-specific search, Search prefixes (`ytsearch:`, `ytsearchdate:`)**\***, Mixes, and Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`, `:ytnotif`) * Supports Clips, Stories (`ytstories:<channel UCID>`), Search (including filters)**\***, YouTube Music Search, Channel-specific search, Search prefix (`ytsearch:`)**\***, Mixes, and Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`, `:ytnotif`)
* Fix for [n-sig based throttling](https://github.com/ytdl-org/youtube-dl/issues/29326) **\*** * Fix for [n-sig based throttling](https://github.com/ytdl-org/youtube-dl/issues/29326) **\***
* Download livestreams from the start using `--live-from-start` (*experimental*) * Download livestreams from the start using `--live-from-start` (*experimental*)
* Channel URLs download all uploads of the channel, including shorts and live * Channel URLs download all uploads of the channel, including shorts and live
@@ -2329,7 +2338,7 @@ Some of yt-dlp's default options are different from that of youtube-dl and youtu
* Passing `--simulate` (or calling `extract_info` with `download=False`) no longer alters the default format selection. See [#9843](https://github.com/yt-dlp/yt-dlp/issues/9843) for details. * Passing `--simulate` (or calling `extract_info` with `download=False`) no longer alters the default format selection. See [#9843](https://github.com/yt-dlp/yt-dlp/issues/9843) for details.
* yt-dlp no longer applies the server modified time to downloaded files by default. Use `--mtime` or `--compat-options mtime-by-default` to revert this. * yt-dlp no longer applies the server modified time to downloaded files by default. Use `--mtime` or `--compat-options mtime-by-default` to revert this.
For ease of use, a few more compat options are available: For convenience, there are some compat option aliases available to use:
* `--compat-options all`: Use all compat options (**Do NOT use this!**) * `--compat-options all`: Use all compat options (**Do NOT use this!**)
* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams,-playlist-match-filter,-manifest-filesize-approx,-allow-unsafe-ext,-prefer-vp9-sort` * `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams,-playlist-match-filter,-manifest-filesize-approx,-allow-unsafe-ext,-prefer-vp9-sort`
@@ -2337,7 +2346,10 @@ For ease of use, a few more compat options are available:
* `--compat-options 2021`: Same as `--compat-options 2022,no-certifi,filename-sanitization` * `--compat-options 2021`: Same as `--compat-options 2022,no-certifi,filename-sanitization`
* `--compat-options 2022`: Same as `--compat-options 2023,playlist-match-filter,no-external-downloader-progress,prefer-legacy-http-handler,manifest-filesize-approx` * `--compat-options 2022`: Same as `--compat-options 2023,playlist-match-filter,no-external-downloader-progress,prefer-legacy-http-handler,manifest-filesize-approx`
* `--compat-options 2023`: Same as `--compat-options 2024,prefer-vp9-sort` * `--compat-options 2023`: Same as `--compat-options 2024,prefer-vp9-sort`
* `--compat-options 2024`: Same as `--compat-options mtime-by-default`. Use this to enable all future compat options * `--compat-options 2024`: Same as `--compat-options 2025,mtime-by-default`
* `--compat-options 2025`: Currently does nothing. Use this to enable all future compat options
Using one of the yearly compat option aliases will pin yt-dlp's default behavior to what it was at the *end* of that calendar year.
The following compat options restore vulnerable behavior from before security patches: The following compat options restore vulnerable behavior from before security patches:

View File

@@ -26,7 +26,7 @@ services:
platforms: platforms:
- "linux/amd64" - "linux/amd64"
args: args:
VERIFYIMAGE: quay.io/pypa/manylinux2014_x86_64:latest VERIFYIMAGE: quay.io/pypa/manylinux2014_x86_64:2025.12.19-1@sha256:b716645f9aecd0c1418283af930804bbdbd68a73d855a60101c5aab8548d737d
environment: environment:
EXE_NAME: ${EXE_NAME:?} EXE_NAME: ${EXE_NAME:?}
UPDATE_TO: UPDATE_TO:
@@ -61,7 +61,7 @@ services:
platforms: platforms:
- "linux/arm64" - "linux/arm64"
args: args:
VERIFYIMAGE: quay.io/pypa/manylinux2014_aarch64:latest VERIFYIMAGE: quay.io/pypa/manylinux2014_aarch64:2025.12.19-1@sha256:36cbe6638c7c605c2b44a92e35751baa537ec8902112f790139d89c7e1ccd2a4
environment: environment:
EXE_NAME: ${EXE_NAME:?} EXE_NAME: ${EXE_NAME:?}
UPDATE_TO: UPDATE_TO:
@@ -97,7 +97,7 @@ services:
platforms: platforms:
- "linux/arm/v7" - "linux/arm/v7"
args: args:
VERIFYIMAGE: arm32v7/debian:bullseye VERIFYIMAGE: arm32v7/debian:bullseye@sha256:9d544bf6ff73e36b8df1b7e415f6c8ee40ed84a0f3a26970cac8ea88b0ccf2ac
environment: environment:
EXE_NAME: ${EXE_NAME:?} EXE_NAME: ${EXE_NAME:?}
UPDATE_TO: UPDATE_TO:
@@ -132,7 +132,7 @@ services:
platforms: platforms:
- "linux/amd64" - "linux/amd64"
args: args:
VERIFYIMAGE: alpine:3.22 VERIFYIMAGE: alpine:3.23.2@sha256:865b95f46d98cf867a156fe4a135ad3fe50d2056aa3f25ed31662dff6da4eb62
environment: environment:
EXE_NAME: ${EXE_NAME:?} EXE_NAME: ${EXE_NAME:?}
UPDATE_TO: UPDATE_TO:
@@ -168,7 +168,7 @@ services:
platforms: platforms:
- "linux/arm64" - "linux/arm64"
args: args:
VERIFYIMAGE: alpine:3.22 VERIFYIMAGE: alpine:3.23.2@sha256:865b95f46d98cf867a156fe4a135ad3fe50d2056aa3f25ed31662dff6da4eb62
environment: environment:
EXE_NAME: ${EXE_NAME:?} EXE_NAME: ${EXE_NAME:?}
UPDATE_TO: UPDATE_TO:

View File

@@ -6,43 +6,35 @@ if [[ -z "${PYTHON_VERSION:-}" ]]; then
echo "Defaulting to using Python ${PYTHON_VERSION}" echo "Defaulting to using Python ${PYTHON_VERSION}"
fi fi
function runpy {
"/opt/shared-cpython-${PYTHON_VERSION}/bin/python${PYTHON_VERSION}" "$@"
}
function venvpy {
"python${PYTHON_VERSION}" "$@"
}
INCLUDES=( INCLUDES=(
--include-extra pyinstaller --include-extra pyinstaller
--include-extra secretstorage --include-extra secretstorage
) )
if [[ -z "${EXCLUDE_CURL_CFFI:-}" ]]; then if [[ -z "${EXCLUDE_CURL_CFFI:-}" ]]; then
INCLUDES+=(--include-extra curl-cffi) INCLUDES+=(--include-extra build-curl-cffi)
fi fi
runpy -m venv /yt-dlp-build-venv py"${PYTHON_VERSION}" -m venv /yt-dlp-build-venv
# shellcheck disable=SC1091 # shellcheck disable=SC1091
source /yt-dlp-build-venv/bin/activate source /yt-dlp-build-venv/bin/activate
# Inside the venv we use venvpy instead of runpy # Inside the venv we can use python instead of py3.13 or py3.14 etc
venvpy -m ensurepip --upgrade --default-pip python -m devscripts.install_deps "${INCLUDES[@]}"
venvpy -m devscripts.install_deps --omit-default --include-extra build python -m devscripts.make_lazy_extractors
venvpy -m devscripts.install_deps "${INCLUDES[@]}" python devscripts/update-version.py -c "${CHANNEL}" -r "${ORIGIN}" "${VERSION}"
venvpy -m devscripts.make_lazy_extractors
venvpy devscripts/update-version.py -c "${CHANNEL}" -r "${ORIGIN}" "${VERSION}"
if [[ -z "${SKIP_ONEDIR_BUILD:-}" ]]; then if [[ -z "${SKIP_ONEDIR_BUILD:-}" ]]; then
mkdir -p /build mkdir -p /build
venvpy -m bundle.pyinstaller --onedir --distpath=/build python -m bundle.pyinstaller --onedir --distpath=/build
pushd "/build/${EXE_NAME}" pushd "/build/${EXE_NAME}"
chmod +x "${EXE_NAME}" chmod +x "${EXE_NAME}"
venvpy -m zipfile -c "/yt-dlp/dist/${EXE_NAME}.zip" ./ python -m zipfile -c "/yt-dlp/dist/${EXE_NAME}.zip" ./
popd popd
fi fi
if [[ -z "${SKIP_ONEFILE_BUILD:-}" ]]; then if [[ -z "${SKIP_ONEFILE_BUILD:-}" ]]; then
venvpy -m bundle.pyinstaller python -m bundle.pyinstaller
chmod +x "./dist/${EXE_NAME}" chmod +x "./dist/${EXE_NAME}"
fi fi
deactivate

View File

@@ -325,5 +325,22 @@
"when": "c63b4e2a2b81cc78397c8709ef53ffd29bada213", "when": "c63b4e2a2b81cc78397c8709ef53ffd29bada213",
"short": "[cleanup] Misc (#14767)", "short": "[cleanup] Misc (#14767)",
"authors": ["bashonly", "seproDev", "matyb08"] "authors": ["bashonly", "seproDev", "matyb08"]
},
{
"action": "change",
"when": "abf29e3e72e8a4dcae61e2ceaf37ce8405af61ab",
"short": "[ie/youtube] Fix `player_skip=js` extractor-arg (#15428)",
"authors": ["bashonly"]
},
{
"action": "change",
"when": "e2ea6bd6ab639f910b99e55add18856974ff4c3a",
"short": "[ie] Fix prioritization of Youtube URL matching (#15596)",
"authors": ["Grub4K"]
},
{
"action": "add",
"when": "1fbbe29b99dc61375bf6d786f824d9fcf6ea9c1a",
"short": "[priority] Security: [[CVE-2026-26331](https://nvd.nist.gov/vuln/detail/CVE-2026-26331)] [Arbitrary command injection with the `--netrc-cmd` option](https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-g3gw-q23r-pgqm)\n - The argument passed to the command in `--netrc-cmd` is now limited to a safe subset of characters"
} }
] ]

View File

@@ -21,8 +21,6 @@ def setup_variables(environment):
SOURCE_PYPI_PROJECT, SOURCE_PYPI_SUFFIX, SOURCE_PYPI_PROJECT, SOURCE_PYPI_SUFFIX,
TARGET_PYPI_PROJECT, TARGET_PYPI_SUFFIX, TARGET_PYPI_PROJECT, TARGET_PYPI_SUFFIX,
SOURCE_ARCHIVE_REPO, TARGET_ARCHIVE_REPO, SOURCE_ARCHIVE_REPO, TARGET_ARCHIVE_REPO,
HAS_SOURCE_ARCHIVE_REPO_TOKEN,
HAS_TARGET_ARCHIVE_REPO_TOKEN,
HAS_ARCHIVE_REPO_TOKEN HAS_ARCHIVE_REPO_TOKEN
`INPUTS` must contain these keys: `INPUTS` must contain these keys:
@@ -37,8 +35,6 @@ def setup_variables(environment):
PROCESSED = json.loads(environment['PROCESSED']) PROCESSED = json.loads(environment['PROCESSED'])
source_channel = None source_channel = None
does_not_have_needed_token = False
target_repo_token = None
pypi_project = None pypi_project = None
pypi_suffix = None pypi_suffix = None
@@ -81,28 +77,19 @@ def setup_variables(environment):
target_repo = REPOSITORY target_repo = REPOSITORY
if target_repo != REPOSITORY: if target_repo != REPOSITORY:
target_repo = environment['TARGET_ARCHIVE_REPO'] target_repo = environment['TARGET_ARCHIVE_REPO']
target_repo_token = f'{PROCESSED["target_repo"].upper()}_ARCHIVE_REPO_TOKEN'
if not json.loads(environment['HAS_TARGET_ARCHIVE_REPO_TOKEN']):
does_not_have_needed_token = True
pypi_project = environment['TARGET_PYPI_PROJECT'] or None pypi_project = environment['TARGET_PYPI_PROJECT'] or None
pypi_suffix = environment['TARGET_PYPI_SUFFIX'] or None pypi_suffix = environment['TARGET_PYPI_SUFFIX'] or None
else: else:
target_tag = source_tag or version target_tag = source_tag or version
if source_channel: if source_channel:
target_repo = source_channel target_repo = source_channel
target_repo_token = f'{PROCESSED["source_repo"].upper()}_ARCHIVE_REPO_TOKEN'
if not json.loads(environment['HAS_SOURCE_ARCHIVE_REPO_TOKEN']):
does_not_have_needed_token = True
pypi_project = environment['SOURCE_PYPI_PROJECT'] or None pypi_project = environment['SOURCE_PYPI_PROJECT'] or None
pypi_suffix = environment['SOURCE_PYPI_SUFFIX'] or None pypi_suffix = environment['SOURCE_PYPI_SUFFIX'] or None
else: else:
target_repo = REPOSITORY target_repo = REPOSITORY
if does_not_have_needed_token: if target_repo != REPOSITORY and not json.loads(environment['HAS_ARCHIVE_REPO_TOKEN']):
if not json.loads(environment['HAS_ARCHIVE_REPO_TOKEN']): return None
print(f'::error::Repository access secret {target_repo_token} not found')
return None
target_repo_token = 'ARCHIVE_REPO_TOKEN'
if target_repo == REPOSITORY and not INPUTS['prerelease']: if target_repo == REPOSITORY and not INPUTS['prerelease']:
pypi_project = environment['PYPI_PROJECT'] or None pypi_project = environment['PYPI_PROJECT'] or None
@@ -111,7 +98,6 @@ def setup_variables(environment):
'channel': resolved_source, 'channel': resolved_source,
'version': version, 'version': version,
'target_repo': target_repo, 'target_repo': target_repo,
'target_repo_token': target_repo_token,
'target_tag': target_tag, 'target_tag': target_tag,
'pypi_project': pypi_project, 'pypi_project': pypi_project,
'pypi_suffix': pypi_suffix, 'pypi_suffix': pypi_suffix,
@@ -147,6 +133,7 @@ if __name__ == '__main__':
outputs = setup_variables(dict(os.environ)) outputs = setup_variables(dict(os.environ))
if not outputs: if not outputs:
print('::error::Repository access secret ARCHIVE_REPO_TOKEN not found')
sys.exit(1) sys.exit(1)
print('::group::Output variables') print('::group::Output variables')

View File

@@ -9,8 +9,10 @@ import json
from devscripts.setup_variables import STABLE_REPOSITORY, process_inputs, setup_variables from devscripts.setup_variables import STABLE_REPOSITORY, process_inputs, setup_variables
from devscripts.utils import calculate_version from devscripts.utils import calculate_version
GENERATE_TEST_DATA = object()
def _test(github_repository, note, repo_vars, repo_secrets, inputs, expected=None, ignore_revision=False):
def _test(github_repository, note, repo_vars, repo_secrets, inputs, expected, ignore_revision=False):
inp = inputs.copy() inp = inputs.copy()
inp.setdefault('linux_armv7l', True) inp.setdefault('linux_armv7l', True)
inp.setdefault('prerelease', False) inp.setdefault('prerelease', False)
@@ -33,16 +35,19 @@ def _test(github_repository, note, repo_vars, repo_secrets, inputs, expected=Non
'TARGET_PYPI_SUFFIX': variables.get(f'{target_repo}_PYPI_SUFFIX') or '', 'TARGET_PYPI_SUFFIX': variables.get(f'{target_repo}_PYPI_SUFFIX') or '',
'SOURCE_ARCHIVE_REPO': variables.get(f'{source_repo}_ARCHIVE_REPO') or '', 'SOURCE_ARCHIVE_REPO': variables.get(f'{source_repo}_ARCHIVE_REPO') or '',
'TARGET_ARCHIVE_REPO': variables.get(f'{target_repo}_ARCHIVE_REPO') or '', 'TARGET_ARCHIVE_REPO': variables.get(f'{target_repo}_ARCHIVE_REPO') or '',
'HAS_SOURCE_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get(f'{source_repo}_ARCHIVE_REPO_TOKEN'))),
'HAS_TARGET_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get(f'{target_repo}_ARCHIVE_REPO_TOKEN'))),
'HAS_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get('ARCHIVE_REPO_TOKEN'))), 'HAS_ARCHIVE_REPO_TOKEN': json.dumps(bool(secrets.get('ARCHIVE_REPO_TOKEN'))),
} }
result = setup_variables(env) result = setup_variables(env)
if not expected:
if expected is GENERATE_TEST_DATA:
print(' {\n' + '\n'.join(f' {k!r}: {v!r},' for k, v in result.items()) + '\n }') print(' {\n' + '\n'.join(f' {k!r}: {v!r},' for k, v in result.items()) + '\n }')
return return
if expected is None:
assert result is None, f'expected error/None but got dict: {github_repository} {note}'
return
exp = expected.copy() exp = expected.copy()
if ignore_revision: if ignore_revision:
assert len(result['version']) == len(exp['version']), f'revision missing: {github_repository} {note}' assert len(result['version']) == len(exp['version']), f'revision missing: {github_repository} {note}'
@@ -77,7 +82,6 @@ def test_setup_variables():
'channel': 'stable', 'channel': 'stable',
'version': DEFAULT_VERSION, 'version': DEFAULT_VERSION,
'target_repo': STABLE_REPOSITORY, 'target_repo': STABLE_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION, 'target_tag': DEFAULT_VERSION,
'pypi_project': 'yt-dlp', 'pypi_project': 'yt-dlp',
'pypi_suffix': None, 'pypi_suffix': None,
@@ -91,7 +95,6 @@ def test_setup_variables():
'channel': 'nightly', 'channel': 'nightly',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-nightly-builds', 'target_repo': 'yt-dlp/yt-dlp-nightly-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': 'yt-dlp', 'pypi_project': 'yt-dlp',
'pypi_suffix': 'dev', 'pypi_suffix': 'dev',
@@ -106,7 +109,6 @@ def test_setup_variables():
'channel': 'nightly', 'channel': 'nightly',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-nightly-builds', 'target_repo': 'yt-dlp/yt-dlp-nightly-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': 'yt-dlp', 'pypi_project': 'yt-dlp',
'pypi_suffix': 'dev', 'pypi_suffix': 'dev',
@@ -120,7 +122,6 @@ def test_setup_variables():
'channel': 'master', 'channel': 'master',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-master-builds', 'target_repo': 'yt-dlp/yt-dlp-master-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -135,7 +136,6 @@ def test_setup_variables():
'channel': 'master', 'channel': 'master',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': 'yt-dlp/yt-dlp-master-builds', 'target_repo': 'yt-dlp/yt-dlp-master-builds',
'target_repo_token': 'ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -149,7 +149,6 @@ def test_setup_variables():
'channel': 'stable', 'channel': 'stable',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': STABLE_REPOSITORY, 'target_repo': STABLE_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental', 'target_tag': 'experimental',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -163,7 +162,6 @@ def test_setup_variables():
'channel': 'stable', 'channel': 'stable',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': STABLE_REPOSITORY, 'target_repo': STABLE_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental', 'target_tag': 'experimental',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -175,7 +173,6 @@ def test_setup_variables():
'channel': FORK_REPOSITORY, 'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -186,7 +183,6 @@ def test_setup_variables():
'channel': FORK_REPOSITORY, 'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -201,7 +197,6 @@ def test_setup_variables():
'channel': f'{FORK_REPOSITORY}@nightly', 'channel': f'{FORK_REPOSITORY}@nightly',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'nightly', 'target_tag': 'nightly',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -216,7 +211,6 @@ def test_setup_variables():
'channel': f'{FORK_REPOSITORY}@master', 'channel': f'{FORK_REPOSITORY}@master',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'master', 'target_tag': 'master',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -227,7 +221,6 @@ def test_setup_variables():
'channel': FORK_REPOSITORY, 'channel': FORK_REPOSITORY,
'version': f'{DEFAULT_VERSION[:10]}.123', 'version': f'{DEFAULT_VERSION[:10]}.123',
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': f'{DEFAULT_VERSION[:10]}.123', 'target_tag': f'{DEFAULT_VERSION[:10]}.123',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -239,7 +232,6 @@ def test_setup_variables():
'channel': FORK_REPOSITORY, 'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION, 'version': DEFAULT_VERSION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION, 'target_tag': DEFAULT_VERSION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -250,19 +242,16 @@ def test_setup_variables():
'channel': FORK_REPOSITORY, 'channel': FORK_REPOSITORY,
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
}, ignore_revision=True) }, ignore_revision=True)
_test( _test(
FORK_REPOSITORY, 'fork w/NIGHTLY_ARCHIVE_REPO_TOKEN, nightly', { FORK_REPOSITORY, 'fork, nightly', {
'NIGHTLY_ARCHIVE_REPO': f'{FORK_ORG}/yt-dlp-nightly-builds', 'NIGHTLY_ARCHIVE_REPO': f'{FORK_ORG}/yt-dlp-nightly-builds',
'PYPI_PROJECT': 'yt-dlp-test', 'PYPI_PROJECT': 'yt-dlp-test',
}, { }, BASE_REPO_SECRETS, {
'NIGHTLY_ARCHIVE_REPO_TOKEN': '1',
}, {
'source': f'{FORK_ORG}/yt-dlp-nightly-builds', 'source': f'{FORK_ORG}/yt-dlp-nightly-builds',
'target': 'nightly', 'target': 'nightly',
'prerelease': True, 'prerelease': True,
@@ -270,19 +259,16 @@ def test_setup_variables():
'channel': f'{FORK_ORG}/yt-dlp-nightly-builds', 'channel': f'{FORK_ORG}/yt-dlp-nightly-builds',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': f'{FORK_ORG}/yt-dlp-nightly-builds', 'target_repo': f'{FORK_ORG}/yt-dlp-nightly-builds',
'target_repo_token': 'NIGHTLY_ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
}, ignore_revision=True) }, ignore_revision=True)
_test( _test(
FORK_REPOSITORY, 'fork w/MASTER_ARCHIVE_REPO_TOKEN, master', { FORK_REPOSITORY, 'fork, master', {
'MASTER_ARCHIVE_REPO': f'{FORK_ORG}/yt-dlp-master-builds', 'MASTER_ARCHIVE_REPO': f'{FORK_ORG}/yt-dlp-master-builds',
'MASTER_PYPI_PROJECT': 'yt-dlp-test', 'MASTER_PYPI_PROJECT': 'yt-dlp-test',
'MASTER_PYPI_SUFFIX': 'dev', 'MASTER_PYPI_SUFFIX': 'dev',
}, { }, BASE_REPO_SECRETS, {
'MASTER_ARCHIVE_REPO_TOKEN': '1',
}, {
'source': f'{FORK_ORG}/yt-dlp-master-builds', 'source': f'{FORK_ORG}/yt-dlp-master-builds',
'target': 'master', 'target': 'master',
'prerelease': True, 'prerelease': True,
@@ -290,7 +276,6 @@ def test_setup_variables():
'channel': f'{FORK_ORG}/yt-dlp-master-builds', 'channel': f'{FORK_ORG}/yt-dlp-master-builds',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': f'{FORK_ORG}/yt-dlp-master-builds', 'target_repo': f'{FORK_ORG}/yt-dlp-master-builds',
'target_repo_token': 'MASTER_ARCHIVE_REPO_TOKEN',
'target_tag': DEFAULT_VERSION_WITH_REVISION, 'target_tag': DEFAULT_VERSION_WITH_REVISION,
'pypi_project': 'yt-dlp-test', 'pypi_project': 'yt-dlp-test',
'pypi_suffix': 'dev', 'pypi_suffix': 'dev',
@@ -302,7 +287,6 @@ def test_setup_variables():
'channel': f'{FORK_REPOSITORY}@experimental', 'channel': f'{FORK_REPOSITORY}@experimental',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental', 'target_tag': 'experimental',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
@@ -317,8 +301,15 @@ def test_setup_variables():
'channel': 'stable', 'channel': 'stable',
'version': DEFAULT_VERSION_WITH_REVISION, 'version': DEFAULT_VERSION_WITH_REVISION,
'target_repo': FORK_REPOSITORY, 'target_repo': FORK_REPOSITORY,
'target_repo_token': None,
'target_tag': 'experimental', 'target_tag': 'experimental',
'pypi_project': None, 'pypi_project': None,
'pypi_suffix': None, 'pypi_suffix': None,
}, ignore_revision=True) }, ignore_revision=True)
_test(
STABLE_REPOSITORY, 'official vars but no ARCHIVE_REPO_TOKEN, nightly',
BASE_REPO_VARS, {}, {
'source': 'nightly',
'target': 'nightly',
'prerelease': True,
}, None)

View File

@@ -9,10 +9,9 @@ authors = [
] ]
maintainers = [ maintainers = [
{email = "maintainers@yt-dlp.org"}, {email = "maintainers@yt-dlp.org"},
{name = "Grub4K", email = "contact@grub4k.xyz"}, {name = "Grub4K", email = "contact@grub4k.dev"},
{name = "bashonly", email = "bashonly@protonmail.com"}, {name = "bashonly", email = "bashonly@protonmail.com"},
{name = "coletdjnz", email = "coletdjnz@protonmail.com"}, {name = "coletdjnz", email = "coletdjnz@protonmail.com"},
{name = "sepro", email = "sepro@sepr0.com"},
] ]
description = "A feature-rich command-line audio/video downloader" description = "A feature-rich command-line audio/video downloader"
readme = "README.md" readme = "README.md"
@@ -56,15 +55,22 @@ default = [
"requests>=2.32.2,<3", "requests>=2.32.2,<3",
"urllib3>=2.0.2,<3", "urllib3>=2.0.2,<3",
"websockets>=13.0", "websockets>=13.0",
"yt-dlp-ejs==0.3.2", "yt-dlp-ejs==0.5.0",
] ]
curl-cffi = [ curl-cffi = [
"curl-cffi>=0.5.10,!=0.6.*,!=0.7.*,!=0.8.*,!=0.9.*,<0.14; implementation_name=='cpython'", "curl-cffi>=0.5.10,!=0.6.*,!=0.7.*,!=0.8.*,!=0.9.*,<0.15; implementation_name=='cpython'",
]
build-curl-cffi = [
"curl-cffi==0.13.0; sys_platform=='darwin' or (sys_platform=='linux' and platform_machine!='armv7l')",
"curl-cffi==0.14.0; sys_platform=='win32' or (sys_platform=='linux' and platform_machine=='armv7l')",
] ]
secretstorage = [ secretstorage = [
"cffi", "cffi",
"secretstorage", "secretstorage",
] ]
deno = [
"deno>=2.6.6", # v2.6.5 fixes compatibility, v2.6.6 adds integrity check
]
build = [ build = [
"build", "build",
"hatchling>=1.27.0", "hatchling>=1.27.0",
@@ -79,7 +85,7 @@ dev = [
] ]
static-analysis = [ static-analysis = [
"autopep8~=2.0", "autopep8~=2.0",
"ruff~=0.14.0", "ruff~=0.15.0",
] ]
test = [ test = [
"pytest~=8.1", "pytest~=8.1",

View File

@@ -85,11 +85,10 @@ The only reliable way to check if a site is supported is to try it.
- **ant1newsgr:embed**: ant1news.gr embedded videos - **ant1newsgr:embed**: ant1news.gr embedded videos
- **antenna:watch**: antenna.gr and ant1news.gr videos - **antenna:watch**: antenna.gr and ant1news.gr videos
- **Anvato** - **Anvato**
- **aol.com**: Yahoo screen and movies (**Currently broken**) - **aol.com**: (**Currently broken**)
- **APA** - **APA**
- **Aparat** - **Aparat**
- **apple:music:connect**: Apple Music Connect - **apple:music:connect**: Apple Music Connect
- **AppleDaily**: 臺灣蘋果日報
- **ApplePodcasts** - **ApplePodcasts**
- **appletrailers** - **appletrailers**
- **appletrailers:section** - **appletrailers:section**
@@ -306,6 +305,7 @@ The only reliable way to check if a site is supported is to try it.
- **cpac:playlist** - **cpac:playlist**
- **Cracked** - **Cracked**
- **Craftsy** - **Craftsy**
- **croatian.film**
- **CrooksAndLiars** - **CrooksAndLiars**
- **CrowdBunker** - **CrowdBunker**
- **CrowdBunkerChannel** - **CrowdBunkerChannel**
@@ -415,6 +415,7 @@ The only reliable way to check if a site is supported is to try it.
- **Erocast** - **Erocast**
- **EroProfile**: [*eroprofile*](## "netrc machine") - **EroProfile**: [*eroprofile*](## "netrc machine")
- **EroProfile:album** - **EroProfile:album**
- **ERRArhiiv**
- **ERRJupiter** - **ERRJupiter**
- **ertflix**: ERTFLIX videos - **ertflix**: ERTFLIX videos
- **ertflix:codename**: ERTFLIX videos by codename - **ertflix:codename**: ERTFLIX videos by codename
@@ -433,7 +434,7 @@ The only reliable way to check if a site is supported is to try it.
- **EWETVRecordings**: [*ewetv*](## "netrc machine") - **EWETVRecordings**: [*ewetv*](## "netrc machine")
- **Expressen** - **Expressen**
- **EyedoTV** - **EyedoTV**
- **facebook**: [*facebook*](## "netrc machine") - **facebook**
- **facebook:ads** - **facebook:ads**
- **facebook:reel** - **facebook:reel**
- **FacebookPluginsVideo** - **FacebookPluginsVideo**
@@ -448,6 +449,7 @@ The only reliable way to check if a site is supported is to try it.
- **fc2:live** - **fc2:live**
- **Fczenit** - **Fczenit**
- **Fifa** - **Fifa**
- **FilmArchiv**: FILMARCHIV ON
- **filmon** - **filmon**
- **filmon:channel** - **filmon:channel**
- **Filmweb** - **Filmweb**
@@ -470,10 +472,10 @@ The only reliable way to check if a site is supported is to try it.
- **fptplay**: fptplay.vn - **fptplay**: fptplay.vn
- **FrancaisFacile** - **FrancaisFacile**
- **FranceCulture** - **FranceCulture**
- **franceinfo**: franceinfo.fr (formerly francetvinfo.fr)
- **FranceInter** - **FranceInter**
- **francetv** - **francetv**
- **francetv:site** - **francetv:site**
- **francetvinfo.fr**
- **Freesound** - **Freesound**
- **freespeech.org** - **freespeech.org**
- **freetv:series** - **freetv:series**
@@ -504,7 +506,8 @@ The only reliable way to check if a site is supported is to try it.
- **GDCVault**: [*gdcvault*](## "netrc machine") (**Currently broken**) - **GDCVault**: [*gdcvault*](## "netrc machine") (**Currently broken**)
- **GediDigital** - **GediDigital**
- **gem.cbc.ca**: [*cbcgem*](## "netrc machine") - **gem.cbc.ca**: [*cbcgem*](## "netrc machine")
- **gem.cbc.ca:live** - **gem.cbc.ca:live**: [*cbcgem*](## "netrc machine")
- **gem.cbc.ca:olympics**: [*cbcgem*](## "netrc machine")
- **gem.cbc.ca:playlist**: [*cbcgem*](## "netrc machine") - **gem.cbc.ca:playlist**: [*cbcgem*](## "netrc machine")
- **Genius** - **Genius**
- **GeniusLyrics** - **GeniusLyrics**
@@ -621,7 +624,7 @@ The only reliable way to check if a site is supported is to try it.
- **IPrimaCNN** - **IPrimaCNN**
- **iq.com**: International version of iQiyi - **iq.com**: International version of iQiyi
- **iq.com:album** - **iq.com:album**
- **iqiyi**: [*iqiyi*](## "netrc machine") 爱奇艺 - **iqiyi**: 爱奇艺
- **IslamChannel** - **IslamChannel**
- **IslamChannelSeries** - **IslamChannelSeries**
- **IsraelNationalNews** - **IsraelNationalNews**
@@ -732,6 +735,8 @@ The only reliable way to check if a site is supported is to try it.
- **Livestreamfails** - **Livestreamfails**
- **Lnk** - **Lnk**
- **loc**: Library of Congress - **loc**: Library of Congress
- **Locipo**
- **LocipoPlaylist**
- **Loco** - **Loco**
- **loom** - **loom**
- **loom:folder**: (**Currently broken**) - **loom:folder**: (**Currently broken**)
@@ -755,15 +760,13 @@ The only reliable way to check if a site is supported is to try it.
- **mangomolo:live** - **mangomolo:live**
- **mangomolo:video** - **mangomolo:video**
- **MangoTV**: 芒果TV - **MangoTV**: 芒果TV
- **ManotoTV**: Manoto TV (Episode)
- **ManotoTVLive**: Manoto TV (Live)
- **ManotoTVShow**: Manoto TV (Show)
- **ManyVids** - **ManyVids**
- **MaoriTV** - **MaoriTV**
- **Markiza**: (**Currently broken**) - **Markiza**: (**Currently broken**)
- **MarkizaPage**: (**Currently broken**) - **MarkizaPage**: (**Currently broken**)
- **massengeschmack.tv** - **massengeschmack.tv**
- **Masters** - **Masters**
- **MatchiTV**
- **MatchTV** - **MatchTV**
- **mave** - **mave**
- **mave:channel** - **mave:channel**
@@ -893,6 +896,7 @@ The only reliable way to check if a site is supported is to try it.
- **NDTV**: (**Currently broken**) - **NDTV**: (**Currently broken**)
- **nebula:channel**: [*watchnebula*](## "netrc machine") - **nebula:channel**: [*watchnebula*](## "netrc machine")
- **nebula:media**: [*watchnebula*](## "netrc machine") - **nebula:media**: [*watchnebula*](## "netrc machine")
- **nebula:season**: [*watchnebula*](## "netrc machine")
- **nebula:subscriptions**: [*watchnebula*](## "netrc machine") - **nebula:subscriptions**: [*watchnebula*](## "netrc machine")
- **nebula:video**: [*watchnebula*](## "netrc machine") - **nebula:video**: [*watchnebula*](## "netrc machine")
- **NekoHacker** - **NekoHacker**
@@ -914,15 +918,12 @@ The only reliable way to check if a site is supported is to try it.
- **Netverse** - **Netverse**
- **NetversePlaylist** - **NetversePlaylist**
- **NetverseSearch**: "netsearch:" prefix - **NetverseSearch**: "netsearch:" prefix
- **Netzkino**: (**Currently broken**) - **Netzkino**
- **Newgrounds**: [*newgrounds*](## "netrc machine") - **Newgrounds**: [*newgrounds*](## "netrc machine")
- **Newgrounds:playlist** - **Newgrounds:playlist**
- **Newgrounds:user** - **Newgrounds:user**
- **NewsPicks** - **NewsPicks**
- **Newsy** - **Newsy**
- **NextMedia**: 蘋果日報
- **NextMediaActionNews**: 蘋果日報 - 動新聞
- **NextTV**: 壹電視 (**Currently broken**)
- **Nexx** - **Nexx**
- **NexxEmbed** - **NexxEmbed**
- **nfb**: nfb.ca and onf.ca films and episodes - **nfb**: nfb.ca and onf.ca films and episodes
@@ -1042,6 +1043,7 @@ The only reliable way to check if a site is supported is to try it.
- **PalcoMP3:artist** - **PalcoMP3:artist**
- **PalcoMP3:song** - **PalcoMP3:song**
- **PalcoMP3:video** - **PalcoMP3:video**
- **PandaTv**: pandalive.co.kr (팬더티비)
- **Panopto** - **Panopto**
- **PanoptoList** - **PanoptoList**
- **PanoptoPlaylist** - **PanoptoPlaylist**
@@ -1285,13 +1287,13 @@ The only reliable way to check if a site is supported is to try it.
- **Sangiin**: 参議院インターネット審議中継 (archive) - **Sangiin**: 参議院インターネット審議中継 (archive)
- **Sapo**: SAPO Vídeos - **Sapo**: SAPO Vídeos
- **SaucePlus**: Sauce+ - **SaucePlus**: Sauce+
- **SaucePlusChannel**
- **SBS**: sbs.com.au - **SBS**: sbs.com.au
- **sbs.co.kr** - **sbs.co.kr**
- **sbs.co.kr:allvod_program** - **sbs.co.kr:allvod_program**
- **sbs.co.kr:programs_vod** - **sbs.co.kr:programs_vod**
- **schooltv** - **schooltv**
- **ScienceChannel** - **ScienceChannel**
- **screen.yahoo:search**: Yahoo screen search; "yvsearch:" prefix
- **Screen9** - **Screen9**
- **Screencast** - **Screencast**
- **Screencastify** - **Screencastify**
@@ -1300,8 +1302,6 @@ The only reliable way to check if a site is supported is to try it.
- **ScrippsNetworks** - **ScrippsNetworks**
- **scrippsnetworks:watch** - **scrippsnetworks:watch**
- **Scrolller** - **Scrolller**
- **SCTE**: [*scte*](## "netrc machine") (**Currently broken**)
- **SCTECourse**: [*scte*](## "netrc machine") (**Currently broken**)
- **sejm** - **sejm**
- **Sen** - **Sen**
- **SenalColombiaLive**: (**Currently broken**) - **SenalColombiaLive**: (**Currently broken**)
@@ -1429,6 +1429,9 @@ The only reliable way to check if a site is supported is to try it.
- **TapTapAppIntl** - **TapTapAppIntl**
- **TapTapMoment** - **TapTapMoment**
- **TapTapPostIntl** - **TapTapPostIntl**
- **tarangplus:episodes**
- **tarangplus:playlist**
- **tarangplus:video**
- **Tass**: (**Currently broken**) - **Tass**: (**Currently broken**)
- **TBS** - **TBS**
- **TBSJPEpisode** - **TBSJPEpisode**
@@ -1541,8 +1544,8 @@ The only reliable way to check if a site is supported is to try it.
- **tv2playseries.hu** - **tv2playseries.hu**
- **TV4**: tv4.se and tv4play.se - **TV4**: tv4.se and tv4play.se
- **TV5MONDE** - **TV5MONDE**
- **tv5unis**: (**Currently broken**) - **tv5unis**
- **tv5unis:video**: (**Currently broken**) - **tv5unis:video**
- **tv8.it** - **tv8.it**
- **tv8.it:live**: TV8 Live - **tv8.it:live**: TV8 Live
- **tv8.it:playlist**: TV8 Playlist - **tv8.it:playlist**: TV8 Playlist
@@ -1552,10 +1555,12 @@ The only reliable way to check if a site is supported is to try it.
- **TVC** - **TVC**
- **TVCArticle** - **TVCArticle**
- **TVer** - **TVer**
- **tver:olympic**
- **tvigle**: Интернет-телевидение Tvigle.ru - **tvigle**: Интернет-телевидение Tvigle.ru
- **TVIPlayer** - **TVIPlayer**
- **TVN24**: (**Currently broken**) - **TVN24**: (**Currently broken**)
- **tvnoe**: Televize Noe - **tvnoe**: Televize Noe
- **TVO**
- **tvopengr:embed**: tvopen.gr embedded videos - **tvopengr:embed**: tvopen.gr embedded videos
- **tvopengr:watch**: tvopen.gr (and ethnos.gr) videos - **tvopengr:watch**: tvopen.gr (and ethnos.gr) videos
- **tvp**: Telewizja Polska - **tvp**: Telewizja Polska
@@ -1579,12 +1584,12 @@ The only reliable way to check if a site is supported is to try it.
- **twitch:videos:clips**: [*twitch*](## "netrc machine") - **twitch:videos:clips**: [*twitch*](## "netrc machine")
- **twitch:videos:collections**: [*twitch*](## "netrc machine") - **twitch:videos:collections**: [*twitch*](## "netrc machine")
- **twitch:vod**: [*twitch*](## "netrc machine") - **twitch:vod**: [*twitch*](## "netrc machine")
- **twitter**: [*twitter*](## "netrc machine") - **twitter**
- **twitter:amplify**: [*twitter*](## "netrc machine") - **twitter:amplify**
- **twitter:broadcast**: [*twitter*](## "netrc machine") - **twitter:broadcast**
- **twitter:card** - **twitter:card**
- **twitter:shortener**: [*twitter*](## "netrc machine") - **twitter:shortener**
- **twitter:spaces**: [*twitter*](## "netrc machine") - **twitter:spaces**
- **Txxx** - **Txxx**
- **udemy**: [*udemy*](## "netrc machine") - **udemy**: [*udemy*](## "netrc machine")
- **udemy:course**: [*udemy*](## "netrc machine") - **udemy:course**: [*udemy*](## "netrc machine")
@@ -1666,6 +1671,7 @@ The only reliable way to check if a site is supported is to try it.
- **ViMP:Playlist** - **ViMP:Playlist**
- **Viously** - **Viously**
- **Viqeo**: (**Currently broken**) - **Viqeo**: (**Currently broken**)
- **Visir**: Vísir
- **Viu** - **Viu**
- **viu:ott**: [*viu*](## "netrc machine") - **viu:ott**: [*viu*](## "netrc machine")
- **viu:playlist** - **viu:playlist**
@@ -1681,7 +1687,9 @@ The only reliable way to check if a site is supported is to try it.
- **VODPlatform** - **VODPlatform**
- **voicy**: (**Currently broken**) - **voicy**: (**Currently broken**)
- **voicy:channel**: (**Currently broken**) - **voicy:channel**: (**Currently broken**)
- **VolejTV** - **volejtv:category**
- **volejtv:club**
- **volejtv:match**
- **VoxMedia** - **VoxMedia**
- **VoxMediaVolume** - **VoxMediaVolume**
- **vpro**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl - **vpro**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
@@ -1774,8 +1782,9 @@ The only reliable way to check if a site is supported is to try it.
- **XVideos** - **XVideos**
- **xvideos:quickies** - **xvideos:quickies**
- **XXXYMovies** - **XXXYMovies**
- **Yahoo**: Yahoo screen and movies - **yahoo**
- **yahoo:japannews**: Yahoo! Japan News - **yahoo:japannews**: Yahoo! Japan News
- **yahoo:search**: "yvsearch:" prefix
- **YandexDisk** - **YandexDisk**
- **yandexmusic:album**: Яндекс.Музыка - Альбом - **yandexmusic:album**: Яндекс.Музыка - Альбом
- **yandexmusic:artist:albums**: Яндекс.Музыка - Артист - Альбомы - **yandexmusic:artist:albums**: Яндекс.Музыка - Артист - Альбомы
@@ -1811,7 +1820,6 @@ The only reliable way to check if a site is supported is to try it.
- **youtube:playlist**: [*youtube*](## "netrc machine") YouTube playlists - **youtube:playlist**: [*youtube*](## "netrc machine") YouTube playlists
- **youtube:recommended**: [*youtube*](## "netrc machine") YouTube recommended videos; ":ytrec" keyword - **youtube:recommended**: [*youtube*](## "netrc machine") YouTube recommended videos; ":ytrec" keyword
- **youtube:search**: [*youtube*](## "netrc machine") YouTube search; "ytsearch:" prefix - **youtube:search**: [*youtube*](## "netrc machine") YouTube search; "ytsearch:" prefix
- **youtube:search:date**: [*youtube*](## "netrc machine") YouTube search, newest videos first; "ytsearchdate:" prefix
- **youtube:search_url**: [*youtube*](## "netrc machine") YouTube search URLs with sorting and filter support - **youtube:search_url**: [*youtube*](## "netrc machine") YouTube search URLs with sorting and filter support
- **youtube:shorts:pivot:audio**: [*youtube*](## "netrc machine") YouTube Shorts audio pivot (Shorts using audio of a given video) - **youtube:shorts:pivot:audio**: [*youtube*](## "netrc machine") YouTube Shorts audio pivot (Shorts using audio of a given video)
- **youtube:subscriptions**: [*youtube*](## "netrc machine") YouTube subscriptions feed; ":ytsubs" keyword (requires cookies) - **youtube:subscriptions**: [*youtube*](## "netrc machine") YouTube subscriptions feed; ":ytsubs" keyword (requires cookies)

View File

@@ -261,9 +261,9 @@ def sanitize_got_info_dict(got_dict):
def expect_info_dict(self, got_dict, expected_dict): def expect_info_dict(self, got_dict, expected_dict):
ALLOWED_KEYS_SORT_ORDER = ( ALLOWED_KEYS_SORT_ORDER = (
# NB: Keep in sync with the docstring of extractor/common.py # NB: Keep in sync with the docstring of extractor/common.py
'id', 'ext', 'direct', 'display_id', 'title', 'alt_title', 'description', 'media_type', 'ie_key', 'url', 'id', 'ext', 'direct', 'display_id', 'title', 'alt_title', 'description', 'media_type',
'uploader', 'uploader_id', 'uploader_url', 'channel', 'channel_id', 'channel_url', 'channel_is_verified', 'uploader', 'uploader_id', 'uploader_url', 'channel', 'channel_id', 'channel_url', 'channel_is_verified',
'channel_follower_count', 'comment_count', 'view_count', 'concurrent_view_count', 'channel_follower_count', 'comment_count', 'view_count', 'concurrent_view_count', 'save_count',
'like_count', 'dislike_count', 'repost_count', 'average_rating', 'age_limit', 'duration', 'thumbnail', 'heatmap', 'like_count', 'dislike_count', 'repost_count', 'average_rating', 'age_limit', 'duration', 'thumbnail', 'heatmap',
'chapters', 'chapter', 'chapter_number', 'chapter_id', 'start_time', 'end_time', 'section_start', 'section_end', 'chapters', 'chapter', 'chapter_number', 'chapter_id', 'start_time', 'end_time', 'section_start', 'section_end',
'categories', 'tags', 'cast', 'composers', 'artists', 'album_artists', 'creators', 'genres', 'categories', 'tags', 'cast', 'composers', 'artists', 'album_artists', 'creators', 'genres',
@@ -294,7 +294,7 @@ def expect_info_dict(self, got_dict, expected_dict):
missing_keys = sorted( missing_keys = sorted(
test_info_dict.keys() - expected_dict.keys(), test_info_dict.keys() - expected_dict.keys(),
key=lambda x: ALLOWED_KEYS_SORT_ORDER.index(x)) key=ALLOWED_KEYS_SORT_ORDER.index)
if missing_keys: if missing_keys:
def _repr(v): def _repr(v):
if isinstance(v, str): if isinstance(v, str):

View File

@@ -76,6 +76,8 @@ class TestInfoExtractor(unittest.TestCase):
self.assertEqual(ie._get_netrc_login_info(netrc_machine='empty_pass'), ('user', '')) self.assertEqual(ie._get_netrc_login_info(netrc_machine='empty_pass'), ('user', ''))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='both_empty'), ('', '')) self.assertEqual(ie._get_netrc_login_info(netrc_machine='both_empty'), ('', ''))
self.assertEqual(ie._get_netrc_login_info(netrc_machine='nonexistent'), (None, None)) self.assertEqual(ie._get_netrc_login_info(netrc_machine='nonexistent'), (None, None))
with self.assertRaises(ExtractorError):
ie._get_netrc_login_info(netrc_machine=';echo rce')
def test_html_search_regex(self): def test_html_search_regex(self):
html = '<p id="foo">Watch this <a href="http://www.youtube.com/watch?v=BaW_jenozKc">video</a></p>' html = '<p id="foo">Watch this <a href="http://www.youtube.com/watch?v=BaW_jenozKc">video</a></p>'

View File

@@ -205,8 +205,8 @@ class TestLenientSimpleCookie(unittest.TestCase):
), ),
( (
'Test quoted cookie', 'Test quoted cookie',
'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"', 'keebler="E=mc2; L=\\"Loves\\"; fudge=;"',
{'keebler': 'E=mc2; L="Loves"; fudge=\012;'}, {'keebler': 'E=mc2; L="Loves"; fudge=;'},
), ),
( (
"Allow '=' in an unquoted value", "Allow '=' in an unquoted value",
@@ -328,4 +328,30 @@ class TestLenientSimpleCookie(unittest.TestCase):
'Key=Value; [Invalid]=Value; Another=Value', 'Key=Value; [Invalid]=Value; Another=Value',
{'Key': 'Value', 'Another': 'Value'}, {'Key': 'Value', 'Another': 'Value'},
), ),
# Ref: https://github.com/python/cpython/issues/143919
(
'Test invalid cookie name w/ control character',
'foo\012=bar;',
{},
),
(
'Test invalid cookie name w/ control character 2',
'foo\015baz=bar',
{},
),
(
'Test invalid cookie name w/ control character followed by valid cookie',
'foo\015=bar; x=y;',
{'x': 'y'},
),
(
'Test invalid cookie value w/ control character',
'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"',
{},
),
(
'Test invalid quoted attribute value w/ control character',
'Customer="WILE_E_COYOTE"; Version="1\\012"; Path="/acme"',
{},
),
) )

View File

@@ -227,9 +227,13 @@ class TestDevalue(unittest.TestCase):
{'a': 'b'}, 'revivers (indirect)') {'a': 'b'}, 'revivers (indirect)')
self.assertEqual( self.assertEqual(
devalue.parse([['parse', 1], '{"a":0}'], revivers={'parse': lambda x: json.loads(x)}), devalue.parse([['parse', 1], '{"a":0}'], revivers={'parse': json.loads}),
{'a': 0}, 'revivers (parse)') {'a': 0}, 'revivers (parse)')
self.assertEqual(
devalue.parse([{'a': 1, 'b': 3}, ['EmptyRef', 2], 'false', ['EmptyRef', 2]], revivers={'EmptyRef': json.loads}),
{'a': False, 'b': False}, msg='revivers (duplicate EmptyRef)')
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()

View File

@@ -1,44 +0,0 @@
#!/usr/bin/env python3
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import FakeYDL, is_download_test
from yt_dlp.extractor import IqiyiIE
class WarningLogger:
def __init__(self):
self.messages = []
def warning(self, msg):
self.messages.append(msg)
def debug(self, msg):
pass
def error(self, msg):
pass
@is_download_test
class TestIqiyiSDKInterpreter(unittest.TestCase):
def test_iqiyi_sdk_interpreter(self):
"""
Test the functionality of IqiyiSDKInterpreter by trying to log in
If `sign` is incorrect, /validate call throws an HTTP 556 error
"""
logger = WarningLogger()
ie = IqiyiIE(FakeYDL({'logger': logger}))
ie._perform_login('foo', 'bar')
self.assertTrue('unable to log in:' in logger.messages[0])
if __name__ == '__main__':
unittest.main()

View File

@@ -33,10 +33,12 @@ class Variant(enum.Enum):
tce = 'player_ias_tce.vflset/en_US/base.js' tce = 'player_ias_tce.vflset/en_US/base.js'
es5 = 'player_es5.vflset/en_US/base.js' es5 = 'player_es5.vflset/en_US/base.js'
es6 = 'player_es6.vflset/en_US/base.js' es6 = 'player_es6.vflset/en_US/base.js'
es6_tcc = 'player_es6_tcc.vflset/en_US/base.js'
es6_tce = 'player_es6_tce.vflset/en_US/base.js'
tv = 'tv-player-ias.vflset/tv-player-ias.js' tv = 'tv-player-ias.vflset/tv-player-ias.js'
tv_es6 = 'tv-player-es6.vflset/tv-player-es6.js' tv_es6 = 'tv-player-es6.vflset/tv-player-es6.js'
phone = 'player-plasma-ias-phone-en_US.vflset/base.js' phone = 'player-plasma-ias-phone-en_US.vflset/base.js'
tablet = 'player-plasma-ias-tablet-en_US.vflset/base.js' house = 'house_brand_player.vflset/en_US/base.js'
@dataclasses.dataclass @dataclasses.dataclass
@@ -88,6 +90,81 @@ CHALLENGES: list[Challenge] = [
'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt': 'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt':
'MhudCuAuP-6fByOk1_GNXN7gNHHShjyXS2VOgsEItAJz0tipeav0OmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt', 'MhudCuAuP-6fByOk1_GNXN7gNHHShjyXS2VOgsEItAJz0tipeav0OmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt',
}), }),
# c1c87fb0: tce variant broke sig solving; n and main variant are added only for regression testing
Challenge('c1c87fb0', Variant.main, JsChallengeType.N, {
'ZdZIqFPQK-Ty8wId': 'jCHBK5GuAFNa2',
}),
Challenge('c1c87fb0', Variant.main, JsChallengeType.SIG, {
'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt':
'ttJC2JfQdSswRAIgGBCxZyAfKyi0cjXCb3DqEctUw-NYdNmOEvaepit0zJAtIEsgOV2SXZjhSHMNy0NXNGa1kOyBf6HPuAuCduh-_',
}),
Challenge('c1c87fb0', Variant.tce, JsChallengeType.N, {
'ZdZIqFPQK-Ty8wId': 'jCHBK5GuAFNa2',
}),
Challenge('c1c87fb0', Variant.tce, JsChallengeType.SIG, {
'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt':
'ttJC2JfQdSswRAIgGBCxZyAfKyi0cjXCb3DqEctUw-NYdNmOEvaepit0zJAtIEsgOV2SXZjhSHMNy0NXNGa1kOyBf6HPuAuCduh-_',
}),
# 4e51e895: main variant broke sig solving; n challenge is added only for regression testing
Challenge('4e51e895', Variant.main, JsChallengeType.N, {
'0eRGgQWJGfT5rFHFj': 't5kO23_msekBur',
}),
Challenge('4e51e895', Variant.main, JsChallengeType.SIG, {
'AL6p_8AwdY9yAhRzK8rYA_9n97Kizf7_9n97Kizf7_9n97Kizf7_9n97Kizf7_9n97Kizf7_9n97Kizf7':
'AwdY9yAhRzK8rYA_9n97Kizf7_9n97Kizf7_9n9pKizf7_9n97Kizf7_9n97Kizf7_9n97Kizf7',
}),
# 42c5570b: tce variant broke sig solving; n challenge is added only for regression testing
Challenge('42c5570b', Variant.tce, JsChallengeType.N, {
'ZdZIqFPQK-Ty8wId': 'CRoXjB-R-R',
}),
Challenge('42c5570b', Variant.tce, JsChallengeType.SIG, {
'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt':
'EN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavcOmNdYN-wUtgEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt',
}),
# 54bd1de4: tce variant broke sig solving; n challenge is added only for regression testing
Challenge('54bd1de4', Variant.tce, JsChallengeType.N, {
'ZdZIqFPQK-Ty8wId': 'ka-slAQ31sijFN',
}),
Challenge('54bd1de4', Variant.tce, JsChallengeType.SIG, {
'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0tipeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtt':
'gN7a-hudCuAuPH6fByOk1_GNXN0yNMHShjZXS2VOgsEItAJz0titeavEOmNdYN-wUtcEqD3bCXjc0iyKfAyZxCBGgIARwsSdQfJ2CJtp',
}),
# 94667337: tce and es6 variants broke sig solving; n and main/tv variants are added only for regression testing
Challenge('94667337', Variant.main, JsChallengeType.N, {
'BQoJvGBkC2nj1ZZLK-': 'ib1ShEOGoFXIIw',
}),
Challenge('94667337', Variant.main, JsChallengeType.SIG, {
'NJAJEij0EwRgIhAI0KExTgjfPk-MPM9MAdzyyPRt=BM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=gwzz':
'AJEij0EwRgIhAI0KExTgjfPk-MPM9MNdzyyPRtzBM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=',
}),
Challenge('94667337', Variant.tv, JsChallengeType.N, {
'BQoJvGBkC2nj1ZZLK-': 'ib1ShEOGoFXIIw',
}),
Challenge('94667337', Variant.tv, JsChallengeType.SIG, {
'NJAJEij0EwRgIhAI0KExTgjfPk-MPM9MAdzyyPRt=BM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=gwzz':
'AJEij0EwRgIhAI0KExTgjfPk-MPM9MNdzyyPRtzBM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=',
}),
Challenge('94667337', Variant.es6, JsChallengeType.N, {
'BQoJvGBkC2nj1ZZLK-': 'ib1ShEOGoFXIIw',
}),
Challenge('94667337', Variant.es6, JsChallengeType.SIG, {
'NJAJEij0EwRgIhAI0KExTgjfPk-MPM9MAdzyyPRt=BM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=gwzz':
'AJEij0EwRgIhAI0KExTgjfPk-MPM9MNdzyyPRtzBM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=',
}),
Challenge('94667337', Variant.tce, JsChallengeType.N, {
'BQoJvGBkC2nj1ZZLK-': 'ib1ShEOGoFXIIw',
}),
Challenge('94667337', Variant.tce, JsChallengeType.SIG, {
'NJAJEij0EwRgIhAI0KExTgjfPk-MPM9MAdzyyPRt=BM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=gwzz':
'AJEij0EwRgIhAI0KExTgjfPk-MPM9MNdzyyPRtzBM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=',
}),
Challenge('94667337', Variant.es6_tce, JsChallengeType.N, {
'BQoJvGBkC2nj1ZZLK-': 'ib1ShEOGoFXIIw',
}),
Challenge('94667337', Variant.es6_tce, JsChallengeType.SIG, {
'NJAJEij0EwRgIhAI0KExTgjfPk-MPM9MAdzyyPRt=BM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=gwzz':
'AJEij0EwRgIhAI0KExTgjfPk-MPM9MNdzyyPRtzBM8-XO5tm5hlMCSVpAiEAv7eP3CURqZNSPow8BXXAoazVoXgeMP7gH9BdylHCwgw=',
}),
] ]
requests: list[JsChallengeRequest] = [] requests: list[JsChallengeRequest] = []

View File

@@ -9,7 +9,12 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import math import math
from yt_dlp.jsinterp import JS_Undefined, JSInterpreter, js_number_to_string from yt_dlp.jsinterp import (
JS_Undefined,
JSInterpreter,
int_to_int32,
js_number_to_string,
)
class NaN: class NaN:
@@ -101,8 +106,16 @@ class TestJSInterpreter(unittest.TestCase):
self._test('function f(){return 5 ^ 9;}', 12) self._test('function f(){return 5 ^ 9;}', 12)
self._test('function f(){return 0.0 << NaN}', 0) self._test('function f(){return 0.0 << NaN}', 0)
self._test('function f(){return null << undefined}', 0) self._test('function f(){return null << undefined}', 0)
# TODO: Does not work due to number too large self._test('function f(){return -12616 ^ 5041}', -8951)
# self._test('function f(){return 21 << 4294967297}', 42) self._test('function f(){return 21 << 4294967297}', 42)
def test_string_concat(self):
self._test('function f(){return "a" + "b";}', 'ab')
self._test('function f(){let x = "a"; x += "b"; return x;}', 'ab')
self._test('function f(){return "a" + 1;}', 'a1')
self._test('function f(){let x = "a"; x += 1; return x;}', 'a1')
self._test('function f(){return 2 + "b";}', '2b')
self._test('function f(){let x = 2; x += "b"; return x;}', '2b')
def test_array_access(self): def test_array_access(self):
self._test('function f(){var x = [1,2,3]; x[0] = 4; x[0] = 5; x[2.0] = 7; return x;}', [5, 2, 7]) self._test('function f(){var x = [1,2,3]; x[0] = 4; x[0] = 5; x[2.0] = 7; return x;}', [5, 2, 7])
@@ -325,6 +338,7 @@ class TestJSInterpreter(unittest.TestCase):
self._test('function f() { let a = {m1: 42, m2: 0 }; return [a["m1"], a.m2]; }', [42, 0]) self._test('function f() { let a = {m1: 42, m2: 0 }; return [a["m1"], a.m2]; }', [42, 0])
self._test('function f() { let a; return a?.qq; }', JS_Undefined) self._test('function f() { let a; return a?.qq; }', JS_Undefined)
self._test('function f() { let a = {m1: 42, m2: 0 }; return a?.qq; }', JS_Undefined) self._test('function f() { let a = {m1: 42, m2: 0 }; return a?.qq; }', JS_Undefined)
self._test('function f() { let a = {"1": 123}; return a[1]; }', 123)
def test_regex(self): def test_regex(self):
self._test('function f() { let a=/,,[/,913,/](,)}/; }', None) self._test('function f() { let a=/,,[/,913,/](,)}/; }', None)
@@ -447,6 +461,22 @@ class TestJSInterpreter(unittest.TestCase):
def test_splice(self): def test_splice(self):
self._test('function f(){var T = ["0", "1", "2"]; T["splice"](2, 1, "0")[0]; return T }', ['0', '1', '0']) self._test('function f(){var T = ["0", "1", "2"]; T["splice"](2, 1, "0")[0]; return T }', ['0', '1', '0'])
def test_int_to_int32(self):
for inp, exp in [
(0, 0),
(1, 1),
(-1, -1),
(-8951, -8951),
(2147483647, 2147483647),
(2147483648, -2147483648),
(2147483649, -2147483647),
(-2147483649, 2147483647),
(-2147483648, -2147483648),
(-16799986688, 379882496),
(39570129568, 915423904),
]:
assert int_to_int32(inp) == exp
def test_js_number_to_string(self): def test_js_number_to_string(self):
for test, radix, expected in [ for test, radix, expected in [
(0, None, '0'), (0, None, '0'),

View File

@@ -1004,6 +1004,7 @@ class TestUrllibRequestHandler(TestRequestHandlerBase):
@pytest.mark.parametrize('handler', ['Requests'], indirect=True) @pytest.mark.parametrize('handler', ['Requests'], indirect=True)
class TestRequestsRequestHandler(TestRequestHandlerBase): class TestRequestsRequestHandler(TestRequestHandlerBase):
# ruff: disable[PLW0108] `requests` and/or `urllib3` may not be available
@pytest.mark.parametrize('raised,expected', [ @pytest.mark.parametrize('raised,expected', [
(lambda: requests.exceptions.ConnectTimeout(), TransportError), (lambda: requests.exceptions.ConnectTimeout(), TransportError),
(lambda: requests.exceptions.ReadTimeout(), TransportError), (lambda: requests.exceptions.ReadTimeout(), TransportError),
@@ -1017,8 +1018,10 @@ class TestRequestsRequestHandler(TestRequestHandlerBase):
# catch-all: https://github.com/psf/requests/blob/main/src/requests/adapters.py#L535 # catch-all: https://github.com/psf/requests/blob/main/src/requests/adapters.py#L535
(lambda: urllib3.exceptions.HTTPError(), TransportError), (lambda: urllib3.exceptions.HTTPError(), TransportError),
(lambda: requests.exceptions.RequestException(), RequestError), (lambda: requests.exceptions.RequestException(), RequestError),
# (lambda: requests.exceptions.TooManyRedirects(), HTTPError) - Needs a response object # Needs a response object
# (lambda: requests.exceptions.TooManyRedirects(), HTTPError),
]) ])
# ruff: enable[PLW0108]
def test_request_error_mapping(self, handler, monkeypatch, raised, expected): def test_request_error_mapping(self, handler, monkeypatch, raised, expected):
with handler() as rh: with handler() as rh:
def mock_get_instance(*args, **kwargs): def mock_get_instance(*args, **kwargs):
@@ -1034,6 +1037,7 @@ class TestRequestsRequestHandler(TestRequestHandlerBase):
assert exc_info.type is expected assert exc_info.type is expected
# ruff: disable[PLW0108] `urllib3` may not be available
@pytest.mark.parametrize('raised,expected,match', [ @pytest.mark.parametrize('raised,expected,match', [
(lambda: urllib3.exceptions.SSLError(), SSLError, None), (lambda: urllib3.exceptions.SSLError(), SSLError, None),
(lambda: urllib3.exceptions.TimeoutError(), TransportError, None), (lambda: urllib3.exceptions.TimeoutError(), TransportError, None),
@@ -1052,6 +1056,7 @@ class TestRequestsRequestHandler(TestRequestHandlerBase):
'3 bytes read, 5 more expected', '3 bytes read, 5 more expected',
), ),
]) ])
# ruff: enable[PLW0108]
def test_response_error_mapping(self, handler, monkeypatch, raised, expected, match): def test_response_error_mapping(self, handler, monkeypatch, raised, expected, match):
from requests.models import Response as RequestsResponse from requests.models import Response as RequestsResponse
from urllib3.response import HTTPResponse as Urllib3Response from urllib3.response import HTTPResponse as Urllib3Response

View File

@@ -29,6 +29,11 @@ class TestMetadataFromField(unittest.TestCase):
MetadataParserPP.format_to_regex('%(title)s - %(artist)s'), MetadataParserPP.format_to_regex('%(title)s - %(artist)s'),
r'(?P<title>.+)\ \-\ (?P<artist>.+)') r'(?P<title>.+)\ \-\ (?P<artist>.+)')
self.assertEqual(MetadataParserPP.format_to_regex(r'(?P<x>.+)'), r'(?P<x>.+)') self.assertEqual(MetadataParserPP.format_to_regex(r'(?P<x>.+)'), r'(?P<x>.+)')
self.assertEqual(MetadataParserPP.format_to_regex(r'text (?P<x>.+)'), r'text (?P<x>.+)')
self.assertEqual(MetadataParserPP.format_to_regex('x'), r'(?s)(?P<x>.+)')
self.assertEqual(MetadataParserPP.format_to_regex('Field_Name1'), r'(?s)(?P<Field_Name1>.+)')
self.assertEqual(MetadataParserPP.format_to_regex('é'), r'(?s)(?P<é>.+)')
self.assertEqual(MetadataParserPP.format_to_regex('invalid '), 'invalid ')
def test_field_to_template(self): def test_field_to_template(self):
self.assertEqual(MetadataParserPP.field_to_template('title'), '%(title)s') self.assertEqual(MetadataParserPP.field_to_template('title'), '%(title)s')

View File

@@ -239,6 +239,7 @@ class TestTraversal:
'accept matching `expected_type` type' 'accept matching `expected_type` type'
assert traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int) is None, \ assert traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int) is None, \
'reject non matching `expected_type` type' 'reject non matching `expected_type` type'
# ruff: noqa: PLW0108 `type`s get special treatment, so wrap in lambda
assert traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)) == '0', \ assert traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)) == '0', \
'transform type using type function' 'transform type using type function'
assert traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=lambda _: 1 / 0) is None, \ assert traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=lambda _: 1 / 0) is None, \

View File

@@ -489,6 +489,10 @@ class TestUtil(unittest.TestCase):
self.assertEqual(unified_timestamp('Wednesday 31 December 1969 18:01:26 MDT'), 86) self.assertEqual(unified_timestamp('Wednesday 31 December 1969 18:01:26 MDT'), 86)
self.assertEqual(unified_timestamp('12/31/1969 20:01:18 EDT', False), 78) self.assertEqual(unified_timestamp('12/31/1969 20:01:18 EDT', False), 78)
self.assertEqual(unified_timestamp('2026-01-01 00:00:00', tz_offset=0), 1767225600)
self.assertEqual(unified_timestamp('2026-01-01 00:00:00', tz_offset=8), 1767196800)
self.assertEqual(unified_timestamp('2026-01-01 00:00:00 +0800', tz_offset=-5), 1767196800)
def test_determine_ext(self): def test_determine_ext(self):
self.assertEqual(determine_ext('http://example.com/foo/bar.mp4/?download'), 'mp4') self.assertEqual(determine_ext('http://example.com/foo/bar.mp4/?download'), 'mp4')
self.assertEqual(determine_ext('http://example.com/foo/bar/?download', None), None) self.assertEqual(determine_ext('http://example.com/foo/bar/?download', None), None)
@@ -920,6 +924,7 @@ class TestUtil(unittest.TestCase):
self.assertEqual(month_by_name(None), None) self.assertEqual(month_by_name(None), None)
self.assertEqual(month_by_name('December', 'en'), 12) self.assertEqual(month_by_name('December', 'en'), 12)
self.assertEqual(month_by_name('décembre', 'fr'), 12) self.assertEqual(month_by_name('décembre', 'fr'), 12)
self.assertEqual(month_by_name('desember', 'is'), 12)
self.assertEqual(month_by_name('December'), 12) self.assertEqual(month_by_name('December'), 12)
self.assertEqual(month_by_name('décembre'), None) self.assertEqual(month_by_name('décembre'), None)
self.assertEqual(month_by_name('Unknown', 'unknown'), None) self.assertEqual(month_by_name('Unknown', 'unknown'), None)
@@ -1276,6 +1281,9 @@ class TestUtil(unittest.TestCase):
on = js_to_json('[new Date("spam"), \'("eggs")\']') on = js_to_json('[new Date("spam"), \'("eggs")\']')
self.assertEqual(json.loads(on), ['spam', '("eggs")'], msg='Date regex should match a single string') self.assertEqual(json.loads(on), ['spam', '("eggs")'], msg='Date regex should match a single string')
on = js_to_json('[0.077, 7.06, 29.064, 169.0072]')
self.assertEqual(json.loads(on), [0.077, 7.06, 29.064, 169.0072])
def test_js_to_json_malformed(self): def test_js_to_json_malformed(self):
self.assertEqual(js_to_json('42a1'), '42"a1"') self.assertEqual(js_to_json('42a1'), '42"a1"')
self.assertEqual(js_to_json('42a-1'), '42"a"-1') self.assertEqual(js_to_json('42a-1'), '42"a"-1')

View File

@@ -448,6 +448,7 @@ def create_fake_ws_connection(raised):
@pytest.mark.parametrize('handler', ['Websockets'], indirect=True) @pytest.mark.parametrize('handler', ['Websockets'], indirect=True)
class TestWebsocketsRequestHandler: class TestWebsocketsRequestHandler:
# ruff: disable[PLW0108] `websockets` may not be available
@pytest.mark.parametrize('raised,expected', [ @pytest.mark.parametrize('raised,expected', [
# https://websockets.readthedocs.io/en/stable/reference/exceptions.html # https://websockets.readthedocs.io/en/stable/reference/exceptions.html
(lambda: websockets.exceptions.InvalidURI(msg='test', uri='test://'), RequestError), (lambda: websockets.exceptions.InvalidURI(msg='test', uri='test://'), RequestError),
@@ -459,13 +460,14 @@ class TestWebsocketsRequestHandler:
(lambda: websockets.exceptions.NegotiationError(), TransportError), (lambda: websockets.exceptions.NegotiationError(), TransportError),
# Catch-all # Catch-all
(lambda: websockets.exceptions.WebSocketException(), TransportError), (lambda: websockets.exceptions.WebSocketException(), TransportError),
(lambda: TimeoutError(), TransportError), (TimeoutError, TransportError),
# These may be raised by our create_connection implementation, which should also be caught # These may be raised by our create_connection implementation, which should also be caught
(lambda: OSError(), TransportError), (OSError, TransportError),
(lambda: ssl.SSLError(), SSLError), (ssl.SSLError, SSLError),
(lambda: ssl.SSLCertVerificationError(), CertificateVerifyError), (ssl.SSLCertVerificationError, CertificateVerifyError),
(lambda: socks.ProxyError(), ProxyError), (socks.ProxyError, ProxyError),
]) ])
# ruff: enable[PLW0108]
def test_request_error_mapping(self, handler, monkeypatch, raised, expected): def test_request_error_mapping(self, handler, monkeypatch, raised, expected):
import websockets.sync.client import websockets.sync.client
@@ -482,11 +484,12 @@ class TestWebsocketsRequestHandler:
@pytest.mark.parametrize('raised,expected,match', [ @pytest.mark.parametrize('raised,expected,match', [
# https://websockets.readthedocs.io/en/stable/reference/sync/client.html#websockets.sync.client.ClientConnection.send # https://websockets.readthedocs.io/en/stable/reference/sync/client.html#websockets.sync.client.ClientConnection.send
(lambda: websockets.exceptions.ConnectionClosed(None, None), TransportError, None), (lambda: websockets.exceptions.ConnectionClosed(None, None), TransportError, None),
(lambda: RuntimeError(), TransportError, None), (RuntimeError, TransportError, None),
(lambda: TimeoutError(), TransportError, None), (TimeoutError, TransportError, None),
(lambda: TypeError(), RequestError, None), (TypeError, RequestError, None),
(lambda: socks.ProxyError(), ProxyError, None), (socks.ProxyError, ProxyError, None),
# Catch-all # Catch-all
# ruff: noqa: PLW0108 `websockets` may not be available
(lambda: websockets.exceptions.WebSocketException(), TransportError, None), (lambda: websockets.exceptions.WebSocketException(), TransportError, None),
]) ])
def test_ws_send_error_mapping(self, handler, monkeypatch, raised, expected, match): def test_ws_send_error_mapping(self, handler, monkeypatch, raised, expected, match):
@@ -499,10 +502,11 @@ class TestWebsocketsRequestHandler:
@pytest.mark.parametrize('raised,expected,match', [ @pytest.mark.parametrize('raised,expected,match', [
# https://websockets.readthedocs.io/en/stable/reference/sync/client.html#websockets.sync.client.ClientConnection.recv # https://websockets.readthedocs.io/en/stable/reference/sync/client.html#websockets.sync.client.ClientConnection.recv
(lambda: websockets.exceptions.ConnectionClosed(None, None), TransportError, None), (lambda: websockets.exceptions.ConnectionClosed(None, None), TransportError, None),
(lambda: RuntimeError(), TransportError, None), (RuntimeError, TransportError, None),
(lambda: TimeoutError(), TransportError, None), (TimeoutError, TransportError, None),
(lambda: socks.ProxyError(), ProxyError, None), (socks.ProxyError, ProxyError, None),
# Catch-all # Catch-all
# ruff: noqa: PLW0108 `websockets` may not be available
(lambda: websockets.exceptions.WebSocketException(), TransportError, None), (lambda: websockets.exceptions.WebSocketException(), TransportError, None),
]) ])
def test_ws_recv_error_mapping(self, handler, monkeypatch, raised, expected, match): def test_ws_recv_error_mapping(self, handler, monkeypatch, raised, expected, match):

View File

@@ -595,7 +595,7 @@ class YoutubeDL:
'width', 'height', 'asr', 'audio_channels', 'fps', 'width', 'height', 'asr', 'audio_channels', 'fps',
'tbr', 'abr', 'vbr', 'filesize', 'filesize_approx', 'tbr', 'abr', 'vbr', 'filesize', 'filesize_approx',
'timestamp', 'release_timestamp', 'available_at', 'timestamp', 'release_timestamp', 'available_at',
'duration', 'view_count', 'like_count', 'dislike_count', 'repost_count', 'duration', 'view_count', 'like_count', 'dislike_count', 'repost_count', 'save_count',
'average_rating', 'comment_count', 'age_limit', 'average_rating', 'comment_count', 'age_limit',
'start_time', 'end_time', 'start_time', 'end_time',
'chapter_number', 'season_number', 'episode_number', 'chapter_number', 'season_number', 'episode_number',
@@ -1602,8 +1602,10 @@ class YoutubeDL:
if ret is NO_DEFAULT: if ret is NO_DEFAULT:
while True: while True:
filename = self._format_screen(self.prepare_filename(info_dict), self.Styles.FILENAME) filename = self._format_screen(self.prepare_filename(info_dict), self.Styles.FILENAME)
reply = input(self._format_screen( self.to_screen(
f'Download "{filename}"? (Y/n): ', self.Styles.EMPHASIS)).lower().strip() self._format_screen(f'Download "{filename}"? (Y/n): ', self.Styles.EMPHASIS),
skip_eol=True)
reply = input().lower().strip()
if reply in {'y', ''}: if reply in {'y', ''}:
return None return None
elif reply == 'n': elif reply == 'n':
@@ -3026,9 +3028,14 @@ class YoutubeDL:
format_selector = self.format_selector format_selector = self.format_selector
while True: while True:
if interactive_format_selection: if interactive_format_selection:
req_format = input(self._format_screen('\nEnter format selector ', self.Styles.EMPHASIS) if not formats:
+ '(Press ENTER for default, or Ctrl+C to quit)' # Bypass interactive format selection if no formats & --ignore-no-formats-error
+ self._format_screen(': ', self.Styles.EMPHASIS)) formats_to_download = None
break
self.to_screen(self._format_screen('\nEnter format selector ', self.Styles.EMPHASIS)
+ '(Press ENTER for default, or Ctrl+C to quit)'
+ self._format_screen(': ', self.Styles.EMPHASIS), skip_eol=True)
req_format = input()
try: try:
format_selector = self.build_format_selector(req_format) if req_format else None format_selector = self.build_format_selector(req_format) if req_format else None
except SyntaxError as err: except SyntaxError as err:
@@ -3474,11 +3481,12 @@ class YoutubeDL:
if dl_filename is not None: if dl_filename is not None:
self.report_file_already_downloaded(dl_filename) self.report_file_already_downloaded(dl_filename)
elif fd: elif fd:
for f in info_dict['requested_formats'] if fd != FFmpegFD else []: if fd != FFmpegFD and temp_filename != '-':
f['filepath'] = fname = prepend_extension( for f in info_dict['requested_formats']:
correct_ext(temp_filename, info_dict['ext']), f['filepath'] = fname = prepend_extension(
'f{}'.format(f['format_id']), info_dict['ext']) correct_ext(temp_filename, info_dict['ext']),
downloaded.append(fname) 'f{}'.format(f['format_id']), info_dict['ext'])
downloaded.append(fname)
info_dict['url'] = '\n'.join(f['url'] for f in info_dict['requested_formats']) info_dict['url'] = '\n'.join(f['url'] for f in info_dict['requested_formats'])
success, real_download = self.dl(temp_filename, info_dict) success, real_download = self.dl(temp_filename, info_dict)
info_dict['__real_download'] = real_download info_dict['__real_download'] = real_download

View File

@@ -1168,6 +1168,7 @@ class LenientSimpleCookie(http.cookies.SimpleCookie):
# We use Morsel's legal key chars to avoid errors on setting values # We use Morsel's legal key chars to avoid errors on setting values
_LEGAL_KEY_CHARS = r'\w\d' + re.escape('!#$%&\'*+-.:^_`|~') _LEGAL_KEY_CHARS = r'\w\d' + re.escape('!#$%&\'*+-.:^_`|~')
_LEGAL_VALUE_CHARS = _LEGAL_KEY_CHARS + re.escape('(),/<=>?@[]{}') _LEGAL_VALUE_CHARS = _LEGAL_KEY_CHARS + re.escape('(),/<=>?@[]{}')
_LEGAL_KEY_RE = re.compile(rf'[{_LEGAL_KEY_CHARS}]+', re.ASCII)
_RESERVED = { _RESERVED = {
'expires', 'expires',
@@ -1185,17 +1186,17 @@ class LenientSimpleCookie(http.cookies.SimpleCookie):
# Added 'bad' group to catch the remaining value # Added 'bad' group to catch the remaining value
_COOKIE_PATTERN = re.compile(r''' _COOKIE_PATTERN = re.compile(r'''
\s* # Optional whitespace at start of cookie [ ]* # Optional whitespace at start of cookie
(?P<key> # Start of group 'key' (?P<key> # Start of group 'key'
[''' + _LEGAL_KEY_CHARS + r''']+?# Any word of at least one letter [^ =;]+ # Match almost anything here for now and validate later
) # End of group 'key' ) # End of group 'key'
( # Optional group: there may not be a value. ( # Optional group: there may not be a value.
\s*=\s* # Equal Sign [ ]*=[ ]* # Equal Sign
( # Start of potential value ( # Start of potential value
(?P<val> # Start of group 'val' (?P<val> # Start of group 'val'
"(?:[^\\"]|\\.)*" # Any doublequoted string "(?:[^\\"]|\\.)*" # Any doublequoted string
| # or | # or
\w{3},\s[\w\d\s-]{9,11}\s[\d:]{8}\sGMT # Special case for "expires" attr \w{3},\ [\w\d -]{9,11}\ [\d:]{8}\ GMT # Special case for "expires" attr
| # or | # or
[''' + _LEGAL_VALUE_CHARS + r''']* # Any word or empty string [''' + _LEGAL_VALUE_CHARS + r''']* # Any word or empty string
) # End of group 'val' ) # End of group 'val'
@@ -1203,10 +1204,14 @@ class LenientSimpleCookie(http.cookies.SimpleCookie):
(?P<bad>(?:\\;|[^;])*?) # 'bad' group fallback for invalid values (?P<bad>(?:\\;|[^;])*?) # 'bad' group fallback for invalid values
) # End of potential value ) # End of potential value
)? # End of optional value group )? # End of optional value group
\s* # Any number of spaces. [ ]* # Any number of spaces.
(\s+|;|$) # Ending either at space, semicolon, or EOS. ([ ]+|;|$) # Ending either at space, semicolon, or EOS.
''', re.ASCII | re.VERBOSE) ''', re.ASCII | re.VERBOSE)
# http.cookies.Morsel raises on values w/ control characters in Python 3.14.3+ & 3.13.12+
# Ref: https://github.com/python/cpython/issues/143919
_CONTROL_CHARACTER_RE = re.compile(r'[\x00-\x1F\x7F]')
def load(self, data): def load(self, data):
# Workaround for https://github.com/yt-dlp/yt-dlp/issues/4776 # Workaround for https://github.com/yt-dlp/yt-dlp/issues/4776
if not isinstance(data, str): if not isinstance(data, str):
@@ -1219,6 +1224,9 @@ class LenientSimpleCookie(http.cookies.SimpleCookie):
continue continue
key, value = match.group('key', 'val') key, value = match.group('key', 'val')
if not self._LEGAL_KEY_RE.fullmatch(key):
morsel = None
continue
is_attribute = False is_attribute = False
if key.startswith('$'): if key.startswith('$'):
@@ -1237,6 +1245,14 @@ class LenientSimpleCookie(http.cookies.SimpleCookie):
value = True value = True
else: else:
value, _ = self.value_decode(value) value, _ = self.value_decode(value)
# Guard against control characters in quoted attribute values
if self._CONTROL_CHARACTER_RE.search(value):
# While discarding the entire morsel is not very lenient,
# it's better than http.cookies.Morsel raising a CookieError
# and it's probably better to err on the side of caution
self.pop(morsel.key, None)
morsel = None
continue
morsel[key] = value morsel[key] = value
@@ -1246,6 +1262,10 @@ class LenientSimpleCookie(http.cookies.SimpleCookie):
elif value is not None: elif value is not None:
morsel = self.get(key, http.cookies.Morsel()) morsel = self.get(key, http.cookies.Morsel())
real_value, coded_value = self.value_decode(value) real_value, coded_value = self.value_decode(value)
# Guard against control characters in quoted cookie values
if self._CONTROL_CHARACTER_RE.search(real_value):
morsel = None
continue
morsel.set(key, real_value, coded_value) morsel.set(key, real_value, coded_value)
self[key] = morsel self[key] = morsel

View File

@@ -36,6 +36,7 @@ from .rtsp import RtspFD
from .websocket import WebSocketFragmentFD from .websocket import WebSocketFragmentFD
from .youtube_live_chat import YoutubeLiveChatFD from .youtube_live_chat import YoutubeLiveChatFD
from .bunnycdn import BunnyCdnFD from .bunnycdn import BunnyCdnFD
from .soop import SoopVodFD
PROTOCOL_MAP = { PROTOCOL_MAP = {
'rtmp': RtmpFD, 'rtmp': RtmpFD,
@@ -56,6 +57,7 @@ PROTOCOL_MAP = {
'youtube_live_chat': YoutubeLiveChatFD, 'youtube_live_chat': YoutubeLiveChatFD,
'youtube_live_chat_replay': YoutubeLiveChatFD, 'youtube_live_chat_replay': YoutubeLiveChatFD,
'bunnycdn': BunnyCdnFD, 'bunnycdn': BunnyCdnFD,
'soopvod': SoopVodFD,
} }

61
yt_dlp/downloader/soop.py Normal file
View File

@@ -0,0 +1,61 @@
import threading
import time
from .common import FileDownloader
from . import HlsFD
from ..extractor.afreecatv import _cloudfront_auth_request
from ..networking.exceptions import network_exceptions
class SoopVodFD(FileDownloader):
"""
Downloads Soop subscription VODs with required cookie refresh requests
Note, this is not a part of public API, and will be removed without notice.
DO NOT USE
"""
def real_download(self, filename, info_dict):
self.to_screen(f'[{self.FD_NAME}] Downloading Soop subscription VOD HLS')
fd = HlsFD(self.ydl, self.params)
refresh_params = info_dict['_cookie_refresh_params']
referer_url = info_dict['webpage_url']
stop_event = threading.Event()
refresh_thread = threading.Thread(
target=self._cookie_refresh_thread,
args=(stop_event, refresh_params, referer_url),
)
refresh_thread.start()
try:
return fd.real_download(filename, info_dict)
finally:
stop_event.set()
def _cookie_refresh_thread(self, stop_event, refresh_params, referer_url):
m3u8_url = refresh_params['m3u8_url']
strm_id = refresh_params['strm_id']
video_id = refresh_params['video_id']
def _get_cloudfront_cookie_expiration(m3u8_url):
cookies = self.ydl.cookiejar.get_cookies_for_url(m3u8_url)
return min((cookie.expires for cookie in cookies if 'CloudFront' in cookie.name and cookie.expires), default=0)
while not stop_event.wait(5):
current_time = time.time()
expiration_time = _get_cloudfront_cookie_expiration(m3u8_url)
last_refresh_check = refresh_params.get('_last_refresh', 0)
# Cookie TTL is 90 seconds, but let's give ourselves a 15-second cushion
should_refresh = (
(expiration_time and current_time >= expiration_time - 15)
or (not expiration_time and current_time - last_refresh_check >= 75)
)
if should_refresh:
try:
self.ydl.urlopen(_cloudfront_auth_request(
m3u8_url, strm_id, video_id, referer_url)).read()
refresh_params['_last_refresh'] = current_time
except network_exceptions as e:
self.to_screen(f'[{self.FD_NAME}] Cookie refresh attempt failed: {e}')

View File

@@ -1,32 +1,4 @@
# flake8: noqa: F401 # flake8: noqa: F401
# isort: off
from .youtube import ( # Youtube is moved to the top to improve performance
YoutubeIE,
YoutubeClipIE,
YoutubeFavouritesIE,
YoutubeNotificationsIE,
YoutubeHistoryIE,
YoutubeTabIE,
YoutubeLivestreamEmbedIE,
YoutubePlaylistIE,
YoutubeRecommendedIE,
YoutubeSearchDateIE,
YoutubeSearchIE,
YoutubeSearchURLIE,
YoutubeMusicSearchURLIE,
YoutubeSubscriptionsIE,
YoutubeTruncatedIDIE,
YoutubeTruncatedURLIE,
YoutubeYtBeIE,
YoutubeYtUserIE,
YoutubeWatchLaterIE,
YoutubeShortsAudioPivotIE,
YoutubeConsentRedirectIE,
)
# isort: on
from .abc import ( from .abc import (
ABCIE, ABCIE,
ABCIViewIE, ABCIViewIE,
@@ -339,8 +311,10 @@ from .canalsurmas import CanalsurmasIE
from .caracoltv import CaracolTvPlayIE from .caracoltv import CaracolTvPlayIE
from .cbc import ( from .cbc import (
CBCIE, CBCIE,
CBCGemContentIE,
CBCGemIE, CBCGemIE,
CBCGemLiveIE, CBCGemLiveIE,
CBCGemOlympicsIE,
CBCGemPlaylistIE, CBCGemPlaylistIE,
CBCListenIE, CBCListenIE,
CBCPlayerIE, CBCPlayerIE,
@@ -431,6 +405,7 @@ from .cpac import (
) )
from .cracked import CrackedIE from .cracked import CrackedIE
from .craftsy import CraftsyIE from .craftsy import CraftsyIE
from .croatianfilm import CroatianFilmIE
from .crooksandliars import CrooksAndLiarsIE from .crooksandliars import CrooksAndLiarsIE
from .crowdbunker import ( from .crowdbunker import (
CrowdBunkerChannelIE, CrowdBunkerChannelIE,
@@ -591,7 +566,10 @@ from .eroprofile import (
EroProfileAlbumIE, EroProfileAlbumIE,
EroProfileIE, EroProfileIE,
) )
from .err import ERRJupiterIE from .err import (
ERRArhiivIE,
ERRJupiterIE,
)
from .ertgr import ( from .ertgr import (
ERTFlixCodenameIE, ERTFlixCodenameIE,
ERTFlixIE, ERTFlixIE,
@@ -638,6 +616,7 @@ from .fc2 import (
) )
from .fczenit import FczenitIE from .fczenit import FczenitIE
from .fifa import FifaIE from .fifa import FifaIE
from .filmarchiv import FilmArchivIE
from .filmon import ( from .filmon import (
FilmOnChannelIE, FilmOnChannelIE,
FilmOnIE, FilmOnIE,
@@ -1052,6 +1031,10 @@ from .livestream import (
) )
from .livestreamfails import LivestreamfailsIE from .livestreamfails import LivestreamfailsIE
from .lnk import LnkIE from .lnk import LnkIE
from .locipo import (
LocipoIE,
LocipoPlaylistIE,
)
from .loco import LocoIE from .loco import LocoIE
from .loom import ( from .loom import (
LoomFolderIE, LoomFolderIE,
@@ -1086,11 +1069,6 @@ from .mangomolo import (
MangomoloLiveIE, MangomoloLiveIE,
MangomoloVideoIE, MangomoloVideoIE,
) )
from .manoto import (
ManotoTVIE,
ManotoTVLiveIE,
ManotoTVShowIE,
)
from .manyvids import ManyVidsIE from .manyvids import ManyVidsIE
from .maoritv import MaoriTVIE from .maoritv import MaoriTVIE
from .markiza import ( from .markiza import (
@@ -1099,6 +1077,7 @@ from .markiza import (
) )
from .massengeschmacktv import MassengeschmackTVIE from .massengeschmacktv import MassengeschmackTVIE
from .masters import MastersIE from .masters import MastersIE
from .matchitv import MatchiTVIE
from .matchtv import MatchTVIE from .matchtv import MatchTVIE
from .mave import ( from .mave import (
MaveChannelIE, MaveChannelIE,
@@ -1278,6 +1257,7 @@ from .nebula import (
NebulaChannelIE, NebulaChannelIE,
NebulaClassIE, NebulaClassIE,
NebulaIE, NebulaIE,
NebulaSeasonIE,
NebulaSubscriptionsIE, NebulaSubscriptionsIE,
) )
from .nekohacker import NekoHackerIE from .nekohacker import NekoHackerIE
@@ -1312,12 +1292,6 @@ from .newgrounds import (
) )
from .newspicks import NewsPicksIE from .newspicks import NewsPicksIE
from .newsy import NewsyIE from .newsy import NewsyIE
from .nextmedia import (
AppleDailyIE,
NextMediaActionNewsIE,
NextMediaIE,
NextTVIE,
)
from .nexx import ( from .nexx import (
NexxEmbedIE, NexxEmbedIE,
NexxIE, NexxIE,
@@ -1486,6 +1460,7 @@ from .palcomp3 import (
PalcoMP3IE, PalcoMP3IE,
PalcoMP3VideoIE, PalcoMP3VideoIE,
) )
from .pandatv import PandaTvIE
from .panopto import ( from .panopto import (
PanoptoIE, PanoptoIE,
PanoptoListIE, PanoptoListIE,
@@ -1817,7 +1792,10 @@ from .safari import (
from .saitosan import SaitosanIE from .saitosan import SaitosanIE
from .samplefocus import SampleFocusIE from .samplefocus import SampleFocusIE
from .sapo import SapoIE from .sapo import SapoIE
from .sauceplus import SaucePlusIE from .sauceplus import (
SaucePlusChannelIE,
SaucePlusIE,
)
from .sbs import SBSIE from .sbs import SBSIE
from .sbscokr import ( from .sbscokr import (
SBSCoKrAllvodProgramIE, SBSCoKrAllvodProgramIE,
@@ -1834,10 +1812,6 @@ from .scrippsnetworks import (
ScrippsNetworksWatchIE, ScrippsNetworksWatchIE,
) )
from .scrolller import ScrolllerIE from .scrolller import ScrolllerIE
from .scte import (
SCTEIE,
SCTECourseIE,
)
from .sejmpl import SejmIE from .sejmpl import SejmIE
from .sen import SenIE from .sen import SenIE
from .senalcolombia import SenalColombiaLiveIE from .senalcolombia import SenalColombiaLiveIE
@@ -2019,6 +1993,11 @@ from .taptap import (
TapTapMomentIE, TapTapMomentIE,
TapTapPostIntlIE, TapTapPostIntlIE,
) )
from .tarangplus import (
TarangPlusEpisodesIE,
TarangPlusPlaylistIE,
TarangPlusVideoIE,
)
from .tass import TassIE from .tass import TassIE
from .tbs import TBSIE from .tbs import TBSIE
from .tbsjp import ( from .tbsjp import (
@@ -2205,11 +2184,15 @@ from .tvc import (
TVCIE, TVCIE,
TVCArticleIE, TVCArticleIE,
) )
from .tver import TVerIE from .tver import (
TVerIE,
TVerOlympicIE,
)
from .tvigle import TvigleIE from .tvigle import TvigleIE
from .tviplayer import TVIPlayerIE from .tviplayer import TVIPlayerIE
from .tvn24 import TVN24IE from .tvn24 import TVN24IE
from .tvnoe import TVNoeIE from .tvnoe import TVNoeIE
from .tvo import TvoIE
from .tvopengr import ( from .tvopengr import (
TVOpenGrEmbedIE, TVOpenGrEmbedIE,
TVOpenGrWatchIE, TVOpenGrWatchIE,
@@ -2374,6 +2357,7 @@ from .vimm import (
) )
from .viously import ViouslyIE from .viously import ViouslyIE
from .viqeo import ViqeoIE from .viqeo import ViqeoIE
from .visir import VisirIE
from .viu import ( from .viu import (
ViuIE, ViuIE,
ViuOTTIE, ViuOTTIE,
@@ -2394,7 +2378,11 @@ from .voicy import (
VoicyChannelIE, VoicyChannelIE,
VoicyIE, VoicyIE,
) )
from .volejtv import VolejTVIE from .volejtv import (
VolejTVCategoryPlaylistIE,
VolejTVClubPlaylistIE,
VolejTVIE,
)
from .voxmedia import ( from .voxmedia import (
VoxMediaIE, VoxMediaIE,
VoxMediaVolumeIE, VoxMediaVolumeIE,
@@ -2557,6 +2545,28 @@ from .youporn import (
YouPornTagIE, YouPornTagIE,
YouPornVideosIE, YouPornVideosIE,
) )
from .youtube import (
YoutubeClipIE,
YoutubeConsentRedirectIE,
YoutubeFavouritesIE,
YoutubeHistoryIE,
YoutubeIE,
YoutubeLivestreamEmbedIE,
YoutubeMusicSearchURLIE,
YoutubeNotificationsIE,
YoutubePlaylistIE,
YoutubeRecommendedIE,
YoutubeSearchIE,
YoutubeSearchURLIE,
YoutubeShortsAudioPivotIE,
YoutubeSubscriptionsIE,
YoutubeTabIE,
YoutubeTruncatedIDIE,
YoutubeTruncatedURLIE,
YoutubeWatchLaterIE,
YoutubeYtBeIE,
YoutubeYtUserIE,
)
from .zaiko import ( from .zaiko import (
ZaikoETicketIE, ZaikoETicketIE,
ZaikoIE, ZaikoIE,

View File

@@ -5,10 +5,12 @@ from ..utils import (
ExtractorError, ExtractorError,
GeoRestrictedError, GeoRestrictedError,
int_or_none, int_or_none,
make_archive_id,
remove_start, remove_start,
traverse_obj,
update_url_query, update_url_query,
url_or_none,
) )
from ..utils.traversal import traverse_obj
class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
@@ -29,6 +31,19 @@ class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
'historyvault.com': (None, 'historyvault', None), 'historyvault.com': (None, 'historyvault', None),
'biography.com': (None, 'biography', None), 'biography.com': (None, 'biography', None),
} }
_GRAPHQL_QUERY = '''
query getUserVideo($videoId: ID!) {
video(id: $videoId) {
title
publicUrl
programId
tvSeasonNumber
tvSeasonEpisodeNumber
series {
title
}
}
}'''
def _extract_aen_smil(self, smil_url, video_id, auth=None): def _extract_aen_smil(self, smil_url, video_id, auth=None):
query = { query = {
@@ -73,19 +88,39 @@ class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
def _extract_aetn_info(self, domain, filter_key, filter_value, url): def _extract_aetn_info(self, domain, filter_key, filter_value, url):
requestor_id, brand, software_statement = self._DOMAIN_MAP[domain] requestor_id, brand, software_statement = self._DOMAIN_MAP[domain]
if filter_key == 'canonical':
webpage = self._download_webpage(url, filter_value)
graphql_video_id = self._search_regex(
r'<meta\b[^>]+\bcontent="[^"]*\btpid/(\d+)"', webpage,
'id') or self._html_search_meta('videoId', webpage, 'GraphQL video ID', fatal=True)
else:
graphql_video_id = filter_value
result = self._download_json( result = self._download_json(
f'https://feeds.video.aetnd.com/api/v2/{brand}/videos', 'https://yoga.appsvcs.aetnd.com/', graphql_video_id,
filter_value, query={f'filter[{filter_key}]': filter_value}) query={
result = traverse_obj( 'brand': brand,
result, ('results', 'mode': 'live',
lambda k, v: k == 0 and v[filter_key] == filter_value), 'platform': 'web',
get_all=False) },
if not result: data=json.dumps({
'operationName': 'getUserVideo',
'variables': {
'videoId': graphql_video_id,
},
'query': self._GRAPHQL_QUERY,
}).encode(),
headers={
'Content-Type': 'application/json',
})
result = traverse_obj(result, ('data', 'video', {dict}))
media_url = traverse_obj(result, ('publicUrl', {url_or_none}))
if not media_url:
raise ExtractorError('Show not found in A&E feed (too new?)', expected=True, raise ExtractorError('Show not found in A&E feed (too new?)', expected=True,
video_id=remove_start(filter_value, '/')) video_id=remove_start(filter_value, '/'))
title = result['title'] title = result['title']
video_id = result['id'] video_id = result['programId']
media_url = result['publicUrl']
theplatform_metadata = self._download_theplatform_metadata(self._search_regex( theplatform_metadata = self._download_theplatform_metadata(self._search_regex(
r'https?://link\.theplatform\.com/s/([^?]+)', media_url, 'theplatform_path'), video_id) r'https?://link\.theplatform\.com/s/([^?]+)', media_url, 'theplatform_path'), video_id)
info = self._parse_theplatform_metadata(theplatform_metadata) info = self._parse_theplatform_metadata(theplatform_metadata)
@@ -100,9 +135,13 @@ class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
info.update(self._extract_aen_smil(media_url, video_id, auth)) info.update(self._extract_aen_smil(media_url, video_id, auth))
info.update({ info.update({
'title': title, 'title': title,
'series': result.get('seriesName'), 'display_id': graphql_video_id,
'season_number': int_or_none(result.get('tvSeasonNumber')), '_old_archive_ids': [make_archive_id(self, graphql_video_id)],
'episode_number': int_or_none(result.get('tvSeasonEpisodeNumber')), **traverse_obj(result, {
'series': ('series', 'title', {str}),
'season_number': ('tvSeasonNumber', {int_or_none}),
'episode_number': ('tvSeasonEpisodeNumber', {int_or_none}),
}),
}) })
return info return info
@@ -116,7 +155,7 @@ class AENetworksIE(AENetworksBaseIE):
(?:shows/[^/?#]+/)?videos/[^/?#]+ (?:shows/[^/?#]+/)?videos/[^/?#]+
)''' )'''
_TESTS = [{ _TESTS = [{
'url': 'http://www.history.com/shows/mountain-men/season-1/episode-1', 'url': 'https://www.history.com/shows/mountain-men/season-1/episode-1',
'info_dict': { 'info_dict': {
'id': '22253814', 'id': '22253814',
'ext': 'mp4', 'ext': 'mp4',
@@ -139,11 +178,11 @@ class AENetworksIE(AENetworksBaseIE):
}, },
'params': {'skip_download': 'm3u8'}, 'params': {'skip_download': 'm3u8'},
'add_ie': ['ThePlatform'], 'add_ie': ['ThePlatform'],
'skip': 'Geo-restricted - This content is not available in your location.', 'skip': 'This content requires a valid, unexpired auth token',
}, { }, {
'url': 'http://www.aetv.com/shows/duck-dynasty/season-9/episode-1', 'url': 'https://www.aetv.com/shows/duck-dynasty/season-9/episode-1',
'info_dict': { 'info_dict': {
'id': '600587331957', 'id': '147486',
'ext': 'mp4', 'ext': 'mp4',
'title': 'Inlawful Entry', 'title': 'Inlawful Entry',
'description': 'md5:57c12115a2b384d883fe64ca50529e08', 'description': 'md5:57c12115a2b384d883fe64ca50529e08',
@@ -160,6 +199,8 @@ class AENetworksIE(AENetworksBaseIE):
'season_number': 9, 'season_number': 9,
'series': 'Duck Dynasty', 'series': 'Duck Dynasty',
'age_limit': 0, 'age_limit': 0,
'display_id': '600587331957',
'_old_archive_ids': ['aenetworks 600587331957'],
}, },
'params': {'skip_download': 'm3u8'}, 'params': {'skip_download': 'm3u8'},
'add_ie': ['ThePlatform'], 'add_ie': ['ThePlatform'],
@@ -186,6 +227,7 @@ class AENetworksIE(AENetworksBaseIE):
}, },
'params': {'skip_download': 'm3u8'}, 'params': {'skip_download': 'm3u8'},
'add_ie': ['ThePlatform'], 'add_ie': ['ThePlatform'],
'skip': '404 Not Found',
}, { }, {
'url': 'https://www.aetv.com/specials/hunting-jonbenets-killer-the-untold-story', 'url': 'https://www.aetv.com/specials/hunting-jonbenets-killer-the-untold-story',
'info_dict': { 'info_dict': {
@@ -209,6 +251,7 @@ class AENetworksIE(AENetworksBaseIE):
}, },
'params': {'skip_download': 'm3u8'}, 'params': {'skip_download': 'm3u8'},
'add_ie': ['ThePlatform'], 'add_ie': ['ThePlatform'],
'skip': 'This content requires a valid, unexpired auth token',
}, { }, {
'url': 'http://www.fyi.tv/shows/tiny-house-nation/season-1/episode-8', 'url': 'http://www.fyi.tv/shows/tiny-house-nation/season-1/episode-8',
'only_matching': True, 'only_matching': True,
@@ -259,7 +302,7 @@ class AENetworksListBaseIE(AENetworksBaseIE):
domain, slug = self._match_valid_url(url).groups() domain, slug = self._match_valid_url(url).groups()
_, brand, _ = self._DOMAIN_MAP[domain] _, brand, _ = self._DOMAIN_MAP[domain]
playlist = self._call_api(self._RESOURCE, slug, brand, self._FIELDS) playlist = self._call_api(self._RESOURCE, slug, brand, self._FIELDS)
base_url = f'http://watch.{domain}' base_url = f'https://watch.{domain}'
entries = [] entries = []
for item in (playlist.get(self._ITEMS_KEY) or []): for item in (playlist.get(self._ITEMS_KEY) or []):

View File

@@ -1,5 +1,6 @@
import datetime as dt import datetime as dt
import functools import functools
import time
from .common import InfoExtractor from .common import InfoExtractor
from ..networking import Request from ..networking import Request
@@ -16,7 +17,23 @@ from ..utils import (
urlencode_postdata, urlencode_postdata,
urljoin, urljoin,
) )
from ..utils.traversal import traverse_obj from ..utils.traversal import require, traverse_obj
def _cloudfront_auth_request(m3u8_url, strm_id, video_id, referer_url):
return Request(
'https://live.sooplive.co.kr/api/private_auth.php',
method='POST',
headers={
'Referer': referer_url,
'Origin': 'https://vod.sooplive.co.kr',
},
data=urlencode_postdata({
'type': 'vod',
'strm_id': strm_id,
'title_no': video_id,
'url': m3u8_url,
}))
class AfreecaTVBaseIE(InfoExtractor): class AfreecaTVBaseIE(InfoExtractor):
@@ -153,6 +170,13 @@ class AfreecaTVIE(AfreecaTVBaseIE):
'nApiLevel': 10, 'nApiLevel': 10,
}))['data'] }))['data']
initial_refresh_time = 0
strm_id = None
# For subscriber-only VODs, we need to call private_auth.php to get CloudFront cookies
needs_private_auth = traverse_obj(data, ('sub_upload_type', {str}))
if needs_private_auth:
strm_id = traverse_obj(data, ('bj_id', {str}, {require('stream ID')}))
error_code = traverse_obj(data, ('code', {int})) error_code = traverse_obj(data, ('code', {int}))
if error_code == -6221: if error_code == -6221:
raise ExtractorError('The VOD does not exist', expected=True) raise ExtractorError('The VOD does not exist', expected=True)
@@ -172,9 +196,23 @@ class AfreecaTVIE(AfreecaTVBaseIE):
traverse_obj(data, ('files', lambda _, v: url_or_none(v['file']))), start=1): traverse_obj(data, ('files', lambda _, v: url_or_none(v['file']))), start=1):
file_url = file_element['file'] file_url = file_element['file']
if determine_ext(file_url) == 'm3u8': if determine_ext(file_url) == 'm3u8':
if needs_private_auth:
self._request_webpage(
_cloudfront_auth_request(file_url, strm_id, video_id, url),
video_id, 'Requesting CloudFront cookies', 'Failed to get CloudFront cookies')
initial_refresh_time = time.time()
formats = self._extract_m3u8_formats( formats = self._extract_m3u8_formats(
file_url, video_id, 'mp4', m3u8_id='hls', file_url, video_id, 'mp4', m3u8_id='hls',
note=f'Downloading part {file_num} m3u8 information') note=f'Downloading part {file_num} m3u8 information')
if needs_private_auth:
for fmt in formats:
fmt['protocol'] = 'soopvod'
fmt['_cookie_refresh_params'] = {
'm3u8_url': file_url,
'strm_id': strm_id,
'video_id': video_id,
'_last_refresh': initial_refresh_time,
}
else: else:
formats = [{ formats = [{
'url': file_url, 'url': file_url,

View File

@@ -11,18 +11,18 @@ from ..utils.traversal import traverse_obj
class ApplePodcastsIE(InfoExtractor): class ApplePodcastsIE(InfoExtractor):
_VALID_URL = r'https?://podcasts\.apple\.com/(?:[^/]+/)?podcast(?:/[^/]+){1,2}.*?\bi=(?P<id>\d+)' _VALID_URL = r'https?://podcasts\.apple\.com/(?:[^/]+/)?podcast(?:/[^/]+){1,2}.*?\bi=(?P<id>\d+)'
_TESTS = [{ _TESTS = [{
'url': 'https://podcasts.apple.com/us/podcast/ferreck-dawn-to-the-break-of-dawn-117/id1625658232?i=1000665010654', 'url': 'https://podcasts.apple.com/us/podcast/urbana-podcast-724-by-david-penn/id1531349107?i=1000748574256',
'md5': '82cc219b8cc1dcf8bfc5a5e99b23b172', 'md5': 'f8a6f92735d0cfbd5e6a7294151e28d8',
'info_dict': { 'info_dict': {
'id': '1000665010654', 'id': '1000748574256',
'ext': 'mp3', 'ext': 'm4a',
'title': 'Ferreck Dawn - To The Break of Dawn 117', 'title': 'URBANA PODCAST 724 BY DAVID PENN',
'episode': 'Ferreck Dawn - To The Break of Dawn 117', 'episode': 'URBANA PODCAST 724 BY DAVID PENN',
'description': 'md5:8c4f5c2c30af17ed6a98b0b9daf15b76', 'description': 'md5:fec77bacba32db8c9b3dda5486ed085f',
'upload_date': '20240812', 'upload_date': '20260206',
'timestamp': 1723449600, 'timestamp': 1770400801,
'duration': 3596, 'duration': 3602,
'series': 'Ferreck Dawn - To The Break of Dawn', 'series': 'Urbana Radio Show',
'thumbnail': 're:.+[.](png|jpe?g|webp)', 'thumbnail': 're:.+[.](png|jpe?g|webp)',
}, },
}, { }, {
@@ -57,22 +57,22 @@ class ApplePodcastsIE(InfoExtractor):
webpage = self._download_webpage(url, episode_id) webpage = self._download_webpage(url, episode_id)
server_data = self._search_json( server_data = self._search_json(
r'<script [^>]*\bid=["\']serialized-server-data["\'][^>]*>', webpage, r'<script [^>]*\bid=["\']serialized-server-data["\'][^>]*>', webpage,
'server data', episode_id, contains_pattern=r'\[{(?s:.+)}\]')[0]['data'] 'server data', episode_id)['data'][0]['data']
model_data = traverse_obj(server_data, ( model_data = traverse_obj(server_data, (
'headerButtonItems', lambda _, v: v['$kind'] == 'share' and v['modelType'] == 'EpisodeLockup', 'headerButtonItems', lambda _, v: v['$kind'] == 'share' and v['modelType'] == 'EpisodeLockup',
'model', {dict}, any)) 'model', {dict}, any))
return { return {
'id': episode_id, 'id': episode_id,
**self._json_ld(
traverse_obj(server_data, ('seoData', 'schemaContent', {dict}))
or self._yield_json_ld(webpage, episode_id, fatal=False), episode_id, fatal=False),
**traverse_obj(model_data, { **traverse_obj(model_data, {
'title': ('title', {str}), 'title': ('title', {str}),
'description': ('summary', {clean_html}), 'description': ('summary', {clean_html}),
'url': ('playAction', 'episodeOffer', 'streamUrl', {clean_podcast_url}), 'url': ('playAction', 'episodeOffer', 'streamUrl', {clean_podcast_url}),
'timestamp': ('releaseDate', {parse_iso8601}), 'timestamp': ('releaseDate', {parse_iso8601}),
'duration': ('duration', {int_or_none}), 'duration': ('duration', {int_or_none}),
'episode': ('title', {str}),
'episode_number': ('episodeNumber', {int_or_none}),
'series': ('showTitle', {str}),
}), }),
'thumbnail': self._og_search_thumbnail(webpage), 'thumbnail': self._og_search_thumbnail(webpage),
'vcodec': 'none', 'vcodec': 'none',

View File

@@ -279,7 +279,7 @@ class ArchiveOrgIE(InfoExtractor):
'url': 'https://archive.org/' + track['file'].lstrip('/'), 'url': 'https://archive.org/' + track['file'].lstrip('/'),
} }
metadata = self._download_json('http://archive.org/metadata/' + identifier, identifier) metadata = self._download_json(f'https://archive.org/metadata/{identifier}', identifier)
m = metadata['metadata'] m = metadata['metadata']
identifier = m['identifier'] identifier = m['identifier']

View File

@@ -5,16 +5,18 @@ import time
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ( from ..utils import (
KNOWN_EXTENSIONS,
ExtractorError, ExtractorError,
clean_html, clean_html,
extract_attributes, extract_attributes,
float_or_none, float_or_none,
format_field,
int_or_none, int_or_none,
join_nonempty,
parse_filesize, parse_filesize,
parse_qs,
str_or_none, str_or_none,
strftime_or_none,
try_get, try_get,
unified_strdate,
unified_timestamp, unified_timestamp,
update_url_query, update_url_query,
url_or_none, url_or_none,
@@ -411,70 +413,67 @@ class BandcampAlbumIE(BandcampIE): # XXX: Do not subclass from concrete IE
class BandcampWeeklyIE(BandcampIE): # XXX: Do not subclass from concrete IE class BandcampWeeklyIE(BandcampIE): # XXX: Do not subclass from concrete IE
IE_NAME = 'Bandcamp:weekly' IE_NAME = 'Bandcamp:weekly'
_VALID_URL = r'https?://(?:www\.)?bandcamp\.com/?\?(?:.*?&)?show=(?P<id>\d+)' _VALID_URL = r'https?://(?:www\.)?bandcamp\.com/radio/?\?(?:[^#]+&)?show=(?P<id>\d+)'
_TESTS = [{ _TESTS = [{
'url': 'https://bandcamp.com/?show=224', 'url': 'https://bandcamp.com/radio?show=224',
'md5': '61acc9a002bed93986b91168aa3ab433', 'md5': '61acc9a002bed93986b91168aa3ab433',
'info_dict': { 'info_dict': {
'id': '224', 'id': '224',
'ext': 'mp3', 'ext': 'mp3',
'title': 'BC Weekly April 4th 2017 - Magic Moments', 'title': 'Bandcamp Weekly, 2017-04-04',
'description': 'md5:5d48150916e8e02d030623a48512c874', 'description': 'md5:5d48150916e8e02d030623a48512c874',
'duration': 5829.77, 'thumbnail': 'https://f4.bcbits.com/img/9982549_0.jpg',
'release_date': '20170404',
'series': 'Bandcamp Weekly', 'series': 'Bandcamp Weekly',
'episode': 'Magic Moments',
'episode_id': '224', 'episode_id': '224',
'release_timestamp': 1491264000,
'release_date': '20170404',
'duration': 5829.77,
}, },
'params': { 'params': {
'format': 'mp3-128', 'format': 'mp3-128',
}, },
}, { }, {
'url': 'https://bandcamp.com/?blah/blah@&show=228', 'url': 'https://bandcamp.com/radio/?foo=bar&show=224',
'only_matching': True, 'only_matching': True,
}] }]
def _real_extract(self, url): def _real_extract(self, url):
show_id = self._match_id(url) show_id = self._match_id(url)
webpage = self._download_webpage(url, show_id) audio_data = self._download_json(
'https://bandcamp.com/api/bcradio_api/1/get_show',
show_id, 'Downloading radio show JSON',
data=json.dumps({'id': show_id}).encode(),
headers={'Content-Type': 'application/json'})['radioShowAudio']
blob = self._extract_data_attr(webpage, show_id, 'blob') stream_url = audio_data['streamUrl']
format_id = traverse_obj(stream_url, ({parse_qs}, 'enc', -1))
encoding, _, bitrate_str = (format_id or '').partition('-')
show = blob['bcw_data'][show_id] webpage = self._download_webpage(url, show_id, fatal=False)
metadata = traverse_obj(
self._extract_data_attr(webpage, show_id, 'blob', fatal=False),
('appData', 'shows', lambda _, v: str(v['showId']) == show_id, any)) or {}
formats = [] series_title = audio_data.get('title') or metadata.get('title')
for format_id, format_url in show['audio_stream'].items(): release_timestamp = unified_timestamp(audio_data.get('date')) or unified_timestamp(metadata.get('date'))
if not url_or_none(format_url):
continue
for known_ext in KNOWN_EXTENSIONS:
if known_ext in format_id:
ext = known_ext
break
else:
ext = None
formats.append({
'format_id': format_id,
'url': format_url,
'ext': ext,
'vcodec': 'none',
})
title = show.get('audio_title') or 'Bandcamp Weekly'
subtitle = show.get('subtitle')
if subtitle:
title += f' - {subtitle}'
return { return {
'id': show_id, 'id': show_id,
'title': title,
'description': show.get('desc') or show.get('short_desc'),
'duration': float_or_none(show.get('audio_duration')),
'is_live': False,
'release_date': unified_strdate(show.get('published_date')),
'series': 'Bandcamp Weekly',
'episode': show.get('subtitle'),
'episode_id': show_id, 'episode_id': show_id,
'formats': formats, 'title': join_nonempty(series_title, strftime_or_none(release_timestamp, '%Y-%m-%d'), delim=', '),
'series': series_title,
'thumbnail': format_field(metadata, 'imageId', 'https://f4.bcbits.com/img/%s_0.jpg', default=None),
'description': metadata.get('desc') or metadata.get('short_desc'),
'duration': float_or_none(audio_data.get('duration')),
'release_timestamp': release_timestamp,
'formats': [{
'url': stream_url,
'format_id': format_id,
'ext': encoding or 'mp3',
'acodec': encoding or None,
'vcodec': 'none',
'abr': int_or_none(bitrate_str),
}],
} }

View File

@@ -1,5 +1,5 @@
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ExtractorError, urlencode_postdata from ..utils import ExtractorError, UserNotLive, urlencode_postdata
class BigoIE(InfoExtractor): class BigoIE(InfoExtractor):
@@ -40,7 +40,7 @@ class BigoIE(InfoExtractor):
info = info_raw.get('data') or {} info = info_raw.get('data') or {}
if not info.get('alive'): if not info.get('alive'):
raise ExtractorError('This user is offline.', expected=True) raise UserNotLive(video_id=user_id)
formats, subs = self._extract_m3u8_formats_and_subtitles( formats, subs = self._extract_m3u8_formats_and_subtitles(
info.get('hls_src'), user_id, 'mp4', 'm3u8') info.get('hls_src'), user_id, 'mp4', 'm3u8')

View File

@@ -124,7 +124,7 @@ class BilibiliBaseIE(InfoExtractor):
**traverse_obj(play_info, { **traverse_obj(play_info, {
'quality': ('quality', {int_or_none}), 'quality': ('quality', {int_or_none}),
'format_id': ('quality', {str_or_none}), 'format_id': ('quality', {str_or_none}),
'format_note': ('quality', {lambda x: format_names.get(x)}), 'format_note': ('quality', {format_names.get}),
'duration': ('timelength', {float_or_none(scale=1000)}), 'duration': ('timelength', {float_or_none(scale=1000)}),
}), }),
**parse_resolution(format_names.get(play_info.get('quality'))), **parse_resolution(format_names.get(play_info.get('quality'))),

View File

@@ -21,21 +21,44 @@ class BoostyIE(InfoExtractor):
'url': 'https://boosty.to/kuplinov/posts/e55d050c-e3bb-4873-a7db-ac7a49b40c38', 'url': 'https://boosty.to/kuplinov/posts/e55d050c-e3bb-4873-a7db-ac7a49b40c38',
'info_dict': { 'info_dict': {
'id': 'd7473824-352e-48e2-ae53-d4aa39459968', 'id': 'd7473824-352e-48e2-ae53-d4aa39459968',
'title': 'phasma_3', 'title': 'Бан? А! Бан! (Phasmophobia)',
'alt_title': 'Бан? А! Бан! (Phasmophobia)',
'channel': 'Kuplinov', 'channel': 'Kuplinov',
'channel_id': '7958701', 'channel_id': '7958701',
'timestamp': 1655031975, 'timestamp': 1655031975,
'upload_date': '20220612', 'upload_date': '20220612',
'release_timestamp': 1655049000, 'release_timestamp': 1655049000,
'release_date': '20220612', 'release_date': '20220612',
'modified_timestamp': 1668680993, 'modified_timestamp': 1743328648,
'modified_date': '20221117', 'modified_date': '20250330',
'tags': ['куплинов', 'phasmophobia'], 'tags': ['куплинов', 'phasmophobia'],
'like_count': int, 'like_count': int,
'ext': 'mp4', 'ext': 'mp4',
'duration': 105, 'duration': 105,
'view_count': int, 'view_count': int,
'thumbnail': r're:^https://i\.mycdn\.me/videoPreview\?', 'thumbnail': r're:^https://iv\.okcdn\.ru/videoPreview\?',
},
}, {
# single ok_video with truncated title
'url': 'https://boosty.to/kuplinov/posts/cc09b7f9-121e-40b8-9392-4a075ef2ce53',
'info_dict': {
'id': 'fb5ea762-6303-4557-9a17-157947326810',
'title': 'Какая там активность была? Не слышу! Повтори еще пару раз! (Phas',
'alt_title': 'Какая там активность была? Не слышу! Повтори еще пару раз! (Phasmophobia)',
'channel': 'Kuplinov',
'channel_id': '7958701',
'timestamp': 1655031930,
'upload_date': '20220612',
'release_timestamp': 1655048400,
'release_date': '20220612',
'modified_timestamp': 1743328616,
'modified_date': '20250330',
'tags': ['куплинов', 'phasmophobia'],
'like_count': int,
'ext': 'mp4',
'duration': 39,
'view_count': int,
'thumbnail': r're:^https://iv\.okcdn\.ru/videoPreview\?',
}, },
}, { }, {
# multiple ok_video # multiple ok_video
@@ -109,36 +132,41 @@ class BoostyIE(InfoExtractor):
'thumbnail': r're:^https://i\.mycdn\.me/videoPreview\?', 'thumbnail': r're:^https://i\.mycdn\.me/videoPreview\?',
}, },
}], }],
'skip': 'post has been deleted',
}, { }, {
# single external video (youtube) # single external video (youtube)
'url': 'https://boosty.to/denischuzhoy/posts/6094a487-bcec-4cf8-a453-43313b463c38', 'url': 'https://boosty.to/futuremusicproduction/posts/32a8cae2-3252-49da-b285-0e014bc6e565',
'info_dict': { 'info_dict': {
'id': 'EXelTnve5lY', 'id': '-37FW_YQ3B4',
'title': 'Послание Президента Федеральному Собранию | Класс народа', 'title': 'Afro | Deep House FREE FLP',
'upload_date': '20210425', 'media_type': 'video',
'channel': 'Денис Чужой', 'upload_date': '20250829',
'tags': 'count:10', 'timestamp': 1756466005,
'channel': 'Future Music Production',
'tags': 'count:0',
'like_count': int, 'like_count': int,
'ext': 'mp4', 'ext': 'm4a',
'duration': 816, 'duration': 170,
'view_count': int, 'view_count': int,
'thumbnail': r're:^https://i\.ytimg\.com/', 'thumbnail': r're:^https://i\.ytimg\.com/',
'age_limit': 0, 'age_limit': 0,
'availability': 'public', 'availability': 'public',
'categories': list, 'categories': list,
'channel_follower_count': int, 'channel_follower_count': int,
'channel_id': 'UCCzVNbWZfYpBfyofCCUD_0w', 'channel_id': 'UCKVYrFBYmci1e-T8NeHw2qg',
'channel_is_verified': bool,
'channel_url': r're:^https://www\.youtube\.com/', 'channel_url': r're:^https://www\.youtube\.com/',
'comment_count': int, 'comment_count': int,
'description': str, 'description': str,
'heatmap': 'count:100',
'live_status': str, 'live_status': str,
'playable_in_embed': bool, 'playable_in_embed': bool,
'uploader': str, 'uploader': str,
'uploader_id': str, 'uploader_id': str,
'uploader_url': r're:^https://www\.youtube\.com/', 'uploader_url': r're:^https://www\.youtube\.com/',
}, },
'expected_warnings': [
'Remote components challenge solver script',
'n challenge solving failed',
],
}] }]
_MP4_TYPES = ('tiny', 'lowest', 'low', 'medium', 'high', 'full_hd', 'quad_hd', 'ultra_hd') _MP4_TYPES = ('tiny', 'lowest', 'low', 'medium', 'high', 'full_hd', 'quad_hd', 'ultra_hd')
@@ -207,13 +235,14 @@ class BoostyIE(InfoExtractor):
video_id = item.get('id') or post_id video_id = item.get('id') or post_id
entries.append({ entries.append({
'id': video_id, 'id': video_id,
'alt_title': post_title,
'formats': self._extract_formats(item.get('playerUrls'), video_id), 'formats': self._extract_formats(item.get('playerUrls'), video_id),
**common_metadata, **common_metadata,
**traverse_obj(item, { **traverse_obj(item, {
'title': ('title', {str}), 'title': ('title', {str}),
'duration': ('duration', {int_or_none}), 'duration': ('duration', {int_or_none}),
'view_count': ('viewsCounter', {int_or_none}), 'view_count': ('viewsCounter', {int_or_none}),
'thumbnail': (('previewUrl', 'defaultPreview'), {url_or_none}), 'thumbnail': (('preview', 'defaultPreview'), {url_or_none}),
}, get_all=False)}) }, get_all=False)})
if not entries and not post.get('hasAccess'): if not entries and not post.get('hasAccess'):

View File

@@ -10,6 +10,7 @@ from ..utils import (
ExtractorError, ExtractorError,
float_or_none, float_or_none,
int_or_none, int_or_none,
join_nonempty,
js_to_json, js_to_json,
jwt_decode_hs256, jwt_decode_hs256,
mimetype2ext, mimetype2ext,
@@ -25,6 +26,7 @@ from ..utils import (
url_basename, url_basename,
url_or_none, url_or_none,
urlencode_postdata, urlencode_postdata,
urljoin,
) )
from ..utils.traversal import require, traverse_obj, trim_str from ..utils.traversal import require, traverse_obj, trim_str
@@ -105,7 +107,7 @@ class CBCIE(InfoExtractor):
# multiple CBC.APP.Caffeine.initInstance(...) # multiple CBC.APP.Caffeine.initInstance(...)
'url': 'http://www.cbc.ca/news/canada/calgary/dog-indoor-exercise-winter-1.3928238', 'url': 'http://www.cbc.ca/news/canada/calgary/dog-indoor-exercise-winter-1.3928238',
'info_dict': { 'info_dict': {
'title': 'Keep Rover active during the deep freeze with doggie pushups and other fun indoor tasks', # FIXME: actual title includes " | CBC News" 'title': 'Keep Rover active during the deep freeze with doggie pushups and other fun indoor tasks',
'id': 'dog-indoor-exercise-winter-1.3928238', 'id': 'dog-indoor-exercise-winter-1.3928238',
'description': 'md5:c18552e41726ee95bd75210d1ca9194c', 'description': 'md5:c18552e41726ee95bd75210d1ca9194c',
}, },
@@ -134,6 +136,13 @@ class CBCIE(InfoExtractor):
title = (self._og_search_title(webpage, default=None) title = (self._og_search_title(webpage, default=None)
or self._html_search_meta('twitter:title', webpage, 'title', default=None) or self._html_search_meta('twitter:title', webpage, 'title', default=None)
or self._html_extract_title(webpage)) or self._html_extract_title(webpage))
title = self._search_regex(
r'^(?P<title>.+?)(?:\s*[|-]\s*CBC.*)?$',
title, 'cleaned title', group='title', default=title)
data = self._search_json(
r'window\.__INITIAL_STATE__\s*=', webpage,
'initial state', display_id, default={}, transform_source=js_to_json)
entries = [ entries = [
self._extract_player_init(player_init, display_id) self._extract_player_init(player_init, display_id)
for player_init in re.findall(r'CBC\.APP\.Caffeine\.initInstance\(({.+?})\);', webpage)] for player_init in re.findall(r'CBC\.APP\.Caffeine\.initInstance\(({.+?})\);', webpage)]
@@ -143,6 +152,11 @@ class CBCIE(InfoExtractor):
r'<div[^>]+\bid=["\']player-(\d+)', r'<div[^>]+\bid=["\']player-(\d+)',
r'guid["\']\s*:\s*["\'](\d+)'): r'guid["\']\s*:\s*["\'](\d+)'):
media_ids.extend(re.findall(media_id_re, webpage)) media_ids.extend(re.findall(media_id_re, webpage))
media_ids.extend(traverse_obj(data, (
'detail', 'content', 'body', ..., 'content',
lambda _, v: v['type'] == 'polopoly_media', 'content', 'sourceId', {str})))
if content_id := traverse_obj(data, ('app', 'contentId', {str})):
media_ids.append(content_id)
entries.extend([ entries.extend([
self.url_result(f'cbcplayer:{media_id}', 'CBCPlayer', media_id) self.url_result(f'cbcplayer:{media_id}', 'CBCPlayer', media_id)
for media_id in orderedSet(media_ids)]) for media_id in orderedSet(media_ids)])
@@ -268,7 +282,7 @@ class CBCPlayerIE(InfoExtractor):
'duration': 2692.833, 'duration': 2692.833,
'subtitles': { 'subtitles': {
'en-US': [{ 'en-US': [{
'name': 'English Captions', 'name': r're:English',
'url': 'https://cbchls.akamaized.net/delivery/news-shows/2024/06/17/NAT_JUN16-00-55-00/NAT_JUN16_cc.vtt', 'url': 'https://cbchls.akamaized.net/delivery/news-shows/2024/06/17/NAT_JUN16-00-55-00/NAT_JUN16_cc.vtt',
}], }],
}, },
@@ -322,6 +336,7 @@ class CBCPlayerIE(InfoExtractor):
'categories': ['Olympics Summer Soccer', 'Summer Olympics Replays', 'Summer Olympics Soccer Replays'], 'categories': ['Olympics Summer Soccer', 'Summer Olympics Replays', 'Summer Olympics Soccer Replays'],
'location': 'Canada', 'location': 'Canada',
}, },
'skip': 'Video no longer available',
'params': {'skip_download': 'm3u8'}, 'params': {'skip_download': 'm3u8'},
}, { }, {
'url': 'https://www.cbc.ca/player/play/video/9.6459530', 'url': 'https://www.cbc.ca/player/play/video/9.6459530',
@@ -380,7 +395,8 @@ class CBCPlayerIE(InfoExtractor):
video_id = self._match_id(url) video_id = self._match_id(url)
webpage = self._download_webpage(f'https://www.cbc.ca/player/play/{video_id}', video_id) webpage = self._download_webpage(f'https://www.cbc.ca/player/play/{video_id}', video_id)
data = self._search_json( data = self._search_json(
r'window\.__INITIAL_STATE__\s*=', webpage, 'initial state', video_id)['video']['currentClip'] r'window\.__INITIAL_STATE__\s*=', webpage,
'initial state', video_id, transform_source=js_to_json)['video']['currentClip']
assets = traverse_obj( assets = traverse_obj(
data, ('media', 'assets', lambda _, v: url_or_none(v['key']) and v['type'])) data, ('media', 'assets', lambda _, v: url_or_none(v['key']) and v['type']))
@@ -492,12 +508,14 @@ class CBCPlayerPlaylistIE(InfoExtractor):
'info_dict': { 'info_dict': {
'id': 'news/tv shows/the national/latest broadcast', 'id': 'news/tv shows/the national/latest broadcast',
}, },
'skip': 'Playlist no longer available',
}, { }, {
'url': 'https://www.cbc.ca/player/news/Canada/North', 'url': 'https://www.cbc.ca/player/news/Canada/North',
'playlist_mincount': 25, 'playlist_mincount': 25,
'info_dict': { 'info_dict': {
'id': 'news/canada/north', 'id': 'news/canada/north',
}, },
'skip': 'Playlist no longer available',
}] }]
def _real_extract(self, url): def _real_extract(self, url):
@@ -524,6 +542,32 @@ class CBCGemBaseIE(InfoExtractor):
f'https://services.radio-canada.ca/ott/catalog/v2/gem/show/{item_id}', f'https://services.radio-canada.ca/ott/catalog/v2/gem/show/{item_id}',
display_id or item_id, query={'device': 'web'}) display_id or item_id, query={'device': 'web'})
def _call_media_api(self, media_id, app_code='gem', display_id=None, headers=None):
media_data = self._download_json(
'https://services.radio-canada.ca/media/validation/v2/',
display_id or media_id, headers=headers, query={
'appCode': app_code,
'connectionType': 'hd',
'deviceType': 'ipad',
'multibitrate': 'true',
'output': 'json',
'tech': 'hls',
'manifestVersion': '2',
'manifestType': 'desktop',
'idMedia': media_id,
})
error_code = traverse_obj(media_data, ('errorCode', {int}))
if error_code == 1:
self.raise_geo_restricted(countries=self._GEO_COUNTRIES)
if error_code == 35:
self.raise_login_required(method='password')
if error_code != 0:
error_message = join_nonempty(error_code, media_data.get('message'), delim=' - ')
raise ExtractorError(f'{self.IE_NAME} said: {error_message}')
return media_data
def _extract_item_info(self, item_info): def _extract_item_info(self, item_info):
episode_number = None episode_number = None
title = traverse_obj(item_info, ('title', {str})) title = traverse_obj(item_info, ('title', {str}))
@@ -551,7 +595,7 @@ class CBCGemBaseIE(InfoExtractor):
class CBCGemIE(CBCGemBaseIE): class CBCGemIE(CBCGemBaseIE):
IE_NAME = 'gem.cbc.ca' IE_NAME = 'gem.cbc.ca'
_VALID_URL = r'https?://gem\.cbc\.ca/(?:media/)?(?P<id>[0-9a-z-]+/s(?P<season>[0-9]+)[a-z][0-9]+)' _VALID_URL = r'https?://gem\.cbc\.ca/(?:media/)?(?P<id>[0-9a-z-]+/s(?P<season>[0-9]+)[a-z][0-9]{2,4})/?(?:[?#]|$)'
_TESTS = [{ _TESTS = [{
# This is a normal, public, TV show video # This is a normal, public, TV show video
'url': 'https://gem.cbc.ca/media/schitts-creek/s06e01', 'url': 'https://gem.cbc.ca/media/schitts-creek/s06e01',
@@ -693,29 +737,10 @@ class CBCGemIE(CBCGemBaseIE):
if claims_token := self._fetch_claims_token(): if claims_token := self._fetch_claims_token():
headers['x-claims-token'] = claims_token headers['x-claims-token'] = claims_token
m3u8_info = self._download_json( m3u8_url = self._call_media_api(
'https://services.radio-canada.ca/media/validation/v2/', item_info['idMedia'], display_id=video_id, headers=headers)['url']
video_id, headers=headers, query={
'appCode': 'gem',
'connectionType': 'hd',
'deviceType': 'ipad',
'multibitrate': 'true',
'output': 'json',
'tech': 'hls',
'manifestVersion': '2',
'manifestType': 'desktop',
'idMedia': item_info['idMedia'],
})
if m3u8_info.get('errorCode') == 1:
self.raise_geo_restricted(countries=['CA'])
elif m3u8_info.get('errorCode') == 35:
self.raise_login_required(method='password')
elif m3u8_info.get('errorCode') != 0:
raise ExtractorError(f'{self.IE_NAME} said: {m3u8_info.get("errorCode")} - {m3u8_info.get("message")}')
formats = self._extract_m3u8_formats( formats = self._extract_m3u8_formats(
m3u8_info['url'], video_id, 'mp4', m3u8_id='hls', query={'manifestType': ''}) m3u8_url, video_id, 'mp4', m3u8_id='hls', query={'manifestType': ''})
self._remove_duplicate_formats(formats) self._remove_duplicate_formats(formats)
for fmt in formats: for fmt in formats:
@@ -785,7 +810,128 @@ class CBCGemPlaylistIE(CBCGemBaseIE):
}), series=traverse_obj(show_info, ('title', {str}))) }), series=traverse_obj(show_info, ('title', {str})))
class CBCGemLiveIE(InfoExtractor): class CBCGemContentIE(CBCGemBaseIE):
IE_NAME = 'gem.cbc.ca:content'
IE_DESC = False # Do not list
_VALID_URL = r'https?://gem\.cbc\.ca/(?P<id>[0-9a-z-]+)/?(?:[?#]|$)'
_TESTS = [{
# Series URL; content_type == 'Season'
'url': 'https://gem.cbc.ca/the-tunnel',
'playlist_count': 3,
'info_dict': {
'id': 'the-tunnel',
},
}, {
# Miniseries URL; content_type == 'Parts'
'url': 'https://gem.cbc.ca/summit-72',
'playlist_count': 1,
'info_dict': {
'id': 'summit-72',
},
}, {
# Olympics URL; content_type == 'Standalone'
'url': 'https://gem.cbc.ca/ski-jumping-nh-individual-womens-final-30086',
'info_dict': {
'id': 'ski-jumping-nh-individual-womens-final-30086',
'ext': 'mp4',
'title': 'Ski Jumping: NH Individual (Women\'s) - Final',
'description': 'md5:411c07c8a9a4a36344530b0c726bf8ab',
'duration': 12793,
'thumbnail': r're:https://[^.]+\.cbc\.ca/.+\.jpg',
'release_timestamp': 1770482100,
'release_date': '20260207',
'live_status': 'was_live',
},
}, {
# Movie URL; content_type == 'Standalone'; requires authentication
'url': 'https://gem.cbc.ca/copa-71',
'only_matching': True,
}]
def _real_extract(self, url):
display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id)
data = self._search_nextjs_data(webpage, display_id)['props']['pageProps']['data']
content_type = data['contentType']
self.write_debug(f'Routing for content type "{content_type}"')
if content_type == 'Standalone':
new_url = traverse_obj(data, (
'header', 'cta', 'media', 'url', {urljoin('https://gem.cbc.ca/')}))
if CBCGemOlympicsIE.suitable(new_url):
return self.url_result(new_url, CBCGemOlympicsIE)
# Manually construct non-Olympics standalone URLs to avoid returning trailer URLs
return self.url_result(f'https://gem.cbc.ca/{display_id}/s01e01', CBCGemIE)
# Handle series URLs (content_type == 'Season') and miniseries URLs (content_type == 'Parts')
def entries():
for playlist_url in traverse_obj(data, (
'content', ..., 'lineups', ..., 'url', {urljoin('https://gem.cbc.ca/')},
{lambda x: x if CBCGemPlaylistIE.suitable(x) else None},
)):
yield self.url_result(playlist_url, CBCGemPlaylistIE)
return self.playlist_result(entries(), display_id)
class CBCGemOlympicsIE(CBCGemBaseIE):
IE_NAME = 'gem.cbc.ca:olympics'
_VALID_URL = r'https?://gem\.cbc\.ca/(?P<id>(?:[0-9a-z]+-)+[0-9]{5,})/s01e(?P<media_id>[0-9]{5,})'
_TESTS = [{
'url': 'https://gem.cbc.ca/ski-jumping-nh-individual-womens-final-30086/s01e30086',
'info_dict': {
'id': 'ski-jumping-nh-individual-womens-final-30086',
'ext': 'mp4',
'title': 'Ski Jumping: NH Individual (Women\'s) - Final',
'description': 'md5:411c07c8a9a4a36344530b0c726bf8ab',
'duration': 12793,
'thumbnail': r're:https://[^.]+\.cbc\.ca/.+\.jpg',
'release_timestamp': 1770482100,
'release_date': '20260207',
'live_status': 'was_live',
},
}]
def _real_extract(self, url):
video_id, media_id = self._match_valid_url(url).group('id', 'media_id')
video_info = self._call_show_api(video_id)
item_info = traverse_obj(video_info, (
'content', ..., 'lineups', ..., 'items',
lambda _, v: v['formattedIdMedia'] == media_id, any, {require('item info')}))
live_status = {
'LiveEvent': 'is_live',
'Replay': 'was_live',
}.get(item_info.get('type'))
release_timestamp = traverse_obj(item_info, (
'metadata', (('live', 'startDate'), ('replay', 'airDate')), {parse_iso8601}, any))
if live_status == 'is_live' and release_timestamp and release_timestamp > time.time():
formats = []
live_status = 'is_upcoming'
self.raise_no_formats('This livestream has not yet started', expected=True)
else:
m3u8_url = self._call_media_api(media_id, 'medianetlive', video_id)['url']
formats = self._extract_m3u8_formats(m3u8_url, video_id, 'mp4', live=live_status == 'is_live')
return {
'id': video_id,
'formats': formats,
'live_status': live_status,
'release_timestamp': release_timestamp,
**traverse_obj(item_info, {
'title': ('title', {str}),
'description': ('description', {str}),
'thumbnail': ('images', 'card', 'url', {url_or_none}),
'duration': ('metadata', 'replay', 'duration', {int_or_none}),
}),
}
class CBCGemLiveIE(CBCGemBaseIE):
IE_NAME = 'gem.cbc.ca:live' IE_NAME = 'gem.cbc.ca:live'
_VALID_URL = r'https?://gem\.cbc\.ca/live(?:-event)?/(?P<id>\d+)' _VALID_URL = r'https?://gem\.cbc\.ca/live(?:-event)?/(?P<id>\d+)'
_TESTS = [ _TESTS = [
@@ -855,7 +1001,6 @@ class CBCGemLiveIE(InfoExtractor):
'only_matching': True, 'only_matching': True,
}, },
] ]
_GEO_COUNTRIES = ['CA']
def _real_extract(self, url): def _real_extract(self, url):
video_id = self._match_id(url) video_id = self._match_id(url)
@@ -884,19 +1029,8 @@ class CBCGemLiveIE(InfoExtractor):
live_status = 'is_upcoming' live_status = 'is_upcoming'
self.raise_no_formats('This livestream has not yet started', expected=True) self.raise_no_formats('This livestream has not yet started', expected=True)
else: else:
stream_data = self._download_json( m3u8_url = self._call_media_api(video_stream_id, 'medianetlive', video_id)['url']
'https://services.radio-canada.ca/media/validation/v2/', video_id, query={ formats = self._extract_m3u8_formats(m3u8_url, video_id, 'mp4', live=live_status == 'is_live')
'appCode': 'medianetlive',
'connectionType': 'hd',
'deviceType': 'ipad',
'idMedia': video_stream_id,
'multibitrate': 'true',
'output': 'json',
'tech': 'hls',
'manifestType': 'desktop',
})
formats = self._extract_m3u8_formats(
stream_data['url'], video_id, 'mp4', live=live_status == 'is_live')
return { return {
'id': video_id, 'id': video_id,

View File

@@ -18,23 +18,41 @@ class CCCIE(InfoExtractor):
'id': '1839', 'id': '1839',
'ext': 'mp4', 'ext': 'mp4',
'title': 'Introduction to Processor Design', 'title': 'Introduction to Processor Design',
'creator': 'byterazor', 'creators': ['byterazor'],
'description': 'md5:df55f6d073d4ceae55aae6f2fd98a0ac', 'description': 'md5:df55f6d073d4ceae55aae6f2fd98a0ac',
'thumbnail': r're:^https?://.*\.jpg$', 'thumbnail': r're:^https?://.*\.jpg$',
'upload_date': '20131228', 'upload_date': '20131228',
'timestamp': 1388188800, 'timestamp': 1388188800,
'duration': 3710, 'duration': 3710,
'tags': list, 'tags': list,
'display_id': '30C3_-_5443_-_en_-_saal_g_-_201312281830_-_introduction_to_processor_design_-_byterazor',
'view_count': int,
}, },
}, { }, {
'url': 'https://media.ccc.de/v/32c3-7368-shopshifting#download', 'url': 'https://media.ccc.de/v/32c3-7368-shopshifting#download',
'only_matching': True, 'only_matching': True,
}, {
'url': 'https://media.ccc.de/v/39c3-schlechte-karten-it-sicherheit-im-jahr-null-der-epa-fur-alle',
'info_dict': {
'id': '16261',
'ext': 'mp4',
'title': 'Schlechte Karten - IT-Sicherheit im Jahr null der ePA für alle',
'display_id': '39c3-schlechte-karten-it-sicherheit-im-jahr-null-der-epa-fur-alle',
'description': 'md5:719a5a9a52630249d606219c55056cbf',
'view_count': int,
'duration': 3619,
'thumbnail': 'https://static.media.ccc.de/media/congress/2025/2403-2b5a6a8e-327e-594d-8f92-b91201d18a02.jpg',
'tags': list,
'creators': ['Bianca Kastl'],
'timestamp': 1767024900,
'upload_date': '20251229',
},
}] }]
def _real_extract(self, url): def _real_extract(self, url):
display_id = self._match_id(url) display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id) webpage = self._download_webpage(url, display_id)
event_id = self._search_regex(r"data-id='(\d+)'", webpage, 'event id') event_id = self._search_regex(r"data-id=(['\"])(?P<event_id>\d+)\1", webpage, 'event id', group='event_id')
event_data = self._download_json(f'https://media.ccc.de/public/events/{event_id}', event_id) event_data = self._download_json(f'https://media.ccc.de/public/events/{event_id}', event_id)
formats = [] formats = []

View File

@@ -27,7 +27,7 @@ from ..utils.traversal import traverse_obj
class CDAIE(InfoExtractor): class CDAIE(InfoExtractor):
_VALID_URL = r'https?://(?:(?:www\.)?cda\.pl/video|ebd\.cda\.pl/[0-9]+x[0-9]+)/(?P<id>[0-9a-z]+)' _VALID_URL = r'https?://(?:(?:(?:www|m)\.)?cda\.pl/video|ebd\.cda\.pl/[0-9]+x[0-9]+)/(?P<id>[0-9a-z]+)'
_NETRC_MACHINE = 'cdapl' _NETRC_MACHINE = 'cdapl'
_BASE_URL = 'https://www.cda.pl' _BASE_URL = 'https://www.cda.pl'
@@ -110,6 +110,9 @@ class CDAIE(InfoExtractor):
}, { }, {
'url': 'http://ebd.cda.pl/0x0/5749950c', 'url': 'http://ebd.cda.pl/0x0/5749950c',
'only_matching': True, 'only_matching': True,
}, {
'url': 'https://m.cda.pl/video/617297677',
'only_matching': True,
}] }]
def _download_age_confirm_page(self, url, video_id, *args, **kwargs): def _download_age_confirm_page(self, url, video_id, *args, **kwargs):
@@ -367,35 +370,35 @@ class CDAIE(InfoExtractor):
class CDAFolderIE(InfoExtractor): class CDAFolderIE(InfoExtractor):
_MAX_PAGE_SIZE = 36 _MAX_PAGE_SIZE = 36
_VALID_URL = r'https?://(?:www\.)?cda\.pl/(?P<channel>[\w-]+)/folder/(?P<id>\d+)' _VALID_URL = r'https?://(?:(?:www|m)\.)?cda\.pl/(?P<channel>[\w-]+)/folder/(?P<id>\d+)'
_TESTS = [ _TESTS = [{
{ 'url': 'https://www.cda.pl/domino264/folder/31188385',
'url': 'https://www.cda.pl/domino264/folder/31188385', 'info_dict': {
'info_dict': { 'id': '31188385',
'id': '31188385', 'title': 'SERIA DRUGA',
'title': 'SERIA DRUGA',
},
'playlist_mincount': 13,
}, },
{ 'playlist_mincount': 13,
'url': 'https://www.cda.pl/smiechawaTV/folder/2664592/vfilm', }, {
'info_dict': { 'url': 'https://www.cda.pl/smiechawaTV/folder/2664592/vfilm',
'id': '2664592', 'info_dict': {
'title': 'VideoDowcipy - wszystkie odcinki', 'id': '2664592',
}, 'title': 'VideoDowcipy - wszystkie odcinki',
'playlist_mincount': 71,
}, },
{ 'playlist_mincount': 71,
'url': 'https://www.cda.pl/DeliciousBeauty/folder/19129979/vfilm', }, {
'info_dict': { 'url': 'https://www.cda.pl/DeliciousBeauty/folder/19129979/vfilm',
'id': '19129979', 'info_dict': {
'title': 'TESTY KOSMETYKÓW', 'id': '19129979',
}, 'title': 'TESTY KOSMETYKÓW',
'playlist_mincount': 139, },
}, { 'playlist_mincount': 139,
'url': 'https://www.cda.pl/FILMY-SERIALE-ANIME-KRESKOWKI-BAJKI/folder/18493422', }, {
'only_matching': True, 'url': 'https://www.cda.pl/FILMY-SERIALE-ANIME-KRESKOWKI-BAJKI/folder/18493422',
}] 'only_matching': True,
}, {
'url': 'https://m.cda.pl/smiechawaTV/folder/2664592/vfilm',
'only_matching': True,
}]
def _real_extract(self, url): def _real_extract(self, url):
folder_id, channel = self._match_valid_url(url).group('id', 'channel') folder_id, channel = self._match_valid_url(url).group('id', 'channel')

View File

@@ -348,6 +348,7 @@ class InfoExtractor:
duration: Length of the video in seconds, as an integer or float. duration: Length of the video in seconds, as an integer or float.
view_count: How many users have watched the video on the platform. view_count: How many users have watched the video on the platform.
concurrent_view_count: How many users are currently watching the video on the platform. concurrent_view_count: How many users are currently watching the video on the platform.
save_count: Number of times the video has been saved or bookmarked
like_count: Number of positive ratings of the video like_count: Number of positive ratings of the video
dislike_count: Number of negative ratings of the video dislike_count: Number of negative ratings of the video
repost_count: Number of reposts of the video repost_count: Number of reposts of the video
@@ -660,9 +661,11 @@ class InfoExtractor:
if not self._ready: if not self._ready:
self._initialize_pre_login() self._initialize_pre_login()
if self.supports_login(): if self.supports_login():
username, password = self._get_login_info() # try login only if it would actually do anything
if username: if type(self)._perform_login is not InfoExtractor._perform_login:
self._perform_login(username, password) username, password = self._get_login_info()
if username:
self._perform_login(username, password)
elif self.get_param('username') and False not in (self.IE_DESC, self._NETRC_MACHINE): elif self.get_param('username') and False not in (self.IE_DESC, self._NETRC_MACHINE):
self.report_warning(f'Login with password is not supported for this website. {self._login_hint("cookies")}') self.report_warning(f'Login with password is not supported for this website. {self._login_hint("cookies")}')
self._real_initialize() self._real_initialize()
@@ -1384,6 +1387,11 @@ class InfoExtractor:
def _get_netrc_login_info(self, netrc_machine=None): def _get_netrc_login_info(self, netrc_machine=None):
netrc_machine = netrc_machine or self._NETRC_MACHINE netrc_machine = netrc_machine or self._NETRC_MACHINE
if not netrc_machine:
raise ExtractorError(f'Missing netrc_machine and {type(self).__name__}._NETRC_MACHINE')
ALLOWED = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_'
if netrc_machine.startswith(('-', '_')) or not all(c in ALLOWED for c in netrc_machine):
raise ExtractorError(f'Invalid netrc machine: {netrc_machine!r}', expected=True)
cmd = self.get_param('netrc_cmd') cmd = self.get_param('netrc_cmd')
if cmd: if cmd:

View File

@@ -0,0 +1,79 @@
from .common import InfoExtractor
from .vimeo import VimeoIE
from ..utils import (
ExtractorError,
join_nonempty,
)
from ..utils.traversal import traverse_obj
class CroatianFilmIE(InfoExtractor):
IE_NAME = 'croatian.film'
_VALID_URL = r'https://?(?:www\.)?croatian\.film/[a-z]{2}/[^/?#]+/(?P<id>\d+)'
_GEO_COUNTRIES = ['HR']
_TESTS = [{
'url': 'https://www.croatian.film/hr/films/72472',
'info_dict': {
'id': '1078340774',
'ext': 'mp4',
'title': '“ŠKAFETIN”, r. Paško Vukasović',
'uploader': 'croatian.film',
'uploader_id': 'user94192658',
'uploader_url': 'https://vimeo.com/user94192658',
'duration': 1357,
'thumbnail': 'https://i.vimeocdn.com/video/2008556407-40eb1315ec11be5fcb8dda4d7059675b0881e182b9fc730892e267db72cb57f5-d',
},
'params': {'skip_download': 'm3u8'},
'expected_warnings': ['Failed to parse XML: not well-formed'],
}, {
# geo-restricted but works with xff
'url': 'https://www.croatian.film/en/films/77144',
'info_dict': {
'id': '1144997795',
'ext': 'mp4',
'title': '“ROKO” r. Ivana Marinić Kragić',
'uploader': 'croatian.film',
'uploader_id': 'user94192658',
'uploader_url': 'https://vimeo.com/user94192658',
'duration': 1023,
'thumbnail': 'https://i.vimeocdn.com/video/2093793231-11c2928698ff8347489e679b4d563a576e7acd0681ce95b383a9a25f6adb5e8f-d',
},
'params': {'skip_download': 'm3u8'},
'expected_warnings': ['Failed to parse XML: not well-formed'],
}, {
'url': 'https://www.croatian.film/en/films/75904/watch',
'info_dict': {
'id': '1134883757',
'ext': 'mp4',
'title': '"CARPE DIEM" r. Nina Damjanović',
'uploader': 'croatian.film',
'uploader_id': 'user94192658',
'uploader_url': 'https://vimeo.com/user94192658',
'duration': 1123,
'thumbnail': 'https://i.vimeocdn.com/video/2080022187-bb691c470c28c4d979258cf235e594bf9a11c14b837a0784326c25c95edd83f9-d',
},
'params': {'skip_download': 'm3u8'},
'expected_warnings': ['Failed to parse XML: not well-formed'],
}]
def _real_extract(self, url):
display_id = self._match_id(url)
api_data = self._download_json(
f'https://api.croatian.film/api/videos/{display_id}',
display_id)
if errors := traverse_obj(api_data, ('errors', lambda _, v: v['code'])):
codes = traverse_obj(errors, (..., 'code', {str}))
if 'INVALID_COUNTRY' in codes:
self.raise_geo_restricted(countries=self._GEO_COUNTRIES)
raise ExtractorError(join_nonempty(
*(traverse_obj(errors, (..., 'details', {str})) or codes),
delim='; '))
vimeo_id = self._search_regex(
r'/videos/(\d+)', api_data['video']['vimeoURL'], 'vimeo ID')
return self.url_result(
VimeoIE._smuggle_referrer(f'https://player.vimeo.com/video/{vimeo_id}', url),
VimeoIE, vimeo_id)

View File

@@ -1,5 +1,6 @@
import functools import functools
import json import json
import random
import re import re
import urllib.parse import urllib.parse
@@ -363,6 +364,55 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
continue continue
yield update_url(player_url, query=query_string) yield update_url(player_url, query=query_string)
@staticmethod
def _generate_blockbuster_headers():
"""Randomize our HTTP header fingerprint to bust the HTTP Error 403 block"""
def random_letters(minimum, maximum):
# Omit vowels so we don't generate valid header names like 'authorization', etc
return ''.join(random.choices('bcdfghjklmnpqrstvwxz', k=random.randint(minimum, maximum)))
return {
random_letters(8, 24): random_letters(16, 32)
for _ in range(random.randint(2, 8))
}
def _extract_dailymotion_m3u8_formats_and_subtitles(self, media_url, video_id, live=False):
"""See https://github.com/yt-dlp/yt-dlp/issues/15526"""
ERROR_NOTE = 'Unable to download m3u8 information'
last_error = None
for note, kwargs in (
('Downloading m3u8 information with randomized headers', {
'headers': self._generate_blockbuster_headers(),
}),
('Retrying m3u8 download with Chrome impersonation', {
'impersonate': 'chrome',
'require_impersonation': True,
}),
('Retrying m3u8 download with Firefox impersonation', {
'impersonate': 'firefox',
'require_impersonation': True,
}),
):
try:
m3u8_doc = self._download_webpage(media_url, video_id, note, ERROR_NOTE, **kwargs)
break
except ExtractorError as e:
last_error = e.orig_msg
self.write_debug(f'{video_id}: {last_error}')
else:
if 'impersonation' not in last_error:
self.report_warning(last_error, video_id=video_id)
last_error = None
return [], {}, last_error
formats, subtitles = self._parse_m3u8_formats_and_subtitles(
m3u8_doc, media_url, 'mp4', m3u8_id='hls', live=live, fatal=False)
return formats, subtitles, last_error
def _real_extract(self, url): def _real_extract(self, url):
url, smuggled_data = unsmuggle_url(url) url, smuggled_data = unsmuggle_url(url)
video_id, is_playlist, playlist_id = self._match_valid_url(url).group('id', 'is_playlist', 'playlist_id') video_id, is_playlist, playlist_id = self._match_valid_url(url).group('id', 'is_playlist', 'playlist_id')
@@ -416,6 +466,7 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
is_live = media.get('isOnAir') is_live = media.get('isOnAir')
formats = [] formats = []
subtitles = {} subtitles = {}
expected_error = None
for quality, media_list in metadata['qualities'].items(): for quality, media_list in metadata['qualities'].items():
for m in media_list: for m in media_list:
@@ -424,8 +475,8 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
if not media_url or media_type == 'application/vnd.lumberjack.manifest': if not media_url or media_type == 'application/vnd.lumberjack.manifest':
continue continue
if media_type == 'application/x-mpegURL': if media_type == 'application/x-mpegURL':
fmt, subs = self._extract_m3u8_formats_and_subtitles( fmt, subs, expected_error = self._extract_dailymotion_m3u8_formats_and_subtitles(
media_url, video_id, 'mp4', live=is_live, m3u8_id='hls', fatal=False) media_url, video_id, live=is_live)
formats.extend(fmt) formats.extend(fmt)
self._merge_subtitles(subs, target=subtitles) self._merge_subtitles(subs, target=subtitles)
else: else:
@@ -442,6 +493,10 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
'width': width, 'width': width,
}) })
formats.append(f) formats.append(f)
if not formats and expected_error:
self.raise_no_formats(expected_error, expected=True)
for f in formats: for f in formats:
f['url'] = f['url'].split('#')[0] f['url'] = f['url'].split('#')[0]
if not f.get('fps') and f['format_id'].endswith('@60'): if not f.get('fps') and f['format_id'].endswith('@60'):

View File

@@ -14,7 +14,7 @@ from ..utils import (
class DropboxIE(InfoExtractor): class DropboxIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?dropbox\.com/(?:(?:e/)?scl/fi|sh?)/(?P<id>\w+)' _VALID_URL = r'https?://(?:www\.)?dropbox\.com/(?:(?:e/)?scl/f[io]|sh?)/(?P<id>\w+)'
_TESTS = [ _TESTS = [
{ {
'url': 'https://www.dropbox.com/s/nelirfsxnmcfbfh/youtube-dl%20test%20video%20%27%C3%A4%22BaW_jenozKc.mp4?dl=0', 'url': 'https://www.dropbox.com/s/nelirfsxnmcfbfh/youtube-dl%20test%20video%20%27%C3%A4%22BaW_jenozKc.mp4?dl=0',
@@ -35,6 +35,9 @@ class DropboxIE(InfoExtractor):
}, { }, {
'url': 'https://www.dropbox.com/e/scl/fi/r2kd2skcy5ylbbta5y1pz/DJI_0003.MP4?dl=0&rlkey=wcdgqangn7t3lnmmv6li9mu9h', 'url': 'https://www.dropbox.com/e/scl/fi/r2kd2skcy5ylbbta5y1pz/DJI_0003.MP4?dl=0&rlkey=wcdgqangn7t3lnmmv6li9mu9h',
'only_matching': True, 'only_matching': True,
}, {
'url': 'https://www.dropbox.com/scl/fo/zjfqse5txqfd7twa8iewj/AOfZzSYWUSKle2HD7XF7kzQ/A-BEAT%20C.mp4?rlkey=6tg3jkp4tv6a5vt58a6dag0mm&dl=0',
'only_matching': True,
}, },
] ]

View File

@@ -2,6 +2,7 @@ from .common import InfoExtractor
from ..utils import ( from ..utils import (
clean_html, clean_html,
int_or_none, int_or_none,
parse_iso8601,
str_or_none, str_or_none,
url_or_none, url_or_none,
) )
@@ -222,3 +223,70 @@ class ERRJupiterIE(InfoExtractor):
'episode_id': ('id', {str_or_none}), 'episode_id': ('id', {str_or_none}),
}) if data.get('type') == 'episode' else {}), }) if data.get('type') == 'episode' else {}),
} }
class ERRArhiivIE(InfoExtractor):
_VALID_URL = r'https://arhiiv\.err\.ee/video/(?:vaata/)?(?P<id>[^/?#]+)'
_TESTS = [{
'url': 'https://arhiiv.err.ee/video/kontsertpalad',
'info_dict': {
'id': 'kontsertpalad',
'ext': 'mp4',
'title': 'Kontsertpalad: 255 | L. Beethoveni sonaat c-moll, "Pateetiline"',
'description': 'md5:a70f4ff23c3618f3be63f704bccef063',
'series': 'Kontsertpalad',
'episode_id': 255,
'timestamp': 1666152162,
'upload_date': '20221019',
'release_year': 1970,
'modified_timestamp': 1718620982,
'modified_date': '20240617',
},
'params': {'skip_download': 'm3u8'},
}, {
'url': 'https://arhiiv.err.ee/video/vaata/koalitsioonileppe-allkirjastamine',
'info_dict': {
'id': 'koalitsioonileppe-allkirjastamine',
'ext': 'mp4',
'title': 'Koalitsioonileppe allkirjastamine',
'timestamp': 1710728222,
'upload_date': '20240318',
'release_timestamp': 1611532800,
'release_date': '20210125',
},
'params': {'skip_download': 'm3u8'},
}]
def _real_extract(self, url):
video_id = self._match_id(url)
data = self._download_json(
f'https://arhiiv.err.ee/api/v1/content/video/{video_id}', video_id)
formats, subtitles = [], {}
if hls_url := traverse_obj(data, ('media', 'src', 'hls', {url_or_none})):
fmts, subs = self._extract_m3u8_formats_and_subtitles(
hls_url, video_id, 'mp4', m3u8_id='hls', fatal=False)
formats.extend(fmts)
self._merge_subtitles(subs, target=subtitles)
if dash_url := traverse_obj(data, ('media', 'src', 'dash', {url_or_none})):
fmts, subs = self._extract_mpd_formats_and_subtitles(
dash_url, video_id, mpd_id='dash', fatal=False)
formats.extend(fmts)
self._merge_subtitles(subs, target=subtitles)
return {
'id': video_id,
'formats': formats,
'subtitles': subtitles,
**traverse_obj(data, ('info', {
'title': ('title', {str}),
'series': ('seriesTitle', {str}, filter),
'series_id': ('seriesId', {str}, filter),
'episode_id': ('episode', {int_or_none}),
'description': ('synopsis', {str}, filter),
'timestamp': ('uploadDate', {parse_iso8601}),
'modified_timestamp': ('dateModified', {parse_iso8601}),
'release_timestamp': ('date', {parse_iso8601}),
'release_year': ('year', {int_or_none}),
})),
}

View File

@@ -1,4 +1,4 @@
import inspect import itertools
import os import os
from ..globals import LAZY_EXTRACTORS from ..globals import LAZY_EXTRACTORS
@@ -17,12 +17,18 @@ else:
if not _CLASS_LOOKUP: if not _CLASS_LOOKUP:
from . import _extractors from . import _extractors
_CLASS_LOOKUP = { members = tuple(
name: value (name, getattr(_extractors, name))
for name, value in inspect.getmembers(_extractors) for name in dir(_extractors)
if name.endswith('IE') and name != 'GenericIE' if name.endswith('IE')
} )
_CLASS_LOOKUP['GenericIE'] = _extractors.GenericIE _CLASS_LOOKUP = dict(itertools.chain(
# Add Youtube first to improve matching performance
((name, value) for name, value in members if '.youtube' in value.__module__),
# Add Generic last so that it is the fallback
((name, value) for name, value in members if name != 'GenericIE'),
(('GenericIE', _extractors.GenericIE),),
))
# We want to append to the main lookup # We want to append to the main lookup
_current = _extractors_context.value _current = _extractors_context.value

View File

@@ -4,8 +4,7 @@ import urllib.parse
from .common import InfoExtractor from .common import InfoExtractor
from ..compat import compat_etree_fromstring from ..compat import compat_etree_fromstring
from ..networking import Request from ..networking.exceptions import HTTPError
from ..networking.exceptions import network_exceptions
from ..utils import ( from ..utils import (
ExtractorError, ExtractorError,
clean_html, clean_html,
@@ -64,9 +63,6 @@ class FacebookIE(InfoExtractor):
class=(?P<q1>[\'"])[^\'"]*\bfb-(?:video|post)\b[^\'"]*(?P=q1)[^>]+ class=(?P<q1>[\'"])[^\'"]*\bfb-(?:video|post)\b[^\'"]*(?P=q1)[^>]+
data-href=(?P<q2>[\'"])(?P<url>(?:https?:)?//(?:www\.)?facebook.com/.+?)(?P=q2)''', data-href=(?P<q2>[\'"])(?P<url>(?:https?:)?//(?:www\.)?facebook.com/.+?)(?P=q2)''',
] ]
_LOGIN_URL = 'https://www.facebook.com/login.php?next=http%3A%2F%2Ffacebook.com%2Fhome.php&login_attempt=1'
_CHECKPOINT_URL = 'https://www.facebook.com/checkpoint/?next=http%3A%2F%2Ffacebook.com%2Fhome.php&_fb_noscript=1'
_NETRC_MACHINE = 'facebook'
IE_NAME = 'facebook' IE_NAME = 'facebook'
_VIDEO_PAGE_TEMPLATE = 'https://www.facebook.com/video/video.php?v=%s' _VIDEO_PAGE_TEMPLATE = 'https://www.facebook.com/video/video.php?v=%s'
@@ -469,65 +465,6 @@ class FacebookIE(InfoExtractor):
'graphURI': '/api/graphql/', 'graphURI': '/api/graphql/',
} }
def _perform_login(self, username, password):
login_page_req = Request(self._LOGIN_URL)
self._set_cookie('facebook.com', 'locale', 'en_US')
login_page = self._download_webpage(login_page_req, None,
note='Downloading login page',
errnote='Unable to download login page')
lsd = self._search_regex(
r'<input type="hidden" name="lsd" value="([^"]*)"',
login_page, 'lsd')
lgnrnd = self._search_regex(r'name="lgnrnd" value="([^"]*?)"', login_page, 'lgnrnd')
login_form = {
'email': username,
'pass': password,
'lsd': lsd,
'lgnrnd': lgnrnd,
'next': 'http://facebook.com/home.php',
'default_persistent': '0',
'legacy_return': '1',
'timezone': '-60',
'trynum': '1',
}
request = Request(self._LOGIN_URL, urlencode_postdata(login_form))
request.headers['Content-Type'] = 'application/x-www-form-urlencoded'
try:
login_results = self._download_webpage(request, None,
note='Logging in', errnote='unable to fetch login page')
if re.search(r'<form(.*)name="login"(.*)</form>', login_results) is not None:
error = self._html_search_regex(
r'(?s)<div[^>]+class=(["\']).*?login_error_box.*?\1[^>]*><div[^>]*>.*?</div><div[^>]*>(?P<error>.+?)</div>',
login_results, 'login error', default=None, group='error')
if error:
raise ExtractorError(f'Unable to login: {error}', expected=True)
self.report_warning('unable to log in: bad username/password, or exceeded login rate limit (~3/min). Check credentials or wait.')
return
fb_dtsg = self._search_regex(
r'name="fb_dtsg" value="(.+?)"', login_results, 'fb_dtsg', default=None)
h = self._search_regex(
r'name="h"\s+(?:\w+="[^"]+"\s+)*?value="([^"]+)"', login_results, 'h', default=None)
if not fb_dtsg or not h:
return
check_form = {
'fb_dtsg': fb_dtsg,
'h': h,
'name_action_selected': 'dont_save',
}
check_req = Request(self._CHECKPOINT_URL, urlencode_postdata(check_form))
check_req.headers['Content-Type'] = 'application/x-www-form-urlencoded'
check_response = self._download_webpage(check_req, None,
note='Confirming login')
if re.search(r'id="checkpointSubmitButton"', check_response) is not None:
self.report_warning('Unable to confirm login, you have to login in your browser and authorize the login.')
except network_exceptions as err:
self.report_warning(f'unable to log in: {err}')
return
def _extract_from_url(self, url, video_id): def _extract_from_url(self, url, video_id):
webpage = self._download_webpage( webpage = self._download_webpage(
url.replace('://m.facebook.com/', '://www.facebook.com/'), video_id) url.replace('://m.facebook.com/', '://www.facebook.com/'), video_id)
@@ -1081,6 +1018,7 @@ class FacebookAdsIE(InfoExtractor):
'upload_date': '20240812', 'upload_date': '20240812',
'like_count': int, 'like_count': int,
}, },
'skip': 'Invalid URL',
}, { }, {
'url': 'https://www.facebook.com/ads/library/?id=893637265423481', 'url': 'https://www.facebook.com/ads/library/?id=893637265423481',
'info_dict': { 'info_dict': {
@@ -1095,6 +1033,42 @@ class FacebookAdsIE(InfoExtractor):
}, },
'playlist_count': 3, 'playlist_count': 3,
'skip': 'Invalid URL', 'skip': 'Invalid URL',
}, {
'url': 'https://www.facebook.com/ads/library/?id=312304267031140',
'info_dict': {
'id': '312304267031140',
'title': 'Casper Wave Hybrid Mattress',
'uploader': 'Casper',
'uploader_id': '224110981099062',
'uploader_url': 'https://www.facebook.com/Casper/',
'like_count': int,
},
'playlist_count': 2,
}, {
'url': 'https://www.facebook.com/ads/library/?id=874812092000430',
'info_dict': {
'id': '874812092000430',
'title': 'TikTok',
'uploader': 'Case \u00e0 Chocs',
'uploader_id': '112960472096793',
'uploader_url': 'https://www.facebook.com/Caseachocs/',
'like_count': int,
'description': 'md5:f02a255fcf7dce6ed40e9494cf4bc49a',
},
'playlist_count': 3,
}, {
'url': 'https://www.facebook.com/ads/library/?id=1704834754236452',
'info_dict': {
'id': '1704834754236452',
'ext': 'mp4',
'title': 'Get answers now!',
'description': 'Ask the best psychics and get accurate answers on questions that bother you!',
'uploader': 'Your Relationship Advisor',
'uploader_id': '108939234726306',
'uploader_url': 'https://www.facebook.com/100068970634636/',
'like_count': int,
'thumbnail': r're:https://.+/.+\.jpg',
},
}, { }, {
'url': 'https://es-la.facebook.com/ads/library/?id=901230958115569', 'url': 'https://es-la.facebook.com/ads/library/?id=901230958115569',
'only_matching': True, 'only_matching': True,
@@ -1124,15 +1098,45 @@ class FacebookAdsIE(InfoExtractor):
}) })
return formats return formats
def _download_fb_webpage_and_verify(self, url, video_id):
# See https://github.com/yt-dlp/yt-dlp/issues/15577
try:
return self._download_webpage(url, video_id)
except ExtractorError as e:
if (
not isinstance(e.cause, HTTPError)
or e.cause.status != 403
or e.cause.reason != 'Client challenge'
):
raise
error_page = self._webpage_read_content(e.cause.response, url, video_id)
self.write_debug('Received a client challenge response')
challenge_path = self._search_regex(
r'fetch\s*\(\s*["\'](/__rd_verify[^"\']+)["\']',
error_page, 'challenge path')
# Successful response will set the necessary cookie
self._request_webpage(
urljoin(url, challenge_path), video_id, 'Requesting verification cookie',
'Unable to get verification cookie', data=b'')
return self._download_webpage(url, video_id)
def _real_extract(self, url): def _real_extract(self, url):
video_id = self._match_id(url) video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id) webpage = self._download_fb_webpage_and_verify(url, video_id)
post_data = traverse_obj( post_data = traverse_obj(
re.findall(r'data-sjs>({.*?ScheduledServerJS.*?})</script>', webpage), (..., {json.loads})) re.findall(r'data-sjs>({.*?ScheduledServerJS.*?})</script>', webpage), (..., {json.loads}))
data = get_first(post_data, ( data = get_first(post_data, (
'require', ..., ..., ..., '__bbox', 'require', ..., ..., ..., 'require', ..., ..., ..., '__bbox', 'require', ..., ..., ..., (
'entryPointRoot', 'otherProps', 'deeplinkAdCard', 'snapshot', {dict})) ('__bbox', 'result', 'data', 'ad_library_main', 'deeplink_ad_archive_result', 'deeplink_ad_archive'),
# old path
('entryPointRoot', 'otherProps', 'deeplinkAdCard'),
), 'snapshot', {dict}))
if not data: if not data:
raise ExtractorError('Unable to extract ad data') raise ExtractorError('Unable to extract ad data')
@@ -1148,11 +1152,12 @@ class FacebookAdsIE(InfoExtractor):
'title': title, 'title': title,
'description': markup or None, 'description': markup or None,
}, traverse_obj(data, { }, traverse_obj(data, {
'description': ('link_description', {lambda x: x if not x.startswith('{{product.') else None}), 'description': (
(('body', 'text'), 'link_description'),
{lambda x: x if not x.startswith('{{product.') else None}, any),
'uploader': ('page_name', {str}), 'uploader': ('page_name', {str}),
'uploader_id': ('page_id', {str_or_none}), 'uploader_id': ('page_id', {str_or_none}),
'uploader_url': ('page_profile_uri', {url_or_none}), 'uploader_url': ('page_profile_uri', {url_or_none}),
'timestamp': ('creation_time', {int_or_none}),
'like_count': ('page_like_count', {int_or_none}), 'like_count': ('page_like_count', {int_or_none}),
})) }))
@@ -1163,7 +1168,8 @@ class FacebookAdsIE(InfoExtractor):
entries.append({ entries.append({
'id': f'{video_id}_{idx}', 'id': f'{video_id}_{idx}',
'title': entry.get('title') or title, 'title': entry.get('title') or title,
'description': traverse_obj(entry, 'body', 'link_description') or info_dict.get('description'), 'description': traverse_obj(
entry, 'body', 'link_description', expected_type=str) or info_dict.get('description'),
'thumbnail': url_or_none(entry.get('video_preview_image_url')), 'thumbnail': url_or_none(entry.get('video_preview_image_url')),
'formats': self._extract_formats(entry), 'formats': self._extract_formats(entry),
}) })

View File

@@ -0,0 +1,52 @@
from .common import InfoExtractor
from ..utils import clean_html
from ..utils.traversal import (
find_element,
find_elements,
traverse_obj,
)
class FilmArchivIE(InfoExtractor):
IE_DESC = 'FILMARCHIV ON'
_VALID_URL = r'https?://(?:www\.)?filmarchiv\.at/de/filmarchiv-on/video/(?P<id>f_[0-9a-zA-Z]{5,})'
_TESTS = [{
'url': 'https://www.filmarchiv.at/de/filmarchiv-on/video/f_0305p7xKrXUPBwoNE9x6mh',
'md5': '54a6596f6a84624531866008a77fa27a',
'info_dict': {
'id': 'f_0305p7xKrXUPBwoNE9x6mh',
'ext': 'mp4',
'title': 'Der Wurstelprater zur Kaiserzeit',
'description': 'md5:9843f92df5cc9a4975cee7aabcf6e3b2',
'thumbnail': r're:https://cdn\.filmarchiv\.at/f_0305/p7xKrXUPBwoNE9x6mh_v1/poster\.jpg',
},
}, {
'url': 'https://www.filmarchiv.at/de/filmarchiv-on/video/f_0306vI3wO0tJIsfrqYFQXF',
'md5': '595385d7f54cb6529140ee8de7d1c3c7',
'info_dict': {
'id': 'f_0306vI3wO0tJIsfrqYFQXF',
'ext': 'mp4',
'title': 'Vor 70 Jahren: Wettgehen der Briefträger in Wien',
'description': 'md5:b2a2e4230923cd1969d471c552e62811',
'thumbnail': r're:https://cdn\.filmarchiv\.at/f_0306/vI3wO0tJIsfrqYFQXF_v1/poster\.jpg',
},
}]
def _real_extract(self, url):
media_id = self._match_id(url)
webpage = self._download_webpage(url, media_id)
path = '/'.join((media_id[:6], media_id[6:]))
formats, subtitles = self._extract_m3u8_formats_and_subtitles(
f'https://cdn.filmarchiv.at/{path}_v1_sv1/playlist.m3u8', media_id)
return {
'id': media_id,
'title': traverse_obj(webpage, ({find_element(tag='title-div')}, {clean_html})),
'description': traverse_obj(webpage, (
{find_elements(tag='div', attr='class', value=r'.*\bborder-base-content\b', regex=True)}, ...,
{find_elements(tag='div', attr='class', value=r'.*\bprose\b', html=False, regex=True)}, ...,
{clean_html}, any)),
'thumbnail': f'https://cdn.filmarchiv.at/{path}_v1/poster.jpg',
'formats': formats,
'subtitles': subtitles,
}

View File

@@ -3,10 +3,12 @@ import urllib.parse
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ( from ..utils import (
determine_ext, determine_ext,
float_or_none,
int_or_none, int_or_none,
join_nonempty, join_nonempty,
mimetype2ext, mimetype2ext,
parse_qs, parse_qs,
unescapeHTML,
unified_strdate, unified_strdate,
url_or_none, url_or_none,
) )
@@ -107,6 +109,11 @@ class FirstTVIE(InfoExtractor):
'timestamp': ('dvr_begin_at', {int_or_none}), 'timestamp': ('dvr_begin_at', {int_or_none}),
'upload_date': ('date_air', {unified_strdate}), 'upload_date': ('date_air', {unified_strdate}),
'duration': ('duration', {int_or_none}), 'duration': ('duration', {int_or_none}),
'chapters': ('episodes', lambda _, v: float_or_none(v['from']) is not None, {
'start_time': ('from', {float_or_none}),
'title': ('name', {str}, {unescapeHTML}),
'end_time': ('to', {float_or_none}),
}),
}), }),
'id': video_id, 'id': video_id,
'formats': formats, 'formats': formats,

View File

@@ -318,9 +318,48 @@ class FloatplaneIE(FloatplaneBaseIE):
self.raise_login_required() self.raise_login_required()
class FloatplaneChannelIE(InfoExtractor): class FloatplaneChannelBaseIE(InfoExtractor):
"""Subclasses must set _RESULT_IE, _BASE_URL and _PAGE_SIZE"""
def _fetch_page(self, display_id, creator_id, channel_id, page):
query = {
'id': creator_id,
'limit': self._PAGE_SIZE,
'fetchAfter': page * self._PAGE_SIZE,
}
if channel_id:
query['channel'] = channel_id
page_data = self._download_json(
f'{self._BASE_URL}/api/v3/content/creator', display_id,
query=query, note=f'Downloading page {page + 1}')
for post in page_data or []:
yield self.url_result(
f'{self._BASE_URL}/post/{post["id"]}',
self._RESULT_IE, id=post['id'], title=post.get('title'),
release_timestamp=parse_iso8601(post.get('releaseDate')))
def _real_extract(self, url):
creator, channel = self._match_valid_url(url).group('id', 'channel')
display_id = join_nonempty(creator, channel, delim='/')
creator_data = self._download_json(
f'{self._BASE_URL}/api/v3/creator/named',
display_id, query={'creatorURL[0]': creator})[0]
channel_data = traverse_obj(
creator_data, ('channels', lambda _, v: v['urlname'] == channel), get_all=False) or {}
return self.playlist_result(OnDemandPagedList(functools.partial(
self._fetch_page, display_id, creator_data['id'], channel_data.get('id')), self._PAGE_SIZE),
display_id, title=channel_data.get('title') or creator_data.get('title'),
description=channel_data.get('about') or creator_data.get('about'))
class FloatplaneChannelIE(FloatplaneChannelBaseIE):
_VALID_URL = r'https?://(?:(?:www|beta)\.)?floatplane\.com/channel/(?P<id>[\w-]+)/home(?:/(?P<channel>[\w-]+))?' _VALID_URL = r'https?://(?:(?:www|beta)\.)?floatplane\.com/channel/(?P<id>[\w-]+)/home(?:/(?P<channel>[\w-]+))?'
_BASE_URL = 'https://www.floatplane.com'
_PAGE_SIZE = 20 _PAGE_SIZE = 20
_RESULT_IE = FloatplaneIE
_TESTS = [{ _TESTS = [{
'url': 'https://www.floatplane.com/channel/linustechtips/home/ltxexpo', 'url': 'https://www.floatplane.com/channel/linustechtips/home/ltxexpo',
'info_dict': { 'info_dict': {
@@ -346,36 +385,3 @@ class FloatplaneChannelIE(InfoExtractor):
}, },
'playlist_mincount': 200, 'playlist_mincount': 200,
}] }]
def _fetch_page(self, display_id, creator_id, channel_id, page):
query = {
'id': creator_id,
'limit': self._PAGE_SIZE,
'fetchAfter': page * self._PAGE_SIZE,
}
if channel_id:
query['channel'] = channel_id
page_data = self._download_json(
'https://www.floatplane.com/api/v3/content/creator', display_id,
query=query, note=f'Downloading page {page + 1}')
for post in page_data or []:
yield self.url_result(
f'https://www.floatplane.com/post/{post["id"]}',
FloatplaneIE, id=post['id'], title=post.get('title'),
release_timestamp=parse_iso8601(post.get('releaseDate')))
def _real_extract(self, url):
creator, channel = self._match_valid_url(url).group('id', 'channel')
display_id = join_nonempty(creator, channel, delim='/')
creator_data = self._download_json(
'https://www.floatplane.com/api/v3/creator/named',
display_id, query={'creatorURL[0]': creator})[0]
channel_data = traverse_obj(
creator_data, ('channels', lambda _, v: v['urlname'] == channel), get_all=False) or {}
return self.playlist_result(OnDemandPagedList(functools.partial(
self._fetch_page, display_id, creator_data['id'], channel_data.get('id')), self._PAGE_SIZE),
display_id, title=channel_data.get('title') or creator_data.get('title'),
description=channel_data.get('about') or creator_data.get('about'))

View File

@@ -371,15 +371,16 @@ class FranceTVSiteIE(FranceTVBaseInfoExtractor):
class FranceTVInfoIE(FranceTVBaseInfoExtractor): class FranceTVInfoIE(FranceTVBaseInfoExtractor):
IE_NAME = 'francetvinfo.fr' IE_NAME = 'franceinfo'
_VALID_URL = r'https?://(?:www|mobile|france3-regions)\.francetvinfo\.fr/(?:[^/]+/)*(?P<id>[^/?#&.]+)' IE_DESC = 'franceinfo.fr (formerly francetvinfo.fr)'
_VALID_URL = r'https?://(?:www|mobile|france3-regions)\.france(?:tv)?info.fr/(?:[^/?#]+/)*(?P<id>[^/?#&.]+)'
_TESTS = [{ _TESTS = [{
'url': 'https://www.francetvinfo.fr/replay-jt/france-3/soir-3/jt-grand-soir-3-jeudi-22-aout-2019_3561461.html', 'url': 'https://www.francetvinfo.fr/replay-jt/france-3/soir-3/jt-grand-soir-3-jeudi-22-aout-2019_3561461.html',
'info_dict': { 'info_dict': {
'id': 'd12458ee-5062-48fe-bfdd-a30d6a01b793', 'id': 'd12458ee-5062-48fe-bfdd-a30d6a01b793',
'ext': 'mp4', 'ext': 'mp4',
'title': 'Soir 3', 'title': 'Soir 3 - Émission du jeudi 22 août 2019',
'upload_date': '20190822', 'upload_date': '20190822',
'timestamp': 1566510730, 'timestamp': 1566510730,
'thumbnail': r're:^https?://.*\.jpe?g$', 'thumbnail': r're:^https?://.*\.jpe?g$',
@@ -398,7 +399,7 @@ class FranceTVInfoIE(FranceTVBaseInfoExtractor):
'info_dict': { 'info_dict': {
'id': '7d204c9e-a2d3-11eb-9e4c-000d3a23d482', 'id': '7d204c9e-a2d3-11eb-9e4c-000d3a23d482',
'ext': 'mp4', 'ext': 'mp4',
'title': 'Covid-19 : une situation catastrophique à New Dehli - Édition du mercredi 21 avril 2021', 'title': 'Journal 20h00 - Covid-19 : une situation catastrophique à New Dehli',
'thumbnail': r're:^https?://.*\.jpe?g$', 'thumbnail': r're:^https?://.*\.jpe?g$',
'duration': 76, 'duration': 76,
'timestamp': 1619028518, 'timestamp': 1619028518,
@@ -438,6 +439,18 @@ class FranceTVInfoIE(FranceTVBaseInfoExtractor):
'thumbnail': r're:https://[^/?#]+/v/[^/?#]+/x1080', 'thumbnail': r're:https://[^/?#]+/v/[^/?#]+/x1080',
}, },
'add_ie': ['Dailymotion'], 'add_ie': ['Dailymotion'],
'skip': 'Broken Dailymotion link',
}, {
'url': 'https://www.franceinfo.fr/monde/usa/presidentielle/donald-trump/etats-unis-un-risque-d-embrasement-apres-la-mort-d-un-manifestant_7764542.html',
'info_dict': {
'id': 'f920fcc2-fa20-11f0-ac98-57a09c50f7ce',
'ext': 'mp4',
'title': 'Affaires sensibles - Manifestant tué Le risque d\'embrasement',
'duration': 118,
'thumbnail': r're:https?://.+/.+\.jpg',
'timestamp': 1769367756,
'upload_date': '20260125',
},
}, { }, {
'url': 'http://france3-regions.francetvinfo.fr/limousin/emissions/jt-1213-limousin', 'url': 'http://france3-regions.francetvinfo.fr/limousin/emissions/jt-1213-limousin',
'only_matching': True, 'only_matching': True,
@@ -445,6 +458,9 @@ class FranceTVInfoIE(FranceTVBaseInfoExtractor):
# "<figure id=" pattern (#28792) # "<figure id=" pattern (#28792)
'url': 'https://www.francetvinfo.fr/culture/patrimoine/incendie-de-notre-dame-de-paris/notre-dame-de-paris-de-l-incendie-de-la-cathedrale-a-sa-reconstruction_4372291.html', 'url': 'https://www.francetvinfo.fr/culture/patrimoine/incendie-de-notre-dame-de-paris/notre-dame-de-paris-de-l-incendie-de-la-cathedrale-a-sa-reconstruction_4372291.html',
'only_matching': True, 'only_matching': True,
}, {
'url': 'https://www.franceinfo.fr/replay-jt/france-2/20-heures/robert-de-niro-portrait-d-un-monument-du-cinema_7245456.html',
'only_matching': True,
}] }]
def _real_extract(self, url): def _real_extract(self, url):
@@ -460,7 +476,7 @@ class FranceTVInfoIE(FranceTVBaseInfoExtractor):
video_id = ( video_id = (
traverse_obj(webpage, ( traverse_obj(webpage, (
{find_element(tag='button', attr='data-cy', value='francetv-player-wrapper', html=True)}, {find_element(tag='(button|div)', attr='data-cy', value='francetv-player-wrapper', html=True, regex=True)},
{extract_attributes}, 'id')) {extract_attributes}, 'id'))
or self._search_regex( or self._search_regex(
(r'player\.load[^;]+src:\s*["\']([^"\']+)', (r'player\.load[^;]+src:\s*["\']([^"\']+)',

View File

@@ -104,9 +104,9 @@ class FrontroGroupBaseIE(FrontoBaseIE):
class TheChosenIE(FrontroVideoBaseIE): class TheChosenIE(FrontroVideoBaseIE):
_CHANNEL_ID = '12884901895' _CHANNEL_ID = '12884901895'
_VALID_URL = r'https?://(?:www\.)?watch\.thechosen\.tv/video/(?P<id>[0-9]+)' _VALID_URL = r'https?://(?:www\.)?watch\.thechosen\.tv/watch/(?P<id>[0-9]+)'
_TESTS = [{ _TESTS = [{
'url': 'https://watch.thechosen.tv/video/184683594325', 'url': 'https://watch.thechosen.tv/watch/184683594325',
'md5': '3f878b689588c71b38ec9943c54ff5b0', 'md5': '3f878b689588c71b38ec9943c54ff5b0',
'info_dict': { 'info_dict': {
'id': '184683594325', 'id': '184683594325',
@@ -124,7 +124,7 @@ class TheChosenIE(FrontroVideoBaseIE):
'modified_date': str, 'modified_date': str,
}, },
}, { }, {
'url': 'https://watch.thechosen.tv/video/184683596189', 'url': 'https://watch.thechosen.tv/watch/184683596189',
'md5': 'd581562f9d29ce82f5b7770415334151', 'md5': 'd581562f9d29ce82f5b7770415334151',
'info_dict': { 'info_dict': {
'id': '184683596189', 'id': '184683596189',
@@ -147,7 +147,7 @@ class TheChosenIE(FrontroVideoBaseIE):
class TheChosenGroupIE(FrontroGroupBaseIE): class TheChosenGroupIE(FrontroGroupBaseIE):
_CHANNEL_ID = '12884901895' _CHANNEL_ID = '12884901895'
_VIDEO_EXTRACTOR = TheChosenIE _VIDEO_EXTRACTOR = TheChosenIE
_VIDEO_URL_TMPL = 'https://watch.thechosen.tv/video/%s' _VIDEO_URL_TMPL = 'https://watch.thechosen.tv/watch/%s'
_VALID_URL = r'https?://(?:www\.)?watch\.thechosen\.tv/group/(?P<id>[0-9]+)' _VALID_URL = r'https?://(?:www\.)?watch\.thechosen\.tv/group/(?P<id>[0-9]+)'
_TESTS = [{ _TESTS = [{

View File

@@ -821,13 +821,17 @@ class GenericIE(InfoExtractor):
'Referer': smuggled_data.get('referer'), 'Referer': smuggled_data.get('referer'),
}), impersonate=impersonate) }), impersonate=impersonate)
except ExtractorError as e: except ExtractorError as e:
if not (isinstance(e.cause, HTTPError) and e.cause.status == 403 if not isinstance(e.cause, HTTPError) or e.cause.status != 403:
and e.cause.response.get_header('cf-mitigated') == 'challenge' raise
and e.cause.response.extensions.get('impersonate') is None): res = e.cause.response
already_impersonating = res.extensions.get('impersonate') is not None
if already_impersonating or (
res.get_header('cf-mitigated') != 'challenge'
and b'<title>Attention Required! | Cloudflare</title>' not in res.read()
):
raise raise
cf_cookie_domain = traverse_obj( cf_cookie_domain = traverse_obj(
LenientSimpleCookie(e.cause.response.get_header('set-cookie')), LenientSimpleCookie(res.get_header('set-cookie')), ('__cf_bm', 'domain'))
('__cf_bm', 'domain'))
if cf_cookie_domain: if cf_cookie_domain:
self.write_debug(f'Clearing __cf_bm cookie for {cf_cookie_domain}') self.write_debug(f'Clearing __cf_bm cookie for {cf_cookie_domain}')
self.cookiejar.clear(domain=cf_cookie_domain, path='/', name='__cf_bm') self.cookiejar.clear(domain=cf_cookie_domain, path='/', name='__cf_bm')

View File

@@ -59,7 +59,7 @@ class GetCourseRuIE(InfoExtractor):
'marafon.mani-beauty.com', 'marafon.mani-beauty.com',
'on.psbook.ru', 'on.psbook.ru',
] ]
_BASE_URL_RE = rf'https?://(?:(?!player02\.)[^.]+\.getcourse\.(?:ru|io)|{"|".join(map(re.escape, _DOMAINS))})' _BASE_URL_RE = rf'https?://(?:(?!player02\.)[a-zA-Z0-9-]+\.getcourse\.(?:ru|io)|{"|".join(map(re.escape, _DOMAINS))})'
_VALID_URL = [ _VALID_URL = [
rf'{_BASE_URL_RE}/(?!pl/|teach/)(?P<id>[^?#]+)', rf'{_BASE_URL_RE}/(?!pl/|teach/)(?P<id>[^?#]+)',
rf'{_BASE_URL_RE}/(?:pl/)?teach/control/lesson/view\?(?:[^#]+&)?id=(?P<id>\d+)', rf'{_BASE_URL_RE}/(?:pl/)?teach/control/lesson/view\?(?:[^#]+&)?id=(?P<id>\d+)',

View File

@@ -46,6 +46,7 @@ class GofileIE(InfoExtractor):
'videopassword': 'password', 'videopassword': 'password',
}, },
}] }]
_STATIC_TOKEN = '4fd6sg89d7s6' # From https://gofile.io/dist/js/config.js
_TOKEN = None _TOKEN = None
def _real_initialize(self): def _real_initialize(self):
@@ -60,13 +61,16 @@ class GofileIE(InfoExtractor):
self._set_cookie('.gofile.io', 'accountToken', self._TOKEN) self._set_cookie('.gofile.io', 'accountToken', self._TOKEN)
def _entries(self, file_id): def _entries(self, file_id):
query_params = {'wt': '4fd6sg89d7s6'} # From https://gofile.io/dist/js/alljs.js query_params = {}
password = self.get_param('videopassword') if password := self.get_param('videopassword'):
if password:
query_params['password'] = hashlib.sha256(password.encode()).hexdigest() query_params['password'] = hashlib.sha256(password.encode()).hexdigest()
files = self._download_json( files = self._download_json(
f'https://api.gofile.io/contents/{file_id}', file_id, 'Getting filelist', f'https://api.gofile.io/contents/{file_id}', file_id, 'Getting filelist',
query=query_params, headers={'Authorization': f'Bearer {self._TOKEN}'}) query=query_params, headers={
'Authorization': f'Bearer {self._TOKEN}',
'X-Website-Token': self._STATIC_TOKEN,
})
status = files['status'] status = files['status']
if status == 'error-passwordRequired': if status == 'error-passwordRequired':

View File

@@ -27,7 +27,7 @@ class HotStarBaseIE(InfoExtractor):
_TOKEN_NAME = 'userUP' _TOKEN_NAME = 'userUP'
_BASE_URL = 'https://www.hotstar.com' _BASE_URL = 'https://www.hotstar.com'
_API_URL = 'https://api.hotstar.com' _API_URL = 'https://api.hotstar.com'
_API_URL_V2 = 'https://apix.hotstar.com/v2' _API_URL_V2 = 'https://www.hotstar.com/api/internal/bff/v2'
_AKAMAI_ENCRYPTION_KEY = b'\x05\xfc\x1a\x01\xca\xc9\x4b\xc4\x12\xfc\x53\x12\x07\x75\xf9\xee' _AKAMAI_ENCRYPTION_KEY = b'\x05\xfc\x1a\x01\xca\xc9\x4b\xc4\x12\xfc\x53\x12\x07\x75\xf9\xee'
_FREE_HEADERS = { _FREE_HEADERS = {

View File

@@ -9,14 +9,12 @@ from .openload import PhantomJSwrapper
from ..utils import ( from ..utils import (
ExtractorError, ExtractorError,
clean_html, clean_html,
decode_packed_codes,
float_or_none, float_or_none,
format_field, format_field,
get_element_by_attribute, get_element_by_attribute,
get_element_by_id, get_element_by_id,
int_or_none, int_or_none,
js_to_json, js_to_json,
ohdave_rsa_encrypt,
parse_age_limit, parse_age_limit,
parse_duration, parse_duration,
parse_iso8601, parse_iso8601,
@@ -33,143 +31,12 @@ def md5_text(text):
return hashlib.md5(text.encode()).hexdigest() return hashlib.md5(text.encode()).hexdigest()
class IqiyiSDK:
def __init__(self, target, ip, timestamp):
self.target = target
self.ip = ip
self.timestamp = timestamp
@staticmethod
def split_sum(data):
return str(sum(int(p, 16) for p in data))
@staticmethod
def digit_sum(num):
if isinstance(num, int):
num = str(num)
return str(sum(map(int, num)))
def even_odd(self):
even = self.digit_sum(str(self.timestamp)[::2])
odd = self.digit_sum(str(self.timestamp)[1::2])
return even, odd
def preprocess(self, chunksize):
self.target = md5_text(self.target)
chunks = []
for i in range(32 // chunksize):
chunks.append(self.target[chunksize * i:chunksize * (i + 1)])
if 32 % chunksize:
chunks.append(self.target[32 - 32 % chunksize:])
return chunks, list(map(int, self.ip.split('.')))
def mod(self, modulus):
chunks, ip = self.preprocess(32)
self.target = chunks[0] + ''.join(str(p % modulus) for p in ip)
def split(self, chunksize):
modulus_map = {
4: 256,
5: 10,
8: 100,
}
chunks, ip = self.preprocess(chunksize)
ret = ''
for i in range(len(chunks)):
ip_part = str(ip[i] % modulus_map[chunksize]) if i < 4 else ''
if chunksize == 8:
ret += ip_part + chunks[i]
else:
ret += chunks[i] + ip_part
self.target = ret
def handle_input16(self):
self.target = md5_text(self.target)
self.target = self.split_sum(self.target[:16]) + self.target + self.split_sum(self.target[16:])
def handle_input8(self):
self.target = md5_text(self.target)
ret = ''
for i in range(4):
part = self.target[8 * i:8 * (i + 1)]
ret += self.split_sum(part) + part
self.target = ret
def handleSum(self):
self.target = md5_text(self.target)
self.target = self.split_sum(self.target) + self.target
def date(self, scheme):
self.target = md5_text(self.target)
d = time.localtime(self.timestamp)
strings = {
'y': str(d.tm_year),
'm': '%02d' % d.tm_mon,
'd': '%02d' % d.tm_mday,
}
self.target += ''.join(strings[c] for c in scheme)
def split_time_even_odd(self):
even, odd = self.even_odd()
self.target = odd + md5_text(self.target) + even
def split_time_odd_even(self):
even, odd = self.even_odd()
self.target = even + md5_text(self.target) + odd
def split_ip_time_sum(self):
chunks, ip = self.preprocess(32)
self.target = str(sum(ip)) + chunks[0] + self.digit_sum(self.timestamp)
def split_time_ip_sum(self):
chunks, ip = self.preprocess(32)
self.target = self.digit_sum(self.timestamp) + chunks[0] + str(sum(ip))
class IqiyiSDKInterpreter:
def __init__(self, sdk_code):
self.sdk_code = sdk_code
def run(self, target, ip, timestamp):
self.sdk_code = decode_packed_codes(self.sdk_code)
functions = re.findall(r'input=([a-zA-Z0-9]+)\(input', self.sdk_code)
sdk = IqiyiSDK(target, ip, timestamp)
other_functions = {
'handleSum': sdk.handleSum,
'handleInput8': sdk.handle_input8,
'handleInput16': sdk.handle_input16,
'splitTimeEvenOdd': sdk.split_time_even_odd,
'splitTimeOddEven': sdk.split_time_odd_even,
'splitIpTimeSum': sdk.split_ip_time_sum,
'splitTimeIpSum': sdk.split_time_ip_sum,
}
for function in functions:
if re.match(r'mod\d+', function):
sdk.mod(int(function[3:]))
elif re.match(r'date[ymd]{3}', function):
sdk.date(function[4:])
elif re.match(r'split\d+', function):
sdk.split(int(function[5:]))
elif function in other_functions:
other_functions[function]()
else:
raise ExtractorError(f'Unknown function {function}')
return sdk.target
class IqiyiIE(InfoExtractor): class IqiyiIE(InfoExtractor):
IE_NAME = 'iqiyi' IE_NAME = 'iqiyi'
IE_DESC = '爱奇艺' IE_DESC = '爱奇艺'
_VALID_URL = r'https?://(?:(?:[^.]+\.)?iqiyi\.com|www\.pps\.tv)/.+\.html' _VALID_URL = r'https?://(?:(?:[^.]+\.)?iqiyi\.com|www\.pps\.tv)/.+\.html'
_NETRC_MACHINE = 'iqiyi'
_TESTS = [{ _TESTS = [{
'url': 'http://www.iqiyi.com/v_19rrojlavg.html', 'url': 'http://www.iqiyi.com/v_19rrojlavg.html',
# MD5 checksum differs on my machine and Travis CI # MD5 checksum differs on my machine and Travis CI
@@ -234,57 +101,6 @@ class IqiyiIE(InfoExtractor):
'18': 7, # 1080p '18': 7, # 1080p
} }
@staticmethod
def _rsa_fun(data):
# public key extracted from http://static.iqiyi.com/js/qiyiV2/20160129180840/jobs/i18n/i18nIndex.js
N = 0xab86b6371b5318aaa1d3c9e612a9f1264f372323c8c0f19875b5fc3b3fd3afcc1e5bec527aa94bfa85bffc157e4245aebda05389a5357b75115ac94f074aefcd
e = 65537
return ohdave_rsa_encrypt(data, e, N)
def _perform_login(self, username, password):
data = self._download_json(
'http://kylin.iqiyi.com/get_token', None,
note='Get token for logging', errnote='Unable to get token for logging')
sdk = data['sdk']
timestamp = int(time.time())
target = (
f'/apis/reglogin/login.action?lang=zh_TW&area_code=null&email={username}'
f'&passwd={self._rsa_fun(password.encode())}&agenttype=1&from=undefined&keeplogin=0&piccode=&fromurl=&_pos=1')
interp = IqiyiSDKInterpreter(sdk)
sign = interp.run(target, data['ip'], timestamp)
validation_params = {
'target': target,
'server': 'BEA3AA1908656AABCCFF76582C4C6660',
'token': data['token'],
'bird_src': 'f8d91d57af224da7893dd397d52d811a',
'sign': sign,
'bird_t': timestamp,
}
validation_result = self._download_json(
'http://kylin.iqiyi.com/validate?' + urllib.parse.urlencode(validation_params), None,
note='Validate credentials', errnote='Unable to validate credentials')
MSG_MAP = {
'P00107': 'please login via the web interface and enter the CAPTCHA code',
'P00117': 'bad username or password',
}
code = validation_result['code']
if code != 'A00000':
msg = MSG_MAP.get(code)
if not msg:
msg = f'error {code}'
if validation_result.get('msg'):
msg += ': ' + validation_result['msg']
self.report_warning('unable to log in: ' + msg)
return False
return True
def get_raw_data(self, tvid, video_id): def get_raw_data(self, tvid, video_id):
tm = int(time.time() * 1000) tm = int(time.time() * 1000)

View File

@@ -95,6 +95,7 @@ class LBRYBaseIE(InfoExtractor):
'_type': 'url', '_type': 'url',
'id': item['claim_id'], 'id': item['claim_id'],
'url': self._permanent_url(url, item['name'], item['claim_id']), 'url': self._permanent_url(url, item['name'], item['claim_id']),
'ie_key': 'LBRY',
} }
def _playlist_entries(self, url, display_id, claim_param, metadata): def _playlist_entries(self, url, display_id, claim_param, metadata):

View File

@@ -29,7 +29,7 @@ class LearningOnScreenIE(InfoExtractor):
}] }]
def _real_initialize(self): def _real_initialize(self):
if not self._get_cookies('https://learningonscreen.ac.uk/').get('PHPSESSID-BOB-LIVE'): if not self._get_cookies('https://learningonscreen.ac.uk/').get('PHPSESSID-LOS-LIVE'):
self.raise_login_required(method='session_cookies') self.raise_login_required(method='session_cookies')
def _real_extract(self, url): def _real_extract(self, url):

209
yt_dlp/extractor/locipo.py Normal file
View File

@@ -0,0 +1,209 @@
import functools
import math
from .streaks import StreaksBaseIE
from ..networking import HEADRequest
from ..utils import (
InAdvancePagedList,
clean_html,
js_to_json,
parse_iso8601,
parse_qs,
str_or_none,
)
from ..utils.traversal import require, traverse_obj
class LocipoBaseIE(StreaksBaseIE):
_API_BASE = 'https://web-api.locipo.jp'
_BASE_URL = 'https://locipo.jp'
_UUID_RE = r'[\da-f]{8}(?:-[\da-f]{4}){3}-[\da-f]{12}'
def _call_api(self, path, item_id, note, fatal=True):
return self._download_json(
f'{self._API_BASE}/{path}', item_id,
f'Downloading {note} API JSON',
f'Unable to download {note} API JSON',
fatal=fatal)
class LocipoIE(LocipoBaseIE):
_VALID_URL = [
fr'https?://locipo\.jp/creative/(?P<id>{LocipoBaseIE._UUID_RE})',
fr'https?://locipo\.jp/embed/?\?(?:[^#]+&)?id=(?P<id>{LocipoBaseIE._UUID_RE})',
]
_TESTS = [{
'url': 'https://locipo.jp/creative/fb5ffeaa-398d-45ce-bb49-0e221b5f94f1',
'info_dict': {
'id': 'fb5ffeaa-398d-45ce-bb49-0e221b5f94f1',
'ext': 'mp4',
'title': 'リアルカレカノ#4 ~伊達さゆりと勉強しよっ?~',
'description': 'md5:70a40c202f3fb7946b61e55fa015094c',
'display_id': '5a2947fe596441f5bab88a61b0432d0d',
'live_status': 'not_live',
'modified_date': r're:\d{8}',
'modified_timestamp': int,
'release_timestamp': 1711789200,
'release_date': '20240330',
'series': 'リアルカレカノ',
'series_id': '1142',
'tags': 'count:4',
'thumbnail': r're:https?://.+\.(?:jpg|png)',
'timestamp': 1756984919,
'upload_date': '20250904',
'uploader': '東海テレビ',
'uploader_id': 'locipo-prod',
},
}, {
'url': 'https://locipo.jp/embed/?id=71a334a0-2b25-406f-9d96-88f341f571c2',
'info_dict': {
'id': '71a334a0-2b25-406f-9d96-88f341f571c2',
'ext': 'mp4',
'title': '#1 オーディション/ゲスト伊藤美来、豊田萌絵',
'description': 'md5:5bbcf532474700439cf56ceb6a15630e',
'display_id': '0ab32634b884499a84adb25de844c551',
'live_status': 'not_live',
'modified_date': r're:\d{8}',
'modified_timestamp': int,
'release_timestamp': 1751623200,
'release_date': '20250704',
'series': '声優ラジオのウラカブリLocipo出張所',
'series_id': '1454',
'tags': 'count:6',
'thumbnail': r're:https?://.+\.(?:jpg|png)',
'timestamp': 1757002966,
'upload_date': '20250904',
'uploader': 'テレビ愛知',
'uploader_id': 'locipo-prod',
},
}, {
'url': 'https://locipo.jp/creative/bff9950d-229b-4fe9-911a-7fa71a232f35?list=69a5b15c-901f-4828-a336-30c0de7612d3',
'info_dict': {
'id': '69a5b15c-901f-4828-a336-30c0de7612d3',
'title': '見て・乗って・語りたい。 東海の鉄道沼',
},
'playlist_mincount': 3,
}, {
'url': 'https://locipo.jp/creative/a0751a7f-c7dd-4a10-a7f1-e12720bdf16c?list=006cff3f-ba74-42f0-b4fd-241486ebda2b',
'info_dict': {
'id': 'a0751a7f-c7dd-4a10-a7f1-e12720bdf16c',
'ext': 'mp4',
'title': '#839 人間真空パック',
'description': 'md5:9fe190333b6975c5001c8c9cbe20d276',
'display_id': 'c2b4c9f4a6d648bd8e3c320e384b9d56',
'live_status': 'not_live',
'modified_date': r're:\d{8}',
'modified_timestamp': int,
'release_timestamp': 1746239400,
'release_date': '20250503',
'series': 'でんじろう先生のはぴエネ!',
'series_id': '202',
'tags': 'count:3',
'thumbnail': r're:https?://.+\.(?:jpg|png)',
'timestamp': 1756975909,
'upload_date': '20250904',
'uploader': '中京テレビ',
'uploader_id': 'locipo-prod',
},
'params': {'noplaylist': True},
}]
def _real_extract(self, url):
video_id = self._match_id(url)
playlist_id = traverse_obj(parse_qs(url), ('list', -1, {str}))
if self._yes_playlist(playlist_id, video_id):
return self.url_result(
f'{self._BASE_URL}/playlist/{playlist_id}', LocipoPlaylistIE)
creatives = self._call_api(f'creatives/{video_id}', video_id, 'Creatives')
media_id = traverse_obj(creatives, ('media_id', {str}, {require('Streaks media ID')}))
webpage = self._download_webpage(url, video_id)
config = self._search_json(
r'window\.__NUXT__\.config\s*=', webpage, 'config', video_id, transform_source=js_to_json)
api_key = traverse_obj(config, ('public', 'streaksVodPlaybackApiKey', {str}, {require('api key')}))
return {
**self._extract_from_streaks_api('locipo-prod', media_id, headers={
'Origin': 'https://locipo.jp',
'X-Streaks-Api-Key': api_key,
}),
**traverse_obj(creatives, {
'title': ('name', {clean_html}),
'description': ('description', {clean_html}, filter),
'release_timestamp': ('publication_started_at', {parse_iso8601}),
'tags': ('keyword', {clean_html}, {lambda x: x.split(',')}, ..., {str.strip}, filter),
'uploader': ('company', 'name', {clean_html}, filter),
}),
**traverse_obj(creatives, ('series', {
'series': ('name', {clean_html}, filter),
'series_id': ('id', {str_or_none}),
})),
'id': video_id,
}
class LocipoPlaylistIE(LocipoBaseIE):
_VALID_URL = [
fr'https?://locipo\.jp/(?P<type>playlist)/(?P<id>{LocipoBaseIE._UUID_RE})',
r'https?://locipo\.jp/(?P<type>series)/(?P<id>\d+)',
]
_TESTS = [{
'url': 'https://locipo.jp/playlist/35d3dd2b-531d-4824-8575-b1c527d29538',
'info_dict': {
'id': '35d3dd2b-531d-4824-8575-b1c527d29538',
'title': 'レシピ集',
},
'playlist_mincount': 135,
}, {
# Redirects to https://locipo.jp/series/1363
'url': 'https://locipo.jp/playlist/fef7c4fb-741f-4d6a-a3a6-754f354302a2',
'info_dict': {
'id': '1363',
'title': 'CBCアナウンサー公式【みてちょてれび】',
'description': 'md5:50a1b23e63112d5c06c882835c8c1fb1',
},
'playlist_mincount': 38,
}, {
'url': 'https://locipo.jp/series/503',
'info_dict': {
'id': '503',
'title': 'FishingLover東海',
'description': '東海地区の釣り場でフィッシングの魅力を余すところなくご紹介!!',
},
'playlist_mincount': 223,
}]
_PAGE_SIZE = 100
def _fetch_page(self, path, playlist_id, page):
creatives = self._download_json(
f'{self._API_BASE}/{path}/{playlist_id}/creatives',
playlist_id, f'Downloading page {page + 1}', query={
'premium': False,
'live': False,
'limit': self._PAGE_SIZE,
'offset': page * self._PAGE_SIZE,
})
for video_id in traverse_obj(creatives, ('items', ..., 'id', {str})):
yield self.url_result(f'{self._BASE_URL}/creative/{video_id}', LocipoIE)
def _real_extract(self, url):
playlist_type, playlist_id = self._match_valid_url(url).group('type', 'id')
if urlh := self._request_webpage(HEADRequest(url), playlist_id, fatal=False):
playlist_type, playlist_id = self._match_valid_url(urlh.url).group('type', 'id')
path = 'playlists' if playlist_type == 'playlist' else 'series'
creatives = self._call_api(
f'{path}/{playlist_id}/creatives', playlist_id, path.capitalize())
entries = InAdvancePagedList(
functools.partial(self._fetch_page, path, playlist_id),
math.ceil(int(creatives['total']) / self._PAGE_SIZE), self._PAGE_SIZE)
return self.playlist_result(
entries, playlist_id,
**traverse_obj(creatives, ('items', ..., playlist_type, {
'title': ('name', {clean_html}, filter),
'description': ('description', {clean_html}, filter),
}, any)))

View File

@@ -1,128 +0,0 @@
from .common import InfoExtractor
from ..utils import clean_html, int_or_none, traverse_obj
_API_URL = 'https://dak1vd5vmi7x6.cloudfront.net/api/v1/publicrole/{}/{}?id={}'
class ManotoTVIE(InfoExtractor):
IE_DESC = 'Manoto TV (Episode)'
_VALID_URL = r'https?://(?:www\.)?manototv\.com/episode/(?P<id>[0-9]+)'
_TESTS = [{
'url': 'https://www.manototv.com/episode/8475',
'info_dict': {
'id': '8475',
'series': 'خانه های رویایی با برادران اسکات',
'season_number': 7,
'episode_number': 25,
'episode_id': 'My Dream Home S7: Carol & John',
'duration': 3600,
'categories': ['سرگرمی'],
'title': 'کارول و جان',
'description': 'md5:d0fff1f8ba5c6775d312a00165d1a97e',
'thumbnail': r're:^https?://.*\.(jpeg|png|jpg)$',
'ext': 'mp4',
},
'params': {
'skip_download': 'm3u8',
},
}, {
'url': 'https://www.manototv.com/episode/12576',
'info_dict': {
'id': '12576',
'series': 'فیلم های ایرانی',
'episode_id': 'Seh Mah Taatili',
'duration': 5400,
'view_count': int,
'categories': ['سرگرمی'],
'title': 'سه ماه تعطیلی',
'description': 'سه ماه تعطیلی فیلمی به کارگردانی و نویسندگی شاپور قریب ساختهٔ سال ۱۳۵۶ است.',
'thumbnail': r're:^https?://.*\.(jpeg|png|jpg)$',
'ext': 'mp4',
},
'params': {
'skip_download': 'm3u8',
},
}]
def _real_extract(self, url):
video_id = self._match_id(url)
episode_json = self._download_json(_API_URL.format('showmodule', 'episodedetails', video_id), video_id)
details = episode_json.get('details', {})
formats = self._extract_m3u8_formats(details.get('videoM3u8Url'), video_id, 'mp4')
return {
'id': video_id,
'series': details.get('showTitle'),
'season_number': int_or_none(details.get('analyticsSeasonNumber')),
'episode_number': int_or_none(details.get('episodeNumber')),
'episode_id': details.get('analyticsEpisodeTitle'),
'duration': int_or_none(details.get('durationInMinutes'), invscale=60),
'view_count': details.get('viewCount'),
'categories': [details.get('videoCategory')],
'title': details.get('episodeTitle'),
'description': clean_html(details.get('episodeDescription')),
'thumbnail': details.get('episodelandscapeImgIxUrl'),
'formats': formats,
}
class ManotoTVShowIE(InfoExtractor):
IE_DESC = 'Manoto TV (Show)'
_VALID_URL = r'https?://(?:www\.)?manototv\.com/show/(?P<id>[0-9]+)'
_TESTS = [{
'url': 'https://www.manototv.com/show/2526',
'playlist_mincount': 68,
'info_dict': {
'id': '2526',
'title': 'فیلم های ایرانی',
'description': 'مجموعه ای از فیلم های سینمای کلاسیک ایران',
},
}]
def _real_extract(self, url):
show_id = self._match_id(url)
show_json = self._download_json(_API_URL.format('showmodule', 'details', show_id), show_id)
show_details = show_json.get('details', {})
title = show_details.get('showTitle')
description = show_details.get('showSynopsis')
series_json = self._download_json(_API_URL.format('showmodule', 'serieslist', show_id), show_id)
playlist_id = str(traverse_obj(series_json, ('details', 'list', 0, 'id')))
playlist_json = self._download_json(_API_URL.format('showmodule', 'episodelist', playlist_id), playlist_id)
playlist = traverse_obj(playlist_json, ('details', 'list')) or []
entries = [
self.url_result(
'https://www.manototv.com/episode/{}'.format(item['slideID']), ie=ManotoTVIE.ie_key(), video_id=item['slideID'])
for item in playlist]
return self.playlist_result(entries, show_id, title, description)
class ManotoTVLiveIE(InfoExtractor):
IE_DESC = 'Manoto TV (Live)'
_VALID_URL = r'https?://(?:www\.)?manototv\.com/live/'
_TEST = {
'url': 'https://www.manototv.com/live/',
'info_dict': {
'id': 'live',
'title': 'Manoto TV Live',
'ext': 'mp4',
'is_live': True,
},
'params': {
'skip_download': 'm3u8',
},
}
def _real_extract(self, url):
video_id = 'live'
json = self._download_json(_API_URL.format('livemodule', 'details', ''), video_id)
details = json.get('details', {})
video_url = details.get('liveUrl')
formats = self._extract_m3u8_formats(video_url, video_id, 'mp4', live=True)
return {
'id': video_id,
'title': 'Manoto TV Live',
'is_live': True,
'formats': formats,
}

View File

@@ -0,0 +1,38 @@
from .common import InfoExtractor
from ..utils import join_nonempty, unified_strdate
from ..utils.traversal import traverse_obj
class MatchiTVIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?matchi\.tv/watch/?\?(?:[^#]+&)?s=(?P<id>[0-9a-zA-Z]+)'
_TESTS = [{
'url': 'https://matchi.tv/watch?s=0euhjzrxsjm',
'info_dict': {
'id': '0euhjzrxsjm',
'ext': 'mp4',
'title': 'Court 2 at Stratford Padel Club 2024-07-13T18:32:24',
'thumbnail': 'https://thumbnails.padelgo.tv/0euhjzrxsjm.jpg',
'upload_date': '20240713',
},
}, {
'url': 'https://matchi.tv/watch?s=FkKDJ9SvAx1',
'only_matching': True,
}]
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
loaded_media = traverse_obj(
self._search_nextjs_data(webpage, video_id, fatal=False),
('props', 'pageProps', 'loadedMedia', {dict})) or {}
start_date_time = traverse_obj(loaded_media, ('startDateTime', {str}))
return {
'id': video_id,
'title': join_nonempty(loaded_media.get('courtDescription'), start_date_time, delim=' '),
'thumbnail': f'https://thumbnails.padelgo.tv/{video_id}.jpg',
'upload_date': unified_strdate(start_date_time),
'formats': self._extract_m3u8_formats(
f'https://streams.padelgo.tv/v2/streams/m3u8/{video_id}/anonymous/playlist.m3u8',
video_id, 'mp4', m3u8_id='hls'),
}

View File

@@ -25,7 +25,7 @@ class MixcloudBaseIE(InfoExtractor):
%s %s
} }
}''' % (lookup_key, username, f', slug: "{slug}"' if slug else '', object_fields), # noqa: UP031 }''' % (lookup_key, username, f', slug: "{slug}"' if slug else '', object_fields), # noqa: UP031
})['data'][lookup_key] }, impersonate=True)['data'][lookup_key]
class MixcloudIE(MixcloudBaseIE): class MixcloudIE(MixcloudBaseIE):

View File

@@ -478,3 +478,64 @@ class NebulaChannelIE(NebulaBaseIE):
playlist_id=collection_slug, playlist_id=collection_slug,
playlist_title=channel.get('title'), playlist_title=channel.get('title'),
playlist_description=channel.get('description')) playlist_description=channel.get('description'))
class NebulaSeasonIE(NebulaBaseIE):
IE_NAME = 'nebula:season'
_VALID_URL = rf'{_BASE_URL_RE}/(?P<series>[\w-]+)/season/(?P<season_number>[\w-]+)'
_TESTS = [{
'url': 'https://nebula.tv/jetlag/season/15',
'info_dict': {
'id': 'jetlag_15',
'title': 'Tag: All Stars',
'description': 'md5:5aa5b8abf3de71756448dc44ffebb674',
},
'playlist_count': 8,
}, {
'url': 'https://nebula.tv/jetlag/season/14',
'info_dict': {
'id': 'jetlag_14',
'title': 'Snake',
'description': 'md5:6da9040f1c2ac559579738bfb6919d1e',
},
'playlist_count': 8,
}, {
'url': 'https://nebula.tv/jetlag/season/13-5',
'info_dict': {
'id': 'jetlag_13-5',
'title': 'Hide + Seek Across NYC',
'description': 'md5:5b87bb9acc6dcdff289bb4c71a2ad59f',
},
'playlist_count': 3,
}]
def _build_url_result(self, item):
url = (
traverse_obj(item, ('share_url', {url_or_none}))
or urljoin('https://nebula.tv/', item.get('app_path'))
or f'https://nebula.tv/videos/{item["slug"]}')
return self.url_result(
smuggle_url(url, {'id': item['id']}),
NebulaIE, url_transparent=True,
**self._extract_video_metadata(item))
def _entries(self, data):
for episode in traverse_obj(data, ('episodes', lambda _, v: v['video']['id'], 'video')):
yield self._build_url_result(episode)
for extra in traverse_obj(data, ('extras', ..., 'items', lambda _, v: v['id'])):
yield self._build_url_result(extra)
for trailer in traverse_obj(data, ('trailers', lambda _, v: v['id'])):
yield self._build_url_result(trailer)
def _real_extract(self, url):
series, season_id = self._match_valid_url(url).group('series', 'season_number')
playlist_id = f'{series}_{season_id}'
data = self._call_api(
f'https://content.api.nebula.app/content/{series}/season/{season_id}', playlist_id)
return self.playlist_result(
self._entries(data), playlist_id,
**traverse_obj(data, {
'title': ('title', {str}),
'description': ('description', {str}),
}))

View File

@@ -156,18 +156,36 @@ class NetEaseMusicIE(NetEaseMusicBaseIE):
'id': '17241424', 'id': '17241424',
'ext': 'mp3', 'ext': 'mp3',
'title': 'Opus 28', 'title': 'Opus 28',
'upload_date': '20080211', 'upload_date': '20060912',
'timestamp': 1202745600, 'timestamp': 1158076800,
'duration': 263, 'duration': 263,
'thumbnail': r're:^http.*\.jpg', 'thumbnail': r're:^http.*\.jpg',
'album': 'Piano Solos Vol. 2', 'album': 'Piano Solos, Vol. 2',
'album_artist': 'Dustin O\'Halloran', 'album_artist': 'Dustin O\'Halloran',
'average_rating': int, 'average_rating': int,
'description': '[00:05.00]纯音乐,请欣赏\n', 'description': 'md5:b566b92c55ca348df65d206c5d689576',
'album_artists': ['Dustin O\'Halloran'], 'album_artists': ['Dustin O\'Halloran'],
'creators': ['Dustin O\'Halloran'], 'creators': ['Dustin O\'Halloran'],
'subtitles': {'lyrics': [{'ext': 'lrc'}]}, 'subtitles': {'lyrics': [{'ext': 'lrc'}]},
}, },
}, {
'url': 'https://music.163.com/#/song?id=2755669231',
'info_dict': {
'id': '2755669231',
'ext': 'mp3',
'title': '十二月-Departure',
'upload_date': '20251111',
'timestamp': 1762876800,
'duration': 188,
'thumbnail': r're:^http.*\.jpg',
'album': '',
'album_artist': 'ひとひら',
'average_rating': int,
'description': 'md5:deee249c8c9c3e2c54ecdab36e87d174',
'album_artists': ['ひとひら'],
'creators': ['ひとひら'],
'subtitles': {'lyrics': [{'ext': 'lrc', 'data': 'md5:d32b4425a5d6c9fa249ca6e803dd0401'}]},
},
}, { }, {
'url': 'https://y.music.163.com/m/song?app_version=8.8.45&id=95670&uct2=sKnvS4+0YStsWkqsPhFijw%3D%3D&dlt=0846', 'url': 'https://y.music.163.com/m/song?app_version=8.8.45&id=95670&uct2=sKnvS4+0YStsWkqsPhFijw%3D%3D&dlt=0846',
'md5': 'b896be78d8d34bd7bb665b26710913ff', 'md5': 'b896be78d8d34bd7bb665b26710913ff',
@@ -241,9 +259,16 @@ class NetEaseMusicIE(NetEaseMusicBaseIE):
'lyrics': [{'data': original, 'ext': 'lrc'}], 'lyrics': [{'data': original, 'ext': 'lrc'}],
} }
lyrics_expr = r'(\[[0-9]{2}:[0-9]{2}\.[0-9]{2,}\])([^\n]+)' def collect_lyrics(lrc):
original_ts_texts = re.findall(lyrics_expr, original) lyrics_expr = r'\[([0-9]{2}):([0-9]{2})[:.]([0-9]{2,})\]([^\n]+)'
translation_ts_dict = dict(re.findall(lyrics_expr, translated)) matches = re.findall(lyrics_expr, lrc)
return (
(f'[{minute}:{sec}.{msec}]', text)
for minute, sec, msec, text in matches
)
original_ts_texts = collect_lyrics(original)
translation_ts_dict = dict(collect_lyrics(translated))
merged = '\n'.join( merged = '\n'.join(
join_nonempty(f'{timestamp}{text}', translation_ts_dict.get(timestamp, ''), delim=' / ') join_nonempty(f'{timestamp}{text}', translation_ts_dict.get(timestamp, ''), delim=' / ')
@@ -528,7 +553,7 @@ class NetEaseMusicMvIE(NetEaseMusicBaseIE):
class NetEaseMusicProgramIE(NetEaseMusicBaseIE): class NetEaseMusicProgramIE(NetEaseMusicBaseIE):
IE_NAME = 'netease:program' IE_NAME = 'netease:program'
IE_DESC = '网易云音乐 - 电台节目' IE_DESC = '网易云音乐 - 电台节目'
_VALID_URL = r'https?://music\.163\.com/(?:#/)?program\?id=(?P<id>[0-9]+)' _VALID_URL = r'https?://music\.163\.com/(?:#/)?(?:dj|program)\?id=(?P<id>[0-9]+)'
_TESTS = [{ _TESTS = [{
'url': 'http://music.163.com/#/program?id=10109055', 'url': 'http://music.163.com/#/program?id=10109055',
'info_dict': { 'info_dict': {
@@ -572,6 +597,9 @@ class NetEaseMusicProgramIE(NetEaseMusicBaseIE):
'params': { 'params': {
'noplaylist': True, 'noplaylist': True,
}, },
}, {
'url': 'https://music.163.com/#/dj?id=3706179315',
'only_matching': True,
}] }]
def _real_extract(self, url): def _real_extract(self, url):

View File

@@ -2,84 +2,59 @@ from .common import InfoExtractor
from ..utils import ( from ..utils import (
clean_html, clean_html,
int_or_none, int_or_none,
js_to_json, url_or_none,
parse_iso8601, urljoin,
) )
from ..utils.traversal import traverse_obj
class NetzkinoIE(InfoExtractor): class NetzkinoIE(InfoExtractor):
_WORKING = False _GEO_COUNTRIES = ['DE']
_VALID_URL = r'https?://(?:www\.)?netzkino\.de/\#!/[^/]+/(?P<id>[^/]+)' _VALID_URL = r'https?://(?:www\.)?netzkino\.de/details/(?P<id>[^/?#]+)'
_TESTS = [{ _TESTS = [{
'url': 'https://www.netzkino.de/#!/scifikino/rakete-zum-mond', 'url': 'https://www.netzkino.de/details/snow-beast',
'md5': '92a3f8b76f8d7220acce5377ea5d4873', 'md5': '1a4c90fe40d3ccabce163287e45e56dd',
'info_dict': { 'info_dict': {
'id': 'rakete-zum-mond', 'id': 'snow-beast',
'ext': 'mp4', 'ext': 'mp4',
'title': 'Rakete zum Mond \u2013 Jules Verne', 'title': 'Snow Beast',
'description': 'md5:f0a8024479618ddbfa450ff48ffa6c60',
'upload_date': '20120813',
'thumbnail': r're:https?://.*\.jpg$',
'timestamp': 1344858571,
'age_limit': 12, 'age_limit': 12,
}, 'alt_title': 'Snow Beast',
'params': { 'cast': 'count:3',
'skip_download': 'Download only works from Germany', 'categories': 'count:7',
}, 'creators': 'count:2',
}, { 'description': 'md5:e604a954a7f827a80e96a3a97d48b269',
'url': 'https://www.netzkino.de/#!/filme/dr-jekyll-mrs-hyde-2', 'location': 'US',
'md5': 'c7728b2dadd04ff6727814847a51ef03', 'release_year': 2011,
'info_dict': { 'thumbnail': r're:https?://.+\.jpg',
'id': 'dr-jekyll-mrs-hyde-2',
'ext': 'mp4',
'title': 'Dr. Jekyll & Mrs. Hyde 2',
'description': 'md5:c2e9626ebd02de0a794b95407045d186',
'upload_date': '20190130',
'thumbnail': r're:https?://.*\.jpg$',
'timestamp': 1548849437,
'age_limit': 18,
},
'params': {
'skip_download': 'Download only works from Germany',
}, },
}] }]
def _real_extract(self, url): def _real_extract(self, url):
mobj = self._match_valid_url(url) video_id = self._match_id(url)
video_id = mobj.group('id') webpage = self._download_webpage(url, video_id)
next_js_data = self._search_nextjs_data(webpage, video_id)
api_url = f'https://api.netzkino.de.simplecache.net/capi-2.0a/movies/{video_id}.json?d=www' query = traverse_obj(next_js_data, (
info = self._download_json(api_url, video_id) 'props', '__dehydratedState', 'queries', ..., 'state',
custom_fields = info['custom_fields'] 'data', 'data', lambda _, v: v['__typename'] == 'CmsMovie', any))
if 'DRM' in traverse_obj(query, ('licenses', 'nodes', ..., 'properties', {str})):
production_js = self._download_webpage( self.report_drm(video_id)
'http://www.netzkino.de/beta/dist/production.min.js', video_id,
note='Downloading player code')
avo_js = self._search_regex(
r'var urlTemplate=(\{.*?"\})',
production_js, 'URL templates')
templates = self._parse_json(
avo_js, video_id, transform_source=js_to_json)
suffix = {
'hds': '.mp4/manifest.f4m',
'hls': '.mp4/master.m3u8',
'pmd': '.mp4',
}
film_fn = custom_fields['Streaming'][0]
formats = [{
'format_id': key,
'ext': 'mp4',
'url': tpl.replace('{}', film_fn) + suffix[key],
} for key, tpl in templates.items()]
return { return {
'id': video_id, 'id': video_id,
'formats': formats, **traverse_obj(query, {
'title': info['title'], 'title': ('originalTitle', {clean_html}),
'age_limit': int_or_none(custom_fields.get('FSK')[0]), 'age_limit': ('fskRating', {int_or_none}),
'timestamp': parse_iso8601(info.get('date'), delimiter=' '), 'alt_title': ('originalTitle', {clean_html}, filter),
'description': clean_html(info.get('content')), 'cast': ('cast', 'nodes', ..., 'person', 'name', {clean_html}, filter),
'thumbnail': info.get('thumbnail'), 'creators': (('directors', 'writers'), 'nodes', ..., 'person', 'name', {clean_html}, filter),
'categories': ('categories', 'nodes', ..., 'category', 'title', {clean_html}, filter),
'description': ('longSynopsis', {clean_html}, filter),
'duration': ('runtimeInSeconds', {int_or_none}),
'location': ('productionCountry', {clean_html}, filter),
'release_year': ('productionYear', {int_or_none}),
'thumbnail': ('coverImage', 'masterUrl', {url_or_none}),
'url': ('videoSource', 'pmdUrl', {urljoin('https://pmd.netzkino-seite.netzkino.de/')}),
}),
} }

View File

@@ -1,238 +0,0 @@
import urllib.parse
from .common import InfoExtractor
from ..utils import (
clean_html,
get_element_by_class,
int_or_none,
parse_iso8601,
remove_start,
unified_timestamp,
)
class NextMediaIE(InfoExtractor):
IE_DESC = '蘋果日報'
_VALID_URL = r'https?://hk\.apple\.nextmedia\.com/[^/]+/[^/]+/(?P<date>\d+)/(?P<id>\d+)'
_TESTS = [{
'url': 'http://hk.apple.nextmedia.com/realtime/news/20141108/53109199',
'md5': 'dff9fad7009311c421176d1ac90bfe4f',
'info_dict': {
'id': '53109199',
'ext': 'mp4',
'title': '【佔領金鐘】50外國領事議員撐場 讚學生勇敢香港有希望',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:28222b9912b6665a21011b034c70fcc7',
'timestamp': 1415456273,
'upload_date': '20141108',
},
}]
_URL_PATTERN = r'\{ url: \'(.+)\' \}'
def _real_extract(self, url):
news_id = self._match_id(url)
page = self._download_webpage(url, news_id)
return self._extract_from_nextmedia_page(news_id, url, page)
def _extract_from_nextmedia_page(self, news_id, url, page):
redirection_url = self._search_regex(
r'window\.location\.href\s*=\s*([\'"])(?P<url>(?!\1).+)\1',
page, 'redirection URL', default=None, group='url')
if redirection_url:
return self.url_result(urllib.parse.urljoin(url, redirection_url))
title = self._fetch_title(page)
video_url = self._search_regex(self._URL_PATTERN, page, 'video url')
attrs = {
'id': news_id,
'title': title,
'url': video_url, # ext can be inferred from url
'thumbnail': self._fetch_thumbnail(page),
'description': self._fetch_description(page),
}
timestamp = self._fetch_timestamp(page)
if timestamp:
attrs['timestamp'] = timestamp
else:
attrs['upload_date'] = self._fetch_upload_date(url)
return attrs
def _fetch_title(self, page):
return self._og_search_title(page)
def _fetch_thumbnail(self, page):
return self._og_search_thumbnail(page)
def _fetch_timestamp(self, page):
date_created = self._search_regex('"dateCreated":"([^"]+)"', page, 'created time')
return parse_iso8601(date_created)
def _fetch_upload_date(self, url):
return self._search_regex(self._VALID_URL, url, 'upload date', group='date')
def _fetch_description(self, page):
return self._og_search_property('description', page)
class NextMediaActionNewsIE(NextMediaIE): # XXX: Do not subclass from concrete IE
IE_DESC = '蘋果日報 - 動新聞'
_VALID_URL = r'https?://hk\.dv\.nextmedia\.com/actionnews/[^/]+/(?P<date>\d+)/(?P<id>\d+)/\d+'
_TESTS = [{
'url': 'http://hk.dv.nextmedia.com/actionnews/hit/20150121/19009428/20061460',
'md5': '05fce8ffeed7a5e00665d4b7cf0f9201',
'info_dict': {
'id': '19009428',
'ext': 'mp4',
'title': '【壹週刊】細10年男友偷食 50歲邵美琪再失戀',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:cd802fad1f40fd9ea178c1e2af02d659',
'timestamp': 1421791200,
'upload_date': '20150120',
},
}]
def _real_extract(self, url):
news_id = self._match_id(url)
actionnews_page = self._download_webpage(url, news_id)
article_url = self._og_search_url(actionnews_page)
article_page = self._download_webpage(article_url, news_id)
return self._extract_from_nextmedia_page(news_id, url, article_page)
class AppleDailyIE(NextMediaIE): # XXX: Do not subclass from concrete IE
IE_DESC = '臺灣蘋果日報'
_VALID_URL = r'https?://(www|ent)\.appledaily\.com\.tw/[^/]+/[^/]+/[^/]+/(?P<date>\d+)/(?P<id>\d+)(/.*)?'
_TESTS = [{
'url': 'http://ent.appledaily.com.tw/enews/article/entertainment/20150128/36354694',
'md5': 'a843ab23d150977cc55ef94f1e2c1e4d',
'info_dict': {
'id': '36354694',
'ext': 'mp4',
'title': '周亭羽走過摩鐵陰霾2男陪吃 九把刀孤寒看醫生',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:2acd430e59956dc47cd7f67cb3c003f4',
'upload_date': '20150128',
},
}, {
'url': 'http://www.appledaily.com.tw/realtimenews/article/strange/20150128/550549/%E4%B8%8D%E6%BB%BF%E8%A2%AB%E8%B8%A9%E8%85%B3%E3%80%80%E5%B1%B1%E6%9D%B1%E5%85%A9%E5%A4%A7%E5%AA%BD%E4%B8%80%E8%B7%AF%E6%89%93%E4%B8%8B%E8%BB%8A',
'md5': '86b4e9132d158279c7883822d94ccc49',
'info_dict': {
'id': '550549',
'ext': 'mp4',
'title': '不滿被踩腳 山東兩大媽一路打下車',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:175b4260c1d7c085993474217e4ab1b4',
'upload_date': '20150128',
},
}, {
'url': 'http://www.appledaily.com.tw/animation/realtimenews/new/20150128/5003671',
'md5': '03df296d95dedc2d5886debbb80cb43f',
'info_dict': {
'id': '5003671',
'ext': 'mp4',
'title': '20正妹熱舞 《刀龍傳說Online》火辣上市',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:23c0aac567dc08c9c16a3161a2c2e3cd',
'upload_date': '20150128',
},
'skip': 'redirect to http://www.appledaily.com.tw/animation/',
}, {
# No thumbnail
'url': 'http://www.appledaily.com.tw/animation/realtimenews/new/20150128/5003673/',
'md5': 'b06182cd386ea7bc6115ec7ff0f72aeb',
'info_dict': {
'id': '5003673',
'ext': 'mp4',
'title': '半夜尿尿 好像會看到___',
'description': 'md5:61d2da7fe117fede148706cdb85ac066',
'upload_date': '20150128',
},
'expected_warnings': [
'video thumbnail',
],
'skip': 'redirect to http://www.appledaily.com.tw/animation/',
}, {
'url': 'http://www.appledaily.com.tw/appledaily/article/supplement/20140417/35770334/',
'md5': 'eaa20e6b9df418c912d7f5dec2ba734d',
'info_dict': {
'id': '35770334',
'ext': 'mp4',
'title': '咖啡占卜測 XU裝熟指數',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:7b859991a6a4fedbdf3dd3b66545c748',
'upload_date': '20140417',
},
}, {
'url': 'http://www.appledaily.com.tw/actionnews/appledaily/7/20161003/960588/',
'only_matching': True,
}, {
# Redirected from http://ent.appledaily.com.tw/enews/article/entertainment/20150128/36354694
'url': 'http://ent.appledaily.com.tw/section/article/headline/20150128/36354694',
'only_matching': True,
}]
_URL_PATTERN = r'\{url: \'(.+)\'\}'
def _fetch_title(self, page):
return (self._html_search_regex(r'<h1 id="h1">([^<>]+)</h1>', page, 'news title', default=None)
or self._html_search_meta('description', page, 'news title'))
def _fetch_thumbnail(self, page):
return self._html_search_regex(r"setInitialImage\(\'([^']+)'\)", page, 'video thumbnail', fatal=False)
def _fetch_timestamp(self, page):
return None
def _fetch_description(self, page):
return self._html_search_meta('description', page, 'news description')
class NextTVIE(InfoExtractor):
_WORKING = False
_ENABLED = None # XXX: pass through to GenericIE
IE_DESC = '壹電視'
_VALID_URL = r'https?://(?:www\.)?nexttv\.com\.tw/(?:[^/]+/)+(?P<id>\d+)'
_TEST = {
'url': 'http://www.nexttv.com.tw/news/realtime/politics/11779671',
'info_dict': {
'id': '11779671',
'ext': 'mp4',
'title': '「超收稅」近4千億 藍議員籲發消費券',
'thumbnail': r're:^https?://.*\.jpg$',
'timestamp': 1484825400,
'upload_date': '20170119',
'view_count': int,
},
}
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
title = self._html_search_regex(
r'<h1[^>]*>([^<]+)</h1>', webpage, 'title')
data = self._hidden_inputs(webpage)
video_url = data['ntt-vod-src-detailview']
date_str = get_element_by_class('date', webpage)
timestamp = unified_timestamp(date_str + '+0800') if date_str else None
view_count = int_or_none(remove_start(
clean_html(get_element_by_class('click', webpage)), '點閱:'))
return {
'id': video_id,
'title': title,
'url': video_url,
'thumbnail': data.get('ntt-vod-img-src'),
'timestamp': timestamp,
'view_count': view_count,
}

View File

@@ -9,13 +9,13 @@ from ..utils import (
int_or_none, int_or_none,
qualities, qualities,
smuggle_url, smuggle_url,
traverse_obj,
unescapeHTML, unescapeHTML,
unified_strdate, unified_strdate,
unsmuggle_url, unsmuggle_url,
url_or_none, url_or_none,
urlencode_postdata, urlencode_postdata,
) )
from ..utils.traversal import find_element, traverse_obj
class OdnoklassnikiIE(InfoExtractor): class OdnoklassnikiIE(InfoExtractor):
@@ -264,9 +264,7 @@ class OdnoklassnikiIE(InfoExtractor):
note='Downloading desktop webpage', note='Downloading desktop webpage',
headers={'Referer': smuggled['referrer']} if smuggled.get('referrer') else {}) headers={'Referer': smuggled['referrer']} if smuggled.get('referrer') else {})
error = self._search_regex( error = traverse_obj(webpage, {find_element(cls='vp_video_stub_txt')})
r'[^>]+class="vp_video_stub_txt"[^>]*>([^<]+)<',
webpage, 'error', default=None)
# Direct link from boosty # Direct link from boosty
if (error == 'The author of this video has not been found or is blocked' if (error == 'The author of this video has not been found or is blocked'
and not smuggled.get('referrer') and mode == 'videoembed'): and not smuggled.get('referrer') and mode == 'videoembed'):

View File

@@ -33,7 +33,8 @@ class OpencastBaseIE(InfoExtractor):
vid\.igb\.illinois\.edu| vid\.igb\.illinois\.edu|
cursosabertos\.c3sl\.ufpr\.br| cursosabertos\.c3sl\.ufpr\.br|
mcmedia\.missioncollege\.org| mcmedia\.missioncollege\.org|
clases\.odon\.edu\.uy clases\.odon\.edu\.uy|
oc-p\.uni-jena\.de
)''' )'''
_UUID_RE = r'[\da-fA-F]{8}-[\da-fA-F]{4}-[\da-fA-F]{4}-[\da-fA-F]{4}-[\da-fA-F]{12}' _UUID_RE = r'[\da-fA-F]{8}-[\da-fA-F]{4}-[\da-fA-F]{4}-[\da-fA-F]{4}-[\da-fA-F]{12}'
@@ -106,7 +107,7 @@ class OpencastBaseIE(InfoExtractor):
class OpencastIE(OpencastBaseIE): class OpencastIE(OpencastBaseIE):
_VALID_URL = rf'''(?x) _VALID_URL = rf'''(?x)
https?://(?P<host>{OpencastBaseIE._INSTANCES_RE})/paella/ui/watch\.html\? https?://(?P<host>{OpencastBaseIE._INSTANCES_RE})/paella[0-9]*/ui/watch\.html\?
(?:[^#]+&)?id=(?P<id>{OpencastBaseIE._UUID_RE})''' (?:[^#]+&)?id=(?P<id>{OpencastBaseIE._UUID_RE})'''
_API_BASE = 'https://%s/search/episode.json?id=%s' _API_BASE = 'https://%s/search/episode.json?id=%s'
@@ -131,8 +132,12 @@ class OpencastIE(OpencastBaseIE):
def _real_extract(self, url): def _real_extract(self, url):
host, video_id = self._match_valid_url(url).group('host', 'id') host, video_id = self._match_valid_url(url).group('host', 'id')
return self._parse_mediapackage( response = self._call_api(host, video_id)
self._call_api(host, video_id)['search-results']['result']['mediapackage']) package = traverse_obj(response, (
('search-results', 'result'),
('result', ...), # Path needed for oc-p.uni-jena.de
'mediapackage', {dict}, any)) or {}
return self._parse_mediapackage(package)
class OpencastPlaylistIE(OpencastBaseIE): class OpencastPlaylistIE(OpencastBaseIE):

View File

@@ -0,0 +1,83 @@
from .common import InfoExtractor
from ..utils import (
ExtractorError,
UserNotLive,
filter_dict,
int_or_none,
join_nonempty,
parse_iso8601,
url_or_none,
urlencode_postdata,
)
from ..utils.traversal import traverse_obj
class PandaTvIE(InfoExtractor):
IE_DESC = 'pandalive.co.kr (팬더티비)'
_VALID_URL = r'https?://(?:www\.|m\.)?pandalive\.co\.kr/play/(?P<id>\w+)'
_TESTS = [{
'url': 'https://www.pandalive.co.kr/play/bebenim',
'info_dict': {
'id': 'bebenim',
'ext': 'mp4',
'channel': '릴리ෆ',
'title': r're:앙앙❤ \d{4}-\d{2}-\d{2} \d{2}:\d{2}',
'thumbnail': r're:https://cdn\.pandalive\.co\.kr/ivs/v1/.+/thumb\.jpg',
'concurrent_view_count': int,
'like_count': int,
'live_status': 'is_live',
'upload_date': str,
},
'skip': 'The channel is not currently live',
}]
def _real_extract(self, url):
channel_id = self._match_id(url)
video_meta = self._download_json(
'https://api.pandalive.co.kr/v1/live/play', channel_id,
'Downloading video meta data', 'Unable to download video meta data',
data=urlencode_postdata(filter_dict({
'action': 'watch',
'userId': channel_id,
'password': self.get_param('videopassword'),
})), expected_status=400)
if error_code := traverse_obj(video_meta, ('errorData', 'code', {str})):
if error_code == 'castEnd':
raise UserNotLive(video_id=channel_id)
elif error_code == 'needAdult':
self.raise_login_required('Adult verification is required for this stream')
elif error_code == 'needLogin':
self.raise_login_required('Login is required for this stream')
elif error_code == 'needCoinPurchase':
raise ExtractorError('Coin purchase is required for this stream', expected=True)
elif error_code == 'needUnlimitItem':
raise ExtractorError('Ticket purchase is required for this stream', expected=True)
elif error_code == 'needPw':
raise ExtractorError('Password protected video, use --video-password <password>', expected=True)
elif error_code == 'wrongPw':
raise ExtractorError('Wrong password', expected=True)
else:
error_msg = video_meta.get('message')
raise ExtractorError(join_nonempty(
'API returned error code', error_code,
error_msg and 'with error message:', error_msg,
delim=' '))
http_headers = {'Origin': 'https://www.pandalive.co.kr'}
return {
'id': channel_id,
'is_live': True,
'formats': self._extract_m3u8_formats(
video_meta['PlayList']['hls'][0]['url'], channel_id, 'mp4', headers=http_headers, live=True),
'http_headers': http_headers,
**traverse_obj(video_meta, ('media', {
'title': ('title', {str}),
'release_timestamp': ('startTime', {parse_iso8601(delim=' ')}),
'thumbnail': ('ivsThumbnail', {url_or_none}),
'channel': ('userNick', {str}),
'concurrent_view_count': ('user', {int_or_none}),
'like_count': ('likeCnt', {int_or_none}),
})),
}

View File

@@ -6,7 +6,10 @@ from ..utils.traversal import traverse_obj
class PartiBaseIE(InfoExtractor): class PartiBaseIE(InfoExtractor):
def _call_api(self, path, video_id, note=None): def _call_api(self, path, video_id, note=None):
return self._download_json( return self._download_json(
f'https://api-backend.parti.com/parti_v2/profile/{path}', video_id, note) f'https://prod-api.parti.com/parti_v2/profile/{path}', video_id, note, headers={
'Origin': 'https://parti.com',
'Referer': 'https://parti.com/',
})
class PartiVideoIE(PartiBaseIE): class PartiVideoIE(PartiBaseIE):
@@ -20,7 +23,7 @@ class PartiVideoIE(PartiBaseIE):
'title': 'NOW LIVE ', 'title': 'NOW LIVE ',
'upload_date': '20250327', 'upload_date': '20250327',
'categories': ['Gaming'], 'categories': ['Gaming'],
'thumbnail': 'https://assets.parti.com/351424_eb9e5250-2821-484a-9c5f-ca99aa666c87.png', 'thumbnail': 'https://media.parti.com/351424_eb9e5250-2821-484a-9c5f-ca99aa666c87.png',
'channel': 'ItZTMGG', 'channel': 'ItZTMGG',
'timestamp': 1743044379, 'timestamp': 1743044379,
}, },
@@ -34,7 +37,7 @@ class PartiVideoIE(PartiBaseIE):
return { return {
'id': video_id, 'id': video_id,
'formats': self._extract_m3u8_formats( 'formats': self._extract_m3u8_formats(
urljoin('https://watch.parti.com', data['livestream_recording']), video_id, 'mp4'), urljoin('https://media.parti.com/', data['livestream_recording']), video_id, 'mp4'),
**traverse_obj(data, { **traverse_obj(data, {
'title': ('event_title', {str}), 'title': ('event_title', {str}),
'channel': ('user_name', {str}), 'channel': ('user_name', {str}),
@@ -47,32 +50,27 @@ class PartiVideoIE(PartiBaseIE):
class PartiLivestreamIE(PartiBaseIE): class PartiLivestreamIE(PartiBaseIE):
IE_NAME = 'parti:livestream' IE_NAME = 'parti:livestream'
_VALID_URL = r'https?://(?:www\.)?parti\.com/creator/(?P<service>[\w]+)/(?P<id>[\w/-]+)' _VALID_URL = r'https?://(?:www\.)?parti\.com/(?!video/)(?P<id>[\w/-]+)'
_TESTS = [{ _TESTS = [{
'url': 'https://parti.com/creator/parti/Capt_Robs_Adventures', 'url': 'https://parti.com/247CryptoTracker',
'info_dict': { 'info_dict': {
'id': 'Capt_Robs_Adventures',
'ext': 'mp4', 'ext': 'mp4',
'id': '247CryptoTracker',
'description': 'md5:a78051f3d7e66e6a64c6b1eaf59fd364',
'title': r"re:I'm Live on Parti \d{4}-\d{2}-\d{2} \d{2}:\d{2}", 'title': r"re:I'm Live on Parti \d{4}-\d{2}-\d{2} \d{2}:\d{2}",
'view_count': int, 'thumbnail': r're:https://media\.parti\.com/stream-screenshots/.+\.png',
'thumbnail': r're:https://assets\.parti\.com/.+\.png',
'timestamp': 1743879776,
'upload_date': '20250405',
'live_status': 'is_live', 'live_status': 'is_live',
}, },
'params': {'skip_download': 'm3u8'}, 'params': {'skip_download': 'm3u8'},
}, {
'url': 'https://parti.com/creator/discord/sazboxgaming/0',
'only_matching': True,
}] }]
def _real_extract(self, url): def _real_extract(self, url):
service, creator_slug = self._match_valid_url(url).group('service', 'id') creator_slug = self._match_id(url)
encoded_creator_slug = creator_slug.replace('/', '%23') encoded_creator_slug = creator_slug.replace('/', '%23')
creator_id = self._call_api( creator_id = self._call_api(
f'get_user_by_social_media/{service}/{encoded_creator_slug}', f'user_id_from_name/{encoded_creator_slug}',
creator_slug, note='Fetching user ID') creator_slug, note='Fetching user ID')['user_id']
data = self._call_api( data = self._call_api(
f'get_livestream_channel_info/{creator_id}', creator_id, f'get_livestream_channel_info/{creator_id}', creator_id,
@@ -85,11 +83,7 @@ class PartiLivestreamIE(PartiBaseIE):
return { return {
'id': creator_slug, 'id': creator_slug,
'formats': self._extract_m3u8_formats( 'formats': self._extract_m3u8_formats(channel_info['playback_url'], creator_slug, live=True),
channel_info['playback_url'], creator_slug, live=True, query={
'token': channel_info['playback_auth_token'],
'player_version': '1.17.0',
}),
'is_live': True, 'is_live': True,
**traverse_obj(data, { **traverse_obj(data, {
'title': ('livestream_event_info', 'event_name', {str}), 'title': ('livestream_event_info', 'event_name', {str}),

View File

@@ -1,6 +1,5 @@
import functools import functools
import itertools import itertools
import urllib.parse
from .common import InfoExtractor from .common import InfoExtractor
from .sproutvideo import VidsIoIE from .sproutvideo import VidsIoIE
@@ -11,15 +10,23 @@ from ..utils import (
ExtractorError, ExtractorError,
clean_html, clean_html,
determine_ext, determine_ext,
extract_attributes,
float_or_none,
int_or_none, int_or_none,
mimetype2ext, mimetype2ext,
parse_iso8601, parse_iso8601,
smuggle_url, smuggle_url,
str_or_none, str_or_none,
update_url_query,
url_or_none, url_or_none,
urljoin, urljoin,
) )
from ..utils.traversal import require, traverse_obj, value from ..utils.traversal import (
find_elements,
require,
traverse_obj,
value,
)
class PatreonBaseIE(InfoExtractor): class PatreonBaseIE(InfoExtractor):
@@ -121,6 +128,7 @@ class PatreonIE(PatreonBaseIE):
'channel_is_verified': True, 'channel_is_verified': True,
'chapters': 'count:4', 'chapters': 'count:4',
'timestamp': 1423689666, 'timestamp': 1423689666,
'media_type': 'video',
}, },
'params': { 'params': {
'noplaylist': True, 'noplaylist': True,
@@ -161,7 +169,7 @@ class PatreonIE(PatreonBaseIE):
'uploader_url': 'https://www.patreon.com/loish', 'uploader_url': 'https://www.patreon.com/loish',
'description': 'md5:e2693e97ee299c8ece47ffdb67e7d9d2', 'description': 'md5:e2693e97ee299c8ece47ffdb67e7d9d2',
'title': 'VIDEO // sketchbook flipthrough', 'title': 'VIDEO // sketchbook flipthrough',
'uploader': 'Loish ', 'uploader': 'Loish',
'tags': ['sketchbook', 'video'], 'tags': ['sketchbook', 'video'],
'channel_id': '1641751', 'channel_id': '1641751',
'channel_url': 'https://www.patreon.com/loish', 'channel_url': 'https://www.patreon.com/loish',
@@ -274,8 +282,73 @@ class PatreonIE(PatreonBaseIE):
'channel_id': '9346307', 'channel_id': '9346307',
}, },
'params': {'getcomments': True}, 'params': {'getcomments': True},
}, {
# Inlined media in post; uses _extract_from_media_api
'url': 'https://www.patreon.com/posts/scottfalco-146966245',
'info_dict': {
'id': '146966245',
'ext': 'mp4',
'title': 'scottfalco 1080',
'description': 'md5:a3f29bbd0a46b4821ec3400957c98aa2',
'uploader': 'Insanimate',
'uploader_id': '2828146',
'uploader_url': 'https://www.patreon.com/Insanimate',
'channel_id': '6260877',
'channel_url': 'https://www.patreon.com/Insanimate',
'channel_follower_count': int,
'comment_count': int,
'like_count': int,
'duration': 7.833333,
'timestamp': 1767061800,
'upload_date': '20251230',
},
}] }]
_RETURN_TYPE = 'video' _RETURN_TYPE = 'video'
_HTTP_HEADERS = {
# Must be all-lowercase 'referer' so we can smuggle it to Generic, SproutVideo, and Vimeo.
# patreon.com URLs redirect to www.patreon.com; this matters when requesting mux.com m3u8s
'referer': 'https://www.patreon.com/',
}
def _extract_from_media_api(self, media_id):
attributes = traverse_obj(
self._call_api(f'media/{media_id}', media_id, fatal=False),
('data', 'attributes', {dict}))
if not attributes:
return None
info_dict = traverse_obj(attributes, {
'title': ('file_name', {lambda x: x.rpartition('.')[0]}),
'timestamp': ('created_at', {parse_iso8601}),
'duration': ('display', 'duration', {float_or_none}),
})
info_dict['id'] = media_id
playback_url = traverse_obj(
attributes, ('display', (None, 'viewer_playback_data'), 'url', {url_or_none}, any))
download_url = traverse_obj(attributes, ('download_url', {url_or_none}))
if playback_url and mimetype2ext(attributes.get('mimetype')) == 'm3u8':
info_dict['formats'], info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(
playback_url, media_id, 'mp4', fatal=False, headers=self._HTTP_HEADERS)
for f in info_dict['formats']:
f['http_headers'] = self._HTTP_HEADERS
if transcript_url := traverse_obj(attributes, ('display', 'transcript_url', {url_or_none})):
info_dict['subtitles'].setdefault('en', []).append({
'url': transcript_url,
'ext': 'vtt',
})
elif playback_url or download_url:
info_dict['formats'] = [{
# If playback_url is available, download_url is a duplicate lower resolution format
'url': playback_url or download_url,
'vcodec': 'none' if attributes.get('media_type') != 'video' else None,
}]
if not info_dict.get('formats'):
return None
return info_dict
def _real_extract(self, url): def _real_extract(self, url):
video_id = self._match_id(url) video_id = self._match_id(url)
@@ -299,6 +372,7 @@ class PatreonIE(PatreonBaseIE):
'comment_count': ('comment_count', {int_or_none}), 'comment_count': ('comment_count', {int_or_none}),
}) })
seen_media_ids = set()
entries = [] entries = []
idx = 0 idx = 0
for include in traverse_obj(post, ('included', lambda _, v: v['type'])): for include in traverse_obj(post, ('included', lambda _, v: v['type'])):
@@ -320,6 +394,8 @@ class PatreonIE(PatreonBaseIE):
'url': download_url, 'url': download_url,
'alt_title': traverse_obj(media_attributes, ('file_name', {str})), 'alt_title': traverse_obj(media_attributes, ('file_name', {str})),
}) })
if media_id := traverse_obj(include, ('id', {str})):
seen_media_ids.add(media_id)
elif include_type == 'user': elif include_type == 'user':
info.update(traverse_obj(include, { info.update(traverse_obj(include, {
@@ -340,34 +416,29 @@ class PatreonIE(PatreonBaseIE):
'channel_follower_count': ('attributes', 'patron_count', {int_or_none}), 'channel_follower_count': ('attributes', 'patron_count', {int_or_none}),
})) }))
# Must be all-lowercase 'referer' so we can smuggle it to Generic, SproutVideo, and Vimeo. if embed_url := traverse_obj(attributes, ('embed', 'url', {url_or_none})):
# patreon.com URLs redirect to www.patreon.com; this matters when requesting mux.com m3u8s # Convert useless vimeo.com URLs to useful player.vimeo.com embed URLs
headers = {'referer': 'https://www.patreon.com/'} vimeo_id, vimeo_hash = self._search_regex(
r'//vimeo\.com/(\d+)(?:/([\da-f]+))?', embed_url,
'vimeo id', group=(1, 2), default=(None, None))
if vimeo_id:
embed_url = update_url_query(
f'https://player.vimeo.com/video/{vimeo_id}',
{'h': vimeo_hash or []})
if VimeoIE.suitable(embed_url):
entry = self.url_result(
VimeoIE._smuggle_referrer(embed_url, self._HTTP_HEADERS['referer']),
VimeoIE, url_transparent=True)
else:
entry = self.url_result(smuggle_url(embed_url, self._HTTP_HEADERS))
# handle Vimeo embeds if urlh := self._request_webpage(
if traverse_obj(attributes, ('embed', 'provider')) == 'Vimeo': embed_url, video_id, 'Checking embed URL', headers=self._HTTP_HEADERS,
v_url = urllib.parse.unquote(self._html_search_regex( fatal=False, errnote=False, expected_status=(403, 429), # Ignore Vimeo 429's
r'(https(?:%3A%2F%2F|://)player\.vimeo\.com.+app_id(?:=|%3D)+\d+)', ):
traverse_obj(attributes, ('embed', 'html', {str})), 'vimeo url', fatal=False) or '') # Password-protected vids.io embeds return 403 errors w/o --video-password or session cookie
if url_or_none(v_url) and self._request_webpage( if VidsIoIE.suitable(embed_url) or urlh.status != 403:
v_url, video_id, 'Checking Vimeo embed URL', headers=headers, entries.append(entry)
fatal=False, errnote=False, expected_status=429): # 429 is TLS fingerprint rejection
entries.append(self.url_result(
VimeoIE._smuggle_referrer(v_url, headers['referer']),
VimeoIE, url_transparent=True))
embed_url = traverse_obj(attributes, ('embed', 'url', {url_or_none}))
if embed_url and (urlh := self._request_webpage(
embed_url, video_id, 'Checking embed URL', headers=headers,
fatal=False, errnote=False, expected_status=403)):
# Vimeo's Cloudflare anti-bot protection will return HTTP status 200 for 404, so we need
# to check for "Sorry, we couldn&amp;rsquo;t find that page" in the meta description tag
meta_description = clean_html(self._html_search_meta(
'description', self._webpage_read_content(urlh, embed_url, video_id, fatal=False), default=None))
# Password-protected vids.io embeds return 403 errors w/o --video-password or session cookie
if ((urlh.status != 403 and meta_description != 'Sorry, we couldnt find that page')
or VidsIoIE.suitable(embed_url)):
entries.append(self.url_result(smuggle_url(embed_url, headers)))
post_file = traverse_obj(attributes, ('post_file', {dict})) post_file = traverse_obj(attributes, ('post_file', {dict}))
if post_file: if post_file:
@@ -381,13 +452,27 @@ class PatreonIE(PatreonBaseIE):
}) })
elif name == 'video' or determine_ext(post_file.get('url')) == 'm3u8': elif name == 'video' or determine_ext(post_file.get('url')) == 'm3u8':
formats, subtitles = self._extract_m3u8_formats_and_subtitles( formats, subtitles = self._extract_m3u8_formats_and_subtitles(
post_file['url'], video_id, headers=headers) post_file['url'], video_id, headers=self._HTTP_HEADERS)
for f in formats:
f['http_headers'] = self._HTTP_HEADERS
entries.append({ entries.append({
'id': video_id, 'id': video_id,
'formats': formats, 'formats': formats,
'subtitles': subtitles, 'subtitles': subtitles,
'http_headers': headers,
}) })
if media_id := traverse_obj(post_file, ('media_id', {int}, {str_or_none})):
seen_media_ids.add(media_id)
for media_id in traverse_obj(attributes, (
'content', {find_elements(attr='data-media-id', value=r'\d+', regex=True, html=True)},
..., {extract_attributes}, 'data-media-id',
)):
# Inlined media may be duplicates of what was extracted above
if media_id in seen_media_ids:
continue
if media := self._extract_from_media_api(media_id):
entries.append(media)
seen_media_ids.add(media_id)
can_view_post = traverse_obj(attributes, 'current_user_can_view') can_view_post = traverse_obj(attributes, 'current_user_can_view')
comments = None comments = None

View File

@@ -453,6 +453,23 @@ class PBSIE(InfoExtractor):
'url': 'https://player.pbs.org/portalplayer/3004638221/?uid=', 'url': 'https://player.pbs.org/portalplayer/3004638221/?uid=',
'only_matching': True, 'only_matching': True,
}, },
{
# Next.js v13+, see https://github.com/yt-dlp/yt-dlp/issues/13299
'url': 'https://www.pbs.org/video/caregiving',
'info_dict': {
'id': '3101776876',
'ext': 'mp4',
'title': 'Caregiving - Caregiving',
'description': 'A documentary revealing Americas caregiving crisis through intimate stories and expert insight.',
'display_id': 'caregiving',
'duration': 6783,
'thumbnail': 'https://image.pbs.org/video-assets/BSrSkcc-asset-mezzanine-16x9-nlcxQts.jpg',
'chapters': [],
},
'params': {
'skip_download': True,
},
},
] ]
_ERRORS = { _ERRORS = {
101: 'We\'re sorry, but this video is not yet available.', 101: 'We\'re sorry, but this video is not yet available.',
@@ -506,6 +523,7 @@ class PBSIE(InfoExtractor):
r"(?s)window\.PBS\.playerConfig\s*=\s*{.*?id\s*:\s*'([0-9]+)',", r"(?s)window\.PBS\.playerConfig\s*=\s*{.*?id\s*:\s*'([0-9]+)',",
r'<div[^>]+\bdata-cove-id=["\'](\d+)"', # http://www.pbs.org/wgbh/roadshow/watch/episode/2105-indianapolis-hour-2/ r'<div[^>]+\bdata-cove-id=["\'](\d+)"', # http://www.pbs.org/wgbh/roadshow/watch/episode/2105-indianapolis-hour-2/
r'<iframe[^>]+\bsrc=["\'](?:https?:)?//video\.pbs\.org/widget/partnerplayer/(\d+)', # https://www.pbs.org/wgbh/masterpiece/episodes/victoria-s2-e1/ r'<iframe[^>]+\bsrc=["\'](?:https?:)?//video\.pbs\.org/widget/partnerplayer/(\d+)', # https://www.pbs.org/wgbh/masterpiece/episodes/victoria-s2-e1/
r'\\"videoTPMediaId\\":\\\"(\d+)\\"', # Next.js v13, e.g. https://www.pbs.org/video/caregiving
r'\bhttps?://player\.pbs\.org/[\w-]+player/(\d+)', # last pattern to avoid false positives r'\bhttps?://player\.pbs\.org/[\w-]+player/(\d+)', # last pattern to avoid false positives
] ]

View File

@@ -4,6 +4,7 @@ from .common import InfoExtractor
from ..utils import ( from ..utils import (
ExtractorError, ExtractorError,
str_or_none, str_or_none,
strip_or_none,
traverse_obj, traverse_obj,
update_url, update_url,
) )
@@ -50,7 +51,6 @@ class PicartoIE(InfoExtractor):
if metadata.get('online') == 0: if metadata.get('online') == 0:
raise ExtractorError('Stream is offline', expected=True) raise ExtractorError('Stream is offline', expected=True)
title = metadata['title']
cdn_data = self._download_json(''.join(( cdn_data = self._download_json(''.join((
update_url(data['getLoadBalancerUrl']['url'], scheme='https'), update_url(data['getLoadBalancerUrl']['url'], scheme='https'),
@@ -79,7 +79,7 @@ class PicartoIE(InfoExtractor):
return { return {
'id': channel_id, 'id': channel_id,
'title': title.strip(), 'title': strip_or_none(metadata.get('title')),
'is_live': True, 'is_live': True,
'channel': channel_id, 'channel': channel_id,
'channel_id': metadata.get('id'), 'channel_id': metadata.get('id'),
@@ -159,7 +159,7 @@ class PicartoVodIE(InfoExtractor):
'id': video_id, 'id': video_id,
**traverse_obj(data, { **traverse_obj(data, {
'id': ('id', {str_or_none}), 'id': ('id', {str_or_none}),
'title': ('title', {str}), 'title': ('title', {str.strip}),
'thumbnail': 'video_recording_image_url', 'thumbnail': 'video_recording_image_url',
'channel': ('channel', 'name', {str}), 'channel': ('channel', 'name', {str}),
'age_limit': ('adult', {lambda x: 18 if x else 0}), 'age_limit': ('adult', {lambda x: 18 if x else 0}),

View File

@@ -24,6 +24,7 @@ from ..utils import (
url_or_none, url_or_none,
urlencode_postdata, urlencode_postdata,
) )
from ..utils.traversal import find_elements, traverse_obj
class PornHubBaseIE(InfoExtractor): class PornHubBaseIE(InfoExtractor):
@@ -127,7 +128,7 @@ class PornHubIE(PornHubBaseIE):
_VALID_URL = rf'''(?x) _VALID_URL = rf'''(?x)
https?:// https?://
(?: (?:
(?:[^/]+\.)? (?:[a-zA-Z0-9.-]+\.)?
{PornHubBaseIE._PORNHUB_HOST_RE} {PornHubBaseIE._PORNHUB_HOST_RE}
/(?:(?:view_video\.php|video/show)\?viewkey=|embed/)| /(?:(?:view_video\.php|video/show)\?viewkey=|embed/)|
(?:www\.)?thumbzilla\.com/video/ (?:www\.)?thumbzilla\.com/video/
@@ -137,23 +138,24 @@ class PornHubIE(PornHubBaseIE):
_EMBED_REGEX = [r'<iframe[^>]+?src=["\'](?P<url>(?:https?:)?//(?:www\.)?pornhub(?:premium)?\.(?:com|net|org)/embed/[\da-z]+)'] _EMBED_REGEX = [r'<iframe[^>]+?src=["\'](?P<url>(?:https?:)?//(?:www\.)?pornhub(?:premium)?\.(?:com|net|org)/embed/[\da-z]+)']
_TESTS = [{ _TESTS = [{
'url': 'http://www.pornhub.com/view_video.php?viewkey=648719015', 'url': 'http://www.pornhub.com/view_video.php?viewkey=648719015',
'md5': 'a6391306d050e4547f62b3f485dd9ba9', 'md5': '4d4a4e9178b655776f86cf89ecaf0edf',
'info_dict': { 'info_dict': {
'id': '648719015', 'id': '648719015',
'ext': 'mp4', 'ext': 'mp4',
'title': 'Seductive Indian beauty strips down and fingers her pink pussy', 'title': 'Seductive Indian beauty strips down and fingers her pink pussy',
'uploader': 'Babes', 'uploader': 'BABES-COM',
'uploader_id': '/users/babes-com',
'upload_date': '20130628', 'upload_date': '20130628',
'timestamp': 1372447216, 'timestamp': 1372447216,
'duration': 361, 'duration': 361,
'view_count': int, 'view_count': int,
'like_count': int, 'like_count': int,
'dislike_count': int,
'comment_count': int, 'comment_count': int,
'age_limit': 18, 'age_limit': 18,
'tags': list, 'tags': list,
'categories': list, 'categories': list,
'cast': list, 'cast': list,
'thumbnail': r're:https?://.+',
}, },
}, { }, {
# non-ASCII title # non-ASCII title
@@ -480,13 +482,6 @@ class PornHubIE(PornHubBaseIE):
comment_count = self._extract_count( comment_count = self._extract_count(
r'All Comments\s*<span>\(([\d,.]+)\)', webpage, 'comment') r'All Comments\s*<span>\(([\d,.]+)\)', webpage, 'comment')
def extract_list(meta_key):
div = self._search_regex(
rf'(?s)<div[^>]+\bclass=["\'].*?\b{meta_key}Wrapper[^>]*>(.+?)</div>',
webpage, meta_key, default=None)
if div:
return [clean_html(x).strip() for x in re.findall(r'(?s)<a[^>]+\bhref=[^>]+>.+?</a>', div)]
info = self._search_json_ld(webpage, video_id, default={}) info = self._search_json_ld(webpage, video_id, default={})
# description provided in JSON-LD is irrelevant # description provided in JSON-LD is irrelevant
info['description'] = None info['description'] = None
@@ -505,10 +500,13 @@ class PornHubIE(PornHubBaseIE):
'comment_count': comment_count, 'comment_count': comment_count,
'formats': formats, 'formats': formats,
'age_limit': 18, 'age_limit': 18,
'tags': extract_list('tags'), **traverse_obj(webpage, {
'categories': extract_list('categories'), 'tags': ({find_elements(attr='data-label', value='tag')}, ..., {clean_html}),
'cast': extract_list('pornstars'), 'categories': ({find_elements(attr='data-label', value='category')}, ..., {clean_html}),
'cast': ({find_elements(attr='data-label', value='pornstar')}, ..., {clean_html}),
}),
'subtitles': subtitles, 'subtitles': subtitles,
'http_headers': {'Referer': f'https://www.{host}/'},
}, info) }, info)
@@ -536,7 +534,7 @@ class PornHubPlaylistBaseIE(PornHubBaseIE):
class PornHubUserIE(PornHubPlaylistBaseIE): class PornHubUserIE(PornHubPlaylistBaseIE):
_VALID_URL = rf'(?P<url>https?://(?:[^/]+\.)?{PornHubBaseIE._PORNHUB_HOST_RE}/(?:(?:user|channel)s|model|pornstar)/(?P<id>[^/?#&]+))(?:[?#&]|/(?!videos)|$)' _VALID_URL = rf'(?P<url>https?://(?:[a-zA-Z0-9.-]+\.)?{PornHubBaseIE._PORNHUB_HOST_RE}/(?:(?:user|channel)s|model|pornstar)/(?P<id>[^/?#&]+))(?:[?#&]|/(?!videos)|$)'
_TESTS = [{ _TESTS = [{
'url': 'https://www.pornhub.com/model/zoe_ph', 'url': 'https://www.pornhub.com/model/zoe_ph',
'playlist_mincount': 118, 'playlist_mincount': 118,

View File

@@ -405,7 +405,7 @@ class RumbleChannelIE(InfoExtractor):
for video_url in traverse_obj( for video_url in traverse_obj(
get_elements_html_by_class('videostream__link', webpage), (..., {extract_attributes}, 'href'), get_elements_html_by_class('videostream__link', webpage), (..., {extract_attributes}, 'href'),
): ):
yield self.url_result(urljoin('https://rumble.com', video_url)) yield self.url_result(urljoin('https://rumble.com', video_url), RumbleIE)
def _real_extract(self, url): def _real_extract(self, url):
url, playlist_id = self._match_valid_url(url).groups() url, playlist_id = self._match_valid_url(url).groups()

View File

@@ -1,4 +1,4 @@
from .floatplane import FloatplaneBaseIE from .floatplane import FloatplaneBaseIE, FloatplaneChannelBaseIE
class SaucePlusIE(FloatplaneBaseIE): class SaucePlusIE(FloatplaneBaseIE):
@@ -39,3 +39,19 @@ class SaucePlusIE(FloatplaneBaseIE):
def _real_initialize(self): def _real_initialize(self):
if not self._get_cookies(self._BASE_URL).get('__Host-sp-sess'): if not self._get_cookies(self._BASE_URL).get('__Host-sp-sess'):
self.raise_login_required() self.raise_login_required()
class SaucePlusChannelIE(FloatplaneChannelBaseIE):
_VALID_URL = r'https?://(?:(?:www|beta)\.)?sauceplus\.com/channel/(?P<id>[\w-]+)/home(?:/(?P<channel>[\w-]+))?'
_BASE_URL = 'https://www.sauceplus.com'
_RESULT_IE = SaucePlusIE
_PAGE_SIZE = 20
_TESTS = [{
'url': 'https://www.sauceplus.com/channel/williamosman/home',
'info_dict': {
'id': 'williamosman',
'title': 'William Osman',
'description': 'md5:a67bc961d23c293b2c5308d84f34f26c',
},
'playlist_mincount': 158,
}]

View File

@@ -146,8 +146,8 @@ class SBSIE(InfoExtractor):
'release_year': ('releaseYear', {int_or_none}), 'release_year': ('releaseYear', {int_or_none}),
'duration': ('duration', ({float_or_none}, {parse_duration})), 'duration': ('duration', ({float_or_none}, {parse_duration})),
'is_live': ('liveStream', {bool}), 'is_live': ('liveStream', {bool}),
'age_limit': (('classificationID', 'contentRating'), {str.upper}, { 'age_limit': (
lambda x: self._AUS_TV_PARENTAL_GUIDELINES.get(x)}), # dict.get is unhashable in py3.7 ('classificationID', 'contentRating'), {str.upper}, {self._AUS_TV_PARENTAL_GUIDELINES.get}),
}, get_all=False), }, get_all=False),
**traverse_obj(media, { **traverse_obj(media, {
'categories': (('genres', ...), ('taxonomy', ('genre', 'subgenre'), 'name'), {str}), 'categories': (('genres', ...), ('taxonomy', ('genre', 'subgenre'), 'name'), {str}),

View File

@@ -1,137 +0,0 @@
import re
from .common import InfoExtractor
from ..utils import (
ExtractorError,
decode_packed_codes,
urlencode_postdata,
)
class SCTEBaseIE(InfoExtractor):
_LOGIN_URL = 'https://www.scte.org/SCTE/Sign_In.aspx'
_NETRC_MACHINE = 'scte'
def _perform_login(self, username, password):
login_popup = self._download_webpage(
self._LOGIN_URL, None, 'Downloading login popup')
def is_logged(webpage):
return any(re.search(p, webpage) for p in (
r'class=["\']welcome\b', r'>Sign Out<'))
# already logged in
if is_logged(login_popup):
return
login_form = self._hidden_inputs(login_popup)
login_form.update({
'ctl01$TemplateBody$WebPartManager1$gwpciNewContactSignInCommon$ciNewContactSignInCommon$signInUserName': username,
'ctl01$TemplateBody$WebPartManager1$gwpciNewContactSignInCommon$ciNewContactSignInCommon$signInPassword': password,
'ctl01$TemplateBody$WebPartManager1$gwpciNewContactSignInCommon$ciNewContactSignInCommon$RememberMe': 'on',
})
response = self._download_webpage(
self._LOGIN_URL, None, 'Logging in',
data=urlencode_postdata(login_form))
if '|pageRedirect|' not in response and not is_logged(response):
error = self._html_search_regex(
r'(?s)<[^>]+class=["\']AsiError["\'][^>]*>(.+?)</',
response, 'error message', default=None)
if error:
raise ExtractorError(f'Unable to login: {error}', expected=True)
raise ExtractorError('Unable to log in')
class SCTEIE(SCTEBaseIE):
_WORKING = False
_VALID_URL = r'https?://learning\.scte\.org/mod/scorm/view\.php?.*?\bid=(?P<id>\d+)'
_TESTS = [{
'url': 'https://learning.scte.org/mod/scorm/view.php?id=31484',
'info_dict': {
'title': 'Introduction to DOCSIS Engineering Professional',
'id': '31484',
},
'playlist_count': 5,
'skip': 'Requires account credentials',
}]
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
title = self._search_regex(r'<h1>(.+?)</h1>', webpage, 'title')
context_id = self._search_regex(r'context-(\d+)', webpage, video_id)
content_base = f'https://learning.scte.org/pluginfile.php/{context_id}/mod_scorm/content/8/'
context = decode_packed_codes(self._download_webpage(
f'{content_base}mobile/data.js', video_id))
data = self._parse_xml(
self._search_regex(
r'CreateData\(\s*"(.+?)"', context, 'data').replace(r"\'", "'"),
video_id)
entries = []
for asset in data.findall('.//asset'):
asset_url = asset.get('url')
if not asset_url or not asset_url.endswith('.mp4'):
continue
asset_id = self._search_regex(
r'video_([^_]+)_', asset_url, 'asset id', default=None)
if not asset_id:
continue
entries.append({
'id': asset_id,
'title': title,
'url': content_base + asset_url,
})
return self.playlist_result(entries, video_id, title)
class SCTECourseIE(SCTEBaseIE):
_WORKING = False
_VALID_URL = r'https?://learning\.scte\.org/(?:mod/sub)?course/view\.php?.*?\bid=(?P<id>\d+)'
_TESTS = [{
'url': 'https://learning.scte.org/mod/subcourse/view.php?id=31491',
'only_matching': True,
}, {
'url': 'https://learning.scte.org/course/view.php?id=3639',
'only_matching': True,
}, {
'url': 'https://learning.scte.org/course/view.php?id=3073',
'only_matching': True,
}]
def _real_extract(self, url):
course_id = self._match_id(url)
webpage = self._download_webpage(url, course_id)
title = self._search_regex(
r'<h1>(.+?)</h1>', webpage, 'title', default=None)
entries = []
for mobj in re.finditer(
r'''(?x)
<a[^>]+
href=(["\'])
(?P<url>
https?://learning\.scte\.org/mod/
(?P<kind>scorm|subcourse)/view\.php?(?:(?!\1).)*?
\bid=\d+
)
''',
webpage):
item_url = mobj.group('url')
if item_url == url:
continue
ie = (SCTEIE.ie_key() if mobj.group('kind') == 'scorm'
else SCTECourseIE.ie_key())
entries.append(self.url_result(item_url, ie=ie))
return self.playlist_result(entries, course_id, title)

View File

@@ -6,6 +6,7 @@ import re
from .common import InfoExtractor, SearchInfoExtractor from .common import InfoExtractor, SearchInfoExtractor
from ..networking import HEADRequest from ..networking import HEADRequest
from ..networking.exceptions import HTTPError from ..networking.exceptions import HTTPError
from ..networking.impersonate import ImpersonateTarget
from ..utils import ( from ..utils import (
ExtractorError, ExtractorError,
float_or_none, float_or_none,
@@ -118,9 +119,9 @@ class SoundcloudBaseIE(InfoExtractor):
self.cache.store('soundcloud', 'client_id', client_id) self.cache.store('soundcloud', 'client_id', client_id)
def _update_client_id(self): def _update_client_id(self):
webpage = self._download_webpage('https://soundcloud.com/', None) webpage = self._download_webpage('https://soundcloud.com/', None, 'Downloading main page')
for src in reversed(re.findall(r'<script[^>]+src="([^"]+)"', webpage)): for src in reversed(re.findall(r'<script[^>]+src="([^"]+)"', webpage)):
script = self._download_webpage(src, None, fatal=False) script = self._download_webpage(src, None, 'Downloading JS asset', fatal=False)
if script: if script:
client_id = self._search_regex( client_id = self._search_regex(
r'client_id\s*:\s*"([0-9a-zA-Z]{32})"', r'client_id\s*:\s*"([0-9a-zA-Z]{32})"',
@@ -136,13 +137,13 @@ class SoundcloudBaseIE(InfoExtractor):
if non_fatal: if non_fatal:
del kwargs['fatal'] del kwargs['fatal']
query = kwargs.get('query', {}).copy() query = kwargs.get('query', {}).copy()
for _ in range(2): for is_first_attempt in (True, False):
query['client_id'] = self._CLIENT_ID query['client_id'] = self._CLIENT_ID
kwargs['query'] = query kwargs['query'] = query
try: try:
return self._download_json(*args, **kwargs) return self._download_json(*args, **kwargs)
except ExtractorError as e: except ExtractorError as e:
if isinstance(e.cause, HTTPError) and e.cause.status in (401, 403): if is_first_attempt and isinstance(e.cause, HTTPError) and e.cause.status in (401, 403):
self._store_client_id(None) self._store_client_id(None)
self._update_client_id() self._update_client_id()
continue continue
@@ -152,7 +153,10 @@ class SoundcloudBaseIE(InfoExtractor):
raise raise
def _initialize_pre_login(self): def _initialize_pre_login(self):
self._CLIENT_ID = self.cache.load('soundcloud', 'client_id') or 'a3e059563d7fd3372b49b37f00a00bcf' self._CLIENT_ID = self.cache.load('soundcloud', 'client_id')
if self._CLIENT_ID:
return
self._update_client_id()
def _verify_oauth_token(self, token): def _verify_oauth_token(self, token):
if self._request_webpage( if self._request_webpage(
@@ -830,6 +834,30 @@ class SoundcloudPagedPlaylistBaseIE(SoundcloudBaseIE):
'entries': self._entries(base_url, playlist_id), 'entries': self._entries(base_url, playlist_id),
} }
@functools.cached_property
def _browser_impersonate_target(self):
available_targets = self._downloader._get_available_impersonate_targets()
if not available_targets:
# impersonate=True gives a generic warning when no impersonation targets are available
return True
# Any browser target older than chrome-116 is 403'd by Datadome
MIN_SUPPORTED_TARGET = ImpersonateTarget('chrome', '116', 'windows', '10')
version_as_float = lambda x: float(x.version) if x.version else 0
# Always try to use the newest Chrome target available
filtered = sorted([
target[0] for target in available_targets
if target[0].client == 'chrome' and target[0].os in ('windows', 'macos')
], key=version_as_float)
if not filtered or version_as_float(filtered[-1]) < version_as_float(MIN_SUPPORTED_TARGET):
# All available targets are inadequate or newest available Chrome target is too old, so
# warn the user to upgrade their dependency to a version with the minimum supported target
return MIN_SUPPORTED_TARGET
return filtered[-1]
def _entries(self, url, playlist_id): def _entries(self, url, playlist_id):
# Per the SoundCloud documentation, the maximum limit for a linked partitioning query is 200. # Per the SoundCloud documentation, the maximum limit for a linked partitioning query is 200.
# https://developers.soundcloud.com/blog/offset-pagination-deprecated # https://developers.soundcloud.com/blog/offset-pagination-deprecated
@@ -844,7 +872,9 @@ class SoundcloudPagedPlaylistBaseIE(SoundcloudBaseIE):
try: try:
response = self._call_api( response = self._call_api(
url, playlist_id, query=query, headers=self._HEADERS, url, playlist_id, query=query, headers=self._HEADERS,
note=f'Downloading track page {i + 1}') note=f'Downloading track page {i + 1}',
# See: https://github.com/yt-dlp/yt-dlp/issues/15660
impersonate=self._browser_impersonate_target)
break break
except ExtractorError as e: except ExtractorError as e:
# Downloading page may result in intermittent 502 HTTP error # Downloading page may result in intermittent 502 HTTP error

View File

@@ -3,6 +3,7 @@ import re
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ( from ..utils import (
ExtractorError, ExtractorError,
clean_html,
determine_ext, determine_ext,
merge_dicts, merge_dicts,
parse_duration, parse_duration,
@@ -12,6 +13,7 @@ from ..utils import (
urlencode_postdata, urlencode_postdata,
urljoin, urljoin,
) )
from ..utils.traversal import find_element, traverse_obj, trim_str
class SpankBangIE(InfoExtractor): class SpankBangIE(InfoExtractor):
@@ -122,7 +124,7 @@ class SpankBangIE(InfoExtractor):
}), headers={ }), headers={
'Referer': url, 'Referer': url,
'X-Requested-With': 'XMLHttpRequest', 'X-Requested-With': 'XMLHttpRequest',
}) }, impersonate=True)
for format_id, format_url in stream.items(): for format_id, format_url in stream.items():
if format_url and isinstance(format_url, list): if format_url and isinstance(format_url, list):
@@ -178,9 +180,9 @@ class SpankBangPlaylistIE(InfoExtractor):
def _real_extract(self, url): def _real_extract(self, url):
mobj = self._match_valid_url(url) mobj = self._match_valid_url(url)
playlist_id = mobj.group('id') playlist_id = mobj.group('id')
country = self.get_param('geo_bypass_country') or 'US'
webpage = self._download_webpage( self._set_cookie('.spankbang.com', 'country', country.upper())
url, playlist_id, headers={'Cookie': 'country=US; mobile=on'}) webpage = self._download_webpage(url, playlist_id, impersonate=True)
entries = [self.url_result( entries = [self.url_result(
urljoin(url, mobj.group('path')), urljoin(url, mobj.group('path')),
@@ -189,8 +191,8 @@ class SpankBangPlaylistIE(InfoExtractor):
r'<a[^>]+\bhref=(["\'])(?P<path>/?[\da-z]+-(?P<id>[\da-z]+)/playlist/[^"\'](?:(?!\1).)*)\1', r'<a[^>]+\bhref=(["\'])(?P<path>/?[\da-z]+-(?P<id>[\da-z]+)/playlist/[^"\'](?:(?!\1).)*)\1',
webpage)] webpage)]
title = self._html_search_regex( title = traverse_obj(webpage, (
r'<em>([^<]+)</em>\s+playlist\s*<', webpage, 'playlist title', {find_element(tag='h1', attr='data-testid', value='playlist-title')},
fatal=False) {clean_html}, {trim_str(end=' Playlist')}))
return self.playlist_result(entries, playlist_id, title) return self.playlist_result(entries, playlist_id, title)

View File

@@ -8,15 +8,12 @@ from ..utils import (
extract_attributes, extract_attributes,
join_nonempty, join_nonempty,
js_to_json, js_to_json,
parse_resolution,
str_or_none, str_or_none,
url_basename,
url_or_none, url_or_none,
) )
from ..utils.traversal import ( from ..utils.traversal import find_element, traverse_obj
find_element,
find_elements,
traverse_obj,
trim_str,
)
class SteamIE(InfoExtractor): class SteamIE(InfoExtractor):
@@ -27,7 +24,7 @@ class SteamIE(InfoExtractor):
'id': '105600', 'id': '105600',
'title': 'Terraria', 'title': 'Terraria',
}, },
'playlist_mincount': 3, 'playlist_mincount': 5,
}, { }, {
'url': 'https://store.steampowered.com/app/271590/Grand_Theft_Auto_V/', 'url': 'https://store.steampowered.com/app/271590/Grand_Theft_Auto_V/',
'info_dict': { 'info_dict': {
@@ -37,6 +34,39 @@ class SteamIE(InfoExtractor):
'playlist_mincount': 26, 'playlist_mincount': 26,
}] }]
def _entries(self, app_id, app_name, data_props):
for trailer in traverse_obj(data_props, (
'trailers', lambda _, v: str_or_none(v['id']),
)):
movie_id = str_or_none(trailer['id'])
thumbnails = []
for thumbnail_url in traverse_obj(trailer, (
('poster', 'thumbnail'), {url_or_none},
)):
thumbnails.append({
'url': thumbnail_url,
**parse_resolution(url_basename(thumbnail_url)),
})
formats = []
if hls_manifest := traverse_obj(trailer, ('hlsManifest', {url_or_none})):
formats.extend(self._extract_m3u8_formats(
hls_manifest, app_id, 'mp4', m3u8_id='hls', fatal=False))
for dash_manifest in traverse_obj(trailer, ('dashManifests', ..., {url_or_none})):
formats.extend(self._extract_mpd_formats(
dash_manifest, app_id, mpd_id='dash', fatal=False))
self._remove_duplicate_formats(formats)
yield {
'id': join_nonempty(app_id, movie_id),
'title': join_nonempty(app_name, 'video', movie_id, delim=' '),
'formats': formats,
'series': app_name,
'series_id': app_id,
'thumbnails': thumbnails,
}
def _real_extract(self, url): def _real_extract(self, url):
app_id = self._match_id(url) app_id = self._match_id(url)
@@ -45,32 +75,13 @@ class SteamIE(InfoExtractor):
self._set_cookie('store.steampowered.com', 'lastagecheckage', '1-January-2000') self._set_cookie('store.steampowered.com', 'lastagecheckage', '1-January-2000')
webpage = self._download_webpage(url, app_id) webpage = self._download_webpage(url, app_id)
app_name = traverse_obj(webpage, ({find_element(cls='apphub_AppName')}, {clean_html})) data_props = traverse_obj(webpage, (
{find_element(cls='gamehighlight_desktopcarousel', html=True)},
{extract_attributes}, 'data-props', {json.loads}, {dict}))
app_name = traverse_obj(data_props, ('appName', {clean_html}))
entries = [] return self.playlist_result(
for data_prop in traverse_obj(webpage, ( self._entries(app_id, app_name, data_props), app_id, app_name)
{find_elements(cls='highlight_player_item highlight_movie', html=True)},
..., {extract_attributes}, 'data-props', {json.loads}, {dict},
)):
formats = []
if hls_manifest := traverse_obj(data_prop, ('hlsManifest', {url_or_none})):
formats.extend(self._extract_m3u8_formats(
hls_manifest, app_id, 'mp4', m3u8_id='hls', fatal=False))
for dash_manifest in traverse_obj(data_prop, ('dashManifests', ..., {url_or_none})):
formats.extend(self._extract_mpd_formats(
dash_manifest, app_id, mpd_id='dash', fatal=False))
movie_id = traverse_obj(data_prop, ('id', {trim_str(start='highlight_movie_')}))
entries.append({
'id': movie_id,
'title': join_nonempty(app_name, 'video', movie_id, delim=' '),
'formats': formats,
'series': app_name,
'series_id': app_id,
'thumbnail': traverse_obj(data_prop, ('screenshot', {url_or_none})),
})
return self.playlist_result(entries, app_id, app_name)
class SteamCommunityIE(InfoExtractor): class SteamCommunityIE(InfoExtractor):

View File

@@ -22,7 +22,7 @@ class StreaksBaseIE(InfoExtractor):
_GEO_BYPASS = False _GEO_BYPASS = False
_GEO_COUNTRIES = ['JP'] _GEO_COUNTRIES = ['JP']
def _extract_from_streaks_api(self, project_id, media_id, headers=None, query=None, ssai=False): def _extract_from_streaks_api(self, project_id, media_id, headers=None, query=None, ssai=False, live_from_start=False):
try: try:
response = self._download_json( response = self._download_json(
self._API_URL_TEMPLATE.format('playback', project_id, media_id, ''), self._API_URL_TEMPLATE.format('playback', project_id, media_id, ''),
@@ -83,6 +83,10 @@ class StreaksBaseIE(InfoExtractor):
fmts, subs = self._extract_m3u8_formats_and_subtitles( fmts, subs = self._extract_m3u8_formats_and_subtitles(
src_url, media_id, 'mp4', m3u8_id='hls', fatal=False, live=is_live, query=query) src_url, media_id, 'mp4', m3u8_id='hls', fatal=False, live=is_live, query=query)
for fmt in fmts:
if live_from_start:
fmt.setdefault('downloader_options', {}).update({'ffmpeg_args': ['-live_start_index', '0']})
fmt['is_from_start'] = True
formats.extend(fmts) formats.extend(fmts)
self._merge_subtitles(subs, target=subtitles) self._merge_subtitles(subs, target=subtitles)

View File

@@ -0,0 +1,244 @@
import base64
import binascii
import functools
import re
import urllib.parse
from .common import InfoExtractor
from ..dependencies import Cryptodome
from ..utils import (
ExtractorError,
OnDemandPagedList,
clean_html,
extract_attributes,
url_or_none,
urljoin,
)
from ..utils.traversal import (
find_element,
find_elements,
require,
traverse_obj,
)
class TarangPlusBaseIE(InfoExtractor):
_BASE_URL = 'https://tarangplus.in'
class TarangPlusVideoIE(TarangPlusBaseIE):
IE_NAME = 'tarangplus:video'
_VALID_URL = r'https?://(?:www\.)?tarangplus\.in/(?:movies|[^#?/]+/[^#?/]+)/(?!episodes)(?P<id>[^#?/]+)'
_TESTS = [{
'url': 'https://tarangplus.in/tarangaplus-originals/khitpit/khitpit-ep-10',
'md5': '78ce056cee755687b8a48199909ecf53',
'info_dict': {
'id': '67b8206719521d054c0059b7',
'display_id': 'khitpit-ep-10',
'ext': 'mp4',
'title': 'Khitpit Ep-10',
'description': 'md5:a45b805cb628e15c853d78b0406eab48',
'thumbnail': r're:https?://.+/.+\.jpg',
'duration': 756.0,
'timestamp': 1740355200,
'upload_date': '20250224',
'media_type': 'episode',
'categories': ['Originals'],
},
}, {
'url': 'https://tarangplus.in/tarang-serials/bada-bohu/bada-bohu-ep-233',
'md5': 'b4f9beb15172559bb362203b4f48382e',
'info_dict': {
'id': '680b9d6c19521d054c007782',
'display_id': 'bada-bohu-ep-233',
'ext': 'mp4',
'title': 'Bada Bohu | Ep -233',
'description': 'md5:e6b8e7edc9e60b92c1b390f8789ecd69',
'thumbnail': r're:https?://.+/.+\.jpg',
'duration': 1392.0,
'timestamp': 1745539200,
'upload_date': '20250425',
'media_type': 'episode',
'categories': ['Prime'],
},
}, {
# Decrypted m3u8 URL has trailing control characters that need to be stripped
'url': 'https://tarangplus.in/tarangaplus-originals/ichha/ichha-teaser-1',
'md5': '16ee43fe21ad8b6e652ec65eba38a64e',
'info_dict': {
'id': '5f0f252d3326af0720000342',
'ext': 'mp4',
'display_id': 'ichha-teaser-1',
'title': 'Ichha Teaser',
'description': 'md5:c724b0b0669a2cefdada3711cec792e6',
'media_type': 'episode',
'duration': 21.0,
'thumbnail': r're:https?://.+/.+\.jpg',
'categories': ['Originals'],
'timestamp': 1758153600,
'upload_date': '20250918',
},
}, {
'url': 'https://tarangplus.in/short/ai-maa/ai-maa',
'only_matching': True,
}, {
'url': 'https://tarangplus.in/shows/tarang-cine-utsav-2024/tarang-cine-utsav-2024-seg-1',
'only_matching': True,
}, {
'url': 'https://tarangplus.in/music-videos/chori-chori-bohu-chori-songs/nijara-laguchu-dhire-dhire',
'only_matching': True,
}, {
'url': 'https://tarangplus.in/kids-shows/chhota-jaga/chhota-jaga-ep-33-jamidar-ra-khajana-adaya',
'only_matching': True,
}, {
'url': 'https://tarangplus.in/movies/swayambara',
'only_matching': True,
}]
def decrypt(self, data, key):
if not Cryptodome.AES:
raise ExtractorError('pycryptodomex not found. Please install', expected=True)
iv = binascii.unhexlify('00000000000000000000000000000000')
cipher = Cryptodome.AES.new(base64.b64decode(key), Cryptodome.AES.MODE_CBC, iv)
return cipher.decrypt(base64.b64decode(data)).decode('utf-8')
def _real_extract(self, url):
display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id)
hidden_inputs_data = self._hidden_inputs(webpage)
json_ld_data = self._search_json_ld(webpage, display_id)
json_ld_data.pop('url', None)
iframe_url = traverse_obj(webpage, (
{find_element(tag='iframe', attr='src', value=r'.+[?&]contenturl=.+', html=True, regex=True)},
{extract_attributes}, 'src', {require('iframe URL')}))
# Can't use parse_qs here since it would decode the encrypted base64 `+` chars to spaces
content = self._search_regex(r'[?&]contenturl=(.+)', iframe_url, 'content')
encrypted_data, _, attrs = content.partition('|')
metadata = {
m.group('k'): m.group('v')
for m in re.finditer(r'(?:^|\|)(?P<k>[a-z_]+)=(?P<v>(?:(?!\|[a-z_]+=).)+)', attrs)
}
m3u8_url = urllib.parse.unquote(
self.decrypt(encrypted_data, metadata['key'])).rstrip('\x0e\x0f')
return {
'id': display_id, # Fallback
'display_id': display_id,
**json_ld_data,
**traverse_obj(metadata, {
'id': ('content_id', {str}),
'title': ('title', {str}),
'thumbnail': ('image', {url_or_none}),
}),
**traverse_obj(hidden_inputs_data, {
'id': ('content_id', {str}),
'media_type': ('theme_type', {str}),
'categories': ('genre', {str}, filter, all, filter),
}),
'formats': self._extract_m3u8_formats(m3u8_url, display_id),
}
class TarangPlusEpisodesIE(TarangPlusBaseIE):
IE_NAME = 'tarangplus:episodes'
_VALID_URL = r'https?://(?:www\.)?tarangplus\.in/(?P<type>[^#?/]+)/(?P<id>[^#?/]+)/episodes/?(?:$|[?#])'
_TESTS = [{
'url': 'https://tarangplus.in/tarangaplus-originals/balijatra/episodes',
'info_dict': {
'id': 'balijatra',
'title': 'Balijatra',
},
'playlist_mincount': 7,
}, {
'url': 'https://tarangplus.in/tarang-serials/bada-bohu/episodes',
'info_dict': {
'id': 'bada-bohu',
'title': 'Bada Bohu',
},
'playlist_mincount': 236,
}, {
'url': 'https://tarangplus.in/shows/dr-nonsense/episodes',
'info_dict': {
'id': 'dr-nonsense',
'title': 'Dr. Nonsense',
},
'playlist_mincount': 15,
}]
_PAGE_SIZE = 20
def _entries(self, playlist_url, playlist_id, page):
data = self._download_json(
playlist_url, playlist_id, f'Downloading playlist JSON page {page + 1}',
query={'page_no': page})
for item in traverse_obj(data, ('items', ..., {str})):
yield self.url_result(
urljoin(self._BASE_URL, item.split('$')[3]), TarangPlusVideoIE)
def _real_extract(self, url):
url_type, display_id = self._match_valid_url(url).group('type', 'id')
series_url = f'{self._BASE_URL}/{url_type}/{display_id}'
webpage = self._download_webpage(series_url, display_id)
entries = OnDemandPagedList(
functools.partial(self._entries, f'{series_url}/episodes', display_id),
self._PAGE_SIZE)
return self.playlist_result(
entries, display_id, self._hidden_inputs(webpage).get('title'))
class TarangPlusPlaylistIE(TarangPlusBaseIE):
IE_NAME = 'tarangplus:playlist'
_VALID_URL = r'https?://(?:www\.)?tarangplus\.in/(?P<id>[^#?/]+)/all/?(?:$|[?#])'
_TESTS = [{
'url': 'https://tarangplus.in/chhota-jaga/all',
'info_dict': {
'id': 'chhota-jaga',
'title': 'Chhota Jaga',
},
'playlist_mincount': 33,
}, {
'url': 'https://tarangplus.in/kids-yali-show/all',
'info_dict': {
'id': 'kids-yali-show',
'title': 'Yali',
},
'playlist_mincount': 10,
}, {
'url': 'https://tarangplus.in/trailer/all',
'info_dict': {
'id': 'trailer',
'title': 'Trailer',
},
'playlist_mincount': 57,
}, {
'url': 'https://tarangplus.in/latest-songs/all',
'info_dict': {
'id': 'latest-songs',
'title': 'Latest Songs',
},
'playlist_mincount': 46,
}, {
'url': 'https://tarangplus.in/premium-serials-episodes/all',
'info_dict': {
'id': 'premium-serials-episodes',
'title': 'Primetime Latest Episodes',
},
'playlist_mincount': 100,
}]
def _entries(self, webpage):
for url_path in traverse_obj(webpage, (
{find_elements(cls='item')}, ...,
{find_elements(tag='a', attr='href', value='/.+', html=True, regex=True)},
..., {extract_attributes}, 'href',
)):
yield self.url_result(urljoin(self._BASE_URL, url_path), TarangPlusVideoIE)
def _real_extract(self, url):
display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id)
return self.playlist_result(
self._entries(webpage), display_id,
traverse_obj(webpage, ({find_element(id='al_title')}, {clean_html})))

View File

@@ -102,7 +102,7 @@ class TeachableIE(TeachableBaseIE):
_WORKING = False _WORKING = False
_VALID_URL = r'''(?x) _VALID_URL = r'''(?x)
(?: (?:
{}https?://(?P<site_t>[^/]+)| {}https?://(?P<site_t>[a-zA-Z0-9.-]+)|
https?://(?:www\.)?(?P<site>{}) https?://(?:www\.)?(?P<site>{})
) )
/courses/[^/]+/lectures/(?P<id>\d+) /courses/[^/]+/lectures/(?P<id>\d+)
@@ -211,7 +211,7 @@ class TeachableIE(TeachableBaseIE):
class TeachableCourseIE(TeachableBaseIE): class TeachableCourseIE(TeachableBaseIE):
_VALID_URL = r'''(?x) _VALID_URL = r'''(?x)
(?: (?:
{}https?://(?P<site_t>[^/]+)| {}https?://(?P<site_t>[a-zA-Z0-9.-]+)|
https?://(?:www\.)?(?P<site>{}) https?://(?:www\.)?(?P<site>{})
) )
/(?:courses|p)/(?:enrolled/)?(?P<id>[^/?#&]+) /(?:courses|p)/(?:enrolled/)?(?P<id>[^/?#&]+)

Some files were not shown because too many files have changed in this diff Show More