Commit Graph

599 Commits

Author SHA1 Message Date
wukko
aa376d76f6
api/stream/types: huge refactor & simplification of code
- created render() which handles ffmpeg & piping stuff
- merged remux() and merge() into one function
- simplified and cleaned up arguments
- removed headers since they're handled by internal streams now
- removed outdated arguments
2025-06-24 19:55:50 +06:00
wukko
28b85380c9
api/vk: allow auto generated subs & pick explicitly vtt
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
i couldn't find a single video that had any subtitles other than auto generated ones, so i think this is better than nothing at all
2025-06-24 17:56:04 +06:00
wukko
75691d4bac
api/tests/facebook: replace a dead link 2025-06-24 17:28:05 +06:00
wukko
ff06a10b5c
api/processing/url: improve vk url parsing 2025-06-24 17:21:32 +06:00
wukko
997b06ed0e
api/vk: add support for subtitles 2025-06-24 17:06:19 +06:00
wukko
44f4ea32c6
api/stream/internal: stream vk videos in chunks 2025-06-24 17:04:43 +06:00
wukko
6d62bce92d
api/match-action: don't force local-processing response for pickers
cuz that won't work, at least for now
2025-06-22 20:12:22 +06:00
wukko
21c4a1ebbc
api/match: set alwaysProxy to true if local processing is forced 2025-06-22 20:09:48 +06:00
wukko
0fca9c440c
api/schema: remove deprecated variables 2025-06-22 20:07:37 +06:00
wukko
05fb1601c8
api/match: update forcing local processing via env 2025-06-22 20:06:28 +06:00
wukko
ac85ce86c0
api/processing/request: backwards compat with boolean localProcessing 2025-06-22 16:21:55 +06:00
wukko
28ab2747ce
api/match-action: support forced local processing 2025-06-22 16:21:37 +06:00
wukko
a6b599a828
api/schema: transform localProcessing to enum 2025-06-22 16:20:27 +06:00
wukko
630e4a6e0d
api/tiktok: add support for subtitles 2025-06-20 20:07:50 +06:00
wukko
aff2d22edc
api/language-codes: add reverse lookup (2 to 1) 2025-06-20 20:05:17 +06:00
wukko
d18b22e7ed
api/processing/request: return a unique error code 2025-06-20 19:53:01 +06:00
wukko
ab526c234e
api/loom: add transcription subtitles
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / api sanity check (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
since there's no language selection (at all), we just add the only transcription if a user wants subtitles
2025-06-20 18:59:35 +06:00
wukko
a44bea6b50
api/vimeo: add subtitle parsing from the mobile api 2025-06-20 18:21:00 +06:00
wukko
a5838f3c05
api/stream/types: add subtitles & metadata to remux 2025-06-20 18:16:32 +06:00
wukko
337edfc984
api/request/local-processing: return subtitles boolean 2025-06-20 17:38:49 +06:00
wukko
7ce9d6882b
api/youtube: don't use session if user wants subtitles
cuz they're not currently available anywhere but HLS
2025-06-20 17:27:49 +06:00
wukko
c4e910dd29
api/stream/types: refactor, support mkv, don't duplicate args 2025-06-20 14:39:17 +06:00
wukko
33c801f66b
api/youtube: add support for youtubeVideoContainer 2025-06-20 14:35:43 +06:00
wukko
eb249a3eed
api/match: ignore subtitleLang if it's "none" 2025-06-20 14:35:14 +06:00
wukko
2396462c5c
api/schema: add youtubeVideoContainer 2025-06-20 14:23:16 +06:00
wukko
672b3dcf46
api/match-action: convert ISO 639-1 language codes to ISO 639-2
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-06-19 17:46:15 +06:00
wukko
b91c0c0013
api/stream/types: specify subtitle format for containers other than mp4 2025-06-19 17:19:39 +06:00
wukko
259a0758f1
api: initial subtitles functionality with youtube support
this took way more effort than i expected it to do, mostly because of youtube locking everything down to shit

local processing doesn't function with subtitles yet, wasm needs to be updated
2025-06-18 20:24:13 +06:00
wukko
967552b26b
api/schema: add subtitleLang 2025-06-18 18:37:24 +06:00
wukko
b8801570a9
api/env: SESSION_RATELIMIT -> SESSION_RATELIMIT_MAX 2025-06-17 13:33:50 +06:00
wukko
af99e7218c
api: disable youtube HLS by default & add env to enable it 2025-06-17 13:21:16 +06:00
wukko
5e7f9c53b9
api/package: update youtubei.js to 14.0.0 2025-06-16 13:53:04 +06:00
wukko
ed980e3893
api/package: bump version to 11.1
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-06-08 20:52:12 +06:00
wukko
4b6447cba6
api/youtube: use the original track instead of default
closes #1329

default ≠ original, apparently. not sure why youtube thought it's a good idea to force ai generated dubs as default
2025-06-08 19:30:43 +06:00
jj
adc5b89fc2
api/soundcloud: ignore encrypted protocols, match against prefix 2025-06-08 13:17:14 +00:00
jj
2154f464d7
api/soundcloud: prefer progressive format over hls 2025-06-08 12:44:34 +00:00
wukko
eae6a7aa63
api/tests/soundcloud: update the go+ test link 2025-06-08 18:17:07 +06:00
hyperdefined
495729e174
api/pinterest: return fetch.empty if a link is invalid (#1299) 2025-06-08 18:11:05 +06:00
nexpid
170cf293bf
api/soundcloud: add more metadata fields (#1313) 2025-06-08 18:07:38 +06:00
jj
19c036494f
api/cluster: version check for supportsReusePort()
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2025-06-07 09:24:33 +00:00
jj
212b07394d
api: set up env watcher only after cluster is initialized 2025-06-07 09:23:09 +00:00
jj
e2b6879ea2
api/env: log information about dynamic env changes 2025-06-07 08:35:36 +00:00
wukko
ad96155831
api/package: bump version to 11.0.3 2025-06-07 14:04:39 +06:00
wukko
d6095db619
api/service-config/youtube: add /v/:id link pattern
closes #1327
2025-06-07 13:12:04 +06:00
wukko
df7819daa1
api/tests/twitter: update some test links to more popular ones
so that twitter doesn't require someone with an account to view the post before making it available for logged out access. really annoying & makes tests fail
2025-06-07 12:00:51 +06:00
wukko
10e6b4ec71
api/service-alias: add an alias for twitch
because only clips are supported for now. vods may be supported after we implement HLS local processing
2025-06-07 11:51:44 +06:00
wukko
5cd5013de0
api/service-config/vimeo: add /groups/ link pattern
closes #1324
2025-06-07 11:49:40 +06:00
jj
7f5a9cfa75
api/config: remove unused cluster import
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
Run service tests / test service functionality (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-05-31 13:51:57 +00:00
jj
a7bf5c525d
api/package: bump version to 11.0.2 2025-05-31 13:49:15 +00:00
jj
57eba51959
api/env: broadcast raw contents instead of parsed 2025-05-31 13:43:30 +00:00
jj
1c304457e2
api/package: bump version to 11.0.1
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
Run service tests / test service functionality (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-05-29 17:22:03 +00:00
jj
ed18008493
api/soundcloud: return isHLS flag when appropriate
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2025-05-29 17:08:22 +00:00
jj
ff7eb2639d
api/loom: add support for non-transcoded links, add more tests 2025-05-29 15:29:53 +00:00
wukko
5e3d6107f9
api/stream/manage: fix usage of getInternalTunnel()
fixed a typo
2025-05-29 20:20:56 +06:00
wukko
181669f949
api/package: bump version to 11.0 2025-05-29 20:10:28 +06:00
wukko
3c41585158
api/schema: add old variables from cobalt 10 for backwards compatibility 2025-05-29 17:55:52 +06:00
wukko
c2d76010c5
api/core: remove durationLimit from server info
it's not used, no clue why it was here in the first place
2025-05-27 17:12:58 +06:00
wukko
0b36aa09a7
api/match: limit the duration limit number to 2 decimal places 2025-05-27 17:10:41 +06:00
jj
c392864c82
api/env: unquote variables if needed
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-05-25 12:44:40 +00:00
jj
ba2d266de7
api: dynamic env reloading from path/url 2025-05-24 15:52:27 +00:00
jj
e76ccd1941
api: move env loading into separate file 2025-05-24 14:47:01 +00:00
jj
06ee65b55d
api/api-keys: watch for file changes instead of polling 2025-05-24 14:32:50 +00:00
jj
2eadc3fbd8
api/create-filename: relax sanitizeString and use fullwidth replacements 2025-05-23 07:44:05 +00:00
wukko
f36c749692
api/twitter: add support for saving media from ad cards
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / api sanity check (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
2025-05-23 13:06:51 +06:00
wukko
0ae0bbfa1f
api/core: use new tunnel rate limit env 2025-05-14 19:07:11 +06:00
wukko
a66e789317
api/config: add tunnel rate limit env 2025-05-14 19:06:59 +06:00
wukko
06b865e965
api/config: limit acceptable FORCE_LOCAL_PROCESSING values
and throw an error if the value is wrong
2025-05-06 17:33:06 +06:00
wukko
4d582798bf
api/match: force local processing when configured to do so in env 2025-05-06 16:16:52 +06:00
wukko
c5acb45557
api/core: mark request as session when bearer token is used
and pass it to match() for future consumption
2025-05-06 16:16:29 +06:00
wukko
d0539118ce
api/config: add FORCE_LOCAL_PROCESSING 2025-05-06 16:12:51 +06:00
wukko
a95f87ebfb
api & web: make "basic" the default filename style 2025-04-30 21:33:36 +06:00
wukko
9a3d35185b
merge: soundcloud fix from main 2025-04-30 13:11:02 +06:00
wukko
4b9644ebdf
api/package: bump version to 10.9.4
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-04-29 23:10:18 +06:00
wukko
00b217796f
api/soundcloud: fix short link url & refactor 2025-04-29 22:55:52 +06:00
wukko
4202c954d1
merge: api 10.9.3 from main 2025-04-24 16:27:17 +06:00
wukko
de6b611c41
api/package: bump version to 10.9.3
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-04-24 16:15:09 +06:00
wukko
d0deec546b
api/service-config: add new snapchat link pattern 2025-04-24 15:44:52 +06:00
wukko
7ff6d0036b
api/snapchat: prevent errors if params are undefined 2025-04-24 15:44:34 +06:00
wukko
1335313e39
api/service-patterns: increase xiaohongshu shareId max length 2025-04-24 15:30:52 +06:00
wukko
c1b592430a
merge: api updates from main 2025-04-23 21:10:52 +06:00
jj
6f6f885723
api/youtube: update Session arguments to match new constructor 2025-04-23 14:02:27 +00:00
wukko
678f3a6c57
package: update youtubei.js to 13.4.0 2025-04-23 18:58:00 +06:00
wukko
335cd51eb5
api: add an option to allow better audio from youtube
& an env variable to disable it if needed
2025-04-20 20:45:07 +06:00
wukko
06bc51db54
merge: 10.9 from main
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-04-02 21:48:58 +06:00
wukko
fc050d78e2
api/package: bump version to 10.9.1
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-04-02 21:41:43 +06:00
jj
07f81c5d1d
api/reddit: clean up duplicated headers 2025-04-02 12:35:45 +00:00
jj
f5df78ffec
api/utils: retry getting redirecting url with fetch() if request() fails 2025-04-02 12:29:18 +00:00
Felix Vuong 🍂
a6240d0192
api/url: replace user-agent argument with headers in redirect helpers 2025-04-02 11:44:13 +00:00
Felix Vuong 🍂
b1bde25dee
api/reddit: add support for short links 2025-04-02 11:42:59 +00:00
wukko
1477dcd4e7
api/tests/instagram: allow the private post test to fail
sometimes the visibility status isn't returned
2025-04-02 17:35:01 +06:00
wukko
d352eed85f
api/package: bump version to 10.9 2025-04-01 15:35:25 +06:00
wukko
f70f88bc4c
api/core: customizable session rate limit params
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-03-31 22:32:21 +06:00
jj
1f768df4ec
api: bind session tokens to ip hash 2025-03-30 17:08:34 +00:00
wukko
bf5937e336
api/package: bump version to 10.8.4
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-03-29 19:48:11 +06:00
jj
39274d88f6
api/youtube-session: bypass proxy for requests
usually the session server is hosted locally, which
means the proxy tries to access the wrong url
2025-03-29 10:28:11 +00:00
wukko
0b29121c53
merge: changes from main 2025-03-27 20:02:46 +06:00
wukko
2d38d63003
api/package: update youtubei.js to 13.3.0
Some checks failed
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2025-03-25 19:11:19 +06:00
wukko
5036c492b8
api/service-config/tiktok: remove trailing forward slash from a pattern 2025-03-25 18:32:05 +06:00
wukko
ab13f78326
api/tiktok: normalize short link URL & catch empty patternMatch 2025-03-25 18:31:12 +06:00
Felix Vuong 🍂
2f38260e23 api/service-config: add tiktok lite url pattern 2025-03-25 18:11:49 +07:00