Merge branch 'current' of https://github.com/wukko/cobalt into current

This commit is contained in:
xhaeffer 2024-04-29 23:37:35 +07:00
commit ea98286614
42 changed files with 659 additions and 203 deletions

View File

@ -1,7 +1,7 @@
# cobalt
best way to save what you love: [cobalt.tools](https://cobalt.tools/)
![cobalt logo with repeated logo (double arrow) pattern background](https://raw.githubusercontent.com/wukko/cobalt/current/src/front/icons/pattern.png "cobalt logo with repeated logo (double arrow) pattern background")
![cobalt logo with repeated logo (double arrow) pattern background](/src/front/icons/pattern.png "cobalt logo with repeated logo (double arrow) pattern background")
## what's cobalt?
cobalt is a media downloader that doesn't piss you off. it's fast, friendly, and doesn't have any bullshit that modern web is filled with: ***no ads, trackers, or invasive analytics***.
@ -51,12 +51,15 @@ this list is not final and keeps expanding over time. if support for a service y
| youtube | supports videos, music, and shorts. 8K, 4K, HDR, VR, and high FPS videos. rich metadata & dubs. h264/av1/vp9 codecs. |
## cobalt api
cobalt has an open api that you can use in projects *for completely free~*. it's easy and straightforward to use, [check out the docs](https://github.com/wukko/cobalt/blob/current/docs/api.md) to learn how to use it.
cobalt has an open api that you can use in your projects *for free~*. it's easy and straightforward to use, [check out the docs](/docs/api.md) to learn how to use it.
you can use the main api instance ([co.wuk.sh](https://co.wuk.sh/)) in your projects.
✅ you can use the main api instance ([co.wuk.sh](https://co.wuk.sh/)) in your **personal** projects.
❌ you cannot use the free api commercially (anywhere that's gated behind paywalls or ads). host your own instance for this.
we reserve the right to restrict abusive/excessive access to the main instance api.
## how to run your own instance
if you want to run your own instance for whatever purpose, [follow this guide](https://github.com/wukko/cobalt/blob/current/docs/run-an-instance.md).
if you want to run your own instance for whatever purpose, [follow this guide](/docs/run-an-instance.md).
it's *highly* recommended to use a docker compose method unless you run for developing/debugging purposes.
## sponsors
@ -68,7 +71,7 @@ cobalt is a tool for easing content downloads from internet and takes ***zero li
cobalt is ***NOT*** a piracy tool and cannot be used as such. it can only download free, publicly accessible content. such content can be easily downloaded through any browser's dev tools. pressing one button is easier, so i made a convenient, ad-less tool for such repeated actions.
## cobalt license
cobalt code is licensed under [AGPL-3.0](https://github.com/wukko/cobalt/blob/current/LICENSE).
cobalt code is licensed under [AGPL-3.0](/LICENSE).
cobalt branding, mascots, and other related assets included in the repo are ***copyrighted*** and not covered by the AGPL-3.0 license. you ***cannot*** use them under same terms.

View File

@ -59,7 +59,7 @@ from a successful call to `/api/json`. however, the parameters passed to it are
and **unmodifiable** from your (the api client's) perspective, and can change between versions.
therefore you don't need to worry about what they mean - but if you really want to know, you can
[read the source code](../src/modules/stream/manage.js).
[read the source code](/src/modules/stream/manage.js).
## GET: `/api/serverInfo`
returns current basic server info.

View File

@ -21,6 +21,8 @@ services:
API_URL: "https://co.wuk.sh/"
# replace eu-nl with your instance's distinctive name
API_NAME: "eu-nl"
# if you want to use cookies when fetching data from services, uncomment the next line and the lines under volume
# COOKIE_PATH: "/cookies.json"
# see docs/run-an-instance.md for more information
labels:
- com.centurylinklabs.watchtower.scope=cobalt

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

View File

@ -18,7 +18,7 @@ if you need help with installing docker, follow *only the first step* of these t
```
i'm using `nano` in this example, it may not be available in your distro. you can use any other text editor.
3. copy and paste the [sample config from here](https://github.com/wukko/cobalt/blob/current/docs/examples/docker-compose.example.yml) for either web or api instance (or both, if you wish) and edit it to your needs.
3. copy and paste the [sample config from here](examples/docker-compose.example.yml) for either web or api instance (or both, if you wish) and edit it to your needs.
make sure to replace default URLs with your own or cobalt won't work correctly.
4. finally, start the cobalt container (from cobalt directory):
@ -26,7 +26,7 @@ if you need help with installing docker, follow *only the first step* of these t
docker compose up -d
```
if you want your instance to support services that require authentication to view public content, create `cookies.json` file in the same directory as `docker-compose.yml`. example cookies file [can be found here](https://github.com/wukko/cobalt/blob/current/docs/examples/cookies.example.json).
if you want your instance to support services that require authentication to view public content, create `cookies.json` file in the same directory as `docker-compose.yml`. example cookies file [can be found here](examples/cookies.example.json).
cobalt package will update automatically thanks to watchtower.

View File

@ -5,29 +5,33 @@
if any issues occur while using cobalt, you can fix many of them yourself. this document aims to provide guides on how to fix most complicated of them.
use wiki navigation on right to jump between solutions.
## how to fix clipboard pasting in firefox
## how to fix clipboard pasting in older versions of firefox
```
🎉 firefox finally supports pasting by default starting from version 125.
👍 you don't need to follow this tutorial if you're on the latest version of firefox.
```
you can fix this issue by changing a single preference in `about:config`.
### steps to enable clipboard functionality
1. go to `about:config`:
![screenshot showing about:config entered into address bar](https://github.com/wukko/cobalt/assets/71202418/9ad78612-a372-4949-aeac-99dfc41e273c)
![screenshot showing about:config entered into address bar](images/troubleshooting/clipboard/config.png)
2. if asked, read what firefox has to say and press "accept the risk and continue".
⚠ tinkering with other preferences may break your browser. **do not** edit them unless you know what you're doing.
![screenshot showing about:config security warning that reads: "proceed with caution. changing advanced configuration preferences can impact firefox performance or security." lower there's a pre-checked checkbox that says: "warn me when i attempt to access these preferences". lowest element is a blue button that says "accept the risk and continue"](https://github.com/wukko/cobalt/assets/71202418/02328729-dbfe-4ea4-b2ca-7bcf1998c2ca)
![screenshot showing about:config security warning that reads: "proceed with caution. changing advanced configuration preferences can impact firefox performance or security." lower there's a pre-checked checkbox that says: "warn me when i attempt to access these preferences". lowest element is a blue button that says "accept the risk and continue"](images/troubleshooting/clipboard/risk.png)
3. search for `dom.events.asyncClipboard.readText`
![screenshot showing "dom.events.asyncclipboard.readtext" entered into search on about:config page](https://github.com/wukko/cobalt/assets/71202418/7c7f7e3c-6a6a-40df-8436-277489e72e0b)
![screenshot showing "dom.events.asyncclipboard.readtext" entered into search on about:config page](images/troubleshooting/clipboard/search.png)
4. press the toggle button on very right.
![screenshot showing "dom.events.asyncclipboard.readtext" preference on about:config page with highlighted toggle button on very right](https://github.com/wukko/cobalt/assets/71202418/b45db18e-f4bf-4f1c-9a8c-f13a63a21335)
![screenshot showing "dom.events.asyncclipboard.readtext" preference on about:config page with highlighted toggle button on very right](images/troubleshooting/clipboard/toggle.png)
5. "false" should change to "true".
![screenshot showing "dom.events.asyncclipboard.readtext" preference on about:config page, this one with "true" text highlighted](https://github.com/wukko/cobalt/assets/71202418/4869b4ff-8385-4cd3-ae59-aa2e03a58b5f)
![screenshot showing "dom.events.asyncclipboard.readtext" preference on about:config page, this one with "true" text highlighted](images/troubleshooting/clipboard/toggled.png)
6. go back to cobalt, reload the page, press `paste and download` button again. this time it works! enjoy simpler downloading experience :)
6. go back to cobalt, reload the page, press `paste` button again. this time it works! enjoy simpler downloading experience :)

View File

@ -1,7 +1,7 @@
{
"name": "cobalt",
"description": "save what you love",
"version": "7.12.5",
"version": "7.13",
"author": "wukko",
"exports": "./src/cobalt.js",
"type": "module",
@ -40,6 +40,6 @@
"set-cookie-parser": "2.6.0",
"undici": "^6.7.0",
"url-pattern": "1.0.3",
"youtubei.js": "^9.2.0"
"youtubei.js": "^9.3.0"
}
}

View File

@ -6,6 +6,7 @@ import express from "express";
import { Bright, Green, Red } from "./modules/sub/consoleText.js";
import { getCurrentBranch, shortCommit } from "./modules/sub/currentCommit.js";
import { loadLoc } from "./localization/manager.js";
import { mode } from "./modules/config.js"
import path from 'path';
import { fileURLToPath } from 'url';
@ -22,13 +23,10 @@ app.disable('x-powered-by');
await loadLoc();
const apiMode = process.env.API_URL && !process.env.WEB_URL;
const webMode = process.env.WEB_URL && process.env.API_URL;
if (apiMode) {
if (mode === 'API') {
const { runAPI } = await import('./core/api.js');
runAPI(express, app, gitCommit, gitBranch, __dirname)
} else if (webMode) {
} else if (mode === 'WEB') {
const { runWeb } = await import('./core/web.js');
await runWeb(express, app, gitCommit, gitBranch, __dirname)
} else {

View File

@ -4,18 +4,18 @@ import { randomBytes } from "crypto";
const ipSalt = randomBytes(64).toString('hex');
import { version } from "../modules/config.js";
import { env, version } from "../modules/config.js";
import { getJSON } from "../modules/api.js";
import { apiJSON, checkJSONPost, getIP, languageCode } from "../modules/sub/utils.js";
import { Bright, Cyan } from "../modules/sub/consoleText.js";
import stream from "../modules/stream/stream.js";
import loc from "../localization/manager.js";
import { generateHmac } from "../modules/sub/crypto.js";
import { verifyStream } from "../modules/stream/manage.js";
import { verifyStream, getInternalStream } from "../modules/stream/manage.js";
export function runAPI(express, app, gitCommit, gitBranch, __dirname) {
const corsConfig = process.env.CORS_WILDCARD === '0' ? {
origin: process.env.CORS_URL,
const corsConfig = !env.corsWildcard ? {
origin: env.corsURL,
optionsSuccessStatus: 200
} : {};
@ -123,42 +123,53 @@ export function runAPI(express, app, gitCommit, gitBranch, __dirname) {
app.get('/api/:type', (req, res) => {
try {
let j;
switch (req.params.type) {
case 'stream':
const q = req.query;
const checkQueries = q.t && q.e && q.h && q.s && q.i;
const checkBaseLength = q.t.length === 21 && q.e.length === 13;
const checkSafeLength = q.h.length === 43 && q.s.length === 43 && q.i.length === 22;
if (checkQueries && checkBaseLength && checkSafeLength) {
if (q.p) {
return res.status(200).json({
status: "continue"
})
}
let streamInfo = verifyStream(q.t, q.h, q.e, q.s, q.i);
if (streamInfo.error) {
return res.status(streamInfo.status).json(apiJSON(0, { t: streamInfo.error }).body);
}
if (q.p) {
return res.status(200).json({
status: "continue"
});
}
return stream(res, streamInfo);
} else {
let j = apiJSON(0, {
}
j = apiJSON(0, {
t: "bad request. stream link may be incomplete or corrupted."
})
return res.status(j.status).json(j.body);
}
case 'istream':
if (!req.ip.endsWith('127.0.0.1'))
return res.sendStatus(403);
if (('' + req.query.t).length !== 21)
return res.sendStatus(400);
let streamInfo = getInternalStream(req.query.t);
if (!streamInfo) return res.sendStatus(404);
streamInfo.headers = req.headers;
return stream(res, { type: 'internal', ...streamInfo });
case 'serverInfo':
return res.status(200).json({
version: version,
commit: gitCommit,
branch: gitBranch,
name: process.env.API_NAME || "unknown",
url: process.env.API_URL,
cors: process.env?.CORS_WILDCARD === "0" ? 0 : 1,
name: env.apiName,
url: env.apiURL,
cors: Number(env.corsWildcard),
startTime: `${startTimestamp}`
});
default:
let j = apiJSON(0, {
j = apiJSON(0, {
t: "unknown response type"
})
return res.status(j.status).json(j.body);
@ -183,12 +194,12 @@ export function runAPI(express, app, gitCommit, gitBranch, __dirname) {
res.redirect('/api/json')
});
app.listen(process.env.API_PORT || 9000, () => {
app.listen(env.apiPort, () => {
console.log(`\n` +
`${Cyan("cobalt")} API ${Bright(`v.${version}-${gitCommit} (${gitBranch})`)}\n` +
`Start time: ${Bright(`${startTime.toUTCString()} (${startTimestamp})`)}\n\n` +
`URL: ${Cyan(`${process.env.API_URL}`)}\n` +
`Port: ${process.env.API_PORT || 9000}\n`
`URL: ${Cyan(`${env.apiURL}`)}\n` +
`Port: ${env.apiPort}\n`
)
});
}

View File

@ -1,4 +1,4 @@
import { genericUserAgent, version } from "../modules/config.js";
import { genericUserAgent, version, env } from "../modules/config.js";
import { apiJSON, languageCode } from "../modules/sub/utils.js";
import { Bright, Cyan } from "../modules/sub/consoleText.js";
@ -76,12 +76,12 @@ export async function runWeb(express, app, gitCommit, gitBranch, __dirname) {
return res.redirect('/')
});
app.listen(process.env.WEB_PORT || 9001, () => {
app.listen(env.webPort, () => {
console.log(`\n` +
`${Cyan("cobalt")} WEB ${Bright(`v.${version}-${gitCommit} (${gitBranch})`)}\n` +
`Start time: ${Bright(`${startTime.toUTCString()} (${startTimestamp})`)}\n\n` +
`URL: ${Cyan(`${process.env.WEB_URL}`)}\n` +
`Port: ${process.env.WEB_PORT || 9001}\n`
`URL: ${Cyan(`${env.webURL}`)}\n` +
`Port: ${env.webPort}\n`
)
})
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 815 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1014 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 390 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 569 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 617 B

View File

@ -18,6 +18,48 @@
"sizes": "512x512",
"type": "image/png",
"purpose": "any"
},
{
"src": "/icons/maskable/48.png",
"sizes": "48x48",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/icons/maskable/72.png",
"sizes": "72x72",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/icons/maskable/96.png",
"sizes": "96x96",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/icons/maskable/128.png",
"sizes": "128x128",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/icons/maskable/192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/icons/maskable/384.png",
"sizes": "384x384",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/icons/maskable/512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "maskable"
}
],
"share_target": {

View File

@ -101,10 +101,10 @@
"FollowSupport": "keep in touch with cobalt for news, support, and more:",
"SourceCode": "explore source code, report issues, star or fork the repo:",
"PrivacyPolicy": "cobalt's privacy policy is simple: no data about you is ever collected or stored. zero, zilch, nada, nothing.\nwhat you download is solely your business, not mine or anyone else's.\n\nif your download requires rendering, then data about requested content is encrypted and temporarily stored in server's RAM. it's necessary for this feature to function.\n\nencrypted data is stored for <span class=\"text-backdrop\">90 seconds</span> and then permanently removed.\n\nstored data is only possible to decrypt with unique encryption keys from your download link. furthermore, the official cobalt codebase doesn't provide a way to read temporarily stored data outside of processing functions.\n\nyou can check cobalt's <a class=\"text-backdrop link\" href=\"{repo}\" target=\"_blank\">source code</a> yourself and see that everything is as stated.",
"ErrorYTUnavailable": "this youtube video is unavailable, it could be region or age restricted. try another one!",
"ErrorYTTryOtherCodec": "i couldn't find anything to download with your settings. try another codec or quality!\n\nsometimes youtube api acts unexpectedly. try again or try another settings.",
"ErrorYTUnavailable": "this youtube video is unavailable. it could be age or region restricted. try another one!",
"ErrorYTTryOtherCodec": "i couldn't find anything to download with your settings. try another codec or quality in settings!",
"SettingsCodecSubtitle": "youtube codec",
"SettingsCodecDescription": "h264: generally better player support, but quality tops out at 1080p.\nav1: poor player support, but supports 8k & HDR.\nvp9: usually highest bitrate, preserves most detail. supports 4k & HDR.\n\npick h264 if you want best editor/player/social media compatibility.",
"SettingsCodecDescription": "h264: best support across apps/platforms, average detail level. max quality is 1080p.\nav1: best quality, small file size, most detail. supports 8k & HDR.\nvp9: same quality as av1, but file is x2 bigger. supports 4k & HDR.\n\npick h264 if you want best compatibility.\npick av1 if you want best quality and efficiency.",
"SettingsAudioDub": "youtube audio track",
"SettingsAudioDubDescription": "defines which audio track will be used. if dubbed track isn't available, original video language is used instead.\n\noriginal: original video language is used.\nauto: default browser (and cobalt) language is used.",
"SettingsDubDefault": "original",
@ -113,7 +113,6 @@
"SettingsVimeoPreferDescription": "progressive: direct file link to vimeo's cdn. max quality is 1080p.\ndash: video and audio are merged by cobalt into one file. max quality is 4k.\n\npick \"progressive\" if you want best editor/player/social media compatibility. if progressive download isn't available, dash is used instead.",
"ShareURL": "share",
"ErrorTweetUnavailable": "couldn't find anything about this tweet. this could be because its visibility is limited. try another one!",
"ErrorTwitterRIP": "twitter has restricted access to any content to unauthenticated users. while there's a way to get regular tweets, spaces are, unfortunately, impossible to get at this time. i am looking into possible solutions.",
"PopupCloseDone": "done",
"Accessibility": "accessibility",
"SettingsReduceTransparency": "reduce transparency",

View File

@ -102,11 +102,11 @@
"FollowSupport": "подписывайся на соц.сети кобальта для новостей и поддержки:",
"SourceCode": "шарься в исходнике, пиши о проблемах, или же форкай репозиторий:",
"PrivacyPolicy": "политика конфиденциальности кобальта довольно проста: никакие данные о тебе никогда не собираются и не хранятся. нуль, ноль, нада, ничего.\nто, что ты скачиваешь, - твоё личное дело, а не чьё-либо ещё.\n\nесли твоей загрузке требуется рендер, то зашифрованные данные о ней временно хранятся в ОЗУ сервера. это необходимо для работы данной функции.\n\nзашифрованные данные хранятся в течение <span class=\"text-backdrop\">90 секунд</span> и затем безвозвратно удаляются.\n\ncохранённые данные можно расшифровать только с помощью уникальных ключей шифрования из твоей ссылки на скачивание. кроме того, официальная кодовая база кобальта не предусматривает возможности чтения эти данные вне функций обработки.\n\nты всегда можешь посмотреть <a class=\"text-backdrop link\" href=\"{repo}\" target=\"_blank\">исходный код кобальта</a> и убедиться, что всё так, как заявлено.",
"ErrorYTUnavailable": "это видео недоступно, возможно оно ограничено по региону или доступу. попробуй другое!",
"ErrorYTTryOtherCodec": "я не нашёл того, что мог бы скачать с твоими настройками. попробуй другой кодек или качество!",
"SettingsCodecSubtitle": "кодек для видео с youtube",
"SettingsCodecDescription": "h264: обширная поддержка плеерами, но макс. качество всего лишь 1080p.\nav1: слабая поддержка плеерами, но поддерживает 8k и HDR.\nvp9: обычно наиболее высокий битрейт, лучше сохраняется качество видео. поддерживает 4k и HDR.\n\nвыбирай h264, если тебе нужна наилучшая совместимость с плеерами/редакторами/соцсетями.",
"SettingsAudioDub": "звуковая дорожка для видео с youtube",
"ErrorYTUnavailable": "это видео недоступно. возможно оно ограничено по доступу или региону. попробуй другое!",
"ErrorYTTryOtherCodec": "я не нашёл того, что мог бы скачать с твоими настройками. попробуй другой кодек или качество в настройках!",
"SettingsCodecSubtitle": "кодек для youtube видео",
"SettingsCodecDescription": "h264: лучшая совместимость, средний уровень детализированности. максимальное качество - 1080p.\nav1: лучшее качество, маленький размер файла, наибольшее количество деталей. поддерживает 8k и HDR.\nvp9: такая же детализированность, как и у av1, но файл в 2 раза больше. поддерживает 4k и HDR.\n\nвыбирай h264, если тебе нужна наилучшая совместимость.\nвыбирай av1, если ты хочешь лучшее качество и эффективность.",
"SettingsAudioDub": "звуковая дорожка для youtube видео",
"SettingsAudioDubDescription": "определяет, какая звуковая дорожка используется при скачивании видео. если дублированная дорожка недоступна, то вместо неё используется оригинальная.\n\nоригинал: используется оригинальная дорожка.\nавто: используется язык браузера и интерфейса кобальта.",
"SettingsDubDefault": "оригинал",
"SettingsDubAuto": "авто",
@ -114,7 +114,6 @@
"SettingsVimeoPreferDescription": "progressive: прямая ссылка на файл с сервера vimeo. максимальное качество: 1080p.\ndash: кобальт совмещает видео и аудио в один файл. максимальное качество: 4k.\n\nвыбирай \"progressive\", если тебе нужна наилучшая совместимость с плеерами/редакторами/соцсетями. если \"progressive\" файл недоступен, кобальт скачает \"dash\".",
"ShareURL": "поделиться",
"ErrorTweetUnavailable": "не смог найти что-либо об этом твите. возможно его видимость ограничена. попробуй другой!",
"ErrorTwitterRIP": "твиттер ограничил доступ к любому контенту на сайте для пользователей без аккаунтов. я нашёл лазейку, чтобы доставать обычные твиты, а для spaces, к сожалению, нет. я ищу возможные варианты выхода из ситуации.",
"PopupCloseDone": "готово",
"Accessibility": "общедоступность",
"SettingsReduceTransparency": "уменьшить прозрачность",

View File

@ -12,6 +12,31 @@ Object.values(servicesConfigJson.config).forEach(service => {
)
})
const
apiURL = process.env.API_URL || '',
// WEB mode related environment variables
webEnvs = {
webPort: process.env.WEB_PORT || 9001,
webURL: process.env.WEB_URL || '',
showSponsors: !!process.env.SHOW_SPONSORS,
isBeta: !!process.env.IS_BETA,
plausibleHostname: process.env.PLAUSIBLE_HOSTNAME,
apiURL
},
// API mode related environment variables
apiEnvs = {
apiPort: process.env.API_PORT || 9000,
apiName: process.env.API_NAME || 'unknown',
corsWildcard: process.env.CORS_WILDCARD !== '0',
corsURL: process.env.CORS_URL,
cookiePath: process.env.COOKIE_PATH,
processingPriority: process.env.PROCESSING_PRIORITY && parseInt(process.env.PROCESSING_PRIORITY),
tiktokDeviceInfo: process.env.TIKTOK_DEVICE_INFO && JSON.parse(process.env.TIKTOK_DEVICE_INFO),
apiURL
}
export const
services = servicesConfigJson.config,
audioIgnore = servicesConfigJson.audioIgnore,
@ -26,4 +51,7 @@ export const
supportedAudio = config.supportedAudio,
celebrations = config.celebrations,
links = config.links,
sponsors = config.sponsors
sponsors = config.sponsors,
mode = (apiURL && !webEnvs.webURL) ? 'API' :
(webEnvs.webURL && apiURL) ? 'WEB' : undefined,
env = mode === 'API' ? apiEnvs : webEnvs

View File

@ -1,4 +1,4 @@
import { authorInfo, celebrations, sponsors } from "../config.js";
import { authorInfo, celebrations, sponsors, env } from "../config.js";
import emoji from "../emoji.js";
import { loadFile } from "../sub/loadFromFs.js";
@ -266,5 +266,5 @@ export function sponsoredList() {
}
export function betaTag() {
return process.env.IS_BETA ? '<span class="logo-sub">β</span>' : ''
return env.isBeta ? '<span class="logo-sub">β</span>' : ''
}

View File

@ -1,5 +1,5 @@
import { checkbox, collapsibleList, explanation, footerButtons, multiPagePopup, popup, popupWithBottomButtons, sep, settingsCategory, switcher, socialLink, socialLinks, urgentNotice, keyboardShortcuts, webLoc, sponsoredList, betaTag, linkSVG } from "./elements.js";
import { services as s, authorInfo, version, repo, donations, supportedAudio, links } from "../config.js";
import { services as s, authorInfo, version, repo, donations, supportedAudio, links, env } from "../config.js";
import { getCommitInfo } from "../sub/currentCommit.js";
import loc from "../../localization/manager.js";
import emoji from "../emoji.js";
@ -48,10 +48,10 @@ export default function(obj) {
<title>${t("AppTitleCobalt")}</title>
<meta property="og:url" content="${process.env.WEB_URL}">
<meta property="og:url" content="${env.webURL}">
<meta property="og:title" content="${t("AppTitleCobalt")}">
<meta property="og:description" content="${t('EmbedBriefDescription')}">
<meta property="og:image" content="${process.env.WEB_URL}icons/generic.png">
<meta property="og:image" content="${env.webURL}icons/generic.png">
<meta name="title" content="${t("AppTitleCobalt")}">
<meta name="description" content="${t('AboutSummary')}">
<meta name="theme-color" content="#000000">
@ -75,11 +75,11 @@ export default function(obj) {
<link rel="preload" href="assets/meowbalt/error.png" as="image">
<link rel="preload" href="assets/meowbalt/question.png" as="image">
${process.env.PLAUSIBLE_HOSTNAME ?
${env.plausibleHostname ?
`<script
defer
data-domain="${new URL(process.env.WEB_URL).hostname}"
src="https://${process.env.PLAUSIBLE_HOSTNAME}/js/script.js"
data-domain="${new URL(env.webURL).hostname}"
src="https://${env.plausibleHostname}/js/script.js"
></script>`
: ''}
</head>
@ -169,7 +169,7 @@ export default function(obj) {
name: "privacy",
title: `${emoji("🔒")} ${t("CollapsePrivacy")}`,
body: t("PrivacyPolicy") + `${
process.env.PLAUSIBLE_HOSTNAME ? `<br><br>${t("AnalyticsDescription")}` : ''
env.plausibleHostname ? `<br><br>${t("AnalyticsDescription")}` : ''
}`
}, {
name: "legal",
@ -177,7 +177,7 @@ export default function(obj) {
body: t("FairUse")
}])
},
...(process.env.SHOW_SPONSORS ?
...(env.showSponsors ?
[{
text: t("SponsoredBy"),
classes: ["sponsored-by-text"],
@ -499,7 +499,7 @@ export default function(obj) {
}])
})
+ (() => {
if (process.env.PLAUSIBLE_HOSTNAME) {
if (env.plausibleHostname) {
return settingsCategory({
name: "privacy",
title: t('PrivateAnalytics'),
@ -629,7 +629,7 @@ export default function(obj) {
</footer>
</div>
<script>
let defaultApiUrl = '${process.env.API_URL || ''}';
let defaultApiUrl = '${env.apiURL}';
const loc = ${webLoc(t,
[
'ErrorNoInternet',

View File

@ -1,9 +1,10 @@
import Cookie from './cookie.js';
import { readFile, writeFile } from 'fs/promises';
import { parse as parseSetCookie, splitCookiesString } from 'set-cookie-parser';
import { env } from '../../../modules/config.js'
const WRITE_INTERVAL = 60000,
cookiePath = process.env.COOKIE_PATH,
cookiePath = env.cookiePath,
COUNTER = Symbol('counter');
let cookies = {}, dirty = false, intervalId;

View File

@ -152,6 +152,7 @@ export default async function(host, patternMatch, url, lang, obj) {
case "rutube":
r = await rutube({
id: patternMatch.id,
yappyId: patternMatch.yappyId,
quality: obj.vQuality,
isAudioOnly: isAudioOnly
});

View File

@ -2,11 +2,38 @@ import { createStream } from "../../stream/manage.js";
import { genericUserAgent } from "../../config.js";
import { getCookie, updateCookie } from "../cookie/manager.js";
const commonInstagramHeaders = {
'user-agent': genericUserAgent,
'sec-gpc': '1',
'sec-fetch-site': 'same-origin',
'x-ig-app-id': '936619743392459'
const commonHeaders = {
"user-agent": genericUserAgent,
"sec-gpc": "1",
"sec-fetch-site": "same-origin",
"x-ig-app-id": "936619743392459"
}
const mobileHeaders = {
"x-ig-app-locale": "en_US",
"x-ig-device-locale": "en_US",
"x-ig-mapped-locale": "en_US",
"user-agent": "Instagram 275.0.0.27.98 Android (33/13; 280dpi; 720x1423; Xiaomi; Redmi 7; onclite; qcom; en_US; 458229237)",
"accept-language": "en-US",
"x-fb-http-engine": "Liger",
"x-fb-client-ip": "True",
"x-fb-server-cluster": "True",
"content-length": "0",
}
const embedHeaders = {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"Accept-Language": "en-GB,en;q=0.9",
"Cache-Control": "max-age=0",
"Dnt": "1",
"Priority": "u=0, i",
"Sec-Ch-Ua": 'Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99',
"Sec-Ch-Ua-Mobile": "?0",
"Sec-Ch-Ua-Platform": "macOS",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
}
const cachedDtsg = {
@ -20,7 +47,7 @@ async function findDtsgId(cookie) {
const data = await fetch('https://www.instagram.com/', {
headers: {
...commonInstagramHeaders,
...commonHeaders,
cookie
}
}).then(r => r.text());
@ -38,7 +65,7 @@ async function findDtsgId(cookie) {
async function request(url, cookie, method = 'GET', requestData) {
let headers = {
...commonInstagramHeaders,
...commonHeaders,
'x-ig-www-claim': cookie?._wwwClaim || '0',
'x-csrftoken': cookie?.values()?.csrftoken,
cookie
@ -60,16 +87,51 @@ async function request(url, cookie, method = 'GET', requestData) {
return data.json();
}
async function getPost(id) {
let data;
try {
const cookie = getCookie('instagram');
async function requestMobileApi(id, cookie) {
const oembedURL = new URL('https://i.instagram.com/api/v1/oembed/');
oembedURL.searchParams.set('url', `https://www.instagram.com/p/${id}/`);
const oembed = await fetch(oembedURL, {
headers: {
...mobileHeaders,
cookie
}
}).then(r => r.json()).catch(() => {});
const mediaId = oembed?.media_id;
if (!mediaId) return false;
const mediaInfo = await fetch(`https://i.instagram.com/api/v1/media/${mediaId}/info/`, {
headers: {
...mobileHeaders,
cookie
}
}).then(r => r.json()).catch(() => {});
return mediaInfo?.items?.[0];
}
async function requestHTML(id, cookie) {
const data = await fetch(`https://www.instagram.com/p/${id}/embed/captioned/`, {
headers: {
...embedHeaders,
cookie
}
}).then(r => r.text()).catch(() => {});
let embedData = JSON.parse(data?.match(/"init",\[\],\[(.*?)\]\],/)[1]);
if (!embedData || !embedData?.contextJSON) return false;
embedData = JSON.parse(embedData.contextJSON);
return embedData;
}
async function requestGQL(id, cookie) {
let dtsgId;
if (cookie) {
dtsgId = await findDtsgId(cookie);
}
const url = new URL('https://www.instagram.com/api/graphql/');
const requestData = {
@ -84,15 +146,50 @@ async function getPost(id) {
requestData.fb_dtsg = dtsgId;
}
data = (await request(url, cookie, 'POST', requestData))
return (await request(url, cookie, 'POST', requestData))
.data
?.xdt_api__v1__media__shortcode__web_info
?.items
?.[0];
} catch {}
}
if (!data) return { error: 'ErrorCouldntFetch' };
function extractOldPost(data, id) {
const sidecar = data?.gql_data?.shortcode_media?.edge_sidecar_to_children;
if (sidecar) {
const picker = sidecar.edges.filter(e => e.node?.display_url)
.map(e => {
const type = e.node?.is_video ? "video" : "photo";
const url = type === "video" ? e.node?.video_url : e.node?.display_url;
return {
type, url,
/* thumbnails have `Cross-Origin-Resource-Policy`
** set to `same-origin`, so we need to proxy them */
thumb: createStream({
service: "instagram",
type: "default",
u: e.node?.display_url,
filename: "image.jpg"
})
}
});
if (picker.length) return { picker }
} else if (data?.gql_data?.shortcode_media?.video_url) {
return {
urls: data.gql_data.shortcode_media.video_url,
filename: `instagram_${id}.mp4`,
audioFilename: `instagram_${id}_audio`
}
} else if (data?.gql_data?.shortcode_media?.display_url) {
return {
urls: data.gql_data?.shortcode_media.display_url,
isPhoto: true
}
}
}
function extractNewPost(data, id) {
const carousel = data.carousel_media;
if (carousel) {
const picker = carousel.filter(e => e?.image_versions2)
@ -133,7 +230,35 @@ async function getPost(id) {
isPhoto: true
}
}
}
async function getPost(id) {
let data, result;
try {
const cookie = getCookie('instagram');
// mobile api (no cookie, cookie)
data = await requestMobileApi(id);
if (!data && cookie) data = await requestMobileApi(id, cookie);
// html embed (no cookie, cookie)
if (!data) data = await requestHTML(id);
if (!data && cookie) data = await requestHTML(id, cookie);
// web app graphql api (no cookie, cookie)
if (!data) data = await requestGQL(id);
if (!data && cookie) data = await requestGQL(id, cookie);
} catch {}
if (!data) return { error: 'ErrorCouldntFetch' };
if (data?.gql_data) {
result = extractOldPost(data, id)
} else {
result = extractNewPost(data, id)
}
if (result) return result;
return { error: 'ErrorEmptyDownload' }
}

View File

@ -1,22 +1,16 @@
import { genericUserAgent } from "../../config.js";
const videoLinkBase = {
"regular": "https://v1.pinimg.com/videos/mc/720p/",
"story": "https://v1.pinimg.com/videos/mc/720p/"
}
const linkRegex = /"url":"(https:\/\/v1.pinimg.com\/videos\/.*?)"/g;
export default async function(o) {
let id = o.id, type = "regular";
let id = o.id;
if (!o.id && o.shortLink) {
id = await fetch(`https://api.pinterest.com/url_shortener/${o.shortLink}/redirect/`, { redirect: "manual" }).then((r) => {
return r.headers.get("location").split('pin/')[1].split('/')[0]
}).catch(() => {});
}
if (id.includes("--")) {
id = id.split("--")[1];
type = "story";
}
if (id.includes("--")) id = id.split("--")[1];
if (!id) return { error: 'ErrorCouldntFetch' };
let html = await fetch(`https://www.pinterest.com/pin/${id}/`, {
@ -25,11 +19,14 @@ export default async function(o) {
if (!html) return { error: 'ErrorCouldntFetch' };
let videoLink = html.split(`"url":"${videoLinkBase[type]}`)[1]?.split('"')[0];
if (!html.includes(videoLink)) return { error: 'ErrorEmptyDownload' };
let videoLink = [...html.matchAll(linkRegex)]
.map(([, link]) => link)
.filter(a => a.endsWith('.mp4') && a.includes('720p'))[0];
if (!videoLink) return { error: 'ErrorEmptyDownload' };
return {
urls: `${videoLinkBase[type]}${videoLink}`,
urls: videoLink,
filename: `pinterest_${o.id}.mp4`,
audioFilename: `pinterest_${o.id}_audio`
}

View File

@ -1,18 +1,47 @@
import HLS from 'hls-parser';
import { maxVideoDuration } from "../../config.js";
import { cleanString } from '../../sub/utils.js';
async function requestJSON(url) {
try {
const r = await fetch(url);
return await r.json();
} catch {}
}
export default async function(obj) {
if (obj.yappyId) {
let yappy = await requestJSON(
`https://rutube.ru/pangolin/api/web/yappy/yappypage/?client=wdp&videoId=${obj.yappyId}&page=1&page_size=15`
)
let yappyURL = yappy?.results?.find(r => r.id === obj.yappyId)?.link;
if (!yappyURL) return { error: 'ErrorEmptyDownload' };
return {
urls: yappyURL,
filename: `rutube_yappy_${obj.yappyId}.mp4`,
audioFilename: `rutube_yappy_${obj.yappyId}_audio`
}
}
let quality = obj.quality === "max" ? "9000" : obj.quality;
let play = await fetch(`https://rutube.ru/api/play/options/${obj.id}/?no_404=true&referer&pver=v2`).then((r) => { return r.json() }).catch(() => { return false });
let play = await requestJSON(
`https://rutube.ru/api/play/options/${obj.id}/?no_404=true&referer&pver=v2`
)
if (!play) return { error: 'ErrorCouldntFetch' };
if ("hls" in play.live_streams) return { error: 'ErrorLiveVideo' };
if (!play.video_balancer || play.detail) return { error: 'ErrorEmptyDownload' };
if (play.detail || !play.video_balancer) return { error: 'ErrorEmptyDownload' };
if (play.live_streams?.hls) return { error: 'ErrorLiveVideo' };
if (play.duration > maxVideoDuration) return { error: ['ErrorLengthLimit', maxVideoDuration / 60000] };
if (play.duration > maxVideoDuration)
return { error: ['ErrorLengthLimit', maxVideoDuration / 60000] };
let m3u8 = await fetch(play.video_balancer.m3u8)
.then(r => r.text())
.catch(() => {});
let m3u8 = await fetch(play.video_balancer.m3u8).then((r) => { return r.text() }).catch(() => { return false });
if (!m3u8) return { error: 'ErrorCouldntFetch' };
m3u8 = HLS.parse(m3u8).variants.sort((a, b) => Number(b.bandwidth) - Number(a.bandwidth));
@ -21,6 +50,7 @@ export default async function(obj) {
if (Number(quality) < bestQuality.resolution.height) {
bestQuality = m3u8.find((i) => (Number(quality) === i["resolution"].height));
}
let fileMetadata = {
title: cleanString(play.title.trim()),
artist: cleanString(play.author.name.trim()),
@ -31,7 +61,7 @@ export default async function(obj) {
isM3U8: true,
filenameAttributes: {
service: "rutube",
id: play.id,
id: obj.id,
title: fileMetadata.title,
author: fileMetadata.artist,
resolution: `${bestQuality.resolution.width}x${bestQuality.resolution.height}`,

View File

@ -1,4 +1,4 @@
import { genericUserAgent } from "../../config.js";
import { genericUserAgent, env } from "../../config.js";
const shortDomain = "https://vt.tiktok.com/";
const apiPath = "https://api22-normal-c-alisg.tiktokv.com/aweme/v1/feed/?region=US&carrier_region=US";
@ -7,7 +7,7 @@ const apiUserAgent = "TikTok/338014 CFNetwork/1410.1 Darwin/22.6.0";
export default async function(obj) {
let postId = obj.postId ? obj.postId : false;
if (!process.env.TIKTOK_DEVICE_INFO) return { error: 'ErrorCouldntFetch' };
if (!env.tiktokDeviceInfo) return { error: 'ErrorCouldntFetch' };
if (!postId) {
let html = await fetch(`${shortDomain}${obj.id}`, {
@ -27,8 +27,7 @@ export default async function(obj) {
}
if (!postId) return { error: 'ErrorCantGetID' };
let deviceInfo = JSON.parse(process.env.TIKTOK_DEVICE_INFO);
deviceInfo = new URLSearchParams(deviceInfo).toString();
let deviceInfo = new URLSearchParams(env.tiktokDeviceInfo).toString();
let apiURL = new URL(apiPath);
apiURL.searchParams.append("aweme_id", postId);

View File

@ -23,7 +23,9 @@ const c = {
}
export default async function(o) {
let info, isDubbed, quality = o.quality === "max" ? "9000" : o.quality; //set quality 9000(p) to be interpreted as max
let info, isDubbed,
quality = o.quality === "max" ? "9000" : o.quality; // 9000(p) - max quality
function qual(i) {
if (!i.quality_label) {
return;
@ -33,7 +35,7 @@ export default async function(o) {
}
try {
info = await yt.getBasicInfo(o.id, 'ANDROID');
info = await yt.getBasicInfo(o.id, 'WEB');
} catch (e) {
return { error: 'ErrorCantConnectToServiceAPI' };
}
@ -43,7 +45,18 @@ export default async function(o) {
if (info.playability_status.status !== 'OK') return { error: 'ErrorYTUnavailable' };
if (info.basic_info.is_live) return { error: 'ErrorLiveVideo' };
let bestQuality, hasAudio, adaptive_formats = info.streaming_data.adaptive_formats.filter(e =>
// return a critical error if returned video is "Video Not Available"
// or a similar stub by youtube
if (info.basic_info.id !== o.id) {
return {
error: 'ErrorCantConnectToServiceAPI',
critical: true
}
}
let bestQuality, hasAudio;
let adaptive_formats = info.streaming_data.adaptive_formats.filter(e =>
e.mime_type.includes(c[o.format].codec) || e.mime_type.includes(c[o.format].aCodec)
).sort((a, b) => Number(b.bitrate) - Number(a.bitrate));
@ -87,16 +100,10 @@ export default async function(o) {
youtubeDubName: isDubbed ? o.dubLang : false
}
if (filenameAttributes.title === "Video Not Available" && filenameAttributes.author === "YouTube Viewers")
return {
error: 'ErrorCantConnectToServiceAPI',
critical: true
}
if (hasAudio && o.isAudioOnly) return {
type: "render",
isAudioOnly: true,
urls: audio.url,
urls: audio.decipher(yt.session.player),
filenameAttributes: filenameAttributes,
fileMetadata: fileMetadata
}
@ -108,14 +115,14 @@ export default async function(o) {
if (!o.isAudioOnly && !o.isAudioMuted && o.format === 'h264') {
match = info.streaming_data.formats.find(checkSingle);
type = "bridge";
urls = match?.url;
urls = match?.decipher(yt.session.player);
}
const video = adaptive_formats.find(checkRender);
if (!match && video) {
match = video;
type = "render";
urls = [video.url, audio.url];
urls = [video.decipher(yt.session.player), audio.decipher(yt.session.player)];
}
if (match) {

View File

@ -7,6 +7,7 @@
"video/:comId", "_shortLink/:comShortLink",
"_tv/:lang/video/:tvId", "_tv/video/:tvId"
],
"subdomains": ["m"],
"enabled": true
},
"reddit": {
@ -66,7 +67,7 @@
"enabled": false
},
"vimeo": {
"patterns": [":id", "video/:id", ":id/:password"],
"patterns": [":id", "video/:id", ":id/:password", "/channels/:user/:id"],
"enabled": true,
"bestAudio": "mp3"
},
@ -109,7 +110,7 @@
"rutube": {
"alias": "rutube videos",
"tld": "ru",
"patterns": ["video/:id", "play/embed/:id"],
"patterns": ["video/:id", "play/embed/:id", "shorts/:id", "yappy/:yappyId"],
"enabled": true
},
"dailymotion": {

View File

@ -16,7 +16,7 @@ export const testers = {
"reddit": (patternMatch) =>
patternMatch.sub?.length <= 22 && patternMatch.id?.length <= 10,
"rutube": (patternMatch) =>
patternMatch.id?.length === 32,
patternMatch.id?.length === 32 || patternMatch.yappyId?.length === 32,
"soundcloud": (patternMatch) =>
(patternMatch.author?.length <= 255 && patternMatch.song?.length <= 255)

View File

@ -0,0 +1,101 @@
import { request } from 'undici';
import { Readable } from 'node:stream';
import { assert } from 'console';
import { getHeaders } from './shared.js';
const CHUNK_SIZE = BigInt(8e6); // 8 MB
const min = (a, b) => a < b ? a : b;
async function* readChunks(streamInfo, size) {
let read = 0n;
while (read < size) {
if (streamInfo.controller.signal.aborted) {
throw new Error("controller aborted");
}
const chunk = await request(streamInfo.url, {
headers: {
...getHeaders('youtube'),
Range: `bytes=${read}-${read + CHUNK_SIZE}`
},
signal: streamInfo.controller.signal
});
const expected = min(CHUNK_SIZE, size - read);
const received = BigInt(chunk.headers['content-length']);
if (received < expected / 2n) {
streamInfo.controller.abort();
}
for await (const data of chunk.body) {
yield data;
}
read += received;
}
}
function chunkedStream(streamInfo, size) {
assert(streamInfo.controller instanceof AbortController);
const stream = Readable.from(readChunks(streamInfo, size));
return stream;
}
async function handleYoutubeStream(streamInfo, res) {
try {
const req = await fetch(streamInfo.url, {
headers: getHeaders('youtube'),
method: 'HEAD',
signal: streamInfo.controller.signal
});
streamInfo.url = req.url;
const size = BigInt(req.headers.get('content-length'));
if (req.status !== 200 || !size)
return res.destroy();
const stream = chunkedStream(streamInfo, size);
for (const headerName of ['content-type', 'content-length']) {
const headerValue = req.headers.get(headerName);
if (headerValue) res.setHeader(headerName, headerValue);
}
stream.pipe(res);
stream.on('error', () => res.destroy());
} catch {
res.destroy();
}
}
export async function internalStream(streamInfo, res) {
if (streamInfo.service === 'youtube') {
return handleYoutubeStream(streamInfo, res);
}
try {
const req = await request(streamInfo.url, {
headers: {
...streamInfo.headers,
host: undefined
},
signal: streamInfo.controller.signal,
maxRedirections: 16
});
res.status(req.statusCode);
for (const [ name, value ] of Object.entries(req.headers))
res.setHeader(name, value)
if (req.statusCode < 200 || req.statusCode > 299)
return res.destroy();
req.body.pipe(res);
req.body.on('error', () => res.destroy());
} catch {
streamInfo.controller.abort();
}
}

View File

@ -3,7 +3,10 @@ import { randomBytes } from "crypto";
import { nanoid } from 'nanoid';
import { decryptStream, encryptStream, generateHmac } from "../sub/crypto.js";
import { streamLifespan } from "../config.js";
import { streamLifespan, env } from "../config.js";
import { strict as assert } from "assert";
const M3U_SERVICES = ['dailymotion', 'vimeo', 'rutube'];
const streamNoAccess = {
error: "i couldn't verify if you have access to this stream. go back and try again!",
@ -24,6 +27,7 @@ streamCache.on("expired", (key) => {
streamCache.del(key);
})
const internalStreamCache = {};
const hmacSalt = randomBytes(64).toString('hex');
export function createStream(obj) {
@ -50,7 +54,7 @@ export function createStream(obj) {
encryptStream(streamData, iv, secret)
)
let streamLink = new URL('/api/stream', process.env.API_URL);
let streamLink = new URL('/api/stream', env.apiURL);
const params = {
't': streamID,
@ -67,6 +71,61 @@ export function createStream(obj) {
return streamLink.toString();
}
export function getInternalStream(id) {
return internalStreamCache[id];
}
export function createInternalStream(url, obj = {}) {
assert(typeof url === 'string');
const streamID = nanoid();
internalStreamCache[streamID] = {
url,
service: obj.service,
controller: new AbortController()
};
let streamLink = new URL('/api/istream', `http://127.0.0.1:${env.apiPort}`);
streamLink.searchParams.set('t', streamID);
return streamLink.toString();
}
export function destroyInternalStream(url) {
url = new URL(url);
if (url.hostname !== '127.0.0.1') {
return;
}
const id = url.searchParams.get('t');
if (internalStreamCache[id]) {
internalStreamCache[id].controller.abort();
delete internalStreamCache[id];
}
}
function wrapStream(streamInfo) {
/* m3u8 links are currently not supported
* for internal streams, skip them */
if (M3U_SERVICES.includes(streamInfo.service)) {
return streamInfo;
}
const url = streamInfo.urls;
if (typeof url === 'string') {
streamInfo.urls = createInternalStream(url, streamInfo);
} else if (Array.isArray(url)) {
for (const idx in streamInfo.urls) {
streamInfo.urls[idx] = createInternalStream(
streamInfo.urls[idx], streamInfo
);
}
} else throw 'invalid urls';
return streamInfo;
}
export function verifyStream(id, hmac, exp, secret, iv) {
try {
const ghmac = generateHmac(`${id},${exp},${iv},${secret}`, hmacSalt);
@ -82,9 +141,9 @@ export function verifyStream(id, hmac, exp, secret, iv) {
if (Number(exp) <= new Date().getTime())
return streamNoExist;
return streamInfo;
return wrapStream(streamInfo);
}
catch (e) {
catch {
return {
error: "something went wrong and i couldn't verify this stream. go back and try again!",
status: 500

View File

@ -0,0 +1,21 @@
import { genericUserAgent } from "../config.js";
const defaultHeaders = {
'user-agent': genericUserAgent
}
const serviceHeaders = {
bilibili: {
referer: 'https://www.bilibili.com/'
},
youtube: {
accept: '*/*',
origin: 'https://www.youtube.com',
referer: 'https://www.youtube.com',
DNT: '?1'
}
}
export function getHeaders(service) {
return { ...defaultHeaders, ...serviceHeaders[service] }
}

View File

@ -1,4 +1,5 @@
import { streamAudioOnly, streamDefault, streamLiveRender, streamVideoOnly, convertToGif } from "./types.js";
import { internalStream } from './internal.js';
export default async function(res, streamInfo) {
try {
@ -7,6 +8,8 @@ export default async function(res, streamInfo) {
return;
}
switch (streamInfo.type) {
case "internal":
return await internalStream(streamInfo, res);
case "render":
await streamLiveRender(streamInfo, res);
break;
@ -21,7 +24,7 @@ export default async function(res, streamInfo) {
await streamDefault(streamInfo, res);
break;
}
} catch (e) {
} catch {
res.status(500).json({ status: "error", text: "Internal Server Error" });
}
}

View File

@ -1,10 +1,19 @@
import { spawn } from "child_process";
import ffmpeg from "ffmpeg-static";
import { ffmpegArgs, genericUserAgent } from "../config.js";
import { metadataManager } from "../sub/utils.js";
import { request } from "undici";
import ffmpeg from "ffmpeg-static";
import { spawn } from "child_process";
import { create as contentDisposition } from "content-disposition-header";
import { metadataManager } from "../sub/utils.js";
import { destroyInternalStream } from "./manage.js";
import { env, ffmpegArgs } from "../config.js";
import { getHeaders } from "./shared.js";
function toRawHeaders(headers) {
return Object.entries(headers)
.map(([key, value]) => `${key}: ${value}\r\n`)
.join('');
}
function closeRequest(controller) {
try { controller.abort() } catch {}
}
@ -35,15 +44,19 @@ function pipe(from, to, done) {
}
function getCommand(args) {
if (process.env.PROCESSING_PRIORITY && process.platform !== "win32") {
return ['nice', ['-n', process.env.PROCESSING_PRIORITY, ffmpeg, ...args]]
if (!isNaN(env.processingPriority) && process.platform !== "win32") {
return ['nice', ['-n', env.processingPriority.toString(), ffmpeg, ...args]]
}
return [ffmpeg, args]
}
export async function streamDefault(streamInfo, res) {
const abortController = new AbortController();
const shutdown = () => (closeRequest(abortController), closeResponse(res));
const shutdown = () => (
closeRequest(abortController),
closeResponse(res),
destroyInternalStream(streamInfo.urls)
);
try {
let filename = streamInfo.filename;
@ -53,13 +66,16 @@ export async function streamDefault(streamInfo, res) {
res.setHeader('Content-disposition', contentDisposition(filename));
const { body: stream, headers } = await request(streamInfo.urls, {
headers: { 'user-agent': genericUserAgent },
headers: getHeaders(streamInfo.service),
signal: abortController.signal,
maxRedirections: 16
});
res.setHeader('content-type', headers['content-type']);
res.setHeader('content-length', headers['content-length']);
for (const headerName of ['content-type', 'content-length']) {
if (headers[headerName]) {
res.setHeader(headerName, headers[headerName]);
}
}
pipe(stream, res, shutdown);
} catch {
@ -67,68 +83,53 @@ export async function streamDefault(streamInfo, res) {
}
}
export async function streamLiveRender(streamInfo, res) {
let abortController = new AbortController(), process;
export function streamLiveRender(streamInfo, res) {
let process;
const shutdown = () => (
closeRequest(abortController),
killProcess(process),
closeResponse(res)
closeResponse(res),
streamInfo.urls.map(destroyInternalStream)
);
const headers = getHeaders(streamInfo.service);
const rawHeaders = toRawHeaders(headers);
try {
if (streamInfo.urls.length !== 2) return shutdown();
const { body: audio } = await request(streamInfo.urls[1], {
maxRedirections: 16, signal: abortController.signal,
headers: {
'user-agent': genericUserAgent,
referer: streamInfo.service === 'bilibili'
? 'https://www.bilibili.com/'
: undefined,
}
});
const format = streamInfo.filename.split('.')[streamInfo.filename.split('.').length - 1];
let args = [
'-loglevel', '-8',
'-user_agent', genericUserAgent
];
if (streamInfo.service === 'bilibili') {
args.push(
'-headers', 'Referer: https://www.bilibili.com/\r\n',
)
}
args.push(
'-headers', rawHeaders,
'-i', streamInfo.urls[0],
'-i', 'pipe:3',
'-headers', rawHeaders,
'-i', streamInfo.urls[1],
'-map', '0:v',
'-map', '1:a',
);
]
args = args.concat(ffmpegArgs[format]);
if (streamInfo.metadata) {
args = args.concat(metadataManager(streamInfo.metadata))
}
args.push('-f', format, 'pipe:4');
args.push('-f', format, 'pipe:3');
process = spawn(...getCommand(args), {
windowsHide: true,
stdio: [
'inherit', 'inherit', 'inherit',
'pipe', 'pipe'
'pipe'
],
});
const [,,, audioInput, muxOutput] = process.stdio;
const [,,, muxOutput] = process.stdio;
res.setHeader('Connection', 'keep-alive');
res.setHeader('Content-Disposition', contentDisposition(streamInfo.filename));
audio.on('error', shutdown);
audioInput.on('error', shutdown);
audio.pipe(audioInput);
pipe(muxOutput, res, shutdown);
process.on('close', shutdown);
@ -140,18 +141,20 @@ export async function streamLiveRender(streamInfo, res) {
export function streamAudioOnly(streamInfo, res) {
let process;
const shutdown = () => (killProcess(process), closeResponse(res));
const shutdown = () => (
killProcess(process),
closeResponse(res),
destroyInternalStream(streamInfo.urls)
);
try {
let args = [
'-loglevel', '-8',
'-user_agent', genericUserAgent
];
'-headers', toRawHeaders(getHeaders(streamInfo.service)),
]
if (streamInfo.service === "twitter") {
args.push('-seekable', '0');
} else if (streamInfo.service === 'bilibili') {
args.push('-headers', 'Referer: https://www.bilibili.com/\r\n');
}
args.push(
@ -162,12 +165,12 @@ export function streamAudioOnly(streamInfo, res) {
if (streamInfo.metadata) {
args = args.concat(metadataManager(streamInfo.metadata))
}
let arg = streamInfo.copy ? ffmpegArgs["copy"] : ffmpegArgs["audio"];
args = args.concat(arg);
args = args.concat(ffmpegArgs[streamInfo.copy ? 'copy' : 'audio']);
if (ffmpegArgs[streamInfo.audioFormat]) {
args = args.concat(ffmpegArgs[streamInfo.audioFormat])
}
args.push('-f', streamInfo.audioFormat === "m4a" ? "ipod" : streamInfo.audioFormat, 'pipe:3');
process = spawn(...getCommand(args), {
@ -192,17 +195,20 @@ export function streamAudioOnly(streamInfo, res) {
export function streamVideoOnly(streamInfo, res) {
let process;
const shutdown = () => (killProcess(process), closeResponse(res));
const shutdown = () => (
killProcess(process),
closeResponse(res),
destroyInternalStream(streamInfo.urls)
);
try {
let args = [
'-loglevel', '-8'
'-loglevel', '-8',
'-headers', toRawHeaders(getHeaders(streamInfo.service)),
]
if (streamInfo.service === "twitter") {
args.push('-seekable', '0')
} else if (streamInfo.service === 'bilibili') {
args.push('-headers', 'Referer: https://www.bilibili.com/\r\n')
}
args.push(
@ -222,6 +228,7 @@ export function streamVideoOnly(streamInfo, res) {
if (format === "mp4") {
args.push('-movflags', 'faststart+frag_keyframe+empty_moov')
}
args.push('-f', format, 'pipe:3');
process = spawn(...getCommand(args), {
@ -254,10 +261,12 @@ export function convertToGif(streamInfo, res) {
let args = [
'-loglevel', '-8'
]
if (streamInfo.service === "twitter") {
args.push('-seekable', '0')
}
args.push('-i', streamInfo.urls)
args.push('-i', streamInfo.urls);
args = args.concat(ffmpegArgs["gif"]);
args.push('-f', "gif", 'pipe:3');

View File

@ -1061,6 +1061,22 @@
"code": 200,
"status": "stream"
}
}, {
"name": "yappy",
"url": "https://rutube.ru/yappy/f1771e86294748eea8ecb2ac88e55740/",
"params": {},
"expected": {
"code": 200,
"status": "stream"
}
}, {
"name": "shorts",
"url": "https://rutube.ru/shorts/935c1afafd0e7d52836d671967d53dac/",
"params": {},
"expected": {
"code": 200,
"status": "stream"
}
}, {
"name": "vertical video (isAudioOnly)",
"url": "https://rutube.ru/video/18a281399b96f9184c647455a86f6724/",