mirrors/cobalt
1
0
Fork 0
mirror of https://github.com/imputnet/cobalt.git synced 2025-07-18 19:28:29 +00:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

Merge branch 'imputnet:current' into support/niconico

Browse Source
This commit is contained in:
Mikhail Serebryakov 2024-05-22 14:25:04 +05:00 committed by GitHub
commit cf74d229a3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 104 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -55,7 +55,7 @@ this list is not final and keeps expanding over time. if support for a service y
## cobalt api ## cobalt api
cobalt has an open api that you can use in your projects *for free~*. it's easy and straightforward to use, [check out the docs](/docs/api.md) to learn how to use it. cobalt has an open api that you can use in your projects *for free~*. it's easy and straightforward to use, [check out the docs](/docs/api.md) to learn how to use it.
✅ you can use the main api instance ([co.wuk.sh](https://co.wuk.sh/)) in your **personal** projects. ✅ you can use the main api instance ([api.cobalt.tools](https://api.cobalt.tools/)) in your **personal** projects.
❌ you cannot use the free api commercially (anywhere that's gated behind paywalls or ads). host your own instance for this. ❌ you cannot use the free api commercially (anywhere that's gated behind paywalls or ads). host your own instance for this.
we reserve the right to restrict abusive/excessive access to the main instance api. we reserve the right to restrict abusive/excessive access to the main instance api.
@ -65,7 +65,7 @@ if you want to run your own instance for whatever purpose, [follow this guide](/
it's *highly* recommended to use a docker compose method unless you run for developing/debugging purposes. it's *highly* recommended to use a docker compose method unless you run for developing/debugging purposes.
## partners ## partners
cobalt is sponsored by [royalehosting.net](https://royalehosting.net/), all main instances are currently hosted on their network :) cobalt is sponsored by [royalehosting.net](https://royalehosting.net/?partner=cobalt), all main instances are currently hosted on their network :)
## ethics and disclaimer ## ethics and disclaimer
cobalt is a tool for easing content downloads from internet and takes ***zero liability***. you are responsible for what you download, how you use and distribute that content. please be mindful when using content of others and always credit original creators. fair use and credits benefit everyone. cobalt is a tool for easing content downloads from internet and takes ***zero liability***. you are responsible for what you download, how you use and distribute that content. please be mindful when using content of others and always credit original creators. fair use and credits benefit everyone.

2
docs/api.md
View File

@ -2,7 +2,7 @@
this document provides info about methods and acceptable variables for all cobalt api requests. this document provides info about methods and acceptable variables for all cobalt api requests.
``` ```
👍 you can use co.wuk.sh instance in your projects for free, just don't be an asshole. 👍 you can use api.cobalt.tools in your projects for free, just don't be an asshole.
``` ```
## POST: `/api/json` ## POST: `/api/json`

8
docs/examples/docker-compose.example.yml
View File

@ -17,8 +17,8 @@ services:
#- 127.0.0.1:9000:9000 #- 127.0.0.1:9000:9000
environment: environment:
# replace https://co.wuk.sh/ with your instance's target url in same format # replace https://api.cobalt.tools/ with your instance's target url in same format
API_URL: "https://co.wuk.sh/" API_URL: "https://api.cobalt.tools/"
# replace eu-nl with your instance's distinctive name # replace eu-nl with your instance's distinctive name
API_NAME: "eu-nl" API_NAME: "eu-nl"
# if you want to use cookies when fetching data from services, uncomment the next line and the lines under volume # if you want to use cookies when fetching data from services, uncomment the next line and the lines under volume
@ -49,8 +49,8 @@ services:
environment: environment:
# replace https://cobalt.tools/ with your instance's target url in same format # replace https://cobalt.tools/ with your instance's target url in same format
WEB_URL: "https://cobalt.tools/" WEB_URL: "https://cobalt.tools/"
# replace https://co.wuk.sh/ with preferred api instance url # replace https://api.cobalt.tools/ with preferred api instance url
API_URL: "https://co.wuk.sh/" API_URL: "https://api.cobalt.tools/"
labels: labels:
- com.centurylinklabs.watchtower.scope=cobalt - com.centurylinklabs.watchtower.scope=cobalt

31
docs/run-an-instance.md
View File

@ -41,7 +41,7 @@ setup script installs all needed `npm` dependencies, but you have to install `no
4. done. 4. done.
### ubuntu 22.04 workaround ### ubuntu 22.04 workaround
`nscd` needs to be installed and running so that the `ffmpeg-static` binary can resolve DNS ([#101](https://github.com/wukko/cobalt/issues/101#issuecomment-1494822258)): `nscd` needs to be installed and running so that the `ffmpeg-static` binary can resolve DNS ([#101](https://github.com/imputnet/cobalt/issues/101#issuecomment-1494822258)):
```bash ```bash
sudo apt install nscd sudo apt install nscd
@ -54,12 +54,11 @@ sudo service nscd start
|:----------------------|:----------|:------------------------|:------------| |:----------------------|:----------|:------------------------|:------------|
| `API_PORT` | `9000` | `9000` | changes port from which api server is accessible. | | `API_PORT` | `9000` | `9000` | changes port from which api server is accessible. |
| `API_LISTEN_ADDRESS` | `0.0.0.0` | `127.0.0.1` | changes address from which api server is accessible. **if you are using docker, you usually don't need to configure this.** | | `API_LISTEN_ADDRESS` | `0.0.0.0` | `127.0.0.1` | changes address from which api server is accessible. **if you are using docker, you usually don't need to configure this.** |
| `API_URL` | ➖ | `https://co.wuk.sh/` | changes url from which api server is accessible. <br> ***REQUIRED TO RUN THE API***. | | `API_URL` | ➖ | `https://api.cobalt.tools/` | changes url from which api server is accessible. <br> ***REQUIRED TO RUN THE API***. |
| `API_NAME` | `unknown` | `ams-1` | api server name that is shown in `/api/serverInfo`. | | `API_NAME` | `unknown` | `ams-1` | api server name that is shown in `/api/serverInfo`. |
| `CORS_WILDCARD` | `1` | `0` | toggles cross-origin resource sharing. <br> `0`: disabled. `1`: enabled. | | `CORS_WILDCARD` | `1` | `0` | toggles cross-origin resource sharing. <br> `0`: disabled. `1`: enabled. |
| `CORS_URL` | not used | `https://cobalt.tools/` | cross-origin resource sharing url. api will be available only from this url if `CORS_WILDCARD` is set to `0`. | | `CORS_URL` | not used | `https://cobalt.tools/` | cross-origin resource sharing url. api will be available only from this url if `CORS_WILDCARD` is set to `0`. |
| `COOKIE_PATH` | not used | `/cookies.json` | path for cookie file relative to main folder. | | `COOKIE_PATH` | not used | `/cookies.json` | path for cookie file relative to main folder. |
| `TIKTOK_DEVICE_INFO` | ➖ | *see below* | device info (including `iid` and `device_id`) for tiktok functionality. required for tiktok to work. see below for more info. |
| `PROCESSING_PRIORITY` | not used | `10` | changes `nice` value* for ffmpeg subprocess. available only on unix systems. | | `PROCESSING_PRIORITY` | not used | `10` | changes `nice` value* for ffmpeg subprocess. available only on unix systems. |
| `FREEBIND_CIDR` | ➖ | `2001:db8::/32` | IPv6 prefix used for randomly assigning addresses to cobalt requests. only supported on linux systems. see below for more info. | | `FREEBIND_CIDR` | ➖ | `2001:db8::/32` | IPv6 prefix used for randomly assigning addresses to cobalt requests. only supported on linux systems. see below for more info. |
| `RATELIMIT_WINDOW` | `60` | `120` | rate limit time window in **seconds**. | | `RATELIMIT_WINDOW` | `60` | `120` | rate limit time window in **seconds**. |
@ -68,28 +67,6 @@ sudo service nscd start
\* the higher the nice value, the lower the priority. [read more here](https://en.wikipedia.org/wiki/Nice_(Unix)). \* the higher the nice value, the lower the priority. [read more here](https://en.wikipedia.org/wiki/Nice_(Unix)).
#### TIKTOK_DEVICE_INFO
you need to get your own device info for tiktok functionality to work. this can be done by proxying the app through any request-intercepting proxy (such as [mitmproxy](https://mitmproxy.org)). you need to disable ssl pinning to see requests. there will be no assistance provided by cobalt for this.
example config (replace **ALL** values with ones you got from mitm):
```
'{
"iid": "<install_id here>",
"device_id": "<device_id here>",
"channel": "googleplay",
"app_name": "musical_ly",
"version_code": "310503",
"device_platform": "android",
"device_type": "Redmi+7",
"os_version": "13"
}'
```
you can compress the json to save space. if you're using a `.env` file then the line would would look like this (***note the quotes***):
```
TIKTOK_DEVICE_INFO='{"iid":"<install_id here>","device_id":"<device_id here>","channel":"googleplay","app_name":"musical_ly","version_code":"310503","device_platform":"android","device_type":"Redmi+7","os_version":"13"}'
```
#### FREEBIND_CIDR #### FREEBIND_CIDR
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all
requests it makes for that particular download. to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first. if you configure this option while running cobalt requests it makes for that particular download. to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first. if you configure this option while running cobalt
@ -98,10 +75,10 @@ in a docker container, you also need to set the `API_LISTEN_ADDRESS` env to `127
### variables for web ### variables for web
| variable name | default | example | description | | variable name | default | example | description |
|:---------------------|:---------------------|:------------------------|:--------------------------------------------------------------------------------------| |:---------------------|:----------------------------|:----------------------------|:--------------------------------------------------------------------------------------|
| `WEB_PORT` | `9001` | `9001` | changes port from which frontend server is accessible. | | `WEB_PORT` | `9001` | `9001` | changes port from which frontend server is accessible. |
| `WEB_URL` | ➖ | `https://cobalt.tools/` | changes url from which frontend server is accessible. <br> ***REQUIRED TO RUN WEB***. | | `WEB_URL` | ➖ | `https://cobalt.tools/` | changes url from which frontend server is accessible. <br> ***REQUIRED TO RUN WEB***. |
| `API_URL` | `https://co.wuk.sh/` | `https://co.wuk.sh/` | changes url which is used for api requests by frontend clients. | | `API_URL` | `https://api.cobalt.tools/` | `https://api.cobalt.tools/` | changes url which is used for api requests by frontend clients. |
| `SHOW_SPONSORS` | `0` | `1` | toggles sponsor list in about popup. <br> `0`: disabled. `1`: enabled. | | `SHOW_SPONSORS` | `0` | `1` | toggles sponsor list in about popup. <br> `0`: disabled. `1`: enabled. |
| `IS_BETA` | `0` | `1` | toggles beta tag next to cobalt logo. <br> `0`: disabled. `1`: enabled. | | `IS_BETA` | `0` | `1` | toggles beta tag next to cobalt logo. <br> `0`: disabled. `1`: enabled. |
| `PLAUSIBLE_HOSTNAME` | ➖ | `plausible.io`* | enables plausible analytics with provided hostname as receiver backend. | | `PLAUSIBLE_HOSTNAME` | ➖ | `plausible.io`* | enables plausible analytics with provided hostname as receiver backend. |

2
package.json
View File

@ -1,7 +1,7 @@
{ {
"name": "cobalt", "name": "cobalt",
"description": "save what you love", "description": "save what you love",
"version": "7.14", "version": "7.14.1",
"author": "imput", "author": "imput",
"exports": "./src/cobalt.js", "exports": "./src/cobalt.js",
"type": "module", "type": "module",

2
src/config.json
View File

@ -52,7 +52,7 @@
"saveToGalleryShortcut": "https://www.icloud.com/shortcuts/14e9aebf04b24156acc34ceccf7e6fcd", "saveToGalleryShortcut": "https://www.icloud.com/shortcuts/14e9aebf04b24156acc34ceccf7e6fcd",
"saveToFilesShortcut": "https://www.icloud.com/shortcuts/2134cd9d4d6b41448b2201f933542b2e", "saveToFilesShortcut": "https://www.icloud.com/shortcuts/2134cd9d4d6b41448b2201f933542b2e",
"statusPage": "https://status.cobalt.tools/", "statusPage": "https://status.cobalt.tools/",
"troubleshootingGuide": "https://github.com/wukko/cobalt/blob/current/docs/troubleshooting.md" "troubleshootingGuide": "https://github.com/imputnet/cobalt/blob/current/docs/troubleshooting.md"
}, },
"celebrations": { "celebrations": {
"01-01": "🎄", "01-01": "🎄",

2
src/modules/changelog/changelog.json
View File

@ -9,7 +9,7 @@
"width": 1736, "width": 1736,
"height": 1440 "height": 1440
}, },
"content": "yesterday, cobalt hit 1 million users around the world! it's an absolutely insane milestone for us and we're incredibly grateful to everyone saving and creating what they love with help of cobalt. thank you for being our friends.\n\nin anticipation of 7 figure user count, we've revamped the cobalt codebase and infrastructure to be faster and more reliable than ever. a combination of many changes has resulted into incredible download speeds (up to 30 MB/s, as tested by both developers in europe).\n\nnote: there's no backend instance in asia just yet, so if you're there, you might experience average speeds *for now*. you can help us afford a dedicated server in asia by donating to cobalt in the \"donate\" menu.\n\n<span class=\"text-backdrop\">changes since the last major update</span>\n\nservice improvements:\n*; youtube music support on the main instance is back!\n*; added support for pinterest images and gifs.\n*; cobalt will now use original soundcloud mp3 file when available.\n*; fixed a youtube bug that prevented some videos from downloading.\n\nupdate on tiktok downloads: they're currently not available, but we're working on a fix. follow our <a class=\"text-backdrop link\" href=\"https://x.com/justusecobalt\" target=\"_blank\">twitter/x account</a> or <a class=\"text-backdrop link\" href=\"{repo}/issues/497\" target=\"_blank\">related github issue</a> for timely updates.\n\nui/ux improvements:\n*; cobalt web app is now fully optimized for ipad. you can add it to home screen from share menu to make it act like a native app!\n*; majorly reduced vertical padding when viewing cobalt in mobile web browser, allowing for more content at once. most noticeable on smaller screens.\n*; status bar color is now dynamic in the web browser on ios and web app on android.\n*; web app on android feels way more native than before.\n*; filename style icons are no longer blurry in safari.\n*; changelog notification no longer overlaps with dynamic island on newer iphones when cobalt is installed as a web app.\n*; fixed safe area padding.\n\nother changes:\n*; added support for <a class=\"text-backdrop link\" href=\"https://github.com/imputnet/freebind.js\" target=\"_blank\">freebind</a>, made by one of the cobalt developers.\n*; rate limit and max video length limits are now customizable through <a class=\"text-backdrop link\" href=\"{repo}/blob/current/docs/run-an-instance.md#variables-for-api\" target=\"_blank\">environment variables</a>.\n*; cobalt api now returns rate limit headers at all times.\n*; majorly cleaned up the codebase: removed unnecessary functions, rewrote those that were cryptic and confusing. it's way more comprehensible and contribution-friendly than ever before.\n*; moved the <a class=\"text-backdrop link\" href=\"{repo}\" target=\"_blank\">cobalt repo</a> to our organization on github. everything stayed the same and all old links link back to it.\n\nnote for instance hosters:\nalong with cobalt repo, the docker image also moved! please update the url for it in your config along with watchtower args to include restarting containers (just in case) as seen in <a class=\"text-backdrop link\" href=\"{repo}/blob/current/docs/examples/docker-compose.example.yml\" target=\"_blank\">updated docker compose example</a>. we're mirroring packages to old url for now, but it won't last forever.\n\nthat's it for now! hope you have an amazing day and share the 1 million celebration with us :)\n\njoin our <a class=\"text-backdrop link\" href=\"https://discord.gg/pQPt8HBUPu\" target=\"_blank\">discord server</a> to discuss everything cobalt there" "content": "yesterday, cobalt hit 1 million users around the world! it's an absolutely insane milestone for us and we're incredibly grateful to everyone saving and creating what they love with help of cobalt. thank you for being our friends.\n\nin anticipation of 7 figure user count, we've revamped the cobalt codebase and infrastructure to be faster and more reliable than ever. a combination of many changes has resulted into incredible download speeds (up to 30 MB/s, as tested by both developers in europe).\n\nnote: there's no backend instance in asia just yet, so if you're there, you might experience average speeds *for now*. you can help us afford a dedicated server in asia by donating to cobalt in the \"donate\" menu.\n\n<span class=\"text-backdrop\">changes since the last major update</span>\n\nservice improvements:\n*; youtube music support on the main instance is back!\n*; added support for pinterest images and gifs.\n*; cobalt will now use original soundcloud mp3 file when available.\n*; fixed a youtube bug that prevented some videos from downloading.\n\nui/ux improvements:\n*; cobalt web app is now fully optimized for ipad. you can add it to home screen from share menu to make it act like a native app!\n*; majorly reduced vertical padding when viewing cobalt in mobile web browser, allowing for more content at once. most noticeable on smaller screens.\n*; status bar color is now dynamic in the web browser on ios and web app on android.\n*; web app on android feels way more native than before.\n*; filename style icons are no longer blurry in safari.\n*; changelog notification no longer overlaps with dynamic island on newer iphones when cobalt is installed as a web app.\n*; fixed safe area padding.\n\nother changes:\n*; added support for <a class=\"text-backdrop link\" href=\"https://github.com/imputnet/freebind.js\" target=\"_blank\">freebind</a>, made by one of the cobalt developers.\n*; rate limit and max video length limits are now customizable through <a class=\"text-backdrop link\" href=\"{repo}/blob/current/docs/run-an-instance.md#variables-for-api\" target=\"_blank\">environment variables</a>.\n*; cobalt api now returns rate limit headers at all times.\n*; majorly cleaned up the codebase: removed unnecessary functions, rewrote those that were cryptic and confusing. it's way more comprehensible and contribution-friendly than ever before.\n*; moved the <a class=\"text-backdrop link\" href=\"{repo}\" target=\"_blank\">cobalt repo</a> to our organization on github. everything stayed the same and all old links link back to it.\n\nnote for instance hosters:\nalong with cobalt repo, the docker image also moved! please update the url for it in your config along with watchtower args to include restarting containers (just in case) as seen in <a class=\"text-backdrop link\" href=\"{repo}/blob/current/docs/examples/docker-compose.example.yml\" target=\"_blank\">updated docker compose example</a>. we're mirroring packages to old url for now, but it won't last forever.\n\nthat's it for now! hope you have an amazing day and share the 1 million celebration with us :)\n\njoin our <a class=\"text-backdrop link\" href=\"https://discord.gg/pQPt8HBUPu\" target=\"_blank\">discord server</a> to discuss everything cobalt there"
}, },
"history": [{ "history": [{
"version": "7.13", "version": "7.13",

1
src/modules/config.js
View File

@ -39,7 +39,6 @@ const
corsURL: process.env.CORS_URL, corsURL: process.env.CORS_URL,
cookiePath: process.env.COOKIE_PATH, cookiePath: process.env.COOKIE_PATH,
tiktokDeviceInfo: process.env.TIKTOK_DEVICE_INFO && JSON.parse(process.env.TIKTOK_DEVICE_INFO),
rateLimitWindow: (process.env.RATELIMIT_WINDOW && parseInt(process.env.RATELIMIT_WINDOW)) || 60, rateLimitWindow: (process.env.RATELIMIT_WINDOW && parseInt(process.env.RATELIMIT_WINDOW)) || 60,
rateLimitMax: (process.env.RATELIMIT_MAX && parseInt(process.env.RATELIMIT_MAX)) || 20, rateLimitMax: (process.env.RATELIMIT_MAX && parseInt(process.env.RATELIMIT_MAX)) || 20,

22
src/modules/processing/matchActionDecider.js
View File

@ -2,6 +2,7 @@ import { audioIgnore, services, supportedAudio } from "../config.js";
import { createResponse } from "../processing/request.js"; import { createResponse } from "../processing/request.js";
import loc from "../../localization/manager.js"; import loc from "../../localization/manager.js";
import createFilename from "./createFilename.js"; import createFilename from "./createFilename.js";
import { createStream } from "../stream/manage.js";
export default function(r, host, userFormat, isAudioOnly, lang, isAudioMuted, disableMetadata, filenamePattern, toGif, requestIP) { export default function(r, host, userFormat, isAudioOnly, lang, isAudioMuted, disableMetadata, filenamePattern, toGif, requestIP) {
let action, let action,
@ -68,16 +69,22 @@ export default function(r, host, userFormat, isAudioOnly, lang, isAudioMuted, di
params = { picker: r.picker }; params = { picker: r.picker };
break; break;
case "tiktok": case "tiktok":
let pickerType = "render"; let audioStreamType = "render";
if (audioFormat === "mp3" || audioFormat === "best") { if (r.bestAudio === "mp3" && (audioFormat === "mp3" || audioFormat === "best")) {
audioFormat = "mp3"; audioFormat = "mp3";
pickerType = "bridge" audioStreamType = "bridge"
} }
params = { params = {
type: pickerType,
picker: r.picker, picker: r.picker,
u: Array.isArray(r.urls) ? r.urls[1] : r.urls, u: createStream({
copy: audioFormat === "best" ? true : false service: "tiktok",
type: audioStreamType,
u: r.urls,
filename: r.audioFilename,
isAudioOnly: true,
audioFormat,
}),
copy: audioFormat === "best"
} }
} }
break; break;
@ -147,11 +154,12 @@ export default function(r, host, userFormat, isAudioOnly, lang, isAudioMuted, di
const isTumblrAudio = host === "tumblr" && !r.filename; const isTumblrAudio = host === "tumblr" && !r.filename;
const isSoundCloud = host === "soundcloud"; const isSoundCloud = host === "soundcloud";
const isTiktok = host === "tiktok";
if (isBestAudioDefined || isBestHostAudio) { if (isBestAudioDefined || isBestHostAudio) {
audioFormat = serviceBestAudio; audioFormat = serviceBestAudio;
processType = "bridge"; processType = "bridge";
if (isSoundCloud) { if (isSoundCloud || (isTiktok && audioFormat === "m4a")) {
processType = "render" processType = "render"
copy = true copy = true
} }

83
src/modules/processing/services/tiktok.js
View File

@ -1,13 +1,13 @@
import { genericUserAgent, env } from "../../config.js"; import { genericUserAgent } from "../../config.js";
import { updateCookie } from "../cookie/manager.js";
import { extract } from "../url.js";
import Cookie from "../cookie/cookie.js";
const shortDomain = "https://vt.tiktok.com/"; const shortDomain = "https://vt.tiktok.com/";
const apiPath = "https://api22-normal-c-alisg.tiktokv.com/aweme/v1/feed/?region=US&carrier_region=US"; export const cookie = new Cookie({});
const apiUserAgent = "TikTok/338014 CFNetwork/1410.1 Darwin/22.6.0";
export default async function(obj) { export default async function(obj) {
let postId = obj.postId ? obj.postId : false; let postId = obj.postId;
if (!env.tiktokDeviceInfo) return { error: 'ErrorCouldntFetch' };
if (!postId) { if (!postId) {
let html = await fetch(`${shortDomain}${obj.id}`, { let html = await fetch(`${shortDomain}${obj.id}`, {
@ -19,54 +19,60 @@ export default async function(obj) {
if (!html) return { error: 'ErrorCouldntFetch' }; if (!html) return { error: 'ErrorCouldntFetch' };
if (html.slice(0, 17) === '<a href="https://') { if (html.startsWith('<a href="https://')) {
postId = html.split('<a href="https://')[1].split('?')[0].split('/')[3] const extractedURL = html.split('<a href="')[1].split('?')[0];
} else if (html.slice(0, 32) === '<a href="https://m.tiktok.com/v/' && html.includes('/v/')) { const { patternMatch } = extract(extractedURL);
postId = html.split('/v/')[1].split('.html')[0].replace("/", '') postId = patternMatch.postId
} }
} }
if (!postId) return { error: 'ErrorCantGetID' }; if (!postId) return { error: 'ErrorCantGetID' };
let deviceInfo = new URLSearchParams(env.tiktokDeviceInfo).toString(); // should always be /video/, even for photos
const res = await fetch(`https://tiktok.com/@i/video/${postId}`, {
let apiURL = new URL(apiPath);
apiURL.searchParams.append("aweme_id", postId);
let detail = await fetch(`${apiURL.href}&${deviceInfo}`, {
headers: { headers: {
"user-agent": apiUserAgent "user-agent": genericUserAgent,
cookie,
} }
}).then(r => r.json()).catch(() => {}); })
updateCookie(cookie, res.headers);
detail = detail?.aweme_list?.find(v => v.aweme_id === postId); const html = await res.text();
if (!detail) return { error: 'ErrorCouldntFetch' };
let detail;
try {
const json = html
.split('<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__" type="application/json">')[1]
.split('</script>')[0]
const data = JSON.parse(json)
detail = data["__DEFAULT_SCOPE__"]["webapp.video-detail"]["itemInfo"]["itemStruct"]
} catch {
return { error: 'ErrorCouldntFetch' };
}
let video, videoFilename, audioFilename, audio, images, let video, videoFilename, audioFilename, audio, images,
filenameBase = `tiktok_${detail.author.unique_id}_${postId}`, filenameBase = `tiktok_${detail.author.uniqueId}_${postId}`,
bestAudio = 'm4a'; bestAudio = 'm4a';
images = detail.image_post_info?.images; images = detail.imagePost?.images;
let playAddr = detail.video.play_addr_h264; let playAddr = detail.video.playAddr;
if (obj.h265) { if (obj.h265) {
playAddr = detail.video.bit_rate[0].play_addr const h265PlayAddr = detail.video.bitrateInfo.find(b => b.CodecType.includes("h265"))?.PlayAddr.UrlList[0]
} playAddr = h265PlayAddr || playAddr
if (!playAddr && detail.video.play_addr) {
playAddr = detail.video.play_addr
} }
if (!obj.isAudioOnly && !images) { if (!obj.isAudioOnly && !images) {
video = playAddr.url_list[0]; video = playAddr;
videoFilename = `${filenameBase}.mp4`; videoFilename = `${filenameBase}.mp4`;
} else { } else {
let fallback = playAddr.url_list[0]; audio = playAddr;
audio = fallback;
audioFilename = `${filenameBase}_audio`; audioFilename = `${filenameBase}_audio`;
if (obj.fullAudio || fallback.includes("music")) {
audio = detail.music.play_url.url_list[0] if (obj.fullAudio || !audio) {
audioFilename = `${filenameBase}_audio_original` audio = detail.music.playUrl;
audioFilename += `_original`
} }
if (audio.slice(-4) === ".mp3") bestAudio = 'mp3'; if (audio.includes("mime_type=audio_mpeg")) bestAudio = 'mp3';
} }
if (video) return { if (video) return {
@ -80,12 +86,9 @@ export default async function(obj) {
bestAudio bestAudio
} }
if (images) { if (images) {
let imageLinks = []; let imageLinks = images
for (let i in images) { .map(i => i.imageURL.urlList.find(p => p.includes(".jpeg?")))
let sel = images[i].display_image.url_list; .map(url => ({ url }))
sel = sel.filter(p => p.includes(".jpeg?"))
imageLinks.push({url: sel[0]})
}
return { return {
picker: imageLinks, picker: imageLinks,
urls: audio, urls: audio,

4
src/modules/processing/url.js
View File

@ -127,6 +127,10 @@ export function normalizeURL(url) {
} }
export function extract(url) { export function extract(url) {
if (!(url instanceof URL)) {
url = new URL(url);
}
const host = getHostIfValid(url); const host = getHostIfValid(url);
if (!host || !services[host].enabled) { if (!host || !services[host].enabled) {

6
src/modules/setup.js
View File

@ -36,7 +36,7 @@ function setup() {
rl.question(q, r1 => { rl.question(q, r1 => {
switch (r1.toLowerCase()) { switch (r1.toLowerCase()) {
case 'api': case 'api':
console.log(Bright("\nCool! What's the domain this API instance will be running on? (localhost)\nExample: co.wuk.sh")); console.log(Bright("\nCool! What's the domain this API instance will be running on? (localhost)\nExample: api.cobalt.tools"));
rl.question(q, apiURL => { rl.question(q, apiURL => {
ob.API_URL = `http://localhost:9000/`; ob.API_URL = `http://localhost:9000/`;
@ -83,13 +83,13 @@ function setup() {
if (webPort && (webURL === "localhost" || !webURL)) ob.WEB_URL = `http://localhost:${webPort}/`; if (webPort && (webURL === "localhost" || !webURL)) ob.WEB_URL = `http://localhost:${webPort}/`;
console.log( console.log(
Bright("\nOne last thing: what default API domain should be used? (co.wuk.sh)\nIf it's hosted locally, make sure to include the port:") + Cyan(" localhost:9000") Bright("\nOne last thing: what default API domain should be used? (api.cobalt.tools)\nIf it's hosted locally, make sure to include the port:") + Cyan(" localhost:9000")
); );
rl.question(q, apiURL => { rl.question(q, apiURL => {
ob.API_URL = `https://${apiURL.toLowerCase()}/`; ob.API_URL = `https://${apiURL.toLowerCase()}/`;
if (apiURL.includes(':')) ob.API_URL = `http://${apiURL.toLowerCase()}/`; if (apiURL.includes(':')) ob.API_URL = `http://${apiURL.toLowerCase()}/`;
if (!apiURL) ob.API_URL = "https://co.wuk.sh/"; if (!apiURL) ob.API_URL = "https://api.cobalt.tools/";
final() final()
}) })
}); });

15
src/modules/stream/shared.js
View File

@ -1,4 +1,5 @@
import { genericUserAgent } from "../config.js"; import { genericUserAgent } from "../config.js";
import { cookie as tiktokCookie } from "../processing/services/tiktok.js";
const defaultHeaders = { const defaultHeaders = {
'user-agent': genericUserAgent 'user-agent': genericUserAgent
@ -13,9 +14,19 @@ const serviceHeaders = {
origin: 'https://www.youtube.com', origin: 'https://www.youtube.com',
referer: 'https://www.youtube.com', referer: 'https://www.youtube.com',
DNT: '?1' DNT: '?1'
},
tiktok: {
cookie: tiktokCookie
} }
} }
export function getHeaders(service) { export function closeResponse(res) {
return { ...defaultHeaders, ...serviceHeaders[service] } if (!res.headersSent) res.sendStatus(500);
return res.destroy();
}
export function getHeaders(service) {
// Converting all header values to strings
return Object.entries({ ...defaultHeaders, ...serviceHeaders[service] })
.reduce((p, [key, val]) => ({ ...p, [key]: String(val) }), {})
} }

3
src/modules/stream/stream.js
View File

@ -1,5 +1,6 @@
import { streamAudioOnly, streamDefault, streamLiveRender, streamVideoOnly, convertToGif } from "./types.js"; import { streamAudioOnly, streamDefault, streamLiveRender, streamVideoOnly, convertToGif } from "./types.js";
import { internalStream } from './internal.js'; import { internalStream } from './internal.js';
import { closeResponse } from "./shared.js";
export default async function(res, streamInfo) { export default async function(res, streamInfo) {
try { try {
@ -25,6 +26,6 @@ export default async function(res, streamInfo) {
break; break;
} }
} catch { } catch {
res.sendStatus(500); closeResponse(res)
} }
} }

7
src/modules/stream/types.js
View File

@ -6,7 +6,7 @@ import { create as contentDisposition } from "content-disposition-header";
import { metadataManager } from "../sub/utils.js"; import { metadataManager } from "../sub/utils.js";
import { destroyInternalStream } from "./manage.js"; import { destroyInternalStream } from "./manage.js";
import { env, ffmpegArgs } from "../config.js"; import { env, ffmpegArgs } from "../config.js";
import { getHeaders } from "./shared.js"; import { getHeaders, closeResponse } from "./shared.js";
function toRawHeaders(headers) { function toRawHeaders(headers) {
return Object.entries(headers) return Object.entries(headers)
@ -18,11 +18,6 @@ function closeRequest(controller) {
try { controller.abort() } catch {} try { controller.abort() } catch {}
} }
function closeResponse(res) {
if (!res.headersSent) res.sendStatus(500);
return res.destroy();
}
function killProcess(p) { function killProcess(p) {
// ask the process to terminate itself gracefully // ask the process to terminate itself gracefully
p?.kill('SIGTERM'); p?.kill('SIGTERM');