Commit Graph

45 Commits

Author SHA1 Message Date
syeopite
adcdb8cb92 Fix lint and formatting 2025-02-26 14:18:50 -08:00
syeopite
4ea4878d1a User: Batch notifications together 2025-02-26 13:55:25 -08:00
Samantaz Fox
7c79ee7cc2 Performance: Improve speed of automatic instance redirection (#4193)
The automatic instance redirection implemented in #1940 fetches a new list of
instances each time someone queries the /redirect endpoint. This is extremely
inefficient...

This PR optimizes all that into a background job that only fetches a single
list every 30 minutes. This should performance quite a bit.

No related issue was opened.
2024-10-08 17:31:20 +02:00
⛧-440729 [sophie]
3850739d7f apply review suggestions 2024-08-27 10:48:34 +02:00
Sophie Tauchert
5d0149844f Batch user notifications together 2024-08-26 21:24:27 +02:00
Samantaz Fox
a845752fff Jobs: Remove the signature function update job 2024-07-25 22:13:08 +02:00
syeopite
9980c0e00f Update uptime logic to handle updown.io response 2024-05-22 13:28:15 -07:00
syeopite
aa96cf3453 Fix invalid logic for instance uptime comparison 2024-05-22 13:22:00 -07:00
syeopite
41c978d350 Use HTTP::Client directly in instance list job
The HTTP::Client created via `make_client` is affected by the
force_resolve configuration option. However, api.invidious.io
does not support ipv6 and as such any request with ipv6 to
api.invidious.io will instead raise.

Directly calling the HTTP::Client will ignore the force_resolve option
allowing requests to go through ipv4 when needed.
2024-05-22 13:22:00 -07:00
syeopite
cff25a7b25 Refactor instance fetching logic into separate job 2024-05-22 13:22:00 -07:00
vojkovic
7cca1285aa Fix two swapped function names 2024-01-06 15:51:31 +08:00
Samantaz Fox
d9416a0be5 Jobs: Remove BypassCaptchaJob 2023-11-20 17:39:13 +01:00
syeopite
438467f69a Add playback success rate to /api/v1/stats (#4085)
* Add stats-based /videoplayback blockage status

* Count when YouTube returns wrong video as failure

* Cast playback stats hash type prior to return

* Bump stats refresh timer to 10 minutes
2023-11-04 13:52:30 +00:00
Samantaz Fox
8e8ca4fcc5 Prepare to create a 'Channel' module 2022-12-22 16:13:34 +01:00
Samantaz Fox
3b39b8c772 Add table cleaning job (#3294) 2022-10-12 10:06:36 +02:00
Féry Mathieu (Mathius)
dfab62ce48 Rename new property to channel_refresh_interval
Follow indications :
https://github.com/iv-org/invidious/pull/2915#discussion_r811373503
2022-02-22 00:46:26 +01:00
Féry Mathieu (Mathius)
f75a81c9ee Make configurable time between each RefreshChannelsJob 2022-02-21 10:53:20 +01:00
matthewmcgarvey
a82d21ff78 Cleanup channel helpers code 2022-01-27 20:12:42 -06:00
Samantaz Fox
790b7afcca Fix indefinitely growing database 2022-01-09 22:04:10 +01:00
Samantaz Fox
6704ce3214 Move DB utility functions to the proper module 2022-01-04 17:15:43 +01:00
Samantaz Fox
c021b93b5c Move DB queries related to channels in a separate module 2022-01-04 17:13:52 +01:00
Samantaz Fox
d94d4c2045 Move DB queries related to statistics in a separate module 2022-01-04 17:13:52 +01:00
syeopite
e2bbc9a6fa Switch to #sort_by! in pull_popular_videos job 2021-10-25 01:14:14 -07:00
syeopite
e969c1490a Fix Performance/ChainedCallWithNoBang issues 2021-10-25 01:12:26 -07:00
Samantaz Fox
c6f088d6ca Reduce refresh delay, increase backoff start duration 2021-10-08 18:39:02 +02:00
Samantaz Fox
9ba3e1cdb4 Decrease channel refresh frequency (1 min -> 1 h)
This is a temporary fix to reduce load on instances with many channels and avoid IP being flagged by Google.
2021-09-26 23:31:14 +02:00
Émilien Devos
ce68d09d26 Pick a random video for bypass captcha
pick a random video from the 1000 first rows of the channel_videos table
in order to bypass the captcha more efficiently
2021-06-25 15:37:37 +00:00
Émilien Devos
4803285e50 update video URL for recaptcha detection 2021-06-19 17:38:49 +00:00
syeopite
065c104f27 Upgrade to crystal 1.0.0 2021-06-08 13:10:58 -07:00
namazso
f2cc16009f Fix statistics misreporting register allow state 2021-04-10 18:52:07 +02:00
saltycrys
5f03a583d1 Update code for Crystal 0.36.0
Rename `HTTPClient@socket` to `HTTPClient@io`, see
https://github.com/crystal-lang/crystal/pull/9543.

Rename `URI#full_path` to `URI#request_target`, see
https://github.com/crystal-lang/crystal/pull/10099.
2021-02-02 20:01:02 +01:00
saltycrys
b45f371911 Make config a constant
Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 19:39:04 +01:00
saltycrys
df9e7f284c Adjust log verbosity
The default log level has been changed from `debug` to `info`.
The `debug` log level is now more verbose. `debug` now gives a general overview
of what is happening (where implemented) while `trace` gives all available
details.
2021-01-05 20:51:18 +01:00
saltycrys
6365ee7487 Make logger a constant
Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-05 20:43:19 +01:00
vhuynh3000
3a2bd4e928 add config to decrypt on demand instead of polling 2021-01-03 20:50:52 +01:00
Perflyst
c89632d2a8 Merge pull request #1608 from saltycrys/add-subscription-traces
Add Subscription Traces
2020-12-31 11:30:04 +01:00
Andrew Zhao
e0d25ff887 Close http clients after using
The crystal http client maintains a keepalive connection to the other
server which stays alive for some time. This should be closed if the
client instance is not used again to avoid hogging resources
2020-12-27 19:40:58 -05:00
saltycrys
c4ef055248 Add RefreshChannelsJob traces
Traces can be enabled with `-l trace`.

The problem with subscriptions is that sometimes requests to YouTube never
finish. As soon as that happens `channel-threads` times subscriptions stop
being refreshed. This is most likely a problem with the lsquick bindings.
2020-12-27 05:20:33 +01:00
saltycrys
420ceffbb0 Rename threads to fibers
The config and command line options haven't been changed.
2020-12-27 05:14:33 +01:00
saltycrys
b39f01dcdf Improve logging
Everything that gets logged now has a log level associated with it.

The log level can be set with the new `-l` or `--log-level` arguments.

The defaul log level is `debug` for now. There aren't many things that get
logged but if the logs get spammed in the future it can be set down to `info`.
2020-12-21 16:32:34 +01:00
Émilien Devos
3a2f4d4def allow to set a custom anti-captcha api url (#1473) 2020-11-20 22:21:26 +01:00
TheFrenchGhosty
138f38aade more polish (#1437) 2020-10-24 03:26:17 +00:00
Matthew McGarvey
0520e67303 [refactor] Finish converting jobs to new job setup (#1420)
[refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
Matthew McGarvey
989317e5d3 Extract RefreshChannelsJob (#1403) 2020-10-15 14:22:41 +00:00
Matthew McGarvey
85c212aee3 Provide rough draft of better project organization 2020-10-05 23:57:19 -05:00