feat(Database::Videos): built-in video cache and support for multiple caching backends

I did this to get rid of Redis compatible DBs and for speed purposes.
This is considered experimental, but everything works fine from
what I have tested.

Here are some benchmarks using the built-in benchmark library of
crystal:

\# built-in release
cache get  19.79M ( 50.54ns) (± 4.12%)  32.0B/op  fastest
cache insert   7.88k (126.86µs) (± 2.20%)  65.5kB/op  fastest
cache get   4.31k (232.11µs) (± 5.50%)  104kB/op  fastest

\# redis release
cache get  22.27k ( 44.90µs) (± 6.40%)  264B/op  fastest
cache insert   4.74k (211.01µs) (± 4.72%)  65.7kB/op  fastest
cache get   2.51k (399.11µs) (±13.15%)  129kB/op  fastest

---

OP/s are way higher, and memory usage per call is lower, so it's a win
win.
This commit is contained in:
Fijxu
2025-02-26 17:51:54 -03:00
parent 62cc10d2ca
commit e76867aaba
3 changed files with 164 additions and 46 deletions

View File

@@ -156,19 +156,6 @@ end
OUTPUT = CONFIG.output.upcase == "STDOUT" ? STDOUT : File.open(CONFIG.output, mode: "a")
LOGGER = Invidious::LogHandler.new(OUTPUT, CONFIG.log_level, CONFIG.colorize_logs)
REDIS_DB = begin
LOGGER.info "Connecting to Redis compatible DB"
redis = Redis::PooledClient.new(unixsocket: CONFIG.redis_socket || nil, url: CONFIG.redis_url || nil)
if redis.ping
LOGGER.info "Connected to Redis compatible DB via unix domain socket at '#{CONFIG.redis_socket}'" if CONFIG.redis_socket
LOGGER.info "Connected to Redis compatible DB via TCP socket at '#{CONFIG.redis_url}'" if CONFIG.redis_url
end
redis
rescue ex
LOGGER.error "Failed to connect to a Redis compatible DB. Invidious will store the video cache on the PostgresSQL DB"
nil
end
# Check table integrity
Invidious::Database.check_integrity(CONFIG)