Commit graph

22 commits

Author SHA1 Message Date
Deivid Soto
bc6f85bf39 fix(stream): /critico review fixes for the sidecar cache
- ExtractSubtitlesVTTMulti: distrust output when ffmpeg is killed by signal
  (45-min timeout on a too-big remux) — a truncated WebVTT passed the len>0
  check and got cached as a silently-incomplete track until the media mtime
  changed. Skip all output on signal-kill; keep it on a clean non-zero exit.
- stream handlers: read the sidecar cache BEFORE the ffmpegPath guard so a
  pre-warmed sub/thumbnail still serves if ffmpeg was removed after the cache
  was filled.
- scan: log when the prewarm is skipped because ffmpeg is unavailable (matches
  the daemon; CLAUDE.md wants bootstrap to log on every branch).
- unexport sidecarDir/subtitleCachePath/thumbnailCachePath (no external callers).
- prewarm: surface a sample error in the summary so a systemic ffmpeg failure
  is distinguishable from one corrupt file.
- add unit tests: codec whitelist, cache paths, mtime freshness, atomic write,
  thumb-position dedup.
2026-06-02 13:46:07 +02:00
Deivid Soto
1c8cc1c409 perf(stream): run the subtitle/thumbnail prewarm at idle I/O priority
The prewarm's single big read (a ~14 min sequential pass over a 60GB remux to
demux subtitles) shares the same disk/NFS as live streaming. Lower the prewarm
ffmpeg processes to the Linux IDLE I/O class (ioprio_set) so that background
read yields bandwidth to a user who's actually watching — the prewarm slows
down under contention instead of starving playback, and runs full speed when the
disk is idle.

Applied only to the prewarm-only extractors (ExtractSubtitlesVTTMulti,
ExtractThumbnailJPEG) via Start → setIdleIOPriority(pid) → Wait; the on-demand
/sub + /thumbnail handlers keep normal priority (a user is waiting on those).
Linux-only syscall behind a build tag; a no-op stub elsewhere. Best-effort —
errors ignored, never required for correctness.

Verified: the prewarm ffmpeg shows 'idle' under ionice -p; on-demand stays normal.
2026-06-02 11:51:26 +02:00
Deivid Soto
8a47132f15 perf(stream): extract all text subtitles of a file in one ffmpeg pass
Subtitle extraction is I/O-bound: subtitle packets are interleaved across the
whole container, so ffmpeg must read the entire file to demux a complete track
(measured ~57 MB/s reading a 60GB remux over ~75 MB/s NFS → ~14 min for the
full read). Doing that once per track meant N full reads of a huge file.

ExtractSubtitlesVTTMulti demuxes the container ONCE and routes every text track
to its own WebVTT output, so an N-text-track file costs one read instead of N.
The prewarm now enqueues one job per file (all its text indices) and raises the
per-file deadline to 45 min so even ~200GB remuxes finish the single read in the
background (idempotent; the on-demand /sub keeps its 60s fallback). Thumbnails
are unaffected — a keyframe seek reads a tiny slice (~0.7s even on 60GB).
2026-06-02 10:09:28 +02:00
Deivid Soto
1e5de874cf feat(stream): cache scan-time thumbnail frames to the .unarr sidecar
Pre-extract the file panel's sample frames (10/30/50/70/90% of runtime, w=320)
during the library scan and write-through any on-demand /thumbnail request into
the hidden ".unarr/<name>.t<sec>w<width>.jpg" sidecar. The /thumbnail handler
serves a fresh sidecar instantly, so the characteristics panel and seekbar
previews stop re-running ffmpeg per request.

- mediainfo.sidecar: ThumbnailCachePath, ReadCachedThumbnail, WriteCachedThumbnail,
  ExtractThumbnailJPEG (mirrors engine.buildThumbnailArgs).
- library.PrewarmSidecars: also enqueues the panel frame positions (kept in
  lockstep with the web's THUMB_FRACTIONS / THUMB_WIDTH) per item with a duration.
- thumbnailHandler: cache-read → hit; miss → extract → write-through.
- config: library.cache_thumbnails (default true) + both cache toggles exposed in
  the interactive 'unarr config' library menu.

Local only by design — frames are the user's own content, never uploaded.
2026-06-02 09:20:00 +02:00
Deivid Soto
178c16f458 feat(stream): cache extracted subtitles to a hidden .unarr sidecar
On-demand WebVTT extraction re-ran ffmpeg on every /sub request and, for
50GB+ remuxes, couldn't finish a full text track within the 60s HTTP timeout
→ the web player got a 500 and no subtitles.

Extract each text subtitle ONCE — during the library scan (no HTTP deadline,
generous per-file timeout) and write-through on the first on-demand request —
into a hidden ".unarr/<name>.s<index>.vtt" sidecar next to the media file.
The /sub handler serves a fresh sidecar instantly (mtime-invalidated when the
media is replaced), so playback subtitles are instant and huge files work.

- mediainfo.sidecar: cache paths, mtime freshness, atomic write, ExtractSubtitleVTT,
  IsTextSubtitleCodec (shared classifier, mirrors engine + web whitelists).
- library.PrewarmSidecars: bounded, idempotent, ctx-cancellable background pass
  run after every scan (manual + daemon auto-scan).
- subtitleHandler: cache-read → hit; miss → extract → write-through.
- config: library.cache_subtitles (default true), wired via SetCacheSubtitles.

Local-only by design: nothing extracted is uploaded — the sidecar is the user's
own content, private to their disk.
2026-06-02 09:10:36 +02:00
Deivid Soto
992e16ba05 feat(stream): transcode debrid sources to HLS from a URL (hueco #2/2b)
Non-browser-native debrid content (mkv/HEVC/…) can now stream: ffmpeg reads
the debrid HTTPS link directly (-i <url>) and transcodes to HLS, instead of
2a's raw direct-play which only works for mp4/m4v.

- HLSSessionConfig gains SourceURL + CacheID; sourceRef() feeds ffprobe,
  ffmpeg -i, and subtitle extraction from one place. HTTP-resilience flags
  (-reconnect*, -rw_timeout) are added only for a URL source; a seek-restart
  re-opens the URL with a Range request (-ss before -i = input seek).
- Segment cache keys by CacheID (the torrent info_hash) for URL sessions so
  re-plays hit cache despite the debrid URL changing each resolution
  (KeyForID, no filepath.Abs).
- OnStreamSession: the 2a direct-play branch is now gated on PlayMethod != "hls";
  a new branch handles DirectURL + PlayMethod=="hls" → HLS-from-URL. The
  local-file and both debrid HLS paths share a startHLSPlayback helper.
- ExtractMediaInfo no longer masks a URL probe failure as "file not found"
  (surfaces ffprobe's real stderr, e.g. "Protocol not found" on a TLS-less
  ffmpeg build).
- Bump 0.11.0 -> 0.12.0 as the HLS-from-URL floor the web gates on.

Validated e2e against real AllDebrid: a cached HEVC x265 mkv transcodes
(h264_nvenc) from the debrid URL and plays 1080p in Chrome via hls.js,
subtitles extracted from the remote mkv.
2026-05-31 16:22:14 +02:00
Deivid Soto
9df38c95a3 fix(library): classify resolution by width + height, not height alone
Cinematic widescreen content (1920×804 at 2.39:1, 3840×1600 21:9, etc.)
was being misclassified: a 1080p source presented as 1920×804 fell to
720p because 804 < 900. Same shape for 2160p sources letterboxed below
2000px tall.

ResolveResolution now takes (width, height) and picks the larger of the
width-derived and height-derived buckets, so anamorphic/letterboxed
sources land in the right bucket.
2026-05-27 11:54:29 +02:00
Deivid Soto
ca7de23a56 feat(stream)!: retire WebRTC, HLS-only, bump 0.9.4
Some checks failed
Release / release (push) Failing after 0s
Release / docker (push) Has been skipped
Release / virustotal (push) Failing after 0s
Drops the custom WebRTC DataChannel pipeline + pion deps + WSS signaling
client + wire framing. Every in-browser playback now uses HLS over HTTP
from the daemon (Tailscale/LAN/UPnP). Browser P2P never re-enabled.

Wire renames (incompatible with web < 2026-05-26): agent.WebRTCSession
=> agent.StreamSession, SyncResponse.WebRTCSessions (JSON: webrtcSessions)
=> StreamSessions (JSON: streamSessions). MIN_AGENT_VERSION is bumped
to 0.9.4 on the web side so older agents see an upgrade card.

Also fixes the libx264 'VBV bitrate > level limit' abort by clamping
the encoder bitrate to the effective output height instead of the
requested label (carried over from the prior 0.9.3 unreleased work).

The seed_file vertical (mode=seed_file handler + engine.SeedFile) was
retired with the in-browser P2P player. [downloads.webrtc] config block
deleted; existing TOML files with the section still parse fine.
2026-05-26 18:04:35 +02:00
Deivid Soto
727ab19468 feat(mediainfo): ResolveFFmpeg + DownloadFFmpeg mirroring ffprobe pattern
Adds the ffmpeg-binary half of the resolution stack so the upcoming
WebRTC streaming transcoder (Fase 3.3) has a single point of entry.

Search order matches ResolveFFprobe so operators don't need to learn a
second mental model:
  1. Explicit path  (--ffmpeg flag / library.ffmpeg_path config)
  2. FFMPEG_PATH env var
  3. "ffmpeg" on PATH (system install)
  4. Adjacent to the unarr executable (release tarball bundles it here —
     this is the preferred path; see Fase 3.2 goreleaser changes)
  5. Cache dir (sibling of the cached ffprobe binary)
  6. Auto-download from ffbinaries.com (~70MB) as last resort

Includes:
- internal/library/mediainfo/ffmpeg.go         — ResolveFFmpeg + actionable
  Docker / non-Docker error messages
- internal/library/mediainfo/ffmpeg_download.go — DownloadFFmpeg, reuses
  ffprobePlatformKey + ffprobeAPIClient + ffprobeDLClient + extractFromZip
  helpers; bumps maxZipSize to 200MB (ffmpeg static is ~70-100MB)
- internal/config: LibraryConfig.FFmpegPath toml field for explicit paths
- 4 unit tests: explicit OK, explicit missing, env var, sibling cache path

Tarball bundling and the actual transcoding pipeline land in the next
two commits.
2026-05-06 09:49:32 +02:00
Deivid Soto
f699b26fa6 feat(library): add server-driven file deletion with allow_delete config 2026-04-10 16:35:12 +02:00
Deivid Soto
db316726fd feat(scan): always scan downloads + organize dirs, deduplicate child paths
ResolveScanPaths() collects downloads.dir, organize.movies_dir,
organize.tv_shows_dir, and library.scan_path (if set), then removes
paths that are subdirectories of a parent already in the list.

This ensures the daemon and CLI scan all configured dirs without
relying solely on scan_path being set.
2026-04-10 11:46:20 +02:00
Deivid Soto
fad53a5d84 fix(library): use native arm64 ffprobe on Apple Silicon (osx-arm-64) 2026-04-09 09:26:10 +02:00
Deivid Soto
228564eb7f feat(library): resilient scan for large libraries and better ffprobe errors
- Use a dedicated 10-minute HTTP client for library-sync so libraries
  with hundreds or thousands of items no longer time out
- Show actionable ffprobe-not-found error: detects Docker and suggests
  FFPROBE_PATH env var, config.toml setting, or package install
- Include static ffprobe binary in Docker image (johnvansickle.com)
- Bump version to 0.6.2
2026-04-09 09:13:38 +02:00
Deivid Soto
4d74b8cd8c test(mediainfo): add ffprobe download unit tests 2026-04-06 10:16:27 +02:00
Deivid Soto
3e0f3a5a64 feat(cli): upgrade command, rich status, and version cache
Some checks failed
Release / release (push) Failing after 0s
Release / docker (push) Has been skipped
Release / virustotal (push) Failing after 0s
- Replace `upgrade` stub with real command (alias for `self-update`)
- Also register `update` as alias: `unarr update` works too
- Rewrite `status` to show full config, disk usage, daemon state, and
  update availability with colored sections
- Add version check cache (1h TTL) so `status` is instant on repeat runs
- Guard against division by zero on empty filesystems
- Guard against negative durations from clock skew
- Guard against stale PID via heartbeat recency check (2 min)
- Add comprehensive test coverage across agent, engine, upgrade, usenet,
  arr, library, mediaserver, and UI packages
- Improve Makefile coverage target to exclude cmd/ glue code
- Fix stream handler resource cleanup and ffprobe error handling
2026-03-31 22:05:43 +02:00
Deivid Soto
aed5f0475d fix(lint): use default:none to disable errcheck, fix all gofmt and exhaustive 2026-03-31 00:29:16 +02:00
Deivid Soto
c0fd8d3818 fix(lint): exclude common fire-and-forget patterns from errcheck 2026-03-30 23:34:36 +02:00
Deivid Soto
104820f4fe fix(lint): resolve errcheck and bodyclose warnings for golangci-lint v2 2026-03-30 23:31:06 +02:00
Deivid Soto
5a7449b9e6 chore: rename module from torrentclaw-cli to unarr
- Rename Go module path github.com/torrentclaw/torrentclaw-cli → github.com/torrentclaw/unarr
- Update all imports, ldflags, scripts, docs, and Docker config
- Add GitHub Actions release workflow (goreleaser on tag push)
2026-03-30 13:06:07 +02:00
Deivid Soto
d0f2abcd74 refactor: extract BuildSyncItems to library package, remove duplication 2026-03-29 20:44:33 +02:00
Deivid Soto
386c97f84a fix(torrent): expand tracker list, add DHT persistence and configurable timeouts
- Expand default trackers from 5 to 31 (synced with web tracker-list.ts)
- Add DHT node persistence between sessions (~/.local/share/unarr/dht-nodes.txt)
  Saves known nodes on shutdown, restores on startup for warm DHT bootstrap
- Make metadata_timeout and stall_timeout configurable in config.toml
  Default: 0 (unlimited, like qBittorrent) — users can set custom values
- Fix CleanTitle to handle web domains and format patterns (e.g. pctfenix.com)
2026-03-29 19:09:51 +02:00
Deivid Soto
677a8fe083 feat: add migrate command, media server detection, and debrid auto-config
- Migration wizard from Sonarr/Radarr/Prowlarr (unarr migrate) [pre-beta]
  - Auto-detect instances via Docker, config files, port scan, Prowlarr
  - Import wanted list (monitored+missing movies/series)
  - Import download history and blocklist to avoid re-downloading
  - Extract debrid tokens from *arr download clients
  - Quality profile mapping to preferred_quality config
  - DISTINCT ON PostgreSQL query for optimal torrent selection
  - JSON export with --dry-run --json (text to stderr, JSON to stdout)
- Media server detection (Plex/Jellyfin/Emby) in unarr init
  - Detects library paths and offers them as download directory options
- Debrid auto-configuration in unarr init
  - Scans *arr instances for debrid tokens
  - Validates and saves via API if user confirms
- New preferred_quality setting in config (2160p/1080p/720p)
- Library scan command (unarr scan) with ffprobe metadata extraction
2026-03-29 16:54:32 +02:00