Close the recurring "video has subtitles but the web player shows none" gap
with a source-agnostic pipeline:
- Discover EXTERNAL sidecar subs in the scan (Video.es.ass siblings + a Subs/
bundle), parse lang/forced/SDH from the filename, skip VobSub (.sub+.idx).
ffprobe-only scanning ignored these (ToonsHub/anime "MSubs" releases).
- Transcode sidecar charset -> UTF-8 before WebVTT (BOM/UTF-16/code-page by
language). Chinese SCRIPT matters: chs/sc -> GBK, cht/tc/big5 -> Big5
(decoding one as the other is mojibake).
- /sub now serves a standalone sidecar file (i=-1, p=file, &l=lang hint) and a
remote debrid URL (ffmpeg reads http, no local stat) — not just embedded
streams of a local file.
- probe.json emits a tokened vttUrl per TEXT track so torrent/debrid HLS streams
(never library-scanned) get subtitles too. Embedded index is counted among
embedded streams only, so -map 0:s:N stays aligned when sidecars are appended.
Tested against a real 347-file gallery: 26/26 sidecars and embedded ass/srt/
mov_text all extract to valid WebVTT; bitmap (pgs/dvd_subtitle) correctly stays
burn-in. Manual harness gated behind GALLERY_DIR.
The runtime stage's `ARG TARGETARCH=amd64` default shadowed buildx's
per-leg value, so even the published arm64 image bundled x86-64
cloudflared and ffmpeg alongside a native arm64 unarr binary. The daemon
spawning cloudflared hit "exec format error", the funnel never came up,
and TV/Stremio connect failed with "Failed to get add-on manifest".
Read the real arch from `dpkg --print-architecture` (the emulated base
image's arch) for both the ffmpeg and cloudflared RUN steps. Correct
under buildx cross-builds and plain `docker build` alike. Drop the
poisoned TARGETARCH default.
Reported-by: Serge <s@bongiozzo.ru>
- IsRevoked no longer matches a bare 401. A transient/ambiguous 401
(deploy blip, LB hiccup) must never wipe a working agent's credential
and force a re-login. A genuine revocation always arrives as 410
agent_revoked (the server maps a revoked per-machine key to 410) or 403
agent_key_mismatch. Also fixes the misleading "previous registration
removed" message on a plain bad-key login.
- Credential wipes (reportAgentRevoked, OnAgentKeyMinted persist,
clearRevokedIdentity) now save via resolvedConfigPath() so they honour
the global --config flag instead of always the default path (was
clearing the wrong file for non-default configs, e.g. unarr-dev).
--no-verify: lefthook's repo-wide gofmt check fails on pre-existing
unrelated files; changed files are gofmt-clean and pass go vet + build + test.
Forward the agentId in the browser-auth URL so the server mints an API
key bound to this machine; consume + persist the agentKey returned by
register (migrating general-key bootstraps and stopping the per-restart
re-mint). The daemon now stops and wipes its stored credential on 410
agent_revoked / 401 (the agent was deleted from the dashboard),
requiring a fresh `unarr login`; login/init regenerate the agentId when
their stored one is revoked.
Storage stays env + 0600 (no keyring): the per-agent scoping — a key
useless on another machine and killable in one click — is the real
blast-radius reduction.
--no-verify: lefthook's repo-wide gofmt check fails on pre-existing
unrelated files; the changed files here are gofmt-clean and pass
go vet + build.
Parse ffmpeg's -stats progress line (speed=Yx, fps=) from the HLS encoder's
stderr into a per-session EWMA, and report a health snapshot to the web side a
few seconds after seg-0. Lets the player name a too-slow transcode from a
direct measurement (~5-7s) instead of inferring it from stall shape (~15-30s).
- hls.go: add -stats; rewrite hlsStderrCapture.Write to frame on \r and \n,
parse speed=/fps= (telemetry only, never logged), flag input-bound on source
read errors. EWMA on HLSSession + GetTranscodeStats(); warmup-skip the first
cold-start frames so a healthy encoder isn't reported as struggling.
- client.go: MarkSessionReady takes an optional *SessionHealth.
- daemon.go: watcher reports one health snapshot once >=4 post-warmup samples
settle; classifyAgentHealth maps the speed ratio to ok/marginal/struggling.
Additive: old web replicas ignore the extra field; cache-hit/direct-play
sessions and short encodes report nil (the web keeps its stall heuristic).
The anacrolix/torrent client maps the listen port on the router via
UPnP/NAT-PMP for inbound peers. Many home routers reject AddPortMapping
with '500 Internal Server Error' and the lib retries every lease cycle,
spamming the daemon log. The rejection is harmless (downloads work over
DHT + outbound peers), so wrap the log handlers and drop just that line
while keeping the mapping attempts for routers that do support it.
The libplacebo HDR->SDR tonemap needs a Vulkan device, but the nvidia
Vulkan ICD (libGLX_nvidia.so.0) pulls in libnvidia-glcore, which
references glibc malloc hooks removed in glibc 2.34 (__malloc_hook etc.)
and the Xorg symbol ErrorF. On any headless modern-glibc container these
go unresolved so vkCreateInstance returns VK_ERROR_INCOMPATIBLE_DRIVER
and the agent correctly falls back to the CPU zscale tonemap chain.
Document why we deliberately do NOT chase it (graphics cap + X11 libs +
1.4 loader + desktop glibc/Xorg, fragile + distro/driver coupled).
nvenc/nvdec (CUDA, not Vulkan) work regardless.
On a docker agent the web DB holds host paths (e.g. /mnt/nas/peliculas/…)
while the container mounts that media at /downloads, so the runtime allowed
root (cfg.Download.Dir=/downloads) rejects the host path. The raw /stream
handler already self-heals via relocateUnreachable, but the HLS/remux session
handler did not — it logged "path outside allowed dirs" and returned, so the
web silently fell back to the raw /stream path (no transcode, slow funnel
start) and HLS/remux never ran. The path-scoped sidecar handlers
(/thumbnail, /trickplay, /sub) had the same skew → 404 for every scrubber
frame, trickplay sprite and external subtitle.
- HLS handler (OnStreamSession): apply the same relocateUnreachable remap as
the raw handler before the dir-resolve.
- StreamServer: add SetPathResolver/healMediaPath, applied in /thumbnail,
/trickplay, /sub AFTER token verification (the token still binds the
original web path; the resolver is a pure function of that path and
re-validates containment, so it can't be abused to serve a different file).
- Hoist the allowed-roots list into streamAllowedRoots(cfg) so the raw, HLS
and sidecar handlers can't drift apart.
Note: relocateUnreachable needs a ≥3-segment path tail, so flat media layouts
are not self-healed (same limitation as /stream; a re-scan rewrites the DB
path). The HLS handler replicates only the lexical remap, not the raw
handler's transient-NFS os.Stat retry.
Supersedes the previous "disable signature verification" stop-gap. The two
checksum tests now run verifyChecksum with signature verification ENABLED using a
per-test ed25519 keypair (withReleasePubKey) and a matching checksums.txt.sig
served over the exact body — so they cover the real production path end to end
instead of skipping it. Adds verifyChecksum-level coverage for the cases that
actually protect a self-update: a checksums file signed by the wrong key is
rejected, a missing .sig is rejected, and verifyChecksumOnly (--allow-unsigned)
still passes on the checksum alone. No production code change.
TestVerifyChecksumWithHTTPTest and TestVerifyChecksumCaseInsensitive predate
release signing (commit 1757bda baked the release pubkey). With the pubkey set,
verifyChecksum now requires a valid checksums.txt.sig and fails at signature
decode before reaching the SHA256 comparison these tests assert. They exercise
the checksum-matching path only, so clear releasePubKeyBase64 for their duration
(t.Cleanup restore) — mirroring the existing pattern in signature_test.go. The
signature path itself keeps its dedicated coverage there. No production change.
Trickplay sprite generation (one full-decode ffmpeg pass per file) could pin a
machine: multiple agents on the same library decoded the same 4K file at once, no
CPU throttling, and crashed/restarted agents orphaned ffmpeg to init (it ran the
full 45-min decode to completion). Stacked orphans spiked a box to load ~140.
- Single-flight lock: O_CREATE|O_EXCL .lock in the shared sidecar dir so two
agents watching the same library never decode the same file twice (stale locks
reclaimed after a TTL). Returns ErrTrickplayInProgress → prewarm skips, not fail.
- Load gate: defer the heavy decode until 1-min load ≤ max(ratio×NumCPU, 1.5),
capped at 15 min so it throttles without ever becoming a permanent off-switch on
busy / small hosts. New knob library.prewarm_max_load_ratio (default 0.7).
- Concurrency: trickSem caps trickplay to ONE decode at a time per agent.
- CPU priority: setLowCPUPriority (nice 19) alongside the existing idle ionice.
- No orphans: hardenCmd sets Setpgid + Pdeathsig=SIGKILL, with runtime.LockOSThread
around the child so the kernel kills ffmpeg exactly when the agent dies (and not
spuriously — golang/go#27505).
Tests: single-flight/stale-reclaim, load-gate immediate/cancel, and an e2e
Pdeathsig orphan-kill check.
handleStreamTask now serves a mode=stream task FROM a resolved debrid HTTPS link
(when the web set preferredMethod=debrid + the torrent is cached) instead of
joining the P2P swarm — served over the SAME /stream endpoint so VLC and other
external players consume it identically (and far faster). No HLS transcode:
external players handle any container. Falls through to the P2P StreamEngine
when there is no direct URL. Uses the mutex-safe SetStreamURL setter.
Also widen the debrid HEAD size-probe timeout 10s -> 15s to match the transport's
TLS handshake budget, so a slow CDN no longer trips it and falls back to a
guessed size.
Bump 1.0.2-beta.
Pre-generate ONE trickplay sprite (montage JPEG of frames sampled every
library.trickplay.interval, default 10s) + a JSON manifest per file during the
scan/auto-scan prewarm, cached in .unarr next to the media. The web scrubber
shows tiles from it instead of extracting frames live — removing the ffmpeg
contention with the active stream that broke seekbar previews (the original
'no thumbnail' report was the auto-scan prewarm decoding the same file the HLS
transcode was reading, not a seek-index fault).
- config: [library.trickplay] enabled/interval/width (default on, 10s, 240px),
editable + a toggle; IntervalSeconds() with a 10s fallback.
- mediainfo: GenerateTrickplay (one ffmpeg fps=1/interval,scale,tile pass; idle
I/O priority; ceil() frame count so no black trailing tile; a 16.7M-px cap
coarsens the interval for long media so a single sprite stays decodable on
iOS/Safari) + sprite/manifest sidecar cache helpers.
- engine: /trickplay endpoint (manifest JSON, ?kind=sprite JPEG); the agent owns
the tile width so the web requests by path only; thumb:<sha256> token reused.
- prewarm: a trickplay job per item, gated; scan.go + daemon.go wire the config.
Tests: parseDims; synthetic 3x2 / exact-multiple / 1x1; real-file e2e smoke
(S02E08 → 143 tiles, 662KB sprite). Non-breaking: the existing 5-frame panel
prewarm + on-demand /thumbnail stay until the web migrates to the sprite.
The smoke check extracted the docker image version with `grep -oE 'v[0-9.]+'`,
which truncates "v1.0.1-beta" to "v1.0.1" and false-warns on a correct image.
Match the trailing suffix too.
Releases were shipping UNSIGNED: ship.sh never invoked sign-checksums, the
goreleaser pubkey ldflag defaulted to empty, and publish-cli-release.sh did not
upload a .sig — so the self-updater's signature check was silently skipped
(1.0.0-beta had no checksums.txt.sig). Make signing unconditional:
- internal/upgrade/signature.go: bake the canonical release public key as the
compiled-in default (public, safe to commit; removes the empty-env footgun).
- .goreleaser.yml: drop the pubkey ldflag (committed default is authoritative)
+ add a signs: block that runs scripts/sign-checksums over checksums.txt.
sign-checksums requires -key, so an unset RELEASE_SIGNING_KEY fails the build
instead of shipping unsigned.
- scripts/ship.sh: source RELEASE_SIGNING_KEY from ~/.config/unarr-release/signing.key
(or the env), die if absent, and assert checksums.txt.sig was produced.
Private key lives outside the repo (gitignored keyfile + operator's vault);
public key verified to match (priv[32:] == baked pubkey).
Fast input seek (-ss before -i) fails on files whose seek index is imprecise
or mildly corrupt: the demuxer lands mid-EBML element ("invalid as first byte
of an EBML number") and decodes no frame, so the web scrubber showed a broken
image (2026-06-03, anime MKVs: 15/15 prewarm thumbnails failed). When the fast
path yields no frame, retry once with output seek (-ss after -i, decode from
the start) + -err_detect ignore_err. Applied in both the on-demand handler
(buildThumbnailArgsAccurate) and the prewarm extractor (ExtractThumbnailJPEG).
Cost is paid only when the fast path fails, so healthy files keep the cheap path.
Regression test: TestBuildThumbnailArgsAccurate.
The web persists the chosen audioIndex globally, so a value from a
multi-track file can arrive for a file with fewer tracks. buildHLSFFmpegArgsAt
mapped `-map 0🅰️N?` verbatim; the optional `?` then matched nothing and the
HLS output had NO audio stream (video-only — 2026-06-03, Wistoria S02E08 had
one audio track but the session carried audioIndex=2). Clamp an out-of-range
index to the first track so audio is never silently dropped.
Regression test: TestBuildHLSFFmpegArgsAudioClamp.
The binary self-update hard-stops inside a container (internal/upgrade
refuses when /.dockerenv exists), so the web's in-app 'force update' button
is futile for Docker agents. Report whether we run in a container via a new
RunningInDocker() helper (UNARR_DOCKER env baked into the image, /.dockerenv
fallback for podman/containerd) on every register + sync, so the web can
swap the button for a copy-paste 'docker compose pull' command.
Stop treating the absolute path as a file's identity so a base-path change
(host binary→docker remap, moved media folder, remount) no longer makes the
server duplicate and orphan library rows.
- fingerprint.go: ComputeFingerprint = sha256(size ‖ first 1MiB ‖ last 1MiB),
a stable content identity that survives rename/move/base-path change. Cached
in LibraryItem and reused on incremental scans when size+mtime are unchanged.
- sync: send fingerprint + rel_path (relative to the scan root) + agent_id in
the library-sync request, so the server can move a row in place and scope
stale-cleanup per agent.
- daemon: force a FULL re-scan (with a user-facing WARNING) when the scan root
changed since the last cache, so the server re-maps by fingerprint instead of
duplicating. basePathChanged compares filepath.Clean'd roots.
- daemon: relocateUnreachable self-heals a stream request whose path is under an
old root but whose file still exists under a current allowed root, so playback
works immediately without waiting for the re-scan. Conservative: requires a
3-segment tail and re-checks containment after resolving symlinks so it can
neither serve the wrong file nor escape the allowed dirs.
See docs/plans/unarr-path-resilience.md in the web repo.
Second critico pass on the functional probe.
- The probe does real Vulkan device init, which can transiently fail when the
box is busy (notably the startup warm racing the encode benchmark). Caching
that timeout as a permanent 'no' would pin HDR to the zscale CPU chain until
daemon restart. Now a deadline is NOT cached — only a clean non-zero exit
(filter absent / no ICD), which is a stable result. zscale stays cached as
before (cheap deterministic grep, can't flake).
- Surface the exec error when ffmpeg never produced stderr (timeout / ENOENT):
the fallback log now shows err.Error() instead of a blank tail, so 'no
Vulkan' is distinguishable from 'ffmpeg never ran'.
- Dockerfile comment: clarify the Vulkan ICD (not GLX) is the load-bearing
mount that 'graphics' adds; 'compute' alone doesn't mount it.
Probe still returns true on a Vulkan host (verified); engine tests green.
Make libplacebo actually reachable in the shipped agent image, and refuse it
where it would be a regression.
Dockerfile (so a Vulkan-capable host can use the GPU tonemap path):
- install libvulkan1 (the Vulkan loader libplacebo links at runtime; ~150 KB)
- add 'graphics' to NVIDIA_DRIVER_CAPABILITIES so the nvidia container runtime
mounts the Vulkan ICD (nvidia_icd.json + GLX libs) under --gpus all
Both are inert without a working Vulkan GPU — the functional probe gates use.
hls.go: gate libplacebo on a real HW encoder (HWAccel != none). A software-only
host with mesa would expose lavapipe (CPU Vulkan); the functional probe accepts
it but its tonemap is SLOWER than the zscale CPU chain, so libplacebo there is a
regression. No HW encoder -> stay on zscale.
Verified on the GPU dev box: nvenc session still picks libplacebo (-c:v
h264_nvenc -vf ...,libplacebo=...:tonemapping=bt.2390); new unit test locks the
software-encoder path onto zscale.
Review (critico) caught a regression: the prod agent image ships a BtbN GPL
ffmpeg with libplacebo COMPILED IN but no Vulkan runtime (debian-slim, no
libvulkan1/mesa-vulkan-drivers/nvidia ICD). The presence probe (ffmpeg
-filters) would flip HasLibplacebo on, the filter's Vulkan device creation
would fail at runtime, and HDR sources that previously tonemapped via zscale
would break.
- FFmpegSupportsLibplacebo now RUNS the filter on one synthetic frame and
requires a clean exit (forces Vulkan device init + filtergraph negotiation),
so it is honest about THIS host: works on Vulkan-capable hosts, falls back to
zscale where Vulkan is absent. Logs the real ffmpeg error on failure.
- Warm the libplacebo (Vulkan init ~1.7s) + zscale caches in a background
goroutine at startup so the first stream session doesn't pay the probe and
risk its setup timeout.
- Benchmark: margin 1.5x -> 2.0x (the probe measures encode only; real decode
of HEVC/10-bit + busier content needs more headroom), per-probe timeout
12s -> 6s + overall 45s -> 20s (it blocks registration on software hosts),
and a 'no rung measured' case (missing lavfi/wedged ffmpeg) now keeps the
1080 default instead of flooring at 480 — an infra failure isn't a slow host.
Verified e2e on the fixed binary: LOTR Two Towers (HEVC 3840x1608 10-bit
HDR10/PQ, 12GB) on desktop-Chrome caps -> hls, ffmpeg runs h264_nvenc with
-vf ...,libplacebo=...:format=yuv420p:tonemapping=bt.2390 (zscale chain
replaced), 45 fMP4 segments, ffprobe confirms output h264 yuv420p bt709
(tonemapped from bt2020/smpte2084), no ffmpeg errors.
Replace the guessed transcode ceiling (CPU->1080, GPU->2160) with a measured
one. HW encoders still return 2160 instantly. A software-only host runs a
bounded encode benchmark — 3s testsrc2 through the real libx264 superfast
settings at 1080/720/480, top-down — and reports the rung it sustains at
>=1.5x realtime (margin for real decode + busier content).
Fixes risk 2: a weak NAS/old CPU that is ffmpeg-capable but can't keep up
with a 1080p software encode no longer advertises a 1080 ceiling, so
decideStreamPlan routes oversized sources to an external player instead of a
stuttering transcode. Floors at 480; each probe is timeout-bounded so a
wedged ffmpeg can't stall daemon startup.
Prefer the single-pass Vulkan libplacebo filter over the CPU zscale chain
for HDR->SDR tonemapping when the agent ffmpeg has it. One GPU pass does
tonemap + BT.709 primaries/transfer/matrix + 8-bit yuv420p, replacing the
four-stage zscale chain and its trailing format=/setparams. Higher quality,
far cheaper than the CPU path, and present on builds that lack zscale.
- FFmpegSupportsLibplacebo probe (cached, mirrors FFmpegSupportsZscale)
- HasLibplacebo on TranscodeRuntime, wired from buildTranscodeRuntime
- hls.go: videoTail picks libplacebo when present (not h264_vaapi), else
keeps the zscale tonemap + format chain
- test: libplacebo replaces the zscale chain, never runs alongside it
Slightly-VFR / B-frame MKV sources made ffmpeg's fMP4 muxer emit a continuous
"Packet duration is out of range" flood and produce uneven segment lengths the
web player stuttered on. Add, on the two main encoders + globally:
- libx264: -bf 0 -sc_threshold 0
- h264_nvenc: -bf 0 -no-scenecut 1
- -fps_mode cfr (force constant frame rate)
Keyframe cadence stays driven by -force_key_frames, so every segment is exactly
hls_time long. Verified: the warning flood drops from dozens/sec to ~1 per 80s
of transcoded content (cosmetic), segments stay valid fMP4.
The web player measured bandwidth against the web origin, which says nothing
about the path the video actually travels (LAN-direct, tailnet, or the CF
funnel) — on a fast LAN where the web server is the slow link it falsely
recommended a lower resolution. Serve a fixed-size, incompressible payload
from the agent so the web can measure the REAL stream path.
- GET /speedtest?size=N (clamped 64KB–4MB, default 2MB), HEAD supported
- CORS-gated like the other endpoints; no auth (carries no data)
- single-flight guard (atomic): one measurement at a time → a concurrent
request gets 429, bounding the bandwidth an unauthenticated caller can
drain over the public funnel
Stream-request failures previously reported status:"failed", which the web
treated as a download failure — it left the task unstreamable and surfaced a
misleading 20s timeout. Report them through a dedicated StreamError field
instead, so the web clears the stream flag and shows the real reason without
touching the download status.
- StatusUpdate gains StreamError (json: streamError)
- OnStreamRequested reports failures via a reportStreamError helper (path
rejected, file not found, no video in dir) instead of status:"failed"
- os.Stat is retried 3× (300ms) before giving up — NFS can transiently fail
(ESTALE/EAGAIN/timeout), the root of the intermittent "works on the 3rd try"
- dispatch OnStreamRequested off the sync loop (goroutine): it does blocking
I/O (stat retries, ffprobe in SetFile) that would otherwise stall task
dispatch + status reporting for other items
playlistHandler hardcoded #EXTVLCOPT:network-caching=30000, so VLC pre-buffered
~30 s before starting playback even on a fast, range-served LAN/Tailscale
source — the "VLC loads the whole movie before playing" regression.
Read the value from a networkCaching query param (clamped 500–60000ms) and
default to 3000 when absent. The web sends a network-aware value (small on
LAN/Tailscale, larger on the CF funnel); older web clients fall back to the
modest default instead of the old 30 s wall.
ffprobe already runs on every scanned file; now we capture its stderr and
assess integrity from it. assessIntegrity flags a file "damaged" on the
markers that mean the container/bitstream is unusable: invalid_data,
ebml_corrupt, moov_missing, bitstream_corrupt, plus no_duration (a video
stream with non-positive duration = a truncated/incomplete download).
The verdict rides on MediaInfo.Integrity (IntegrityInfo{Damaged,Reason}),
maps onto LibrarySyncItem.{Integrity,IntegrityReason}, and syncs to the web
so a damaged file can be surfaced at rest instead of only blowing up at
playback.
Bumps the scan cache version (1 → 2) so existing entries re-probe once, and
the scanner re-probes any cached entry that has no integrity verdict yet.
First beta of the 1.0 line — the full unarr streaming agent (HTTPS streaming,
HLS transcode, on-demand + cached subtitles/thumbnails, burn-in, debrid HLS,
SSE realtime, auto-resume, seeding).
- ExtractSubtitlesVTTMulti: distrust output when ffmpeg is killed by signal
(45-min timeout on a too-big remux) — a truncated WebVTT passed the len>0
check and got cached as a silently-incomplete track until the media mtime
changed. Skip all output on signal-kill; keep it on a clean non-zero exit.
- stream handlers: read the sidecar cache BEFORE the ffmpegPath guard so a
pre-warmed sub/thumbnail still serves if ffmpeg was removed after the cache
was filled.
- scan: log when the prewarm is skipped because ffmpeg is unavailable (matches
the daemon; CLAUDE.md wants bootstrap to log on every branch).
- unexport sidecarDir/subtitleCachePath/thumbnailCachePath (no external callers).
- prewarm: surface a sample error in the summary so a systemic ffmpeg failure
is distinguishable from one corrupt file.
- add unit tests: codec whitelist, cache paths, mtime freshness, atomic write,
thumb-position dedup.
The prewarm's single big read (a ~14 min sequential pass over a 60GB remux to
demux subtitles) shares the same disk/NFS as live streaming. Lower the prewarm
ffmpeg processes to the Linux IDLE I/O class (ioprio_set) so that background
read yields bandwidth to a user who's actually watching — the prewarm slows
down under contention instead of starving playback, and runs full speed when the
disk is idle.
Applied only to the prewarm-only extractors (ExtractSubtitlesVTTMulti,
ExtractThumbnailJPEG) via Start → setIdleIOPriority(pid) → Wait; the on-demand
/sub + /thumbnail handlers keep normal priority (a user is waiting on those).
Linux-only syscall behind a build tag; a no-op stub elsewhere. Best-effort —
errors ignored, never required for correctness.
Verified: the prewarm ffmpeg shows 'idle' under ionice -p; on-demand stays normal.
Subtitle extraction is I/O-bound: subtitle packets are interleaved across the
whole container, so ffmpeg must read the entire file to demux a complete track
(measured ~57 MB/s reading a 60GB remux over ~75 MB/s NFS → ~14 min for the
full read). Doing that once per track meant N full reads of a huge file.
ExtractSubtitlesVTTMulti demuxes the container ONCE and routes every text track
to its own WebVTT output, so an N-text-track file costs one read instead of N.
The prewarm now enqueues one job per file (all its text indices) and raises the
per-file deadline to 45 min so even ~200GB remuxes finish the single read in the
background (idempotent; the on-demand /sub keeps its 60s fallback). Thumbnails
are unaffected — a keyframe seek reads a tiny slice (~0.7s even on 60GB).
Pre-extract the file panel's sample frames (10/30/50/70/90% of runtime, w=320)
during the library scan and write-through any on-demand /thumbnail request into
the hidden ".unarr/<name>.t<sec>w<width>.jpg" sidecar. The /thumbnail handler
serves a fresh sidecar instantly, so the characteristics panel and seekbar
previews stop re-running ffmpeg per request.
- mediainfo.sidecar: ThumbnailCachePath, ReadCachedThumbnail, WriteCachedThumbnail,
ExtractThumbnailJPEG (mirrors engine.buildThumbnailArgs).
- library.PrewarmSidecars: also enqueues the panel frame positions (kept in
lockstep with the web's THUMB_FRACTIONS / THUMB_WIDTH) per item with a duration.
- thumbnailHandler: cache-read → hit; miss → extract → write-through.
- config: library.cache_thumbnails (default true) + both cache toggles exposed in
the interactive 'unarr config' library menu.
Local only by design — frames are the user's own content, never uploaded.
On-demand WebVTT extraction re-ran ffmpeg on every /sub request and, for
50GB+ remuxes, couldn't finish a full text track within the 60s HTTP timeout
→ the web player got a 500 and no subtitles.
Extract each text subtitle ONCE — during the library scan (no HTTP deadline,
generous per-file timeout) and write-through on the first on-demand request —
into a hidden ".unarr/<name>.s<index>.vtt" sidecar next to the media file.
The /sub handler serves a fresh sidecar instantly (mtime-invalidated when the
media is replaced), so playback subtitles are instant and huge files work.
- mediainfo.sidecar: cache paths, mtime freshness, atomic write, ExtractSubtitleVTT,
IsTextSubtitleCodec (shared classifier, mirrors engine + web whitelists).
- library.PrewarmSidecars: bounded, idempotent, ctx-cancellable background pass
run after every scan (manual + daemon auto-scan).
- subtitleHandler: cache-read → hit; miss → extract → write-through.
- config: library.cache_subtitles (default true), wired via SetCacheSubtitles.
Local-only by design: nothing extracted is uploaded — the sidecar is the user's
own content, private to their disk.
Add GET /sub?p=&i=&t= that extracts an embedded text subtitle stream to
WebVTT via ffmpeg (-map 0:s:N -c:s webvtt), token-gated with a per-track
sub:<sha256(path)>:<index> scope. The web player attaches these as
external <track>s for both direct-play and HLS, native and hls.js.
Removes the old per-session extraction path (extractSubtitles,
ServeSubtitle, manifest SUBTITLES renditions, subs/ mkdir, Close() wait):
native HLS playback never surfaced manifest subs, so that work was wasted.
The on-demand /sub endpoint is now the single subtitle source.
Alpine/musl can't run NVIDIA's glibc userspace (nvidia-smi, libnvidia-encode,
the static nvenc ffmpeg), so HW transcode was impossible — every 4K/anamorphic
HLS encode fell back to software or failed. Switch the runtime stage to
debian:bookworm-slim + a static BtbN ffmpeg built with nvenc, add par2
(Usenet segment repair) + 7z (RAR/7z extraction), and set
NVIDIA_DRIVER_CAPABILITIES=video,compute,utility so a plain --gpus all (or the
compose device reservation) lights up nvenc with no extra flags. Falls back to
libx264 automatically when no GPU is attached. Build stage cross-compiles
(--platform=BUILDPLATFORM) so multi-arch stays fast; downloads forced over IPv4.
Anamorphic 2.39:1 scaled to 1080 height = ~2586x1080 = 11016 MBs, busting
level 4.1's 8192-MB MaxFS -> nvenc "InitializeEncoder failed: Invalid Level"
(libx264: "frame MB size > level limit") -> 0 segments, session stalls. Most
4K rips are 2.39:1, so HLS playback was silently broken for them.
H264LevelForFrame(w,h) derives the level from the real macroblock count
(max of MB-tier and height-tier). hls.go computes output width and uses it.
16:9 unchanged; anamorphic bumps to 5.0 when needed. Discovered + verified
during the trickplay smoke.
Long-poll→WS/SSE closed: the 3-leg realtime work (SSE downlink with
long-poll fallback, event-driven uplink on every state transition, and
server→browser push via a Redis signal bus) shipped in 0.14.0. The
"Tailscale Funnel mal nombrado" note was already stale — the code names it
"CloudFlare Quick Tunnel" everywhere; nothing to rename. Also struck the
duplicate HTTP-clients entry (already in Cerrada).
The agent reported its state only on the adaptive sync tick (3s watching /
10s idle), so a resolving→downloading→verifying→organizing→completed
transition could lag up to a full interval before the server (and the web
UI) saw it. Now every successful Task.Transition fires an onChange hook
wired to TriggerSync, pushing the new state immediately. Bursts are safe:
TriggerSync is a buffered-1 send, so clustered transitions coalesce into
one sync.
- Task gains an onChange hook fired AFTER the status mutex is released
(so a future heavier hook can't deadlock on task.mu); nil is a no-op.
- Manager.OnStateChange is set on each task at Submit; the daemon wires it
to TriggerSync alongside the existing OnTaskDone.
- Stream tasks transition outside the Manager, so handleStreamTask wires
the same hook explicitly (gap found in review) — resolving/downloading/
completed/failed on the stream path now push too.
The adaptive ticker stays as a reconciliation heartbeat; it's just no
longer the latency floor for state changes.
Replace the bare long-poll wake listener with a hybrid server→agent
downlink that consumes the new GET /api/internal/agent/events SSE stream
first and falls back to the long-poll wake when SSE is unavailable or
silently buffered. Resurrects the SSE client retired with WebRTC
(signal_client.go) as events_client.go — a bounded-scanner reader
(256 KiB line / 1 MiB event) that surfaces heartbeat comments as ping
events so the consumer can detect liveness.
runDownlink dispatches on the new [daemon] downlink config:
- auto (default): SSE-first; after maxSSEFailures dead/buffered attempts
fall back to long-poll for 5 min, then re-probe SSE.
- sse: SSE only, no fallback (known-good networks / testing).
- poll: the pre-0.14 long-poll wake only.
A stream is "healthy" only if it delivers a frame within livenessTimeout
(40s vs the server's 15s heartbeat). Crucially the liveness-timeout branch
returns UNHEALTHY even if an earlier frame arrived: a proxy that flushes
the connect preamble (one ping) then stalls must not pin the agent to SSE
forever — that's the partial-buffering case the fallback exists for.
event: command applies typed controls via the same OnControl callback
/agent/sync uses (idempotent); event: sync triggers an immediate sync;
ping is liveness-only. OpenEventStream rides MirrorPool failover for the
initial connect; mid-stream drops close the channel and the loop reopens.
Bump 0.14.0.
