Review (critico) caught a regression: the prod agent image ships a BtbN GPL
ffmpeg with libplacebo COMPILED IN but no Vulkan runtime (debian-slim, no
libvulkan1/mesa-vulkan-drivers/nvidia ICD). The presence probe (ffmpeg
-filters) would flip HasLibplacebo on, the filter's Vulkan device creation
would fail at runtime, and HDR sources that previously tonemapped via zscale
would break.
- FFmpegSupportsLibplacebo now RUNS the filter on one synthetic frame and
requires a clean exit (forces Vulkan device init + filtergraph negotiation),
so it is honest about THIS host: works on Vulkan-capable hosts, falls back to
zscale where Vulkan is absent. Logs the real ffmpeg error on failure.
- Warm the libplacebo (Vulkan init ~1.7s) + zscale caches in a background
goroutine at startup so the first stream session doesn't pay the probe and
risk its setup timeout.
- Benchmark: margin 1.5x -> 2.0x (the probe measures encode only; real decode
of HEVC/10-bit + busier content needs more headroom), per-probe timeout
12s -> 6s + overall 45s -> 20s (it blocks registration on software hosts),
and a 'no rung measured' case (missing lavfi/wedged ffmpeg) now keeps the
1080 default instead of flooring at 480 — an infra failure isn't a slow host.
Verified e2e on the fixed binary: LOTR Two Towers (HEVC 3840x1608 10-bit
HDR10/PQ, 12GB) on desktop-Chrome caps -> hls, ffmpeg runs h264_nvenc with
-vf ...,libplacebo=...:format=yuv420p:tonemapping=bt.2390 (zscale chain
replaced), 45 fMP4 segments, ffprobe confirms output h264 yuv420p bt709
(tonemapped from bt2020/smpte2084), no ffmpeg errors.
Replace the guessed transcode ceiling (CPU->1080, GPU->2160) with a measured
one. HW encoders still return 2160 instantly. A software-only host runs a
bounded encode benchmark — 3s testsrc2 through the real libx264 superfast
settings at 1080/720/480, top-down — and reports the rung it sustains at
>=1.5x realtime (margin for real decode + busier content).
Fixes risk 2: a weak NAS/old CPU that is ffmpeg-capable but can't keep up
with a 1080p software encode no longer advertises a 1080 ceiling, so
decideStreamPlan routes oversized sources to an external player instead of a
stuttering transcode. Floors at 480; each probe is timeout-bounded so a
wedged ffmpeg can't stall daemon startup.
Prefer the single-pass Vulkan libplacebo filter over the CPU zscale chain
for HDR->SDR tonemapping when the agent ffmpeg has it. One GPU pass does
tonemap + BT.709 primaries/transfer/matrix + 8-bit yuv420p, replacing the
four-stage zscale chain and its trailing format=/setparams. Higher quality,
far cheaper than the CPU path, and present on builds that lack zscale.
- FFmpegSupportsLibplacebo probe (cached, mirrors FFmpegSupportsZscale)
- HasLibplacebo on TranscodeRuntime, wired from buildTranscodeRuntime
- hls.go: videoTail picks libplacebo when present (not h264_vaapi), else
keeps the zscale tonemap + format chain
- test: libplacebo replaces the zscale chain, never runs alongside it
Slightly-VFR / B-frame MKV sources made ffmpeg's fMP4 muxer emit a continuous
"Packet duration is out of range" flood and produce uneven segment lengths the
web player stuttered on. Add, on the two main encoders + globally:
- libx264: -bf 0 -sc_threshold 0
- h264_nvenc: -bf 0 -no-scenecut 1
- -fps_mode cfr (force constant frame rate)
Keyframe cadence stays driven by -force_key_frames, so every segment is exactly
hls_time long. Verified: the warning flood drops from dozens/sec to ~1 per 80s
of transcoded content (cosmetic), segments stay valid fMP4.
The web player measured bandwidth against the web origin, which says nothing
about the path the video actually travels (LAN-direct, tailnet, or the CF
funnel) — on a fast LAN where the web server is the slow link it falsely
recommended a lower resolution. Serve a fixed-size, incompressible payload
from the agent so the web can measure the REAL stream path.
- GET /speedtest?size=N (clamped 64KB–4MB, default 2MB), HEAD supported
- CORS-gated like the other endpoints; no auth (carries no data)
- single-flight guard (atomic): one measurement at a time → a concurrent
request gets 429, bounding the bandwidth an unauthenticated caller can
drain over the public funnel
playlistHandler hardcoded #EXTVLCOPT:network-caching=30000, so VLC pre-buffered
~30 s before starting playback even on a fast, range-served LAN/Tailscale
source — the "VLC loads the whole movie before playing" regression.
Read the value from a networkCaching query param (clamped 500–60000ms) and
default to 3000 when absent. The web sends a network-aware value (small on
LAN/Tailscale, larger on the CF funnel); older web clients fall back to the
modest default instead of the old 30 s wall.
- ExtractSubtitlesVTTMulti: distrust output when ffmpeg is killed by signal
(45-min timeout on a too-big remux) — a truncated WebVTT passed the len>0
check and got cached as a silently-incomplete track until the media mtime
changed. Skip all output on signal-kill; keep it on a clean non-zero exit.
- stream handlers: read the sidecar cache BEFORE the ffmpegPath guard so a
pre-warmed sub/thumbnail still serves if ffmpeg was removed after the cache
was filled.
- scan: log when the prewarm is skipped because ffmpeg is unavailable (matches
the daemon; CLAUDE.md wants bootstrap to log on every branch).
- unexport sidecarDir/subtitleCachePath/thumbnailCachePath (no external callers).
- prewarm: surface a sample error in the summary so a systemic ffmpeg failure
is distinguishable from one corrupt file.
- add unit tests: codec whitelist, cache paths, mtime freshness, atomic write,
thumb-position dedup.
Pre-extract the file panel's sample frames (10/30/50/70/90% of runtime, w=320)
during the library scan and write-through any on-demand /thumbnail request into
the hidden ".unarr/<name>.t<sec>w<width>.jpg" sidecar. The /thumbnail handler
serves a fresh sidecar instantly, so the characteristics panel and seekbar
previews stop re-running ffmpeg per request.
- mediainfo.sidecar: ThumbnailCachePath, ReadCachedThumbnail, WriteCachedThumbnail,
ExtractThumbnailJPEG (mirrors engine.buildThumbnailArgs).
- library.PrewarmSidecars: also enqueues the panel frame positions (kept in
lockstep with the web's THUMB_FRACTIONS / THUMB_WIDTH) per item with a duration.
- thumbnailHandler: cache-read → hit; miss → extract → write-through.
- config: library.cache_thumbnails (default true) + both cache toggles exposed in
the interactive 'unarr config' library menu.
Local only by design — frames are the user's own content, never uploaded.
On-demand WebVTT extraction re-ran ffmpeg on every /sub request and, for
50GB+ remuxes, couldn't finish a full text track within the 60s HTTP timeout
→ the web player got a 500 and no subtitles.
Extract each text subtitle ONCE — during the library scan (no HTTP deadline,
generous per-file timeout) and write-through on the first on-demand request —
into a hidden ".unarr/<name>.s<index>.vtt" sidecar next to the media file.
The /sub handler serves a fresh sidecar instantly (mtime-invalidated when the
media is replaced), so playback subtitles are instant and huge files work.
- mediainfo.sidecar: cache paths, mtime freshness, atomic write, ExtractSubtitleVTT,
IsTextSubtitleCodec (shared classifier, mirrors engine + web whitelists).
- library.PrewarmSidecars: bounded, idempotent, ctx-cancellable background pass
run after every scan (manual + daemon auto-scan).
- subtitleHandler: cache-read → hit; miss → extract → write-through.
- config: library.cache_subtitles (default true), wired via SetCacheSubtitles.
Local-only by design: nothing extracted is uploaded — the sidecar is the user's
own content, private to their disk.
Add GET /sub?p=&i=&t= that extracts an embedded text subtitle stream to
WebVTT via ffmpeg (-map 0:s:N -c:s webvtt), token-gated with a per-track
sub:<sha256(path)>:<index> scope. The web player attaches these as
external <track>s for both direct-play and HLS, native and hls.js.
Removes the old per-session extraction path (extractSubtitles,
ServeSubtitle, manifest SUBTITLES renditions, subs/ mkdir, Close() wait):
native HLS playback never surfaced manifest subs, so that work was wasted.
The on-demand /sub endpoint is now the single subtitle source.
The agent reported its state only on the adaptive sync tick (3s watching /
10s idle), so a resolving→downloading→verifying→organizing→completed
transition could lag up to a full interval before the server (and the web
UI) saw it. Now every successful Task.Transition fires an onChange hook
wired to TriggerSync, pushing the new state immediately. Bursts are safe:
TriggerSync is a buffered-1 send, so clustered transitions coalesce into
one sync.
- Task gains an onChange hook fired AFTER the status mutex is released
(so a future heavier hook can't deadlock on task.mu); nil is a no-op.
- Manager.OnStateChange is set on each task at Submit; the daemon wires it
to TriggerSync alongside the existing OnTaskDone.
- Stream tasks transition outside the Manager, so handleStreamTask wires
the same hook explicitly (gap found in review) — resolving/downloading/
completed/failed on the stream path now push too.
The adaptive ticker stays as a reconciliation heartbeat; it's just no
longer the latency floor for state changes.
- usenet: Par2Verify/Repair return ErrPar2NotInstalled (was nil="verified");
pipeline surfaces it via Result.VerifyNote + WARNING — a download that
shipped parity but couldn't be checked is delivered UNVERIFIED, not verified.
- funnel: pin cloudflared version + verify a baked-in SHA-256 (was `latest` +
ELF-magic only) — a malicious/broken upstream release isn't pulled silently.
- stream: makeReadable verifies the file actually opens after chmod and warns
clearly (NFS root_squash / SMB uid mapping) instead of a cryptic later EPERM.
- WireGuard endpoint pin dropped from the debt list (reseller uses direct
config, no pin).
Foundation for direct, valid-cert browser playback (agent-TLS feature) — the
cert broker + DNS are a later phase; this is inert until a certificate exists.
- StreamServer runs a second TLS listener on https_stream_port (default 11819)
serving the SAME mux as HTTP (11818): same token + CORS gates, no new exposure.
- Certificate is read per-handshake from an atomic holder via tls.Config
GetCertificate, so a cert issued/renewed asynchronously applies without a
restart. SetTLSCertificate / LoadTLSCertificateFromFiles / HasTLSCertificate.
- Daemon arms HTTPS only when a cert pair exists at certs/agent.{crt,key} under
the state dir; without it, no HTTPS port is opened and HTTP + funnel are
unaffected. Shutdown drains the HTTPS server too.
- config: downloads.https_stream_port (default 11819, 0 = disabled).
Tests: real TLS handshake + hot-install (no-cert handshake fails, install →
200), disabled path, missing-cert load error.
SeedRatio/SeedTime were declared on TorrentConfig but never consumed, and
SeedEnabled was hardcoded false in both constructors — the daemon never
seeded, and if forced it seeded forever.
- config: [downloads] seed_enabled/seed_ratio/seed_time (opt-in, off by default)
- daemon: parse seed_time + wire all three; startup log per target shape
- engine: seedTargetReached() (pure) + seedAndDrop() background monitor on a
downloader-scoped seedCtx (not the task ctx, which dies when Download returns);
drops the torrent on ratio (uploaded/size) OR time, whichever first; no target
= seed until shutdown. Configurable check interval (tests lower it).
- fix: cleanup() now always drops — previously leaked the handle on error paths
when seeding was enabled.
- refactor: dropTracked() helper shared by cleanup + post-seeding drop.
Tests: TestSeedTargetReached (9 cases) + ctx/no-target branches + loopback
swarm smoke (-tags smoke). Roadmap hueco closed.
Bitmap subs can't be served as WebVTT, so the user picks one and the daemon
re-encodes with it overlaid. HLSSessionConfig.BurnSubtitleIndex (*int, nil=no
burn) flows into the cache key + a -filter_complex graph:
[0✌️0]<vchain>[base];[0:s:N][base]scale2ref[sub][base2];[base2][sub]overlay[vout]
Overlay after the tonemap (SDR subs keep brightness); scale2ref fits the PGS
canvas to the output. Invalid/text/out-of-range index -> clean-encode fallback.
IsTextSubtitle now includes "text" (parity with the web classifier).
Anamorphic 2.39:1 scaled to 1080 height = ~2586x1080 = 11016 MBs, busting
level 4.1's 8192-MB MaxFS -> nvenc "InitializeEncoder failed: Invalid Level"
(libx264: "frame MB size > level limit") -> 0 segments, session stalls. Most
4K rips are 2.39:1, so HLS playback was silently broken for them.
H264LevelForFrame(w,h) derives the level from the real macroblock count
(max of MB-tier and height-tier). hls.go computes output width and uses it.
16:9 unchanged; anamorphic bumps to 5.0 when needed. Discovered + verified
during the trickplay smoke.
The torrent reader used a static 5 MiB readahead — about 1.9s of a 20 Mbps 4K
stream — so streaming a torrent while it downloaded outran the download and
stalled. anacrolix's reader already prioritises the pieces in the readahead
window ahead of the playhead (and re-prioritises on seek); the window was just
too small. dynamicReadahead sizes it to ~30s of video (clamped 8-96 MiB, 24 MiB
default when bitrate is unknown). The torrent provider probes the bitrate
asynchronously so stream start never blocks on ffprobe; readers created after
the probe resolves pick up the accurate size. Real 4K (20.7 Mbps) -> 73 MiB.
HDR (HDR10/HLG/Dolby Vision) transcoded to SDR came out washed-out and
desaturated because the filter chain never tonemapped. buildHLSFFmpegArgsAt now
inserts a zscale linearise -> hable tonemap -> BT.709 chain after the scale and
before format=, but only when the source is HDR and the ffmpeg build has zscale
(FFmpegSupportsZscale, cached). Builds without zimg keep the old behaviour
(plays, just desaturated) instead of erroring.
It's a CPU filter, valid for every encoder here: the decode hwaccel deliberately
leaves frames in system memory (no -hwaccel_output_format), so zscale runs ahead
of format=/hwupload exactly like the existing scale filter. Verified on a real
4K HDR10 file — vivid colour and deep blacks vs the washed-out baseline.
A daemon restart used to abandon in-flight downloads: the in-memory queue was
lost and the web doesn't re-dispatch a stuck task, so the user had to retry
manually. The bytes already persisted (mmap + anacrolix's piece-completion DB
keyed by info_hash; debrid via Range; usenet via its tracker) — the daemon just
didn't re-attempt the work.
ActiveTaskStore persists each in-flight download's agent.Task payload to
active-tasks.json; the daemon re-submits them on startup so the downloaders
resume the partial data. manager.Submit now dedups (the startup re-submit and a
later web re-dispatch can't both run), and recordFinished removes a task from
the store only on a genuine terminal — shuttingDown (set before Shutdown cancels
the task contexts) keeps shutdown-interrupted tasks so they resume next start.
Stream/seed/upgrade tasks aren't persisted; ForceStart is cleared on resume.
CheckDiskSpace (internal/engine/diskspace.go) refuses a download before
writing when its expected size wouldn't leave a configurable reserve free,
so a download never fills the filesystem to 0 mid-write (which corrupts the
partial file). Wired into all three downloaders ahead of any write — torrent
(DataDir), debrid (outputDir, resume-aware), usenet (outputDir, fresh only).
Reserve from downloads.min_free_disk_mb (default 2048 MiB) via SetMinFreeBytes.
The manager treats an InsufficientDiskError as terminal — no source fallback,
since another source would fill the same disk — and surfaces the clear message.
Best-effort: unknown size or a stat failure doesn't block (ENOSPC stays the
backstop). Also hardens formatBytes against an exabyte-scale out-of-bounds panic.
Add GET /thumbnail to the agent stream server: ffmpeg extracts one frame
at a timestamp (-ss before -i, single-frame MJPEG to stdout) for the web's
file-characteristics panel. Auth via a token scoped thumb:<sha256(path)>
(same HMAC scheme as /stream and /hls; the web mints, the agent verifies),
clamped to a real regular file, 404-no-oracle on a bad token, 20s timeout.
ffmpeg path wired into the stream server from the daemon. Version -> 0.13.0.
Debrid direct links are time-limited; a long playback can outlive the link
the session was created with. When a debrid source dies mid-stream the daemon
now re-resolves a fresh link for the same content and resumes — no torrent
fallback, no playback restart.
- debridFileProvider holds the URL behind a mutex; on an expired-link status
(401/403/404/410) the ranged reader re-resolves via a refresh callback and
retries (bounded: 1 initial + 1 post-refresh attempt). A browser opens
several range connections, so the refresh is coalesced singleflight-style —
N readers hitting the dead link share ONE re-resolution, not N.
- HLS-from-URL: the auto-restart supervisor re-resolves the link before
relaunching ffmpeg (else it just retries the dead URL and burns the retry
budget). The mutable URL lives in s.liveURL under s.mu — restartFromSegment
reads it from the HTTP handler goroutine too (seek-restart), so cfg stays
immutable and the write races nothing.
- agentClient.RefreshStreamURL → POST /api/internal/agent/stream-url.
Cross-source torrent<->debrid swap (the rare "debrid genuinely gone" case) is
intentionally deferred. Reader refresh + coalescing covered by unit tests
(incl. -race); the web endpoint re-resolves against a real AllDebrid account.
Non-browser-native debrid content (mkv/HEVC/…) can now stream: ffmpeg reads
the debrid HTTPS link directly (-i <url>) and transcodes to HLS, instead of
2a's raw direct-play which only works for mp4/m4v.
- HLSSessionConfig gains SourceURL + CacheID; sourceRef() feeds ffprobe,
ffmpeg -i, and subtitle extraction from one place. HTTP-resilience flags
(-reconnect*, -rw_timeout) are added only for a URL source; a seek-restart
re-opens the URL with a Range request (-ss before -i = input seek).
- Segment cache keys by CacheID (the torrent info_hash) for URL sessions so
re-plays hit cache despite the debrid URL changing each resolution
(KeyForID, no filepath.Abs).
- OnStreamSession: the 2a direct-play branch is now gated on PlayMethod != "hls";
a new branch handles DirectURL + PlayMethod=="hls" → HLS-from-URL. The
local-file and both debrid HLS paths share a startHLSPlayback helper.
- ExtractMediaInfo no longer masks a URL probe failure as "file not found"
(surfaces ffprobe's real stderr, e.g. "Protocol not found" on a TLS-less
ffmpeg build).
- Bump 0.11.0 -> 0.12.0 as the HLS-from-URL floor the web gates on.
Validated e2e against real AllDebrid: a cached HEVC x265 mkv transcodes
(h264_nvenc) from the debrid URL and plays 1080p in Chrome via hls.js,
subtitles extracted from the remote mkv.
The daemon can now stream a session straight from a server-resolved debrid
direct URL instead of disk/torrent, delivering the "play instantáneo
cache-fast" promise for cache-confirmed torrents the user never downloaded.
- debridFileProvider: an io.ReadSeekCloser over HTTP Range — network-free
Seek, lazy GET on Read, reopen-on-seek, a HEAD up front for the size, and
a URL-derived name so the served Content-Type is video/mp4 (not
octet-stream) when the web's name lacks an extension.
- OnStreamSession branches on StreamSession.DirectURL before the filePath
checks (no local path, no ffmpeg), builds the provider in a goroutine
(HEAD off the sync loop) and marks the session ready.
- Bump 0.10.0 -> 0.11.0 as the debrid-stream floor the web gates on.
Validated e2e against a real AllDebrid account: a cached mp4 plays 1080p in
Chrome through the agent, including the high-offset seek for a non-faststart
file's moov atom. 2b (HLS-from-URL for mkv/HEVC) + 2c (cache-fast preference
+ mid-stream fallback) remain.
The daemon's baked-in CORS allowlist had the torrentclaw.com family but not
unarr.app — so on the unarr brand the browser dropped every /hls + /stream
response (no Access-Control-Allow-Origin) and the player reported "can't
connect to your agent" even though the agent was reachable. Add unarr.app +
www.unarr.app. (Dev over Tailscale uses cors_extra_origins for the raw IP
origin.) Found while testing the web player from an iPhone over Tailscale.
Hueco #3 / 3c (CLI). NewRemuxSource now copies the video for any
browser-decodable codec: h264, or HEVC/AV1 when the web says the device
decodes them (caps). HEVC is muxed with -tag:v hvc1 (Apple requirement),
and non-aac audio (ac3/eac3/dts) is transcoded to aac while the video is
still copied (ActionRemuxAudio) — this covers the very common h264+ac3 mkv.
Startup instrumentation for time-to-first-frame diagnosis:
- remux branch logs [probe=.. spawn=..]
- transcodeSource logs 'first fMP4 bytes after ..' (ffmpeg → first output)
- serveGrowing logs reads that block >250ms (client seeking ahead of the
live edge) + the first read's offset vs produced/estimated size.
Verified: caps gate (hls without caps, remux with), hvc1 retag (ffprobe of
the /stream output = hevc/hvc1), HEVC playback confirmed on a real iPhone
Safari over Tailscale. LAN timeline: probe 16ms, spawn 1ms, first byte
201ms, no serveGrowing blocks.
Agent side of 3b: serve a growing ffmpeg `-c copy` remux (mkv h264/aac →
fragmented MP4) over /stream with no video re-encode. Dormant until the web
sends PlayMethod="remux" (3b-ii), so this commit changes no live behavior.
- GrowingSource interface + transcodeSource already satisfies it; estimate is
the source file size for copy actions (≈ remux output) vs bitrate×duration
for real transcodes.
- NewRemuxSource: ffmpeg -c copy → growing fMP4 temp, returned as GrowingSource.
- StreamServer.SetGrowingFile + serveGrowing: manual Range responder for a
growing source (http.ServeContent needs a fixed size). 206 with an estimated
total in Content-Range; chunked body while not final (never promise bytes a
running remux might not produce); exact Content-Length once final. Blocks via
ReadAt for not-yet-produced bytes; forward seek waits, backward seek instant.
- daemon OnStreamSession: PlayMethod=="remux" → NewRemuxSource + SetGrowingFile
+ MarkSessionReady (after the ffmpeg check; copy still needs ffmpeg).
- Tests: parseByteRange + serveGrowing (full/offset/bounded/estimate/HEAD/416).
/stream and /hls were served with no auth (only CORS + rate limit), so a
funnel- or UPnP-exposed daemon leaked active downloads to anyone with the URL.
Bind a short-lived HMAC token (scope + 6h expiry) to every stream URL the
daemon hands out and verify it on each request:
- /stream + VLC playlist: ?t= query, agent-minted, scope "stream"
- /hls: path segment /hls/<session>/<token>/<resource>, web-minted with the
agent's reported secret, scope "hls:<session>" — relative playlist URIs
inherit it with no rewriting
- NO loopback exemption: cloudflared relays public funnel traffic over
localhost, so a loopback source address is not a trust signal
- the agent reports its per-run signing key on register only when enforcing
- require_stream_token config (default true); secret fails hard if rand fails
- /playlist.m3u no longer self-mints a token (was an open token oracle)
Roadmap: Docs/plans/unarr-agent-roadmap.md (hueco #1).
Deploy the web HLS-minting change BEFORE shipping this agent release.
funnel: urlPattern matched api.trycloudflare.com before the real quick-tunnel
URL. Cloudflared logs the control-plane endpoint early, so the agent was
advertising a dead URL. Tighten regex to require at least one hyphen — quick
tunnels are always multi-word (e.g. make-appointments-negotiation-blacks).
Covers with funnel_test.go regression test.
download(oneshot): progress reporter called /api/internal/agent/status with a
synthetic "oneshot-<hash>" task ID that is not a UUID, causing the server to
return 400 every 5 s for the entire download. Pass nil client to
NewProgressReporter for one-shot mode; flush/ReportFinal are no-ops when
reporter == nil so terminal output continues unchanged.
torrent: piece-completion SQLite DB (anacrolix) was created inside the download
dir (DataDir). On NFS/SMB mounts SQLite file locking times out, emitting a
warning and falling back to an ephemeral in-memory DB. Add PieceCompletionDir
to TorrentConfig; the daemon now passes config.DataDir() (agent state dir,
always local) so the DB stays off the network mount. One-shot download leaves
the field empty → harmless in-memory fallback as before.
anacrolix mmap storage (storage.NewMMap) creates completed files with
mode 0000. The download succeeds because the agent keeps its own mmap
handle, but any fresh open — direct streaming (/stream :11818), HLS
ffprobe (:11819), or organize-then-reopen — fails with "permission
denied", surfaced in the web UI as "file not found". Both VLC and the
web player were affected.
makeReadable() relaxes the completed file to 0644 (dirs 0755, recursive
for multi-file torrents) right after download finishes, before organize
moves it, so the readable mode survives the rename.
With `-tune ll` NVENC emits long IDR-less GOPs that ignore
`-force_key_frames`, so ffmpeg's HLS muxer keeps writing into seg-0.m4s
forever instead of closing it at the 2 s boundary. Result:
* seg-0.m4s balloons to the full encoded size (1.2 GB on a 48-min movie)
* seg-1.m4s never appears
* daemon's pollSegments needs seg-N+1 to confirm seg-N is closed → never
advances → `mark-ready: timeout` after 60 s
* web player sits on "preparando sesión" until the user gives up
Verified on ffmpeg 6.1.1 + driver 580 / Ryzen 7 7700X + RTX-class GPU:
without `-tune ll`, the same `-preset p3 -rc vbr` cmd produces 39
discrete segments in 15 s at ~27x real-time (was 1 segment / 9 min of
material with `-tune ll` — encoder kept going on a single output).
Introduced by `3b8d77b feat(hls): faster first-start — probe cache +
tighter encoder presets (0.9.9)`. Dropping `-tune ll` costs ~0.5 dB
PSNR at the same bitrate but restores playback. NVENC first-segment
latency remains under 2 s — well within the player's startup budget.
Adds TestBuildHLSFFmpegArgsVAAPIDump alongside the existing assertion
tests. Logs the complete argv buildHLSFFmpegArgsAt emits for a
typical VAAPI session so an operator can paste it into a shell and
reproduce the encode without booting the dev stack — same effect as
`journalctl --user -u unarr-dev | grep ffmpeg`, no daemon needed.
Verified locally against AMD Raphael iGPU on this dev box: the
dumped argv encoded a 5 s 4K source → 720p in 3.1 s wall, produced
3 HLS segments + init.mp4 that decode cleanly under ffprobe.
Closes QW2. Validated against the dev box's AMD Raphael iGPU
(/dev/dri/renderD128, radeonsi/mesa 25.2.8). The "proper" full-GPU
path via scale_vaapi triggers a known mesa 25 + Raphael bug
("Cannot allocate memory" per session start, encode still succeeds
but logs are spammy) — hybrid CPU scale → format=nv12 → hwupload
→ h264_vaapi encode delivers GPU surfaces to the encoder without
poking the broken scaler.
Three concrete changes in buildHLSFFmpegArgsAt:
1. New `case "h264_vaapi"` adds `-vaapi_device /dev/dri/renderD128`.
Multi-GPU hosts (this dev box has NVIDIA on renderD129 + AMD on
renderD128) need it so the encoder doesn't bind to a non-VAAPI
render node — without it the encoder fell back to NULL device
in manual smoke testing.
2. Filter chain branches on codec: VAAPI uses
`scale=…,format=nv12,hwupload` while libx264 / NVENC / QSV
keep the existing `scale=…,format=yuv420p,setparams=…` shape.
The setparams color metadata block is dropped on VAAPI because
VAAPI surfaces don't expose VUI fields and the encoder writes
its own.
3. Two new unit tests lock the argv shape so a future refactor
doesn't accidentally merge the paths back together:
TestBuildHLSFFmpegArgsVAAPI asserts the new flags + the
ABSENCE of scale_vaapi; TestBuildHLSFFmpegArgsLibx264NoRegression
verifies the software path keeps yuv420p + setparams + has
none of the VAAPI extras.
Manual ffmpeg validation on the dev box:
hybrid encode of 5 s 4K → 720p: 0.66 s wall, 472 % CPU, 268 KB
output — no errors logged. scale_vaapi variant in comparison
spammed "Cannot allocate memory" while emitting valid output.
Closes the deferred bajo-priority item from the fase 3.3b critico.
Without this the watcher kept polling a torn-down HLSSession for up
to 60 s — fine in current code paths (Close always pairs with ctx
cancel which makes the select{} branch fire), but the function's
correctness then leaned on a caller invariant rather than its own
state check. Adding IsClosed() as a public wrapper around the
existing isClosed() lets the watcher detect any future
session-shutdown path (registry replace, idle sweep, internal kill)
without touching the unexported helper.
Closes Fase 3.3b. Daemon now tells the server the moment a session's
first HLS segment + init.mp4 land on disk; the web side flips
streaming_session.ready_at = NOW(), which its SSE endpoint pushes to
subscribed players so the loading UI flips from "Preparando…" to
"Stream listo" without polling HEAD on the segment URL.
Surface:
- New Client.MarkSessionReady(ctx, sessionId) HTTP method →
POST /api/internal/agent/session-ready.
- New engine.HLSSession.ReadyCount() + FromCache() accessors so the
watcher goroutine doesn't reach into private state.
- New cmd.watchSessionReady(ctx, client, hsess, sessionId) goroutine
polls ReadyCount every 200 ms with a 60 s deadline + short-circuits
for cache-HIT sessions (ready the moment StartHLSSession returns).
- Daemon callback spawns it right after streamSrv.HLS().Register so
the watcher's lifecycle matches the session's.
Best-effort: a transient network failure on the webhook is logged + the
goroutine exits — the player's existing HEAD-probe retry path still
discovers ready state independently. The webhook is an acceleration,
not a hard dependency.
First-frame latency drops by another 1-2 s on cold-cache plays:
1. HLS segment duration halved from 4 s to 2 s. seg-0 lands in ~half
the wait time — the player paints the first frame as soon as it
arrives. Software encodes on 4K go from ~3 s wait to ~1.5 s; HW
encoders shave ~0.5 s. Trade-off: 2× segment count per source
(~3600 segments for a 2 h movie instead of ~1800), but each is
half the size on disk. Within HLS spec — Apple recommends 6 s, but
2 s is valid; LL-HLS uses 1-2 s.
2. Cache from 0.9.9 self-heals: cached entries used 4 s segments;
VerifyComplete now expects a different highest segment index and
invalidates them, triggering a re-encode on next play. No manual
cleanup needed.
3. OnStreamSession daemon callback now runs StartHLSSession in a
goroutine. Sync HTTP responses return immediately (~50 ms instead
of waiting for the ~0.3-1 s ffprobe). Other pending actions in
the same sync cycle (new tasks, deletes) no longer wait for the
transcoder warmup. Browser HEAD probes already have a 30 s retry
budget that covers the brief gap between playerSessionRegistry.add
and streamSrv.HLS().Register.
Helpers added (engine.segmentDurationFor / segmentStartSec /
segmentCountForDuration) so a future short-first-segment variant or
non-uniform layout can slot in without touching every call site.
Internal: -hls_init_time was investigated but discarded — ffmpeg's
implementation treats it as a min duration, not a target, so it
couldn't deliver a uniformly 2 s first segment on top of a 4 s
steady state. Uniform 2 s is simpler and gets the same first-frame
win.
Addresses items raised by the multi-agent code review of the 0.9.9
HW accel + first-start work:
- EncoderProfile now carries DecodeHwAccel so the demuxer `-hwaccel`
flag and the encoder argv derive from a single resolved profile.
Adding a new backend can no longer leave the two switches out of
sync.
- VAAPI no longer passes `-hwaccel_output_format vaapi`. That option
pinned decoded frames to GPU memory, but the filter chain (scale,
format, setparams) runs on CPU and would fail with "impossible to
convert between formats". Frames now decode HW + flow on CPU; the
encoder uploads back to GPU. Pre-existing bug, never reported because
no one had VAAPI auto-detected in practice.
- readyMax field comment + name: documented that it's a COUNT
(segments ready), not an index. The semantics were correct but the
comment read "highest index" which made `idx < readyMax` look like
an off-by-one to reviewers.
- probe_cache background janitor: 5-minute sweeper that drops expired
entries even when no lookup retouches the key. Lookup-only eviction
was fine for small libraries but unbounded for users who browse and
abandon thousands of files within a TTL window. Lazy + sync.Once.
- probe_cache TTL eviction now re-checks under the write lock so a
concurrent re-insert isn't accidentally evicted.
- probe_cache size-change test now Chtimes the file back to its
original mtime so only `size` differs between store and lookup
keys — properly exercises the size-check path.
- New TestProbeCache_SweepDropsExpired covers the janitor sweep.
- CHANGELOG: backfilled missing compare links 0.6.4 → 0.9.9.
- Stale "line ~1119" reference in VideoToolbox comment dropped; the
bitrate block moved a few lines and the comment was already wrong.
Two issues with the 0.9.9 preset retune:
1. applyDefaults was filling Preset="veryfast" before
ResolveEncoderProfile got to pick the latency-biased default, so the
"superfast" change never reached users with a freshly-generated
config.toml — only those who left the field empty saw it.
2. The configured preset was being passed through to every encoder.
That's only valid for libx264 (ultrafast…veryslow); NVENC uses p1-p7
and rejects anything else, QSV uses its own subset. A user with NVENC
+ preset="veryfast" would have ffmpeg reject the argv.
Now:
- TranscodeConfig.Preset documented as libx264-only with the full
range + advice on quality vs first-start latency.
- Default in applyDefaults is empty (was "veryfast") so the engine
fills in "superfast" on libx264.
- ResolveEncoderProfile ignores configuredPreset for vendor encoders
(NVENC sticks to p3, QSV to veryfast, VideoToolbox has no preset
knob). Test cases updated to lock in this behaviour.
Users who want better quality at slower first-play should set
download.transcode.preset = "veryfast" (previous default) / "faster" /
"fast" / "medium" in their config.toml.
Reduces first-segment latency on cache MISS so the player doesn't sit on
"preparando sesión". Three independent levers:
1. ProbeFile memoised by (path, mtime, size) for 30 min — second play of
the same source skips ffprobe (1-3 s on 50+ GB MKVs).
2. HLS encoder presets biased for latency over quality:
- libx264 default veryfast → superfast (~15-20% faster, marginal
quality loss at 5-25 Mbps target bitrates).
- NVENC: -preset p4 -tune hq → -preset p3 -tune ll. First-segment
~0.8 s on RTX-class GPUs (was ~1.5 s).
- QSV: -preset medium → -preset veryfast (keeps look_ahead=0).
- VideoToolbox: adds -realtime 1 (was unset). Bitrate args still
drive rate control; -q:v dropped to avoid the silent conflict
where ffmpeg ignored it under -b:v.
3. Per-session log surfaces encoder + accel + preset so "first-start
was slow" complaints can be triaged from the journal alone.
Diagnostic helpers (DetectHWAccelDiagnostic + HWAccelDiagnostic) added
for future wiring into daemon startup / agent register; users today can
already inspect via `unarr probe-hwaccel`.
Web: AgentsTab profile page now shows the agent's chosen encoder
(amber if software libx264, green if HW) plus the transcode-resolution
cap. Hidden for pre-0.9.9 agents that haven't reported hwAccel.
Daemon CORS allowlist was hardcoded to torrentclaw.com + localhost. Browsers
playing from any other official mirror (.to, onion, www., staging.) received
200 + body from the daemon's HLS server but no Access-Control-Allow-Origin
header, so the response was dropped client-side. Probe loop treated every
candidate as a failure and surfaced "No se puede conectar con tu agente
— 404 todos los canales" even though the tunnel + ffmpeg were healthy.
Static baseline now includes the full known mirror set (.com / www / app /
staging / .to / www.to / built-in onion). At startup the daemon also fetches
/api/mirrors with IPFS fallback and merges the live origins, so a future
mirror addition does not require a CLI rebuild.
Gives the daemon a public HTTPS hostname (`https://<random>.trycloudflare.com`)
so the in-browser player on torrentclaw.com plays cross-network without
Tailscale or port forwarding — the mixed-content block that was breaking
HTTPS-page → HTTP-daemon fetches is gone. Bytes proxy through CloudFlare,
never through TorrentClaw infra (preserves the aggregator legal posture).
New surface:
• `internal/funnel/` package: subprocess wrapper + auto-download for
cloudflared. Linux amd64/arm64/armhf/386 fetched from GitHub releases
on first run, validated by ELF magic + size sanity, O_EXCL partial
write so concurrent daemons don't clobber each other.
• `unarr funnel on/off/status` cobra command (sibling of `unarr vpn`).
• Daemon supervisor goroutine keeps cloudflared up across crashes + CF's
~6h Quick Tunnel rotation. Exponential backoff (2 s → 5 min). On exit
the reported URL is cleared so the web stops handing out a dead host.
• Wire: agent registers/syncs a FunnelURL field; web prefers it over
Tailscale/LAN for in-browser playback (HlsStreamPlayer + Stremio
addon).
Default ON for fresh installs (NAS/Docker get it without terminal-in);
existing configs that pre-date the feature stay off until the operator
opts in with `unarr funnel on`.
Docker image now bundles cloudflared (built per TARGETARCH via buildx).
Also fixed: libx264 'frame MB size > level limit' on anamorphic >16:9
sources. The level we hint to libx264 was derived from height alone,
which busted on 720p cinemascope (1728×720 = 4860 MBs > level 3.1's
3600). Bumped each tier: 720p → 4.0, 1080p → 4.1.
Version: 0.9.4 → 0.9.5.
Drops the custom WebRTC DataChannel pipeline + pion deps + WSS signaling
client + wire framing. Every in-browser playback now uses HLS over HTTP
from the daemon (Tailscale/LAN/UPnP). Browser P2P never re-enabled.
Wire renames (incompatible with web < 2026-05-26): agent.WebRTCSession
=> agent.StreamSession, SyncResponse.WebRTCSessions (JSON: webrtcSessions)
=> StreamSessions (JSON: streamSessions). MIN_AGENT_VERSION is bumped
to 0.9.4 on the web side so older agents see an upgrade card.
Also fixes the libx264 'VBV bitrate > level limit' abort by clamping
the encoder bitrate to the effective output height instead of the
requested label (carried over from the prior 0.9.3 unreleased work).
The seed_file vertical (mode=seed_file handler + engine.SeedFile) was
retired with the in-browser P2P player. [downloads.webrtc] config block
deleted; existing TOML files with the section still parse fine.
Asking for 2160p quality on a 720p source kept the daemon's qcap.VideoBitrate
at 25 Mbps even after outputHeight was clamped to the source. The level
H264LevelForHeight picks for the 720p output is 3.1 / 4.0, which rejects any
VBV >20 Mbps — libx264 then exited with "VBV bitrate (25000) > level limit"
on every restart, ffmpeg auto-restarted 3 times, master.m3u8 never appeared,
and the player got stuck at "Preparando sesión".
Re-derive the (height, bitrate) cap from the EFFECTIVE outputHeight via the
new capForHeight helper. Result: 720p source asked for 2160p → outputs 720p
with the 3500 kbps bitrate the level actually accepts. ffmpeg runs cleanly,
master.m3u8 appears, playback starts.
The web also clamps effectiveQuality to source resolution before the session
row is written, so the daemon mostly receives sane labels. This change keeps
the daemon defensive against (a) older web clients that still ask for
upscaled qualities, and (b) future quality="original" requests where qcap
is empty and Transcode.VideoBitrate could overshoot the level too.
A failed usenet extract sets task.ErrorMessage to the full unrar/par2
dump (multi-KB). Sent raw, the web /agent/status route rejected it and
the terminal report failed, leaving the task stuck non-terminal.
Cap the reported errorMessage at 2000 bytes (rune-safe) in the status
snapshot, matching the server's stored length.
In-process userspace WireGuard tunnel (wireguard-go + gVisor netstack) for
the managed-VPN add-on. No root, no OS routing changes: only the embedded
anacrolix/torrent client's peer + tracker traffic is routed through the
tunnel, so the swarm and trackers see the VPN IP, not the user's home IP.
unarr's control plane (API, heartbeats) keeps using the normal net.
- internal/vpn: FetchConfig (GET /api/internal/agent/vpn-config, Bearer auth,
typed errors for disabled/not_provisioned/slot_on_device) + Up (parse .conf
→ uapi, CreateNetTUN, device Up) + DialContext/ListenPacket adapters.
- engine/torrent.go: when a tunnel is set, wire TrackerDialContext +
HTTPDialContext + TrackerListenPacket to netstack, DisableUTP, and
AddDialer(NetworkDialer{tcp, netstack}) for peer conns.
- config: downloads.vpn.enabled flag.
- daemon: bring up the tunnel before the torrent client; non-fatal on
failure (logs + downloads in the clear); slot_on_device warns the user.
- version bump 0.8.1 → 0.9.0.
Pairs with the web VPN add-on (dormant behind NEXT_PUBLIC_VPN_ENABLED).
Runtime-verified once a VPNResellers trial provides a live endpoint.
Phase 3 security audit follow-up. Medium and low-severity hardenings
plus a deferred-work plan for the cross-repo stream-token rollout.
Stream server CORS: replace the wildcard Access-Control-Allow-Origin
with an allowlist that echoes back only torrentclaw.com,
app.torrentclaw.com, the local Next dev port (3030 — matches the web
repo package.json) and any extras the operator adds via the new
downloads.cors_extra_origins TOML key. A Vary: Origin header is now
emitted whenever the request carries an Origin header so an
intermediate cache cannot serve a stale ACAO to a different origin.
URL scheme guard: openBrowser and OpenPlayer refuse any URL that is
not http(s). Combined with passing the URL after "--" wherever the
launched helper supports it (open, mpv, vlc, cvlc), this stops a
leading "-" from being parsed as a switch by the spawned process.
State file permissions: WriteState now writes 0o600 so the agent ID,
PID and counters cannot be enumerated by another local user on a
shared host. Matches the existing config file mode.
ZIP slip defense-in-depth: extractZip extracts the safety check into
safeZipPath, which canonicalises the entry name (normalising
backslashes to "/"), rejects "..", "../" prefix and "/../" interior
components, and verifies the final destination stays inside destDir
before opening any file.
Mirror fallback: documented the design for multi-provider
mirrors.json hosting in the comment block on DefaultStaticFallbackURLs
and added a follow-up note about signing it with the same ed25519
release key. The list is kept at one provider until the second host
is provisioned and added to torrentclaw-web's STATIC_FALLBACKS.
Deferred work: a new plan document Docs/plans/security-stream-token.md
covers the per-task stream token (Phase 2.2 of the original audit)
which requires coordinated web + CLI work and ships separately.
Phase 2 security audit follow-up. Three independent hardenings against
the unauthenticated daemon surface, the long-lived agent SSE stream
and the self-update channel.
UPnP is now opt-in. The stream port + /hls endpoints have no auth, so
publishing them on the WAN via the gateway was a default that exposed
active downloads to anyone scanning the operator's external IP. New
config downloads.enable_upnp (default false) gates the mapping; LAN
and Tailscale clients continue to work unchanged. A startup log makes
the new default visible.
The agent SSE reader now uses a bounded bufio.Scanner instead of an
unbounded ReadString. A hostile or buggy server can no longer grow
daemon memory by streaming a single line forever or by emitting
unbounded data: continuation lines — both are capped at 256 KiB and
1 MiB respectively, and an error is surfaced so SignalLoop reconnects.
Self-update now verifies an ed25519 signature over checksums.txt when
the binary was built with a release public key embedded (injected via
goreleaser ldflags from RELEASE_SIGNING_PUBKEY). The companion
scripts/sign-checksums runs in the release workflow when both the
public-key variable and the private-key secret are present, uploading
checksums.txt.sig next to the existing checksums file. Builds without
the embedded key continue to update with SHA256-only verification; a
--allow-unsigned flag is provided so users on a signed build can
still install pre-signing releases or recover from an accidental
unsigned release.
A new scripts/gen-release-key helper documents the one-time keypair
generation procedure required before flipping signing on.
Phase 1 security audit follow-up:
- Reject HLS session IDs that aren't safe filesystem components
(regex allowlist) to defend against path traversal via a buggy or
compromised server. Applied at StartHLSSession and at the /hls URL
handler; invalid IDs share the 404 of unknown sessions so the
accepted format isn't enumerable.
- /health no longer leaks the active filename, taskID prefix or client
IP to non-loopback callers. Uses net.IP.IsLoopback so IPv4-mapped
IPv6 (::ffff:127.0.0.1) is recognised and the empty-string parse
failure stops bypassing the boundary.
- unrar/7z passwords now travel through stdin instead of -p<password>
in argv, removing /proc/<pid>/cmdline disclosure. Control characters
in the password are rejected up front so a hostile NZB cannot feed
extra prompt answers. Both invocations are bounded by a 30-minute
context to stop indefinite hangs if the tool ever decides to prompt.
