Stream extraction¶
services/extractor.py turns a song ID into a playable googlevideo URL using yt-dlp. It knows
nothing about segments or FastAPI; StreamingService is its only caller and never imports yt-dlp
itself.
Output: ExtractedStream¶
@dataclass(frozen=True)
class ExtractedStream:
song_id: str
url: str # googlevideo URL: never log it
itag: str # "140" normally
ext: str # "m4a" | "webm" | "mp4"
mime_type: str # "audio/mp4" | "audio/webm"
acodec: str
is_audio_only: bool
filesize: int | None # format["filesize"], else the URL's clen=
duration_ms: int # info["duration"]
title: str
expires_at: float # URL's expire= minus 5 min
player_client: str # which client worked (diagnostics)
available_at: float # when the URL starts answering (0 = now)
It is never written to disk: the URL is signed and expires after a few hours.
Flow¶
flowchart TD
get["StreamExtractor.get(song_id)"] --> fresh{"cached and<br/>not expired?"}
fresh -- yes --> ret["return it"]
fresh -- no --> failed{"failed recently?<br/>(5 min, 30 min if unavailable)"}
failed -- yes --> raise["raise the cached ExtractionError"]
failed -- no --> sf["SingleFlight: one extraction per song,<br/>concurrent callers wait for it"]
sf --> node{"node on PATH?"}
node -- no --> nm["raise node_missing (not cached)"]
node -- yes --> loop["for client in YTDLP_PLAYER_CLIENTS"]
loop --> ydl["yt-dlp extract_info(www.youtube.com/watch?v=…)"]
ydl --> sel["select_audio_format()"]
sel -- found --> wait["sleep until available_at (≤ 30 s)"] --> store["cache, return"]
sel -- none / error --> next{"unavailable?"}
next -- yes --> stop["raise unavailable"]
next -- no --> loop
Options that yt-dlp actually honours¶
def build_ydl_opts(player_client, fmt, cookie_file):
opts = {
"quiet": True, "no_warnings": True, "skip_download": True, "socket_timeout": 15,
"js_runtimes": {"node": {}}, # top-level dict
}
if player_client: # None = yt-dlp's default selection
opts["extractor_args"] = {"youtube": {"player_client": [player_client]}}
if fmt:
opts["format"] = fmt
if cookie_file:
opts["cookiefile"] = cookie_file
return opts
yt-dlp silently ignores malformed options
extractor_args must be {extractor: {key: [values]}}. An earlier version passed a list
({"youtube": ["player_client=android"]}): yt-dlp dropped it without a warning, so the
client rotation never happened. test_ydl_opts_player_client_applied checks the value
yt-dlp actually sees; keep it.
Each attempt uses format="bestaudio/best" with ignore_no_formats_error=True, so yt-dlp
returns the full format list instead of raising "Requested format is not available".
The choice is then made in Python.
Format selection¶
select_audio_format(info) keeps only audio-only formats that have a URL, then picks:
- the itag in
STREAM_FORMAT(140, AAC 128 kbps in MP4) if present; - else the best m4a/mp4 audio by bitrate;
- else the best audio of any container (e.g.
251, Opus in WebM).
Muxed video (itag 18 = 360p video + audio) is never returned: it would download video for an audio player. A commented-out fallback in the code shows where to enable it.
Player clients¶
YTDLP_PLAYER_CLIENTS defaults to ["visionos", "default"], one extraction per client.
| Client | Audio-only formats | Delay before the URL works | Time |
|---|---|---|---|
visionos |
139, 140, 249, 250, 251 | none | ~1.0 s |
default (yt-dlp picks; tv_samsung's URL wins) |
139, 140, 249–251 | ~5 s | ~1.8 s |
web, web_music, web_safari, tv, ios, mweb, android_vr |
none | ||
android (no PO token) |
only muxed itag 18 |
Measured on 2026-10-31 with yt-dlp 2026.10.26, signed out. Re-measure after a yt-dlp upgrade if extraction gets slow or fails.
Cookies and visionos
visionos has SUPPORTS_COOKIES=False, and yt-dlp silently drops such a client when a
cookie file is passed. So the extractor passes cookies only to clients that support them
(_client_supports_cookies) and runs the others anonymously. When signed in, the cookies
are written to a Netscape-format temp file (mode 0600) and deleted in a finally.
available_at
Some clients (the TV ones that yt-dlp's default uses when a JS runtime is available) return
URLs that answer 403 for a few seconds after extraction. yt-dlp exposes this as
available_at. wait_until_available() sleeps until then, capped at 30 s. This is why
visionos comes first.
Errors¶
yt-dlp errors are mapped to ExtractionError.code by classify_download_error():
| Code | yt-dlp says | Rotation |
|---|---|---|
unavailable |
"Video unavailable", "Private video" | stops: no client will help |
auth_required |
"Sign in to confirm", age-restricted | tries the next client |
no_audio |
formats returned, none audio-only | tries the next client (formats are logged) |
extract_failed |
anything else | tries the next client |
node_missing |
(checked before running yt-dlp) |
After all clients fail, the reported code is auth_required if any client said so, else
no_audio, else extract_failed.
Caching and refresh¶
| Cache | Size | Lifetime |
|---|---|---|
| streams | BoundedDict(256) |
until expires_at (URL expire= − 5 min) |
| failures | BoundedDict(256) |
5 min; 30 min for unavailable; node_missing never cached |
refresh(song_id, stale_url) is called when a segment download gets a 403, which means the
signed URL expired. It runs in the same SingleFlight: if the cached URL already differs from
stale_url, another worker refreshed it and the new one is returned. So N workers hitting 403
at once cause one re-extraction. If the refreshed stream has a different itag or size, a warning
is logged; the segment table handles the change.
get_extraction_dependencies() checks node on PATH, yt_dlp_ejs importable and the yt-dlp
version once at startup, logs them and reports them in /api/health.
SingleFlight and BoundedDict
Both are small helpers in extractor.py. SingleFlight.do(key, fn) runs fn once per key
at a time; concurrent callers block and receive the same result or exception.
BoundedDict(maxsize) is a locked dict that drops its oldest entry beyond maxsize, so
in-memory caches can't grow forever. StreamingService reuses both for stream-info building.