add video playback: theater mode, chapters, webvtt subtitles, transcode cancellation, audio language preference, offline video extras
MREADME.md
@@ -8,6 +8,7 @@ Pico Pixel Player is a lightweight, self-hostable media player with an [ElysiaJS
- **Responsive UI**: Works seamlessly on desktop and mobile
- **Offline Capable**: Progressive Web App (PWA) support and local caching of files
- **Transcoding**: Optionally transcode files at a selected bitrate and format to your client
- **Video playback**: Responsive inline video, theater/fullscreen modes, chapters, and text subtitles
- **Directory Listing**: Browse files in their original directory structure
## Screenshot
@@ -54,6 +55,12 @@ You can configure the backend with the following environment variables:
- `mse` feeds the transcode into a SourceBuffer via Media Source Extensions. This is the only mode that can seek inside a transcode, because a seek simply restarts ffmpeg at an offset. In exchange it only works for containers and codecs that have an MSE byte stream format, which the options menu greys out the rest based on.
- `progressive` hands the transcode straight to the media element as it is produced. It plays whatever the browser can decode - notably video in webm, which MSE cannot handle - but cannot seek.
- `buffered` makes the backend transcode the whole file before sending anything. By default the backend sends the transcoded media as it progresses, which uses "chunked transfer", i.e. without the usual content-length of normal HTTP requests. Some browser (safari cough cough) seem to be very selective about the sources of \<audio\> elements, and choke on sources that are transferred via this method. This mode improves compatibility with those browsers, at the cost of a significant startup delay. It does not enable seeking - the transcode endpoint serves no range requests either way.
- Why does an MKV file stop or fail when transcoding is disabled?
- No-transcoding serves the original file directly, so playback and seeking depend entirely on browser support. Browsers support WebM more consistently than arbitrary Matroska/MKV files; an MKV may start playing and still fail when seeking. Use transcoding or remux the file to a browser-friendly container such as MP4. Remuxing only changes the container and does not fix unsupported codecs.
- Why is AC-3 audio missing?
- AC-3 decoding is not consistently available in browsers. With transcoding enabled, the server re-encodes audio to the selected browser-compatible codec. Disabling transcoding leaves the original AC-3 stream, even if the video itself plays.
- Why are bitmap subtitles such as PGS not shown, and why can FFmpeg not convert them to WebVTT?
- PGS subtitles contain timed bitmap images rather than text. FFmpeg can copy or convert bitmap subtitles to other bitmap formats, but its WebVTT encoder accepts text subtitles only; converting PGS to text requires OCR. Supporting them would require OCR, custom image subtitle rendering, or burning them into a re-encoded video, none of which is currently implemented.
- The playback stops automatically at some point on my mobile device when I put the browser in the background or turning my screen off
- In my testing at least mobile firefox behaves this way, with no apparent solution. It seems to stop after changing the sources of the audio element at some point. Either use a browser that doesn't behave like this (Chrome on Android seems to work), or wrap the PWA in a native app (e.g. https://www.pwabuilder.com/)
- I'm getting "Quota exceeded" errors when trying to sync files *or*
@@ -67,4 +74,4 @@ You can configure the backend with the following environment variables:
## Attributions
- Font: https://github.com/TakWolf/ark-pixel-font
- Icons: https://github.com/hackernoon/pixel-icon-library
- Logo: https://github.com/halfmage/pixelarticons
- Logo: https://github.com/halfmage/pixelarticons
Mserver/src/ffmpeg.ts
@@ -42,17 +42,20 @@ function lookup<T>(table: Record<string, T>, key: string, what: string): T {
return value;
}
//pick the audio track matching the earliest entry of the caller's preference list. with no match at
//all, prefer whichever track the file itself flags as default, and only then fall back to the first
function selectAudioTrack(languages: string, tracks: NonNullable<Metadata["audioTracks"]>): number {
const preferences = languages ? languages.toLocaleLowerCase().split(",") : [];
//Choose the first matching preferred language. Audio always needs a track, so an unmatched
//preference falls back to the source default and then the first audio stream.
function selectAudioTrack(audioLanguagePreference: string, tracks: NonNullable<Metadata["audioTracks"]>): number {
const preferences = audioLanguagePreference
.split(",")
.map((language) => language.trim().toLowerCase())
.filter(Boolean);
let best = -1;
let bestPreference = Number.POSITIVE_INFINITY;
for (let i = 0; i < tracks.length; i++) {
const language = tracks[i].language?.toLocaleLowerCase();
for (let index = 0; index < tracks.length; index++) {
const language = tracks[index].language?.trim().toLowerCase();
const preference = language ? preferences.indexOf(language) : -1;
if (preference >= 0 && preference < bestPreference) {
best = i;
best = index;
bestPreference = preference;
if (preference === 0) break;
}
@@ -119,7 +122,7 @@ export async function convertWithFFmpeg(
requestedVideoCodec: VideoCodec,
requestedVideoEncodingSetting: VideoEncodingSetting,
seekTo: number | undefined,
languages: string,
audioLanguagePreference: string,
metadata: Metadata,
) {
const video = videoStreamInfo(metadata, videoBitrate);
@@ -135,8 +138,7 @@ export async function convertWithFFmpeg(
if (!hasVideo) containerArgs.push("-frag_duration", AUDIO_FRAGMENT_MICROSECONDS.toString());
}
const selectedAudioTrack = selectAudioTrack(languages, metadata.audioTracks || []);
const selectedAudioTrack = selectAudioTrack(audioLanguagePreference, metadata.audioTracks || []);
const args = [
"ffmpeg",
"-nostats",
@@ -162,13 +164,27 @@ export async function convertWithFFmpeg(
console.log(args.join(" "));
const cmd = Bun.spawn(args, { stdin: "ignore", stdout: "pipe", stderr: "pipe" });
const stderrText = new Response(cmd.stderr).text().catch(() => "");
void cmd.exited
.then(async (exitCode) => {
if (exitCode !== 0 && exitCode !== null) {
console.error("ffmpeg failed with status code", exitCode);
console.error(await stderrText);
}
})
.catch(() => {});
return { cmd, mimeType, stderrText };
}
export async function convertSubtitleWithFFmpeg(filePath: string, streamIndex: number) {
const args = [
"ffmpeg",
"-nostats",
"-loglevel",
"error",
"-i",
filePath,
"-map",
`0:s:${streamIndex}`,
"-c:s",
"webvtt",
"-f",
"webvtt",
"-",
];
console.log(args.join(" "));
const cmd = Bun.spawn(args, { stdin: "ignore", stdout: "pipe", stderr: "pipe" });
const stderrText = new Response(cmd.stderr).text().catch(() => "");
return { cmd, mimeType: "text/vtt; charset=utf-8", stderrText };
}
Mserver/src/index.ts
@@ -11,7 +11,7 @@ import {
VideoEncodingSetting,
} from "music-server-shared/types";
import { decodePath } from "music-server-shared/utils";
import { convertWithFFmpeg } from "./ffmpeg";
import { convertSubtitleWithFFmpeg, convertWithFFmpeg } from "./ffmpeg";
import {
allowedTypes,
args,
@@ -26,10 +26,12 @@ import {
probeCache,
ServerError,
username,
videoExtrasCache,
} from "./shared";
import {
findCover,
getPathInfo,
getVideoExtras,
isBelow,
listFiles,
matchesType,
@@ -87,6 +89,29 @@ function transcodeFailed(set: FileHandlerContext["set"], exitCode: number | null
return `Transcoding failed (ffmpeg exit ${exitCode})${reason ? `:\n${reason}` : ""}`;
}
function reportUnexpectedFFmpegExit(label: string, exitCode: number | null, stderr: string): void {
if (exitCode === 0 || exitCode === null) return;
const signal = exitCode > 128 ? ` (signal ${exitCode - 128})` : "";
console.error(`${label} failed with status code ${exitCode}${signal}`);
const reason = stderr.trim();
if (reason) console.error(reason);
}
//Bun does not reliably propagate a disconnect from a streaming Response to request.signal on every
//browser/runtime combination. The frontend sends an explicit cancellation beacon as a second path.
const activeTranscodes = new Map<string, { stop: () => void }>();
const cancelledTranscodes = new Set<string>();
function markTranscodeCancelled(id: string): void {
cancelledTranscodes.add(id);
const timer = setTimeout(() => cancelledTranscodes.delete(id), 60_000);
timer.unref?.();
}
function consumeTranscodeCancellation(id: string | undefined): boolean {
return id !== undefined && cancelledTranscodes.delete(id);
}
const downloadHandler = async ({ params, set }: FileHandlerContext) => {
const resolved = await resolveMediaFile(params["*"]);
if (resolved instanceof ServerError) {
@@ -181,7 +206,23 @@ const app = setup
.post("/reset-cache", () => {
fileTypeCache.clear();
probeCache.clear();
videoExtrasCache.clear();
})
.post(
"/cancel-transcode/:id",
({ params }) => {
const active = activeTranscodes.get(params.id);
if (active) {
activeTranscodes.delete(params.id);
active.stop();
} else {
//The beacon can arrive while the transcode route is still probing the file, before
//there is a child process to register.
markTranscodeCancelled(params.id);
}
},
{ params: t.Object({ id: t.String({ minLength: 1 }) }) },
)
.get("/download/*", downloadHandler)
.head("/download/*", downloadHandler)
.get(
@@ -208,6 +249,7 @@ const app = setup
set.status = "Bad Request";
return "videoBitrate is required when videoCodec is set";
}
if (request.signal.aborted || consumeTranscodeCancellation(query.transcodeId)) return new Uint8Array();
const { cmd, mimeType, stderrText } = await convertWithFFmpeg(
filePath,
audioBitrate,
@@ -217,24 +259,63 @@ const app = setup
query.videoCodec || VideoCodec.none,
query.videoEncodingSetting || VideoEncodingSetting.balanced,
query.seekTo,
query.languages || "",
query.audioLanguagePreference || "",
probe,
);
request.signal.addEventListener("abort", () => cmd.kill("SIGKILL"));
if (consumeTranscodeCancellation(query.transcodeId)) {
cmd.kill("SIGKILL");
return new Uint8Array();
}
let stoppedByClient = false;
const stopForClient = () => {
if (stoppedByClient) return;
stoppedByClient = true;
console.info("ffmpeg cancelled by client", filePath);
cmd.kill("SIGKILL");
};
const activeTranscode = { stop: stopForClient };
if (query.transcodeId) activeTranscodes.set(query.transcodeId, activeTranscode);
const unregister = () => {
if (query.transcodeId && activeTranscodes.get(query.transcodeId) === activeTranscode)
activeTranscodes.delete(query.transcodeId);
};
request.signal.addEventListener("abort", stopForClient, { once: true });
if (request.signal.aborted) stopForClient();
void cmd.exited
.then(async (exitCode) => {
if (!stoppedByClient) reportUnexpectedFFmpegExit("ffmpeg", exitCode, await stderrText);
})
.finally(unregister);
if (query.disableChunkedTranscoding) {
const full = await readStream(cmd.stdout);
if (full.length === 0) return transcodeFailed(set, await cmd.exited, await stderrText);
const full = await readStream(cmd.stdout).catch((error) => {
if (stoppedByClient) return new Uint8Array();
throw error;
});
if (full.length === 0) {
const exitCode = await cmd.exited;
if (stoppedByClient) return full;
return transcodeFailed(set, exitCode, await stderrText);
}
set.headers["content-type"] = mimeType;
return full;
}
//peek ffmpeg to check for failure and return 500
const reader = cmd.stdout.getReader();
const first = await reader.read();
let first: ReadableStreamReadResult<Uint8Array>;
try {
first = await reader.read();
} catch (error) {
reader.releaseLock();
if (stoppedByClient) return new Uint8Array();
throw error;
}
if (first.done) {
reader.releaseLock();
return transcodeFailed(set, await cmd.exited, await stderrText);
const exitCode = await cmd.exited;
if (stoppedByClient) return new Uint8Array();
return transcodeFailed(set, exitCode, await stderrText);
}
set.headers["content-type"] = mimeType;
return new Response(
@@ -248,7 +329,7 @@ const app = setup
else controller.enqueue(value);
},
cancel(reason) {
cmd.kill("SIGKILL");
stopForClient();
void reader.cancel(reason);
},
}),
@@ -257,7 +338,8 @@ const app = setup
{
query: t.Object({
seekTo: t.Optional(t.Number()),
languages: t.Optional(t.String()),
transcodeId: t.Optional(t.String()),
audioLanguagePreference: t.Optional(t.String()),
disableChunkedTranscoding: t.Optional(t.Boolean()),
container: t.Enum(MediaContainer),
videoCodec: t.Optional(t.Enum(VideoCodec)),
@@ -295,6 +377,73 @@ const app = setup
const probeData = await probeFile(dirPath).catch(() => ({}) as Metadata);
return { isVideo: probeData.videoCodec !== undefined } as IsVideoResponse;
})
.get("/video-info/*", async ({ params, set }) => {
const resolved = await resolveMediaFile(params["*"]);
if (resolved instanceof ServerError) {
set.status = resolved.status;
return resolved.error;
}
if (!matchesType(resolved.info.mimeType, mediaTypes)) {
set.status = "Bad Request";
return "Video extras are only available for media files";
}
return await getVideoExtras(resolved.filePath);
})
.get(
"/subtitles/*",
async ({ request, params, query, set }) => {
const resolved = await resolveMediaFile(params["*"]);
if (resolved instanceof ServerError) {
set.status = resolved.status;
return resolved.error;
}
if (!matchesType(resolved.info.mimeType, mediaTypes)) {
set.status = "Bad Request";
return "Subtitles are only available for media files";
}
if (!Number.isInteger(query.track) || query.track < 0) {
set.status = "Bad Request";
return "Invalid subtitle track";
}
const extras = await getVideoExtras(resolved.filePath);
const track = extras.subtitleTracks.find((candidate) => candidate.streamIndex === query.track);
if (!track) {
set.status = "Not Found";
return "Subtitle track not found or unsupported";
}
if (request.signal.aborted) return new Uint8Array();
const { cmd, mimeType, stderrText } = await convertSubtitleWithFFmpeg(resolved.filePath, track.streamIndex);
let stoppedByClient = false;
const stopForClient = () => {
if (stoppedByClient) return;
stoppedByClient = true;
console.info("ffmpeg subtitle conversion cancelled by client", resolved.filePath);
cmd.kill("SIGKILL");
};
request.signal.addEventListener("abort", stopForClient, { once: true });
if (request.signal.aborted) stopForClient();
void cmd.exited.then(async (exitCode) => {
if (!stoppedByClient)
reportUnexpectedFFmpegExit("ffmpeg subtitle conversion", exitCode, await stderrText);
});
const bytes = await readStream(cmd.stdout).catch((error) => {
if (stoppedByClient) return new Uint8Array();
throw error;
});
const exitCode = await cmd.exited;
if (exitCode !== 0 || bytes.length === 0) {
if (stoppedByClient) return bytes;
return transcodeFailed(set, exitCode, await stderrText);
}
set.headers["content-type"] = mimeType;
return bytes;
},
{
query: t.Object({ track: t.Number() }),
},
)
.get(
"/cover/*",
async ({ params, set, query }) => {
Mserver/src/utils.ts
@@ -1,10 +1,17 @@
import { readFile, readdir, stat, writeFile } from "node:fs/promises";
import { readdir, readFile, stat, writeFile } from "node:fs/promises";
import path from "node:path";
import { StatusMap } from "elysia";
import { fileTypeFromBlob } from "file-type";
import { parseFile } from "music-metadata";
import { parseFile, selectCover } from "music-metadata";
import { AsyncSemaphore } from "music-server-shared/semaphore";
import type { FileListingWithStatus, Metadata } from "music-server-shared/types";
import type {
AudioTrack,
Chapter,
FileListingWithStatus,
Metadata,
SubtitleTrack,
VideoExtras,
} from "music-server-shared/types";
import tmp from "tmp";
import {
coverRegex,
@@ -16,6 +23,7 @@ import {
probeCache,
ServerError,
scanConcurrency,
videoExtrasCache,
} from "./shared";
//fs errors are untyped, so pull the errno code out of the unknown catch value
@@ -32,9 +40,7 @@ export async function findAsync<T>(arr: Array<T>, asyncCallback: (value: T) => P
//success carries a bytes thunk so the handler doesn't care whether the cover came from a file on
//disk or from embedded metadata
export type CoverResult =
| { info: PathInfo; bytes: () => Promise<Uint8Array<ArrayBuffer>> }
| { info: ServerError };
export type CoverResult = { info: PathInfo; bytes: () => Promise<Uint8Array<ArrayBuffer>> } | { info: ServerError };
//the mediainfo CLI JSON output emits every value as a string; broken text encodings arrive
//as objects carrying an "@dt" attribute instead (detected in probeFile)
@@ -48,6 +54,11 @@ interface MediaInfoResult {
} | null;
}
//bitmap subtitles can't be converted to vtt
function isBitmapSubtitle(format: string | undefined): boolean {
return /pgs|vobsub|dvd[_ -]?subtitle|hdmv|dvb[_ -]?subtitle|teletext|arib|bitmap/i.test(format || "");
}
const mediaInfoSemaphore = new AsyncSemaphore(scanConcurrency);
export async function findCover(targetPath: string): Promise<CoverResult> {
@@ -55,9 +66,8 @@ export async function findCover(targetPath: string): Promise<CoverResult> {
if ((await stat(targetPath)).isFile()) {
//duration is not needed for cover extraction and would force a full-file parse for some formats
const parsed = await parseFile(targetPath, { duration: false, skipCovers: false });
if (!parsed.common.picture || parsed.common.picture.length === 0) return findCover(path.dirname(targetPath));
const picture = parsed.common.picture[0];
const picture = selectCover(parsed.common.picture);
if (!picture) return findCover(path.dirname(targetPath));
return {
info: new PathInfo(picture.format),
bytes: () => Promise.resolve(picture.data as Uint8Array<ArrayBuffer>),
@@ -133,9 +143,29 @@ export async function probeFile(filePath: string): Promise<Metadata> {
const generalTrack = info.media?.track.find((track) => track["@type"] === "General");
const videoTrack = info.media?.track.find((track) => track["@type"] === "Video");
const audioTrack = info.media?.track.find((track) => track["@type"] === "Audio");
const audioTracks = (info.media?.track || [])
const audioTracks: AudioTrack[] = (info.media?.track || [])
.filter((track) => track["@type"] === "Audio")
.map((track) => ({ language: track.Language, default: track.Default === "Yes" }));
.map((track, streamIndex) => ({
streamIndex,
language: track.Language,
title: track.Title,
default: track.Default === "Yes",
}));
const textTracks = (info.media?.track || []).filter((track) => track["@type"] === "Text");
const subtitleTracks: SubtitleTrack[] = textTracks.flatMap((track, streamIndex) =>
isBitmapSubtitle(track.Format || track.CodecID)
? []
: [
{
streamIndex,
language: track.Language,
title: track.Title,
default: track.Default === "Yes",
forced: track.Forced === "Yes",
codec: track.Format || track.CodecID,
},
],
);
if (!generalTrack?.Duration) throw new Error();
//the CLI emits all values as strings, so numeric fields need explicit coercion
const num = (value: string | undefined) => (value === undefined ? undefined : Number(value));
@@ -162,6 +192,7 @@ export async function probeFile(filePath: string): Promise<Metadata> {
: num(videoTrack?.FrameRate),
interlaced: videoTrack?.ScanType === undefined ? undefined : videoTrack.ScanType !== "Progressive",
audioTracks,
subtitleTracks,
};
//if file has broken encodings in their metadata, the "string" are actually objects, and contain this "@dt" field
@@ -190,6 +221,96 @@ export async function probeFile(filePath: string): Promise<Metadata> {
}
}
interface FFProbeChapter {
start?: number;
start_time?: string;
end?: number;
end_time?: string;
time_base?: string;
tags?: { title?: string };
}
function chapterTime(
time: string | undefined,
ticks: number | undefined,
timeBase: string | undefined,
): number | undefined {
if (time !== undefined) {
const seconds = Number(time);
if (Number.isFinite(seconds)) return seconds;
}
if (ticks === undefined || !timeBase) return undefined;
const [numerator, denominator] = timeBase.split("/").map(Number);
if (!Number.isFinite(numerator) || !Number.isFinite(denominator) || denominator === 0) return undefined;
const seconds = (ticks * numerator) / denominator;
return Number.isFinite(seconds) ? seconds : undefined;
}
async function chaptersFromFFProbe(filePath: string): Promise<Chapter[]> {
const cmd = Bun.spawn(["ffprobe", "-v", "error", "-print_format", "json", "-show_chapters", filePath], {
stdin: "ignore",
stdout: "pipe",
stderr: "pipe",
});
const [output, stderr, exitCode] = await Promise.all([
new Response(cmd.stdout).text(),
new Response(cmd.stderr).text(),
cmd.exited,
]);
if (exitCode !== 0) throw new Error(`ffprobe failed with status code ${exitCode}: ${stderr}`);
const parsed = JSON.parse(output) as { chapters?: FFProbeChapter[] };
return (parsed.chapters || []).flatMap((chapter) => {
const start = chapterTime(chapter.start_time, chapter.start, chapter.time_base);
if (start === undefined) return [];
const end = chapterTime(chapter.end_time, chapter.end, chapter.time_base);
return [
{
start,
...(end !== undefined ? { end } : {}),
title: chapter.tags?.title || "",
},
];
});
}
//Chapter parsing is deliberately lazy: directory scans still use the inexpensive MediaInfo probe, while
//opening a video (or syncing it) pays for chapter extraction only when it is actually useful.
export async function getVideoExtras(filePath: string): Promise<VideoExtras> {
const cached = videoExtrasCache.get(filePath);
if (cached) return cached;
const metadata = await probeFile(filePath);
if (metadata.videoCodec === undefined) {
const empty: VideoExtras = { subtitleTracks: [], chapters: [] };
videoExtrasCache.set(filePath, empty);
return empty;
}
//Use music-metadata first, then ffprobe for containers such as Matroska files whose chapter metadata
//music-metadata does not expose
const parsed = await parseFile(filePath, {
duration: false,
skipCovers: true,
includeChapters: true,
mkvUseIndex: true,
});
const metadataChapters: Chapter[] = (parsed.format.chapters || [])
.filter((chapter) => Number.isFinite(chapter.start))
.map((chapter) => ({
start: chapter.start,
...(chapter.end !== undefined && Number.isFinite(chapter.end) ? { end: chapter.end } : {}),
title: chapter.title || "",
}));
const chapters = metadataChapters.length > 0 ? metadataChapters : await chaptersFromFFProbe(filePath);
const extras: VideoExtras = {
subtitleTracks: metadata.subtitleTracks || [],
chapters,
};
videoExtrasCache.set(filePath, extras);
return extras;
}
export async function getPathInfo(filePath: string): Promise<PathInfo | ServerError | undefined> {
try {
if (excludeExtension.includes(path.extname(filePath).slice(1))) return new ServerError(404, "Excluded extension");
Mwebclient/src/App.tsx
@@ -11,6 +11,7 @@ import type { Component } from "solid-js";
import { batch, createEffect, createResource, createSignal, on, onMount, Show } from "solid-js";
import { createStore, reconcile } from "solid-js/store";
import { BUILD_HASH } from "./build-info";
import { DEFAULT_STREAMING_MODE, playableSettings } from "./capabilities";
import DownloadManager from "./components/DownloadManager";
import FileBrowser from "./components/FileBrowser";
import OptionsModal from "./components/OptionsModal";
@@ -30,9 +31,15 @@ import {
OpenFolderSVG,
PlaylistSVG,
} from "./components/SVG";
import { getCover, getSrc, listFiles } from "./offline";
import { DEFAULT_STREAMING_MODE, playableSettings } from "./capabilities";
import { type AppOptions, FileDisplay, type PlayerState, type PlaylistItem, StreamingMode } from "./types";
import { getCover, getSrc, listFiles, loadVideoExtras, revokeVideoExtras } from "./offline";
import {
type AppOptions,
FileDisplay,
type LoadedVideoExtras,
type PlayerState,
type PlaylistItem,
StreamingMode,
} from "./types";
import { formatFilename, handleUnauthorized, LocalStorageValues, setUnauthorizedHandler, toast } from "./utils";
//TODO: bypass music-metadata if it fails, maybe only use for fallback for files with weird encodings
@@ -88,6 +95,7 @@ const App: Component = () => {
//default settings
enableTranscoding: true,
languages: "",
audioLanguagePreference: "",
maxTranscodes: 4,
maxDownloads: 2,
transcodeCovers: false,
@@ -132,7 +140,42 @@ const App: Component = () => {
},
{ initialValue: [] },
);
const [videoShown, setVideoShown] = createSignal(false);
const [isVideoTrack, setIsVideoTrack] = createSignal(false);
const [videoTheater, setVideoTheater] = createSignal(false);
const [selectedSubtitle, setSelectedSubtitle] = createSignal<"off" | number>("off");
//path the currently served extras belong to; while the resource refetches, it still serves the
//previous track's extras, and this keeps consumers from treating those as the new track's data
let extrasLoadedForPath: string | undefined;
//track path the language-based auto-selection last ran for, so refetches of the same track
//(e.g. offline toggle) don't override a manual subtitle choice
let lastAutoSelectedPath: string | undefined;
const [videoExtras] = createResource<LoadedVideoExtras, { path?: string; offline: boolean }>(
() => ({ path: playerState.currentTrack?.path, offline: isOffline() }),
async (args) => {
if (!args.path || playerState.currentTrack?.metadata.videoCodec === undefined) {
extrasLoadedForPath = args.path;
return { subtitleTracks: [], chapters: [], subtitleSources: {} };
}
try {
const loaded = await loadVideoExtras(args.path, args.offline);
extrasLoadedForPath = args.path;
return loaded;
} catch (error) {
toast("Loading video extras failed", "error", error);
extrasLoadedForPath = args.path;
return { subtitleTracks: [], chapters: [], subtitleSources: {} };
}
},
{ initialValue: { subtitleTracks: [], chapters: [], subtitleSources: {} } },
);
//revoke the previous extras only once the new value has landed (and with it the new <track>
//elements) - revoking at fetch start would leave the still-rendered old tracks pointing at
//dead blob urls for the whole fetch window
createEffect(
on(videoExtras, (_current, previous) => {
if (previous) revokeVideoExtras(previous);
}),
);
async function refreshAuthStatus() {
try {
@@ -186,10 +229,6 @@ const App: Component = () => {
document.fonts.add(new_font);
});
createEffect(() => {
videoElement.style.display = videoShown() ? "block" : "none";
});
//incremented on every playTrack call so a slow/older invocation can detect it has been superseded
//and bail before clobbering the newer track's state
let playGeneration = 0;
@@ -197,11 +236,16 @@ const App: Component = () => {
function stopPlaying() {
//invalidate any in-flight playTrack so a stale invocation bails instead of restarting playback
playGeneration++;
cancelTranscode(activeTranscodeId);
activeTranscodeId = undefined;
batch(() => {
setPlayerState("isPlaying", false);
setPlayerState("currentTime", 0);
setPlayerState("currentTrack", null);
setPlayerState("waitingForStart", false);
setIsVideoTrack(false);
setVideoTheater(false);
setSelectedSubtitle("off");
});
videoElement.pause();
@@ -277,6 +321,79 @@ const App: Component = () => {
}
let videoElement!: HTMLVideoElement;
let activeTranscodeId: string | undefined;
function cancelTranscode(transcodeId: string | undefined) {
if (!transcodeId) return;
const url = `/cancel-transcode/${encodeURIComponent(transcodeId)}`;
try {
if (navigator.sendBeacon(url)) return;
} catch {}
void fetch(url, { method: "POST", credentials: "same-origin", keepalive: true }).catch(() => {});
}
function closeVideoTheater() {
if (document.fullscreenElement === videoElement) void document.exitFullscreen();
setVideoTheater(false);
}
function toggleVideoTheater() {
if (isVideoTrack()) setVideoTheater((open) => !open);
}
function applySubtitleModes() {
if (!videoElement) return;
const extras = videoExtras();
const selected = selectedSubtitle();
for (let index = 0; index < videoElement.textTracks.length; index++) {
const track = extras?.subtitleTracks[index];
const showing = selected !== "off" && track?.streamIndex === selected;
videoElement.textTracks[index].mode = showing ? "showing" : "disabled";
}
}
createEffect(() => {
videoExtras();
selectedSubtitle();
queueMicrotask(applySubtitleModes);
});
function subtitleSelectionForLanguage(extras: LoadedVideoExtras | undefined): "off" | number {
const preferences = options.languages
.split(",")
.map((language) => language.trim().toLowerCase())
.filter(Boolean);
if (!extras || preferences.length === 0) return "off";
let selected: number | undefined;
let selectedPreference = Number.POSITIVE_INFINITY;
for (const track of extras.subtitleTracks) {
const language = track.language?.trim().toLowerCase();
if (!language) continue;
const preference = preferences.indexOf(language);
if (preference >= 0 && preference < selectedPreference) {
selected = track.streamIndex;
selectedPreference = preference;
}
}
return selected ?? "off";
}
createEffect(() => {
const track = playerState.currentTrack;
if (!track) {
lastAutoSelectedPath = undefined;
setSelectedSubtitle("off");
return;
}
//auto-select once per track change, and only against that track's own extras
const extras = videoExtras();
if (!extras || extrasLoadedForPath !== track.path || lastAutoSelectedPath === track.path) return;
lastAutoSelectedPath = track.path;
setSelectedSubtitle(subtitleSelectionForLanguage(extras));
});
createEffect(() => {
if (!isVideoTrack()) closeVideoTheater();
});
function handlePlayerError(e: unknown) {
if (e instanceof Error && e.name === "AbortError") return;
@@ -299,9 +416,7 @@ const App: Component = () => {
settings.container,
isVideo ? options.videoSettings.videoCodec : VideoCodec.none,
settings.audioCodec,
isVideo
? videoStreamInfo(playerState.currentTrack?.metadata, options.videoSettings.videoBitrate)
: undefined,
isVideo ? videoStreamInfo(playerState.currentTrack?.metadata, options.videoSettings.videoBitrate) : undefined,
);
if (!MediaSource.isTypeSupported(mimeType)) return null;
@@ -471,18 +586,22 @@ const App: Component = () => {
let ignoreSeek = false;
// Play a track from the playlist
async function playTrack(track: PlaylistItem, seekTo?: number) {
cancelTranscode(activeTranscodeId);
activeTranscodeId = undefined;
const generation = ++playGeneration;
try {
const srcResult = await getSrc(track.path, track.metadata, options, isOffline(), seekTo || 0);
//undefined means offline and not cached, getSrc already explained that via toast
if (!srcResult) return;
const { src, isVideo } = srcResult;
const { src, isVideo, transcodeId } = srcResult;
if (generation !== playGeneration) {
cancelTranscode(transcodeId);
//superseded while getSrc ran: for a synced file it already minted a blob url that
//nothing will ever assign to the element, so revoke it instead of leaking it
if (src.startsWith("blob:")) URL.revokeObjectURL(src);
return;
}
activeTranscodeId = transcodeId;
//Cleanup
if (videoElement.src.startsWith("blob:")) URL.revokeObjectURL(videoElement.src);
if (mediaSource) {
@@ -506,6 +625,7 @@ const App: Component = () => {
? await getMSEUrl(src, seekTo || 0, isVideo === true)
: null;
if (generation !== playGeneration) {
cancelTranscode(transcodeId);
//superseded while getMSEUrl ran: it created a detached MediaSource + object URL, but its
//sourceopen handler never fired (we never set videoElement.src), so no fetch/transcode was
//started and there's nothing to abort. Just revoke the URL and drop the now-stale module
@@ -516,10 +636,14 @@ const App: Component = () => {
}
return;
}
setVideoShown(false);
setIsVideoTrack(false);
videoElement.removeAttribute("src");
videoElement.srcObject = null;
videoElement.src = mseURL || src;
setIsVideoTrack(
track.metadata.videoCodec !== undefined &&
(!options.enableTranscoding || options.videoSettings.videoCodec !== VideoCodec.none),
);
if (mseURL && seekTo) {
ignoreSeek = true;
videoElement.currentTime = seekTo;
@@ -722,9 +846,6 @@ const App: Component = () => {
}
ignoreSeek = false;
});
videoElement.addEventListener("loadedmetadata", () => {
setVideoShown(videoElement.videoWidth !== 0 || videoElement.videoHeight !== 0);
});
videoElement.addEventListener("click", (e) => {
e.preventDefault();
togglePlay();
@@ -734,12 +855,20 @@ const App: Component = () => {
videoElement.controls = false;
}
});
//Firefox can leave a media fetch alive while tearing down a tab. Remove the source and abort the
//MSE fetch synchronously during pagehide so the server gets a chance to cancel its ffmpeg process.
window.addEventListener("pagehide", stopPlaying);
window.addEventListener("hashchange", () => {
setCurrentDir(decodePath(window.location.hash.slice(1)));
});
document.addEventListener("keydown", (e) => {
if (e.key === "Escape" && videoTheater()) {
closeVideoTheater();
e.preventDefault();
return;
}
const activeElementTag = document.activeElement?.tagName.toLowerCase();
if (activeElementTag === "input") {
return;
@@ -863,7 +992,10 @@ const App: Component = () => {
videoElement={(el) => {
videoElement = el;
}}
videoShown={videoShown}
isVideoTrack={isVideoTrack}
videoTheater={videoTheater}
setVideoTheater={setVideoTheater}
videoExtras={videoExtras}
options={options}
fetchFiles={fetchFiles}
isOffline={isOffline}
@@ -876,6 +1008,7 @@ const App: Component = () => {
togglePlay={togglePlay}
playPrevious={playPrevious}
playNext={playNext}
stopPlaying={stopPlaying}
isShuffled={isShuffled}
setShuffled={setShuffled}
isRepeated={isRepeated}
@@ -883,6 +1016,12 @@ const App: Component = () => {
videoElement={videoElement}
needMSESeek={(seekTo: number) => needMSESeek(seekTo)}
seekTo={(seekTo: number) => playerState.currentTrack && playTrack(playerState.currentTrack, seekTo)}
isVideoTrack={isVideoTrack}
videoExtras={videoExtras}
selectedSubtitle={selectedSubtitle}
setSelectedSubtitle={setSelectedSubtitle}
videoTheater={videoTheater}
toggleVideoTheater={toggleVideoTheater}
options={options}
/>
Mwebclient/src/components/OptionsModal.tsx
@@ -3,19 +3,19 @@ import { type AppOptions, AUDIO_BITRATES, FileDisplay, StreamingMode, VIDEO_BITR
import "./OptionsModal.css";
import "./Dialog.css";
import "toastify-js/src/toastify.css";
import { AudioCodec, MediaContainer, VideoCodec, VideoEncodingSetting } from "music-server-shared/types";
import { type AudioCodec, MediaContainer, VideoCodec, VideoEncodingSetting } from "music-server-shared/types";
import { audioCodecHasBitrate } from "music-server-shared/utils";
import { createStore } from "solid-js/store";
import {
type Combination,
checkContainerSupport,
checkSupport,
type Combination,
containerAudioCodecs,
containerVideoCodecs,
hasMediaSource,
playableSettings,
selectableVideoCodecs,
type Support,
selectableVideoCodecs,
} from "../capabilities";
import { handleUnauthorized, toast } from "../utils";
@@ -166,14 +166,25 @@ export default function OptionsModal(props: OptionsModalProps) {
/>
</div>
<div>
<label for="language-input">Language preference:</label>
<label for="language-input">Subtitle language preference:</label>
<input
id="language-input"
type="text"
placeholder="en,de"
title="Two-letter language codes (ISO 639-1), separated by commas, most preferred first. Three-letter codes like 'eng' do not match."
title="Two-letter language codes (ISO 639-1), separated by commas, most preferred first. A matching subtitle is selected by default; no match leaves subtitles off."
value={newOptions.languages}
onChange={(e) => setNewOptions("languages", e.target.value)}
/>
</div>
<div>
<label for="audio-language-input">Audio language preference:</label>
<input
id="audio-language-input"
type="text"
placeholder="en,de"
title="Two-letter language codes (ISO 639-1), separated by commas, most preferred first. A matching audio track is selected during transcoding; no match uses the source default track."
value={newOptions.audioLanguagePreference}
onChange={(e) => setNewOptions("audioLanguagePreference", e.target.value)}
disabled={!newOptions.enableTranscoding}
/>
</div>
Mwebclient/src/components/PlayerControls.css
@@ -1,4 +1,7 @@
.player-controls {
position: relative;
z-index: 31;
background-color: var(--color-a);
padding: 1rem;
border-top: 1px solid var(--color-c);
display: flex;
@@ -78,6 +81,18 @@
background-color: #4a6da7;
z-index: 1;
}
& > .chapter-marker {
position: absolute;
top: -3px;
width: 5px;
height: calc(100% + 6px);
padding: 0;
border: 1px solid var(--color-b);
background-color: var(--color-c);
box-shadow: 0 0 0 1px var(--text-color);
z-index: 3;
transform: translateX(-2px);
}
& > .progress-handle {
position: absolute;
top: 50%;
Mwebclient/src/components/PlayerControls.tsx
@@ -1,7 +1,7 @@
import "./PlayerControls.css";
import { type Accessor, createEffect, createSignal, For, onMount, type Setter, Show } from "solid-js";
import type { SetStoreFunction } from "solid-js/store";
import type { AppOptions, PlayerState } from "../types";
import type { AppOptions, LoadedVideoExtras, PlayerState } from "../types";
import { formatFilename, formatTime, LocalStorageValues } from "../utils";
import { LoadingSpinner } from "./LoadingScreen";
import {
@@ -15,14 +15,19 @@ import {
PreviousTrackSVG,
RepeatSVG,
ShuffleSVG,
SquareSVG,
TheaterSVG,
UpSVG,
VolumeSVG,
} from "./SVG";
type SubtitleSelection = "off" | number;
interface PlayerControlsProps {
playerState: PlayerState;
setPlayerState: SetStoreFunction<PlayerState>;
togglePlay: () => void;
stopPlaying: () => void;
playPrevious: () => void;
playNext: () => void;
isShuffled: Accessor<boolean>;
@@ -33,6 +38,12 @@ interface PlayerControlsProps {
seekTo: (seekTo: number) => void;
options: AppOptions;
videoElement: HTMLVideoElement;
isVideoTrack: Accessor<boolean>;
videoExtras: Accessor<LoadedVideoExtras | undefined>;
selectedSubtitle: Accessor<SubtitleSelection>;
setSelectedSubtitle: Setter<SubtitleSelection>;
videoTheater: Accessor<boolean>;
toggleVideoTheater: () => void;
}
export default function PlayerControls(props: PlayerControlsProps) {
@@ -71,6 +82,20 @@ export default function PlayerControls(props: PlayerControlsProps) {
}
}
function seekToChapter(start: number) {
const duration = props.playerState.currentTrack?.metadata.duration;
if (!duration || !Number.isFinite(start)) return;
seekTo(Math.max(0, Math.min(1, start / duration)));
}
//percentage position of a chapter marker; bad metadata can carry chapters beyond the reported
//duration, so clamp instead of letting the marker overflow the progress bar
function chapterMarkerPosition(start: number): number {
const duration = props.playerState.currentTrack?.metadata.duration;
if (!duration || !Number.isFinite(start)) return 0;
return Math.max(0, Math.min(100, (start / duration) * 100));
}
const [isDragging, setIsDragging] = createSignal(false);
const [dragPos, setDragPos] = createSignal(0);
createEffect(() => {
@@ -184,6 +209,24 @@ export default function PlayerControls(props: PlayerControlsProps) {
bitrateDisplay(props.playerState.currentTrack?.metadata.videoBitrate, 1000 * 1000),
" mbps",
)}
{MetadataEntry(
"Video dimensions",
props.playerState.currentTrack?.metadata.videoWidth &&
props.playerState.currentTrack?.metadata.videoHeight
? `${props.playerState.currentTrack.metadata.videoWidth} × ${props.playerState.currentTrack.metadata.videoHeight}`
: undefined,
)}
{MetadataEntry("Frame rate", props.playerState.currentTrack?.metadata.frameRate, " fps")}
{MetadataEntry(
"Interlaced",
props.playerState.currentTrack?.metadata.interlaced === undefined
? undefined
: props.playerState.currentTrack.metadata.interlaced
? "yes"
: "no",
)}
{MetadataEntry("Subtitle tracks", props.videoExtras()?.subtitleTracks.length || undefined)}
{MetadataEntry("Chapters", props.videoExtras()?.chapters.length || undefined)}
</div>
</div>
</dialog>
@@ -221,6 +264,22 @@ export default function PlayerControls(props: PlayerControlsProps) {
height: `${props.options.progressBarHeight}px`,
}}
>
<For each={props.isVideoTrack() ? props.videoExtras()?.chapters || [] : []}>
{(chapter, index) => (
<button
class="chapter-marker"
title={chapter.title || `Chapter ${index() + 1}`}
aria-label={`Jump to ${chapter.title || `chapter ${index() + 1}`}`}
style={{
left: `${chapterMarkerPosition(chapter.start)}%`,
}}
onClick={(event) => {
event.stopPropagation();
seekToChapter(chapter.start);
}}
/>
)}
</For>
<For each={props.playerState.buffered}>
{(bufferRange) => {
return (
@@ -281,6 +340,9 @@ export default function PlayerControls(props: PlayerControlsProps) {
<button title="Next track" disabled={controlsDisabled()} onClick={props.playNext}>
<NextTrackSVG disabled={controlsDisabled} />
</button>
<button title="Stop playback" disabled={controlsDisabled()} onClick={props.stopPlaying}>
<SquareSVG disabled={controlsDisabled} />
</button>
<button title="Repeat Playlist" onClick={() => props.setRepeated(!props.isRepeated())}>
<RepeatSVG disabled={() => !props.isRepeated()} />
@@ -311,15 +373,60 @@ export default function PlayerControls(props: PlayerControlsProps) {
<option value={3}>3x</option>
</select>
</div>
<button
title="Toggle Fullscreen"
onClick={() => {
props.videoElement.controls = true;
props.videoElement.requestFullscreen();
}}
>
<FullscreenSVG />
</button>
<Show when={props.isVideoTrack() && props.videoExtras()?.subtitleTracks.length}>
<select
title="Subtitle track"
value={props.selectedSubtitle() === "off" ? "off" : props.selectedSubtitle().toString()}
onchange={(e) => {
const value = e.currentTarget.value;
props.setSelectedSubtitle(value === "off" ? value : Number.parseInt(value, 10));
}}
>
<option value="off">Subtitles: off</option>
<For each={props.videoExtras()?.subtitleTracks || []}>
{(track, index) => (
<option value={track.streamIndex}>{track.title || track.language || `Track ${index() + 1}`}</option>
)}
</For>
</select>
</Show>
<Show when={props.isVideoTrack() && props.videoExtras()?.chapters.length}>
<select
title="Jump to chapter"
value=""
onchange={(e) => {
seekToChapter(Number.parseFloat(e.currentTarget.value));
e.currentTarget.value = "";
}}
>
<option value="">Chapters</option>
<For each={props.videoExtras()?.chapters || []}>
{(chapter, index) => (
<option value={chapter.start}>{chapter.title || `Chapter ${index() + 1}`}</option>
)}
</For>
</select>
</Show>
<Show when={props.isVideoTrack()}>
<button
title={props.videoTheater() ? "Close theater mode" : "Open theater mode"}
aria-label={props.videoTheater() ? "Close theater mode" : "Open theater mode"}
onClick={props.toggleVideoTheater}
>
<TheaterSVG />
</button>
<button
title="Toggle video fullscreen"
onClick={() => {
props.videoElement.controls = true;
void props.videoElement.requestFullscreen();
}}
>
<FullscreenSVG />
</button>
</Show>
</div>
</Show>
</div>
Mwebclient/src/components/Playlist.css
@@ -110,3 +110,55 @@
flex-grow: 1;
min-height: 2rem;
}
.video-stage {
position: relative;
flex: 0 1 auto;
width: 100%;
aspect-ratio: 16 / 9;
max-height: min(45%, 60vh);
min-height: 140px;
border-top: 1px solid var(--color-c);
background: #000;
display: flex;
align-items: center;
justify-content: center;
z-index: 2;
& > .video-element {
display: block;
width: 100%;
height: 100%;
object-fit: contain;
background: #000;
}
& > .video-theater-close {
position: absolute;
top: 0.75rem;
right: 0.75rem;
z-index: 3;
padding: 0;
background: rgba(0, 0, 0, 0.6);
&:hover {
background: rgba(0, 0, 0, 0.85);
}
}
&.theater {
position: fixed;
inset: 0;
width: auto;
height: auto;
max-height: none;
aspect-ratio: auto;
padding: 1rem 1rem 8rem;
background: rgba(0, 0, 0, 0.96);
z-index: 30;
& > .video-element {
max-width: 100%;
max-height: 100%;
}
}
}
Mwebclient/src/components/Playlist.tsx
@@ -5,6 +5,7 @@ import {
createResource,
createSelector,
createSignal,
For,
on,
onMount,
type ResourceActions,
@@ -14,7 +15,7 @@ import {
import "./Playlist.css";
import type { FlatFileListing } from "../App";
import { download, getCover, savePlaylist, storedFilesLive } from "../offline";
import type { AppOptions, PlayerState, PlaylistItem } from "../types";
import type { AppOptions, LoadedVideoExtras, PlayerState, PlaylistItem } from "../types";
import { formatFilename, formatTime, handleUnauthorized, LocalStorageValues, toast } from "../utils";
import { DraggableList } from "./DraggableList";
import { LoadingScreen } from "./LoadingScreen";
@@ -28,7 +29,10 @@ interface PlaylistProps {
playTrack: (item: PlaylistItem) => void;
playNext: () => void;
videoElement: (el: HTMLVideoElement) => void;
videoShown: Accessor<boolean>;
isVideoTrack: Accessor<boolean>;
videoTheater: Accessor<boolean>;
setVideoTheater: Setter<boolean>;
videoExtras: Accessor<LoadedVideoExtras | undefined>;
options: AppOptions;
fetchFiles: ResourceActions<FlatFileListing>;
isOffline: Accessor<boolean>;
@@ -241,19 +245,54 @@ export default function Playlist(props: PlaylistProps) {
</DraggableList>
<div
class="cover"
classList={{
cover: !props.isVideoTrack(),
"video-stage": props.isVideoTrack(),
theater: props.videoTheater(),
}}
style={{
display:
props.videoShown() || (!props.options.hidePlaylistCover && (cover.loading || cover())) ? undefined : "none",
props.isVideoTrack() || (!props.options.hidePlaylistCover && (cover.loading || cover()))
? undefined
: "none",
}}
>
<Show when={!props.videoShown()}>
<Show when={!props.isVideoTrack()}>
<Show when={cover.loading}>
<LoadingScreen />
</Show>
<img src={cover()} alt="" />
</Show>
<video ref={props.videoElement} preload="auto" playsinline />
<video
class="video-element"
ref={props.videoElement}
preload="auto"
playsinline
poster={props.options.hidePlaylistCover ? undefined : cover()}
>
<For each={props.videoExtras()?.subtitleTracks || []}>
{(track, index) => (
<track
kind="subtitles"
src={props.videoExtras()?.subtitleSources[track.streamIndex]}
srclang={track.language || undefined}
label={track.title || track.language || `Track ${index() + 1}`}
/>
)}
</For>
</video>
<Show when={props.isVideoTrack() && props.videoTheater()}>
<button
class="video-theater-close"
title="Close theater mode"
onClick={(event) => {
event.stopPropagation();
props.setVideoTheater(false);
}}
>
<ClearSVG />
</button>
</Show>
</div>
</div>
);
Mwebclient/src/components/SVG.tsx
@@ -235,6 +235,22 @@ export function FullscreenSVG() {
);
}
export function TheaterSVG() {
const [hovered, setHovered] = createSignal();
return (
<svg
xmlns="http://www.w3.org/2000/svg"
width="24"
height="24"
viewBox="0 0 24 24"
onMouseEnter={() => setHovered(true)}
onMouseLeave={() => setHovered(false)}
>
<path fill={hovered() ? highlightColor : defaultColor} fill-rule="evenodd" d="M1 4h22v16H1zm3 3v10h16V7z" />
</svg>
);
}
export function ClearSVG() {
const [hovered, setHovered] = createSignal();
return (
@@ -516,6 +532,23 @@ export function StopSVG() {
);
}
export function SquareSVG(props: Partial<StateProps> = {}) {
const [hovered, setHovered] = createSignal();
return (
<svg
xmlns="http://www.w3.org/2000/svg"
width="24"
height="24"
viewBox="0 0 24 24"
onMouseEnter={() => setHovered(true)}
onMouseLeave={() => setHovered(false)}
>
<path fill={hovered() && !(props?.disabled?.() ?? false) ? highlightColor : defaultColor} d="M4 4h16v16H4z" />
</svg>
);
}
export function FilterSVG() {
const [hovered, setHovered] = createSignal();
Mwebclient/src/offline.ts
@@ -1,24 +1,37 @@
import Dexie, { liveQuery } from "dexie";
import { AsyncSemaphore } from "music-server-shared/semaphore";
import type {
Chapter,
Directory,
DirectoryWithStatus,
FileListing,
FileListingWithStatus,
MediaFile,
Metadata,
SubtitleTrack,
VideoExtras,
} from "music-server-shared/types";
import { AsyncSemaphore } from "music-server-shared/semaphore";
import { audioCodecHasBitrate, encodePath } from "music-server-shared/utils";
import { batch, from, type ResourceActions, untrack } from "solid-js";
import { createStore } from "solid-js/store";
import type { FlatFileListing } from "./App";
import { type AppOptions, type Playlist, StreamingMode } from "./types";
import { type AppOptions, type LoadedVideoExtras, type Playlist, StreamingMode } from "./types";
import { basename, dirname, handleUnauthorized, joinPath, parentPaths, toast, unproxy } from "./utils";
interface StoredSubtitle extends SubtitleTrack {
data: Blob;
}
interface StoredVideoExtras {
chapters: Chapter[];
subtitles: StoredSubtitle[];
}
interface StoredFile {
path: string;
data: Blob;
metadata: Metadata;
videoExtras?: StoredVideoExtras;
}
interface StoredCover {
@@ -334,15 +347,16 @@ export async function getCover(
if (saveToDb) db.covers.add({ path: dir, data: null }).catch(() => {});
//only update a row that already exists, otherwise merely browsing a directory would
//persist its cover - note the await, without it the promise is always truthy
else if (forceRefresh && (await db.covers.get(dir)) !== undefined)
if (!saveToDb && forceRefresh && (await db.covers.get(dir)) !== undefined)
db.covers.put({ path: dir, data: null }).catch(() => {});
return undefined;
}
if (!response.ok) return undefined;
const blob = await response.blob();
coverCache.set(dir, { data: blob });
if (saveToDb) db.covers.add({ path: dir, data: blob }).catch(() => {});
//see above: the await is what makes this an "already stored" check
else if (forceRefresh && (await db.covers.get(dir)) !== undefined)
if (!saveToDb && forceRefresh && (await db.covers.get(dir)) !== undefined)
db.covers.put({ path: dir, data: blob }).catch(() => {});
return URL.createObjectURL(blob);
} catch {
@@ -350,6 +364,78 @@ export async function getCover(
}
}
const emptyVideoExtras = (): LoadedVideoExtras => ({ subtitleTracks: [], chapters: [], subtitleSources: {} });
function subtitleUrl(path: string, streamIndex: number): string {
return `/subtitles/${encodePath(path)}?track=${streamIndex}`;
}
function videoInfoUrl(path: string): string {
return `/video-info/${encodePath(path)}`;
}
function checkVideoExtrasResponse(response: Response | undefined, path: string): asserts response is Response {
if (handleUnauthorized(response)) throw new Error("Unauthorized while fetching video extras");
if (!response?.ok)
throw new Error(`Fetching video extras for ${path} failed: ${response?.statusText || "server unavailable"}`);
}
export async function loadVideoExtras(path: string, offline: boolean): Promise<LoadedVideoExtras> {
const stored = await db.files.get(path);
if (stored?.videoExtras) {
const subtitleSources: Record<number, string> = {};
const subtitleTracks: SubtitleTrack[] = [];
for (const subtitle of stored.videoExtras.subtitles) {
const { data, ...track } = subtitle;
subtitleTracks.push(track);
subtitleSources[track.streamIndex] = URL.createObjectURL(data);
}
return { subtitleTracks, chapters: stored.videoExtras.chapters, subtitleSources };
}
if (offline) return emptyVideoExtras();
const response = await fetch(videoInfoUrl(path), { credentials: "same-origin" }).catch(() => undefined);
checkVideoExtrasResponse(response, path);
const extras = (await response.json()) as VideoExtras;
const subtitleSources: Record<number, string> = {};
for (const track of extras.subtitleTracks) subtitleSources[track.streamIndex] = subtitleUrl(path, track.streamIndex);
return { ...extras, subtitleSources };
}
export function revokeVideoExtras(extras: LoadedVideoExtras | undefined): void {
if (!extras) return;
for (const source of Object.values(extras.subtitleSources)) {
if (source.startsWith("blob:")) URL.revokeObjectURL(source);
}
}
//Unlike normal playback metadata, sync treats extras as part of the file request: a failed chapter probe
//or a failed subtitle conversion rejects before the media blob is committed to IndexedDB.
async function downloadVideoExtras(path: string, signal: AbortSignal): Promise<StoredVideoExtras> {
const response = await fetch(videoInfoUrl(path), { credentials: "same-origin", signal }).catch((error) => {
throw error;
});
checkVideoExtrasResponse(response, path);
const extras = (await response.json()) as VideoExtras;
const subtitles = await Promise.all(
extras.subtitleTracks.map(async (track) => {
const subtitleResponse = await fetch(subtitleUrl(path, track.streamIndex), {
credentials: "same-origin",
signal,
}).catch((error) => {
throw error;
});
if (handleUnauthorized(subtitleResponse) || !subtitleResponse.ok) {
throw new Error(
`Fetching subtitle track ${track.streamIndex} for ${path} failed: ${subtitleResponse?.statusText || "server unavailable"}`,
);
}
return { ...track, data: await subtitleResponse.blob() };
}),
);
return { chapters: extras.chapters, subtitles };
}
//returns an object url to the blob if synced, otherwise the server url
//with forceUrl it always resolves, without it returns undefined for an uncached file in offline mode
export async function getSrc(
@@ -358,23 +444,8 @@ export async function getSrc(
options: AppOptions,
offline: boolean,
seekTo: number,
forceUrl: true,
): Promise<{ src: string; isVideo?: boolean }>;
export async function getSrc(
path: string,
metadata: Metadata,
options: AppOptions,
offline: boolean,
seekTo: number,
): Promise<{ src: string; isVideo?: boolean } | undefined>;
export async function getSrc(
path: string,
metadata: Metadata,
options: AppOptions,
offline: boolean,
seekTo: number,
forceUrl?: boolean,
) {
forceUrl = false,
): Promise<{ src: string; isVideo?: boolean; transcodeId?: string } | undefined> {
if (!forceUrl) {
const dbResult = await db.files.get(path);
if (dbResult) return { src: URL.createObjectURL(dbResult.data), isVideo: undefined };
@@ -390,11 +461,13 @@ export async function getSrc(
const disableChunkedTranscoding = settings.streamingMode === StreamingMode.buffered;
const audioBitrate = audioCodecHasBitrate(settings.audioCodec) ? settings.audioBitrate : undefined;
const transcodeId = crypto.randomUUID();
const query = new URLSearchParams({
disableChunkedTranscoding: String(disableChunkedTranscoding),
container: settings.container,
audioCodec: settings.audioCodec,
languages: options.languages,
audioLanguagePreference: options.audioLanguagePreference,
transcodeId,
});
if (isVideo) {
query.set("videoCodec", options.videoSettings.videoCodec);
@@ -407,6 +480,7 @@ export async function getSrc(
return {
src: `${joinPath("transcode", encodePath(path))}?${query}`,
isVideo: isVideo,
transcodeId,
};
}
return { src: joinPath("download", encodePath(path)), isVideo: undefined };
@@ -474,7 +548,7 @@ export async function deleteFile(
coversToDelete.add(fullPath);
for (const filePath of toDelete) {
for (const dir of parentPaths(filePath)) {
//stop at the deleted root itself, parents of it may still hold other synced files
//stop at the deleted root itself, parents of it may still hold other synced files
if (dir === fullPath) break;
coversToDelete.add(dir);
}
@@ -540,7 +614,9 @@ async function downloadFile(
//finally can't hand out a permit that was never acquired (which would leak a concurrency slot)
await downloadSemaphore.acquire(controller.signal);
try {
const res = await fetch((await getSrc(path, item.metadata, options, false, 0, true)).src, {
const source = await getSrc(path, item.metadata, options, false, 0, true);
if (!source) throw new Error("Unable to create a download URL");
const res = await fetch(source.src, {
signal: controller.signal,
credentials: "same-origin",
});
@@ -568,12 +644,21 @@ async function downloadFile(
setDownloadStatusMap(path, { status: received, abort: abortDownload });
}
const blob = new Blob(chunks, { type: res.headers.get("content-type") ?? undefined });
const isVideo = item.metadata.videoCodec !== undefined;
await db.files.put({
//A video sync is transactional from the user's perspective: subtitle conversion and chapter
//extraction complete before the file row becomes Synced. A failed extra therefore leaves a
//previous redownload untouched and never exposes a half-synced video offline.
const videoExtras = isVideo ? await downloadVideoExtras(path, controller.signal) : undefined;
if (controller.signal.aborted) return;
const storedFile: StoredFile = {
path,
data: blob,
metadata: unproxy(item.metadata),
});
...(videoExtras ? { videoExtras } : {}),
};
await db.files.put(storedFile);
updateFileTree(path, item, true);
stopProgress(path, "Synced");
@@ -688,6 +773,7 @@ export const storageStatsLive = from(
let fileBytes = 0;
await db.files.each((file) => {
fileBytes += file.data.size;
for (const subtitle of file.videoExtras?.subtitles || []) fileBytes += subtitle.data.size;
});
return {
fileCount: await db.files.count(),
Mwebclient/src/style.css
@@ -157,4 +157,7 @@ input {
max-height: 100%;
max-width: 100%;
}
& > .video-element {
display: none;
}
}
Mwebclient/src/types.ts
@@ -4,13 +4,20 @@ import type {
MediaFile,
VideoCodec,
VideoEncodingSetting,
VideoExtras,
} from "music-server-shared/types";
export type { Chapter, SubtitleTrack, VideoExtras } from "music-server-shared/types";
export interface PlaylistItem extends MediaFile {
id: string; //used for determining currently playing item
path: string;
}
export interface LoadedVideoExtras extends VideoExtras {
subtitleSources: Record<number, string>;
}
export interface Playlist {
name: string;
items: Omit<PlaylistItem, "id">[];
@@ -36,8 +43,8 @@ export enum StreamingMode {
//probably support doesn't vary over bitrates/levels (tested for chromium/linux) but is cheap so do it anyway
export const AUDIO_BITRATES = [32_000, 48_000, 64_000, 96_000, 128_000, 192_000, 320_000];
export const VIDEO_BITRATES = [
210_000, 420_000, 720_000, 1_500_000, 3_000_000, 4_000_000, 6_000_000, 8_000_000, 10_000_000, 15_000_000,
20_000_000, 30_000_000, 40_000_000,
210_000, 420_000, 720_000, 1_500_000, 3_000_000, 4_000_000, 6_000_000, 8_000_000, 10_000_000, 15_000_000, 20_000_000,
30_000_000, 40_000_000,
];
export enum FileDisplay {
@@ -49,6 +56,7 @@ export enum FileDisplay {
export interface AppOptions {
enableTranscoding: boolean;
languages: string;
audioLanguagePreference: string;
maxTranscodes: number;
maxDownloads: number;
transcodeCovers: boolean;
Mwebclient/tsconfig.json
@@ -9,6 +9,7 @@
"jsxImportSource": "solid-js",
"types": ["vite/client", "bun-types", "vite-plugin-pwa/client"],
"noEmit": true,
"allowImportingTsExtensions": true,
"isolatedModules": true,
"strict": true,
Mwebclient/vite.common.config.ts
@@ -6,12 +6,7 @@ export const pwaOptions: Partial<VitePWAOptions> = {
workbox: {
globPatterns: ["**/*.{js,css,html,woff2}"],
navigateFallback: undefined,
runtimeCaching: [
{
urlPattern: () => true,
handler: "NetworkOnly",
},
],
//Leave media requests outside Workbox so native Range requests and cancellation are handled directly.
},
manifest: {
id: "/",
Mwebclient/vite.compile.config.ts
@@ -1,7 +1,7 @@
import { defineConfig } from "vite";
import { VitePWA } from "vite-plugin-pwa";
import solidPlugin from "vite-plugin-solid";
import { hashPlugin, pwaOptions } from "./vite.common.config";
import { hashPlugin, pwaOptions } from "./vite.common.config.ts";
export default defineConfig({
css: {
Mwebclient/vite.watch.config.ts
@@ -1,7 +1,7 @@
import { defineConfig } from "vite";
import { VitePWA } from "vite-plugin-pwa";
import solidPlugin from "vite-plugin-solid";
import { hashPlugin, pwaOptions } from "./vite.common.config";
import { hashPlugin, pwaOptions } from "./vite.common.config.ts";
export default defineConfig({
plugins: [solidPlugin(), VitePWA(pwaOptions), hashPlugin],