correctly build codec string, only show actually supported containers and codecs
MREADME.md
@@ -49,8 +49,11 @@ You can configure the backend with the following environment variables:
- **Multiple users**: Right now everything besides the actual file hosting is stored on the server. When syncing is implemented, it will only be single storage. Permissison systems are also not planned. If distinct permissions/syncs are absouletely required, hosting multiple instances is an option
## FAQ
- What is the "Disable chunked transcoding" option for?
- By default the backend sends the transcoded media as it progresses, which uses "chunked transfer", i.e. without the usual content-length of normal HTTP requests. Some browser (safari cough cough) seem to be very selective about the sources of \<audio\> elements, and choke on sources that are transferred via this method. This option forces the server to transcode the whole file before sending it, improving compatibility with these browser, at the cost of increased latency.
- What do the "Streaming mode" options do?
- Each transcoding profile (video and music) picks how its transcode is delivered to the player.
- `mse` feeds the transcode into a SourceBuffer via Media Source Extensions. This is the only mode that can seek inside a transcode, because a seek simply restarts ffmpeg at an offset. In exchange it only works for containers and codecs that have an MSE byte stream format, which the options menu greys out the rest based on.
- `progressive` hands the transcode straight to the media element as it is produced. It plays whatever the browser can decode - notably video in webm, which MSE cannot handle - but cannot seek.
- `buffered` makes the backend transcode the whole file before sending anything. By default the backend sends the transcoded media as it progresses, which uses "chunked transfer", i.e. without the usual content-length of normal HTTP requests. Some browser (safari cough cough) seem to be very selective about the sources of \<audio\> elements, and choke on sources that are transferred via this method. This mode improves compatibility with those browsers, at the cost of a significant startup delay. It does not enable seeking - the transcode endpoint serves no range requests either way.
- The playback stops automatically at some point on my mobile device when I put the browser in the background or turning my screen off
- In my testing at least mobile firefox behaves this way, with no apparent solution. It seems to stop after changing the sources of the audio element at some point. Either use a browser that doesn't behave like this (Chrome on Android seems to work), or wrap the PWA in a native app (e.g. https://www.pwabuilder.com/)
- I'm getting "Quota exceeded" errors when trying to sync files *or*
@@ -58,8 +61,6 @@ You can configure the backend with the following environment variables:
- By default websites aren't granted infinite storage or have persisten data. The frontend requests these permissions by calling `navigator.storage.persist()` on startup, which causes some browsers to show a permissions popup (e.g. firefox), while others just seem to ignore this request and use a heuristic like interaction amount and bookmarking instead. So if you have the error described above you either are running out of actual disk storage, or you need to check what heuristics your browser uses and trigger them.
## Known issues
- Range requests are not correctly handled in Bun right now, and e.g. doesn't send the total byte length correctly. I worked around this by sending a chunked transfer-encoded response, and setting the range and content-length header manually, which seems to work with the browsers I tested, but still might cause issues in some cases
- Some of the transcoded formats behave weirdly in some browsers, e.g. safari seems to dislike ogg containers, and chrome has seeking issues with some webm transcoded files. If you have any code suggestions to improve the handling, feel free to open a PR at https://gitlab.com/Konata390/pico-pixel-player
- Some files (especially mp3) have weird metadata issues (e.g. a padding gap between the ID3 tag and the first frame sync) that stop the `file-type` library from identifying them. The server now falls back to `mediainfo` for MIME detection in that case, so these files list and stream normally without any intervention. If a file still fails to be listed or played, running `ffmpeg -i INPUTFILE -c copy OUTPUTFILE` on it to rewrite the container usually fixes it
Mserver/src/ffmpeg.ts
@@ -1,5 +1,5 @@
import { AudioCodec, MediaContainer, type Metadata, VideoCodec, VideoEncodingSetting } from "music-server-shared/types";
import { createRFC6381String } from "music-server-shared/utils";
import { createRFC6381String, ladderMaxWidth, type VideoStreamInfo } from "music-server-shared/utils";
const KEYFRAME_INTERVAL_SECONDS = 5;
@@ -12,16 +12,9 @@ const AUDIO_ENCODERS: Record<AudioCodec, string> = {
[AudioCodec.mp3]: "libmp3lame",
[AudioCodec.opus]: "libopus",
[AudioCodec.vorbis]: "libvorbis",
[AudioCodec.flac]: "flac",
};
const RESOLUTION_LADDER: { upToBitrate: number; width: number }[] = [
{ upToBitrate: 400_000, width: 640 },
{ upToBitrate: 800_000, width: 854 },
{ upToBitrate: 2_500_000, width: 1280 },
{ upToBitrate: 5_000_000, width: 1920 },
{ upToBitrate: 12_000_000, width: 2560 },
];
//x264 and x265 share the same preset names
const X26X_PRESETS: Record<VideoEncodingSetting, string> = {
[VideoEncodingSetting.veryfast]: "veryfast",
@@ -97,7 +90,7 @@ function videoArgsFor(
throw new Error(`Unsupported video codec: ${requestedVideoCodec}`);
}
const maxWidth = RESOLUTION_LADDER.find((rung) => videoBitrate <= rung.upToBitrate)?.width;
const maxWidth = ladderMaxWidth(videoBitrate);
args.push(
"-b:v",
videoBitrate.toString(), //TODO: better bitrate selection
@@ -118,7 +111,8 @@ function videoArgsFor(
export async function convertWithFFmpeg(
filePath: string,
audioBitrate: number,
//undefined means no -b:a at all, which is the only setting that means anything for a lossless codec
audioBitrate: number | undefined,
videoBitrate: number,
container: MediaContainer,
requestedAudioCodec: AudioCodec,
@@ -128,7 +122,17 @@ export async function convertWithFFmpeg(
languages: string,
metadata: Metadata,
) {
const mimeType = createRFC6381String(container, requestedVideoCodec, requestedAudioCodec);
const videoStreamInfo: VideoStreamInfo | undefined =
metadata.videoWidth && metadata.videoHeight && metadata.frameRate
? {
sourceWidth: metadata.videoWidth,
sourceHeight: metadata.videoHeight,
frameRate: metadata.frameRate,
interlaced: metadata.interlaced === true,
bitrate: videoBitrate,
}
: undefined;
const mimeType = createRFC6381String(container, requestedVideoCodec, requestedAudioCodec, videoStreamInfo);
const audioEncoder = lookup(AUDIO_ENCODERS, requestedAudioCodec, "audio codec");
const hasVideo = requestedVideoCodec !== VideoCodec.none;
@@ -157,8 +161,7 @@ export async function convertWithFFmpeg(
`0:a:${selectedAudioTrack}?`,
"-c:a",
audioEncoder,
"-b:a",
audioBitrate.toString(),
...(audioBitrate === undefined ? [] : ["-b:a", audioBitrate.toString()]),
"-ac",
"2",
"-f",
Mserver/src/index.ts
@@ -180,9 +180,10 @@ const app = setup
const { filePath, info: fileScan } = resolved;
const probe = await probeFile(filePath);
//don't use higher bitrate than what the file has, use requested bitrate if unknown
const audioBitrate = probe.audioBitrate
? Math.min(query.audioBitrate, probe.audioBitrate)
: query.audioBitrate;
const audioBitrate =
query.audioBitrate && probe.audioBitrate
? Math.min(query.audioBitrate, probe.audioBitrate)
: query.audioBitrate;
const videoBitrate =
probe.videoBitrate && query.videoBitrate
? Math.min(query.videoBitrate, probe.videoBitrate)
@@ -253,7 +254,7 @@ const app = setup
videoBitrate: t.Optional(t.Number()),
videoEncodingSetting: t.Optional(t.Enum(VideoEncodingSetting)),
audioCodec: t.Enum(AudioCodec),
audioBitrate: t.Number(),
audioBitrate: t.Optional(t.Number()),
}),
},
)
Mserver/src/utils.ts
@@ -75,7 +75,6 @@ const mediaInfoSemaphore = new Semaphore(scanConcurrency);
export async function findCover(targetPath: string): Promise<CoverResult> {
try {
if ((await stat(targetPath)).isFile()) {
//TODO
//duration is not needed for cover extraction and would force a full-file parse for some formats
const parsed = await parseFile(targetPath, { duration: false, skipCovers: false });
if (!parsed.common.picture || parsed.common.picture.length === 0) return findCover(path.dirname(targetPath));
@@ -173,6 +172,15 @@ export async function probeFile(filePath: string): Promise<Metadata> {
audioBitrate: num(audioTrack?.BitRate),
videoCodec: videoTrack?.Format,
videoBitrate: num(videoTrack?.BitRate),
videoWidth: num(videoTrack?.Width),
videoHeight: num(videoTrack?.Height),
//prefer the exact rational over the rounded FrameRate: 24000/1001 reported as "23.976" is
//not precise enough to land on the right side of a codec level boundary
frameRate:
videoTrack?.FrameRate_Num && videoTrack?.FrameRate_Den
? Number(videoTrack.FrameRate_Num) / Number(videoTrack.FrameRate_Den)
: num(videoTrack?.FrameRate),
interlaced: videoTrack?.ScanType === undefined ? undefined : videoTrack.ScanType !== "Progressive",
audioTracks,
};
Mwebclient/src/App.tsx
@@ -29,7 +29,8 @@ import {
PlaylistSVG,
} from "./components/SVG";
import { getCover, getSrc, listFiles } from "./offline";
import { type AppOptions, FileDisplay, type PlayerState, type PlaylistItem } from "./types";
import { DEFAULT_STREAMING_MODE, playableSettings } from "./capabilities";
import { type AppOptions, FileDisplay, type PlayerState, type PlaylistItem, StreamingMode } from "./types";
import { formatFilename, handleUnauthorized, LocalStorageValues, setUnauthorizedHandler, toast } from "./utils";
//TODO: bypass music-metadata if it fails, maybe only use for fallback for files with weird encodings
@@ -85,7 +86,6 @@ const App: Component = () => {
//default settings
enableTranscoding: true,
languages: "",
disableChunkedTranscoding: false,
maxTranscodes: 4,
maxDownloads: 2,
transcodeCovers: false,
@@ -99,11 +99,11 @@ const App: Component = () => {
controlsDisplay: FileDisplay.TITLE,
progressBarHeight: 10,
useMSE: "MediaSource" in window,
rememberDir: false,
duplicateWarning: true,
videoSettings: {
streamingMode: DEFAULT_STREAMING_MODE,
container: MediaContainer.mp4,
videoCodec: VideoCodec.av1,
videoBitrate: 8000000, //8mbit
@@ -113,6 +113,7 @@ const App: Component = () => {
},
audioSettings: {
streamingMode: DEFAULT_STREAMING_MODE,
container: MediaContainer.webm,
audioBitrate: 128000, //128kbit
audioCodec: AudioCodec.opus,
@@ -293,10 +294,24 @@ const App: Component = () => {
async function getMSEUrl(srcUrl: string, offsetSeconds: number, isVideo: boolean) {
let mimeType: string;
if (isVideo) {
const metadata = playerState.currentTrack?.metadata;
const videoStreamInfo =
metadata?.videoWidth && metadata.videoHeight && metadata.frameRate
? {
sourceWidth: metadata.videoWidth,
sourceHeight: metadata.videoHeight,
frameRate: metadata.frameRate,
interlaced: metadata.interlaced === true,
bitrate: metadata.videoBitrate
? Math.min(options.videoSettings.videoBitrate, metadata.videoBitrate)
: options.videoSettings.videoBitrate,
}
: undefined;
mimeType = createRFC6381String(
options.videoSettings.container,
options.videoSettings.videoCodec,
options.videoSettings.audioCodec,
videoStreamInfo,
);
} else {
mimeType = createRFC6381String(
@@ -500,8 +515,11 @@ const App: Component = () => {
} catch {}
}
//each transcoding profile carries its own streaming mode, so which one applies depends on what
//is being played. a blob url is an already-synced file, which is never streamed
const streamingMode = isVideo ? options.videoSettings.streamingMode : options.audioSettings.streamingMode;
const mseURL =
options.enableTranscoding && options.useMSE && !src.startsWith("blob:")
options.enableTranscoding && streamingMode === StreamingMode.mse && !src.startsWith("blob:")
? await getMSEUrl(src, seekTo || 0, isVideo === true)
: null;
if (generation !== playGeneration) {
@@ -651,8 +669,19 @@ const App: Component = () => {
try {
const savedOptions = localStorage.getItem(LocalStorageValues.options);
if (savedOptions) {
//keep default settings when no override exists
setOptions({ ...options, ...JSON.parse(savedOptions) });
//keep default settings when no override exists. the two transcoding profiles have to be merged
//one level deeper than the rest: a spread would replace them wholesale, so a setting added
//after the options were last saved would come back undefined instead of taking its default
const saved = JSON.parse(savedOptions) as Partial<AppOptions>;
//playableSettings, because nothing downstream re-checks: the transcode url is built straight
//from these, so a combination this browser cannot play would reach the element as a format
//error with no explanation, and one with an unrecognised container would throw on sight
setOptions({
...options,
...saved,
videoSettings: playableSettings({ ...options.videoSettings, ...saved.videoSettings }),
audioSettings: playableSettings({ ...options.audioSettings, ...saved.audioSettings }),
});
}
const savedVolume = Number.parseFloat(localStorage.getItem(LocalStorageValues.volume) ?? "");
if (savedVolume && !Number.isNaN(savedVolume)) {
Awebclient/src/capabilities.ts
@@ -0,0 +1,276 @@
import { AudioCodec, MediaContainer, VideoCodec } from "music-server-shared/types";
import {
audioCodecToken,
containerMimeType,
containerNamesCodecs,
createRFC6381String,
ladderMaxWidth,
type VideoStreamInfo,
videoCodecToken,
} from "music-server-shared/utils";
import { StreamingMode, VIDEO_BITRATES } from "./types";
export interface Support {
supported: boolean;
reason?: string;
}
//what a transcoding profile would produce, which is all it takes to decide whether a browser can play it.
//deliberately the shape of AppOptions' two settings objects, so either can be passed straight in and a
//single field varied with a spread - no argument threading and no positional mix-ups at the call sites
export interface Combination {
container: MediaContainer;
streamingMode: StreamingMode;
audioCodec: AudioCodec;
videoCodec?: VideoCodec;
videoBitrate?: number;
}
export const hasMediaSource = "MediaSource" in window;
export const DEFAULT_STREAMING_MODE = hasMediaSource ? StreamingMode.mse : StreamingMode.progressive;
//the W3C MSE Byte Stream Format Registry is a closed list: ISO BMFF, WebM, MPEG-2 TS and MPEG Audio. a
//container missing from it cannot be fed to a SourceBuffer by any conforming browser, no matter how well
//that same browser plays it progressively - so this is a fact about the spec, not something to probe for
const MSE_BYTE_STREAM_FORMATS: MediaContainer[] = [MediaContainer.mp4, MediaContainer.webm, MediaContainer.mp3];
//audio codecs that are at least plausible in each container - ffmpeg refuses the rest outright, so there
//is no browser verdict to ask for
const CONTAINER_AUDIO_CODECS: Record<MediaContainer, AudioCodec[]> = {
[MediaContainer.mp4]: [AudioCodec.mp3, AudioCodec.opus, AudioCodec.aac, AudioCodec.flac],
[MediaContainer.webm]: [AudioCodec.opus, AudioCodec.vorbis],
[MediaContainer.ogg]: [AudioCodec.opus, AudioCodec.vorbis, AudioCodec.flac],
[MediaContainer.mp3]: [AudioCodec.mp3],
};
const CONTAINER_VIDEO_CODECS: Record<MediaContainer, VideoCodec[]> = {
[MediaContainer.mp4]: [VideoCodec.av1, VideoCodec.h264, VideoCodec.h265, VideoCodec.vp9],
[MediaContainer.webm]: [VideoCodec.av1, VideoCodec.vp9],
[MediaContainer.ogg]: [],
[MediaContainer.mp3]: [],
};
export function containerAudioCodecs(container: MediaContainer): AudioCodec[] {
return CONTAINER_AUDIO_CODECS[container] ?? [];
}
export function containerVideoCodecs(container: MediaContainer): VideoCodec[] {
return [...(CONTAINER_VIDEO_CODECS[container] ?? []), VideoCodec.none];
}
//actual values don't seem to matter, besides for a possible "smoothnes" verdict
//though any valid codec string should result in the same playability result for the same codec
const NOMINAL_FRAMERATE = 30;
const NOMINAL_UNSCALED_WIDTH = 3840;
function nominalVideoStream(videoBitrate: number): VideoStreamInfo {
const width = ladderMaxWidth(videoBitrate) ?? NOMINAL_UNSCALED_WIDTH;
return {
sourceWidth: width,
sourceHeight: Math.round(width / (16 / 9) / 2) * 2,
frameRate: NOMINAL_FRAMERATE,
interlaced: false,
bitrate: videoBitrate,
};
}
let probeElement: HTMLVideoElement | undefined;
function progressivelyPlayable(mimeType: string): boolean {
probeElement ??= document.createElement("video");
return probeElement.canPlayType(mimeType) !== "";
}
//MediaSource.isTypeSupported is the cheap gate but a lenient one: it parses the type and checks the
//codecs in isolation, so it says yes to combinations the decoder then refuses. decodingInfo asks the
//actual decoder pipeline, per track, with the resolution and framerate attached
const decodingCache = new Map<string, { supported: boolean; smooth: boolean }>();
const pending = new Map<string, Promise<void>>();
function startDecodingProbe(key: string, configuration: MediaDecodingConfiguration) {
const existing = pending.get(key);
if (existing) return existing;
const probe = navigator.mediaCapabilities
.decodingInfo(configuration)
.then((info) => {
decodingCache.set(key, { supported: info.supported, smooth: info.smooth });
})
.catch(() => {
//a configuration this browser cannot even parse tells us nothing beyond what the sync checks
//already established, so record the permissive answer rather than disabling a working option
decodingCache.set(key, { supported: true, smooth: true });
})
.finally(() => {
pending.delete(key);
});
pending.set(key, probe);
return probe;
}
export function checkSupport(combination: Combination): Support {
const { container, streamingMode, audioCodec } = combination;
const videoCodec = combination.videoCodec ?? VideoCodec.none;
const hasVideo = videoCodec !== VideoCodec.none;
if (streamingMode === StreamingMode.mse) {
if (!hasMediaSource) return { supported: false, reason: "this browser has no MediaSource support" };
if (!MSE_BYTE_STREAM_FORMATS.includes(container))
return {
supported: false,
reason: `${container} has no registered MSE byte stream format, so no browser can stream it - pick progressive or buffered`,
};
//measured against -live 1, -dash 1, -default_mode infer, explicit dispositions, forced cluster
//limits and both track orders: the append succeeds, updateend fires, and buffered stays empty
if (container === MediaContainer.webm && hasVideo)
return {
supported: false,
reason: "effectively unsupported by browsers in mse",
};
}
const video = hasVideo ? nominalVideoStream(combination.videoBitrate ?? 0) : undefined;
const mimeType = createRFC6381String(container, videoCodec, audioCodec, video);
if (streamingMode === StreamingMode.mse) {
if (!MediaSource.isTypeSupported(mimeType))
return { supported: false, reason: `the browser rejects ${mimeType} in a SourceBuffer` };
} else if (!progressivelyPlayable(mimeType)) {
return { supported: false, reason: `the browser cannot play ${mimeType}` };
}
//decodingInfo needs one content type per track, which is why the tokens are built here rather than
//reusing the combined string above. a container that names no codec has nothing to ask about
if (!containerNamesCodecs(container)) return { supported: true };
const configuration: MediaDecodingConfiguration = {
type: streamingMode === StreamingMode.mse ? "media-source" : "file",
audio: { contentType: `${containerMimeType(container, false)}; codecs="${audioCodecToken(audioCodec)}"` },
...(video
? {
video: {
contentType: `${containerMimeType(container, true)}; codecs="${videoCodecToken(videoCodec, video)}"`,
width: video.sourceWidth,
height: video.sourceHeight,
bitrate: video.bitrate,
framerate: video.frameRate,
},
}
: {}),
};
//keyed on the bitrate only when there is a video track, so an audio-only probe is not re-run once per
//resolution ladder rung for an answer that cannot depend on it
const key = `${configuration.type}|${configuration.audio?.contentType}|${configuration.video?.contentType}|${video?.bitrate}`;
const cached = decodingCache.get(key);
if (!cached) {
void startDecodingProbe(key, configuration);
return { supported: true };
}
if (!cached.supported) return { supported: false, reason: `the browser's decoders reject ${mimeType}` };
if (!cached.smooth)
return { supported: true, reason: "supported, but the browser does not expect to decode this smoothly" };
return { supported: true };
}
//a container is offerable when at least one codec combination inside it works in the current mode
function checkContainer(combination: Combination, videoCodecs: VideoCodec[]): Support {
let firstFailure: Support | undefined;
for (const videoCodec of videoCodecs) {
for (const audioCodec of containerAudioCodecs(combination.container)) {
const support = checkSupport({ ...combination, videoCodec, audioCodec });
if (support.supported) return { supported: true };
firstFailure ??= support;
}
}
return firstFailure ?? { supported: false, reason: `${combination.container} offers no usable codecs` };
}
export function checkVideoContainerSupport(combination: Combination): Support {
return checkContainer(combination, containerVideoCodecs(combination.container));
}
//an audio-only profile must not be told mp4 works because mp4 video works, only because mp4 audio does
export function checkAudioContainerSupport(combination: Combination): Support {
return checkContainer(combination, [VideoCodec.none]);
}
function usableAudioCodecs(combination: Combination): AudioCodec[] {
return containerAudioCodecs(combination.container).filter(
(audioCodec) => checkSupport({ ...combination, audioCodec }).supported,
);
}
function usableVideoCodecs(combination: Combination): VideoCodec[] {
return containerVideoCodecs(combination.container).filter(
(videoCodec) => checkContainer(combination, [videoCodec]).supported,
);
}
function usableContainers(combination: Combination): MediaContainer[] {
const check = combination.videoCodec === undefined ? checkAudioContainerSupport : checkVideoContainerSupport;
return Object.values(MediaContainer).filter((container) => check({ ...combination, container }).supported);
}
//used to decide whether the video section is worth showing at all - it is not when nothing but "none" would
//be selectable
export function selectableVideoCodecs(combination: Combination): VideoCodec[] {
return usableVideoCodecs(combination).filter((videoCodec) => videoCodec !== VideoCodec.none);
}
function firstUsable<T>(current: T, usable: T[]): T | undefined {
return usable.includes(current) ? current : usable[0];
}
function knownMember<T extends string>(values: T[], value: T, fallback: T): T {
return values.includes(value) ? value : fallback;
}
//make sure settings are playable (e.g. might come from an older browser version with different codec support)
export function playableSettings<T extends Combination>(settings: T): T {
const known = {
...settings,
streamingMode: knownMember(Object.values(StreamingMode), settings.streamingMode, DEFAULT_STREAMING_MODE),
container: knownMember(Object.values(MediaContainer), settings.container, MediaContainer.mp4),
audioCodec: knownMember(Object.values(AudioCodec), settings.audioCodec, AudioCodec.opus),
...(settings.videoCodec === undefined
? {}
: { videoCodec: knownMember(Object.values(VideoCodec), settings.videoCodec, VideoCodec.none) }),
};
if (checkSupport(known).supported) return known;
//outermost field first: the container decides which codecs are on offer, and the video codec decides
//which audio codecs are left over
const container = firstUsable(known.container, usableContainers(known));
if (container === undefined) {
if (known.streamingMode === DEFAULT_STREAMING_MODE) return known;
return playableSettings({ ...known, streamingMode: DEFAULT_STREAMING_MODE });
}
const withContainer = { ...known, container };
const videoCodec =
known.videoCodec === undefined ? undefined : firstUsable(known.videoCodec, usableVideoCodecs(withContainer));
const withVideoCodec = videoCodec === undefined ? withContainer : { ...withContainer, videoCodec };
const audioCodec = firstUsable(withVideoCodec.audioCodec, usableAudioCodecs(withVideoCodec));
//the container was only offered because some codec pair inside it works, so this cannot come up empty
if (audioCodec === undefined) return known;
return { ...withVideoCodec, audioCodec };
}
export async function initCapabilities(): Promise<void> {
for (const streamingMode of Object.values(StreamingMode)) {
for (const container of Object.values(MediaContainer)) {
for (const audioCodec of containerAudioCodecs(container)) {
for (const videoCodec of containerVideoCodecs(container)) {
//the bitrate picks the resolution ladder rung and with it the codec level, so each one the
//menu offers is its own probe. an audio-only combination has none to vary - and "none" is
//exactly what an absent videoCodec resolves to in checkSupport
const videoBitrates = videoCodec === VideoCodec.none ? [undefined] : VIDEO_BITRATES;
for (const videoBitrate of videoBitrates) {
checkSupport({ container, streamingMode, audioCodec, videoCodec, videoBitrate });
}
}
}
}
}
//every probe started above resolves into the same map, and settling one cannot start another
await Promise.all(pending.values());
}
Mwebclient/src/components/OptionsModal.tsx
@@ -1,10 +1,23 @@
import { type Accessor, createEffect, createSignal, For, Match, on, type Setter, Switch } from "solid-js";
import { type AppOptions, FileDisplay } from "../types";
import { type Accessor, batch, createEffect, createSignal, For, Match, on, type Setter, Show, Switch } from "solid-js";
import { type AppOptions, AUDIO_BITRATES, FileDisplay, StreamingMode, VIDEO_BITRATES } from "../types";
import "./OptionsModal.css";
import "./Dialog.css";
import "toastify-js/src/toastify.css";
import { AudioCodec, MediaContainer, VideoCodec, VideoEncodingSetting } from "music-server-shared/types";
import { audioCodecHasBitrate } from "music-server-shared/utils";
import { createStore } from "solid-js/store";
import {
checkAudioContainerSupport,
checkSupport,
checkVideoContainerSupport,
type Combination,
containerAudioCodecs,
containerVideoCodecs,
hasMediaSource,
playableSettings,
selectableVideoCodecs,
type Support,
} from "../capabilities";
import { handleUnauthorized, toast } from "../utils";
interface OptionsModalProps {
@@ -14,44 +27,27 @@ interface OptionsModalProps {
setOptions: (options: AppOptions) => void;
}
function supportedVideoCodecs(container: MediaContainer) {
let codecs: VideoCodec[] = [];
switch (container) {
case MediaContainer.mp3:
case MediaContainer.webm:
case MediaContainer.ogg:
break;
case MediaContainer.mp4:
codecs = [VideoCodec.av1, VideoCodec.h264, VideoCodec.h265, VideoCodec.vp9];
break;
default: {
throw new Error(`Unsupported container: ${container}`);
}
}
return codecs.concat([VideoCodec.none]);
const STREAMING_MODE_HINTS: Record<StreamingMode, string> = {
[StreamingMode.mse]:
"Media Source Extensions: the transcode is streamed into a SourceBuffer. The only mode that can seek within a transcode, but limited to the containers and codecs that support MSE.",
[StreamingMode.progressive]:
"The transcode is streamed straight to the player as it is produced. Plays whatever the browser can decode, but cannot seek.",
[StreamingMode.buffered]:
"The server transcodes the whole file before sending anything, so the response is a normal sized download instead of a chunked one. Much slower to start, and only needed for browsers that refuse chunked media sources (Safari).",
};
type Profile = "videoSettings" | "audioSettings";
function bitrateLabel(bitrate: number): string {
return bitrate < 1_000_000 ? `${bitrate / 1000} kbps` : `${bitrate / 1_000_000} Mbps`;
}
function supportedAudioCodecs(container: MediaContainer, forVideo?: boolean) {
let codecs: AudioCodec[] = [];
switch (container) {
case MediaContainer.mp3:
codecs = [AudioCodec.mp3];
break;
case MediaContainer.webm:
codecs = [AudioCodec.opus, AudioCodec.vorbis];
break;
case MediaContainer.ogg:
codecs = [AudioCodec.opus, AudioCodec.vorbis];
break;
case MediaContainer.mp4:
codecs = [AudioCodec.mp3];
if (forVideo) codecs.push(AudioCodec.opus, AudioCodec.aac); //ffmpeg generates garbage if opus or aac is used in mp4 without video
break;
default: {
throw new Error(`Unsupported container: ${container}`);
}
}
return codecs;
function SupportedOption(props: { value: string; support: Support }) {
return (
<option value={props.value} disabled={!props.support.supported} title={props.support.reason}>
{props.support.supported ? props.value : `${props.value} (unsupported)`}
</option>
);
}
export default function OptionsModal(props: OptionsModalProps) {
@@ -61,29 +57,24 @@ export default function OptionsModal(props: OptionsModalProps) {
let dialogElement!: HTMLDialogElement;
let formElement!: HTMLFormElement;
//update to valid codecs on video container change
createEffect(() => {
const suportedVideo = supportedVideoCodecs(newOptions.videoSettings.container);
if (!suportedVideo.includes(newOptions.videoSettings.videoCodec)) {
setNewOptions("videoSettings", "videoCodec", suportedVideo[0]);
}
const suportedAudio = supportedAudioCodecs(
newOptions.videoSettings.container,
newOptions.videoSettings.videoCodec !== VideoCodec.none,
);
if (!suportedAudio.includes(newOptions.videoSettings.audioCodec)) {
setNewOptions("videoSettings", "audioCodec", suportedAudio[0]);
}
});
const videoSupport = () => checkSupport(newOptions.videoSettings);
const audioSupport = () => checkSupport(newOptions.audioSettings);
//update to valid codecs on audio container change
createEffect(() => {
const suportedAudio = supportedAudioCodecs(newOptions.audioSettings.container, false);
if (!suportedAudio.includes(newOptions.audioSettings.audioCodec)) {
setNewOptions("audioSettings", "audioCodec", suportedAudio[0]);
}
});
//keep the selection playable, so the greyed-out entries in the lists are only ever ones the user could
//pick and never the one they are already on. writing a value a store already holds notifies nobody, so
//once the selection is playable this re-runs, finds the same values and settles
function coerceProfile(profile: Profile) {
const playable = playableSettings<Combination>(newOptions[profile]);
batch(() => {
setNewOptions(profile, "streamingMode", playable.streamingMode);
setNewOptions(profile, "container", playable.container);
setNewOptions(profile, "audioCodec", playable.audioCodec);
//only videoSettings has a video codec, which is exactly when playableSettings returns one
if (playable.videoCodec !== undefined) setNewOptions("videoSettings", "videoCodec", playable.videoCodec);
});
}
createEffect(() => coerceProfile("videoSettings"));
createEffect(() => coerceProfile("audioSettings"));
createEffect(
on(props.showOptions, () => {
@@ -197,16 +188,6 @@ export default function OptionsModal(props: OptionsModalProps) {
disabled={!newOptions.enableTranscoding}
/>
</div>
<div>
<label for="transcoding-buffer-checkbox">Disable chunked transcoding:</label>
<input
id="transcoding-buffer-checkbox"
type="checkbox"
checked={newOptions.disableChunkedTranscoding}
onChange={(e) => setNewOptions("disableChunkedTranscoding", e.target.checked)}
disabled={!newOptions.enableTranscoding}
/>
</div>
<div>
<label for="max-transcodes-input">Max concurrent transcodes:</label>
<input
@@ -321,15 +302,6 @@ export default function OptionsModal(props: OptionsModalProps) {
</div>
</Match>
<Match when={optionsPage() === 3}>
<div>
<label for="mse-checkbox">Allow Media Source Extensions (MSE):</label>
<input
id="mse-checkbox"
type="checkbox"
checked={newOptions.useMSE}
onChange={(e) => setNewOptions("useMSE", e.target.checked)}
/>
</div>
<div>
<label for="remember-dir-checkbox">Remember last directory:</label>
<input
@@ -363,72 +335,87 @@ export default function OptionsModal(props: OptionsModalProps) {
Transcoding settings for videos
</div>
<div>
<label for="container-select">Container:</label>
<label for="video-streaming-mode-select">Streaming mode:</label>
<select
id="container-select"
value={newOptions.videoSettings.container}
onChange={(e) => setNewOptions("videoSettings", "container", e.target.value as MediaContainer)}
id="video-streaming-mode-select"
title={STREAMING_MODE_HINTS[newOptions.videoSettings.streamingMode]}
value={newOptions.videoSettings.streamingMode}
onChange={(e) => setNewOptions("videoSettings", "streamingMode", e.target.value as StreamingMode)}
>
<For each={Object.values(MediaContainer)}>
{(format) => {
return <option value={format}>{format}</option>;
}}
</For>
</select>
</div>
<div>
<label for="video-codec-select">Video codec:</label>
<select
id="video-codec-select"
value={newOptions.videoSettings.videoCodec}
onChange={(e) => setNewOptions("videoSettings", "videoCodec", e.target.value as VideoCodec)}
>
<For each={supportedVideoCodecs(newOptions.videoSettings.container)}>
{(codec) => {
return <option value={codec}>{codec}</option>;
}}
<For each={Object.values(StreamingMode)}>
{(mode) => (
<option value={mode} disabled={mode === StreamingMode.mse && !hasMediaSource}>
{mode}
</option>
)}
</For>
</select>
</div>
<div>
<label for="video-settings-select">Video encoding settings:</label>
<label for="container-select">Container:</label>
<select
id="video-settings-select"
value={newOptions.videoSettings.videoEncodingSetting}
onChange={(e) =>
setNewOptions("videoSettings", "videoEncodingSetting", e.target.value as VideoEncodingSetting)
}
id="container-select"
title={videoSupport().reason}
value={newOptions.videoSettings.container}
onChange={(e) => setNewOptions("videoSettings", "container", e.target.value as MediaContainer)}
>
<For each={Object.values(VideoEncodingSetting)}>
{(setting) => {
return <option value={setting}>{setting}</option>;
}}
<For each={Object.values(MediaContainer)}>
{(format) => (
<SupportedOption
value={format}
support={checkVideoContainerSupport({ ...newOptions.videoSettings, container: format })}
/>
)}
</For>
</select>
</div>
<div>
<label for="video-bitrate-select">Video bitrate:</label>
<select
id="video-bitrate-select"
value={newOptions.videoSettings.videoBitrate}
onChange={(e) => setNewOptions("videoSettings", "videoBitrate", Number.parseInt(e.target.value, 10))}
>
<option value="210000">210 kbps</option>
<option value="420000">420 kbps</option>
<option value="720000">720 kbps</option>
<option value="1500000">1.5 Mbps</option>
<option value="3000000">3 Mbps</option>
<option value="4000000">4 Mbps</option>
<option value="6000000">6 Mbps</option>
<option value="8000000">8 Mbps</option>
<option value="10000000">10 Mbps</option>
<option value="15000000">15 Mbps</option>
<option value="20000000">20 Mbps</option>
<option value="30000000">30 Mbps</option>
<option value="40000000">40 Mbps</option>
</select>
</div>
<Show when={selectableVideoCodecs(newOptions.videoSettings).length > 0}>
<div>
<label for="video-codec-select">Video codec:</label>
<select
id="video-codec-select"
value={newOptions.videoSettings.videoCodec}
onChange={(e) => setNewOptions("videoSettings", "videoCodec", e.target.value as VideoCodec)}
>
<For each={containerVideoCodecs(newOptions.videoSettings.container)}>
{(codec) => (
<SupportedOption
value={codec}
support={checkSupport({ ...newOptions.videoSettings, videoCodec: codec })}
/>
)}
</For>
</select>
</div>
</Show>
<Show when={newOptions.videoSettings.videoCodec !== VideoCodec.none}>
<div>
<label for="video-settings-select">Video encoding settings:</label>
<select
id="video-settings-select"
value={newOptions.videoSettings.videoEncodingSetting}
onChange={(e) =>
setNewOptions("videoSettings", "videoEncodingSetting", e.target.value as VideoEncodingSetting)
}
>
<For each={Object.values(VideoEncodingSetting)}>
{(setting) => {
return <option value={setting}>{setting}</option>;
}}
</For>
</select>
</div>
<div>
<label for="video-bitrate-select">Video bitrate:</label>
<select
id="video-bitrate-select"
value={newOptions.videoSettings.videoBitrate}
onChange={(e) => setNewOptions("videoSettings", "videoBitrate", Number.parseInt(e.target.value, 10))}
>
<For each={VIDEO_BITRATES}>{(rate) => <option value={rate}>{bitrateLabel(rate)}</option>}</For>
</select>
</div>
</Show>
<div>
<label for="audio-codec-select">Audio codec:</label>
<select
@@ -436,83 +423,97 @@ export default function OptionsModal(props: OptionsModalProps) {
value={newOptions.videoSettings.audioCodec}
onChange={(e) => setNewOptions("videoSettings", "audioCodec", e.target.value as AudioCodec)}
>
<For
each={supportedAudioCodecs(
newOptions.videoSettings.container,
newOptions.videoSettings.videoCodec !== VideoCodec.none,
<For each={containerAudioCodecs(newOptions.videoSettings.container)}>
{(codec) => (
<SupportedOption
value={codec}
support={checkSupport({ ...newOptions.videoSettings, audioCodec: codec })}
/>
)}
>
{(codec) => {
return <option value={codec}>{codec}</option>;
}}
</For>
</select>
</div>
<div>
<label for="audio-bitrate-select">Audio bitrate:</label>
<select
id="audio-bitrate-select"
value={newOptions.videoSettings.audioBitrate}
onChange={(e) => setNewOptions("videoSettings", "audioBitrate", Number.parseInt(e.target.value, 10))}
>
<option value="32000">32 kbps</option>
<option value="48000">48 kbps</option>
<option value="64000">64 kbps</option>
<option value="96000">96 kbps</option>
<option value="128000">128 kbps</option>
<option value="192000">192 kbps</option>
<option value="320000">320 kbps</option>
</select>
</div>
<Show when={audioCodecHasBitrate(newOptions.videoSettings.audioCodec)}>
<div>
<label for="audio-bitrate-select">Audio bitrate:</label>
<select
id="audio-bitrate-select"
value={newOptions.videoSettings.audioBitrate}
onChange={(e) => setNewOptions("videoSettings", "audioBitrate", Number.parseInt(e.target.value, 10))}
>
<For each={AUDIO_BITRATES}>{(rate) => <option value={rate}>{bitrateLabel(rate)}</option>}</For>
</select>
</div>
</Show>
</Match>
<Match when={optionsPage() === 5}>
<div style={{ "font-size": "1.2rem" }} class="title">
Transcoding settings for music
</div>
<div>
<label for="container-select">Container:</label>
<label for="audio-streaming-mode-select">Streaming mode:</label>
<select
id="container-select"
value={newOptions.audioSettings.container}
onChange={(e) => setNewOptions("audioSettings", "container", e.target.value as MediaContainer)}
id="audio-streaming-mode-select"
title={STREAMING_MODE_HINTS[newOptions.audioSettings.streamingMode]}
value={newOptions.audioSettings.streamingMode}
onChange={(e) => setNewOptions("audioSettings", "streamingMode", e.target.value as StreamingMode)}
>
<For each={Object.values(MediaContainer)}>
{(format) => {
return <option value={format}>{format}</option>;
}}
<For each={Object.values(StreamingMode)}>
{(mode) => (
<option value={mode} disabled={mode === StreamingMode.mse && !hasMediaSource}>
{mode}
</option>
)}
</For>
</select>
</div>
<div>
<label for="audio-codec-select">Audio codec:</label>
<label for="music-container-select">Container:</label>
<select
id="audio-codec-select"
value={newOptions.audioSettings.audioCodec}
onChange={(e) => setNewOptions("audioSettings", "audioCodec", e.target.value as AudioCodec)}
id="music-container-select"
title={audioSupport().reason}
value={newOptions.audioSettings.container}
onChange={(e) => setNewOptions("audioSettings", "container", e.target.value as MediaContainer)}
>
<For each={supportedAudioCodecs(newOptions.audioSettings.container)}>
{(codec) => {
return <option value={codec}>{codec}</option>;
}}
<For each={Object.values(MediaContainer)}>
{(format) => (
<SupportedOption
value={format}
support={checkAudioContainerSupport({ ...newOptions.audioSettings, container: format })}
/>
)}
</For>
</select>
</div>
<div>
<label for="audio-bitrate-select">Audio bitrate:</label>
<label for="music-audio-codec-select">Audio codec:</label>
<select
id="audio-bitrate-select"
value={newOptions.audioSettings.audioBitrate}
onChange={(e) => setNewOptions("audioSettings", "audioBitrate", Number.parseInt(e.target.value, 10))}
id="music-audio-codec-select"
value={newOptions.audioSettings.audioCodec}
onChange={(e) => setNewOptions("audioSettings", "audioCodec", e.target.value as AudioCodec)}
>
<option value="32000">32 kbps</option>
<option value="48000">48 kbps</option>
<option value="64000">64 kbps</option>
<option value="96000">96 kbps</option>
<option value="128000">128 kbps</option>
<option value="192000">192 kbps</option>
<option value="320000">320 kbps</option>
<For each={containerAudioCodecs(newOptions.audioSettings.container)}>
{(codec) => (
<SupportedOption
value={codec}
support={checkSupport({ ...newOptions.audioSettings, audioCodec: codec })}
/>
)}
</For>
</select>
</div>
<Show when={audioCodecHasBitrate(newOptions.audioSettings.audioCodec)}>
<div>
<label for="music-audio-bitrate-select">Audio bitrate:</label>
<select
id="music-audio-bitrate-select"
value={newOptions.audioSettings.audioBitrate}
onChange={(e) => setNewOptions("audioSettings", "audioBitrate", Number.parseInt(e.target.value, 10))}
>
<For each={AUDIO_BITRATES}>{(rate) => <option value={rate}>{bitrateLabel(rate)}</option>}</For>
</select>
</div>
</Show>
</Match>
</Switch>
<div class="option-buttons">
Mwebclient/src/index.tsx
@@ -6,6 +6,7 @@ import "./components/Dialog.css";
import { registerSW } from "virtual:pwa-register";
import App from "./App";
import { initCapabilities } from "./capabilities";
import { loadDataFromDB } from "./offline";
import { toast } from "./utils";
@@ -136,7 +137,7 @@ if (version && version !== currentVersion) {
onNeedReload: () => toast("A new version was installed, reload the page to use it", "info"),
});
loadDataFromDB()
Promise.all([loadDataFromDB(), initCapabilities()])
.then(() => {
root.innerHTML = "";
render(() => <App />, root);
Mwebclient/src/offline.ts
@@ -8,12 +8,12 @@ import type {
MediaFile,
Metadata,
} from "music-server-shared/types";
import { encodePath } from "music-server-shared/utils";
import { audioCodecHasBitrate, encodePath } from "music-server-shared/utils";
import { batch, from, type ResourceActions, untrack } from "solid-js";
import { createStore } from "solid-js/store";
import type { FlatFileListing } from "./App";
import { AsyncSemaphore } from "./semaphore";
import type { AppOptions, Playlist } from "./types";
import { type AppOptions, type Playlist, StreamingMode } from "./types";
import { handleUnauthorized, joinPath, toast, unproxy } from "./utils";
interface StoredFile {
@@ -386,19 +386,21 @@ export async function getSrc(
}
}
if (options.enableTranscoding) {
//the server's own isVideo logic is just metadata.videoCodec !== undefined, so compute it locally
const isVideo = metadata.videoCodec !== undefined;
const settings = isVideo ? options.videoSettings : options.audioSettings;
const disableChunkedTranscoding = settings.streamingMode === StreamingMode.buffered;
const audioBitrate = audioCodecHasBitrate(settings.audioCodec) ? settings.audioBitrate : undefined;
return {
src:
`${joinPath("transcode", encodePath(path))}` +
`?disableChunkedTranscoding=${options.disableChunkedTranscoding}` +
`&container=${isVideo ? options.videoSettings.container : options.audioSettings.container}` +
`?disableChunkedTranscoding=${disableChunkedTranscoding}` +
`&container=${settings.container}` +
`${isVideo ? `&videoCodec=${options.videoSettings.videoCodec}` : ""}` +
`${isVideo ? `&videoBitrate=${options.videoSettings.videoBitrate}` : ""}` +
`${isVideo ? `&videoEncodingSetting=${options.videoSettings.videoEncodingSetting}` : ""}` +
`&audioCodec=${isVideo ? options.videoSettings.audioCodec : options.audioSettings.audioCodec}` +
`&audioBitrate=${isVideo ? options.videoSettings.audioBitrate : options.audioSettings.audioBitrate}` +
`&audioCodec=${settings.audioCodec}` +
`${audioBitrate === undefined ? "" : `&audioBitrate=${audioBitrate}`}` +
`${seekTo === 0 ? "" : `&seekTo=${seekTo}`}` +
`&languages=${options.languages}`,
isVideo: isVideo,
Mwebclient/src/types.ts
@@ -26,6 +26,20 @@ export interface PlayerState {
waitingForStart: boolean;
}
export enum StreamingMode {
mse = "mse",
progressive = "progressive",
// same as progressive, but disables chunked transcoding, to workaround some browser issues
buffered = "buffered",
}
//probably support doesn't vary over bitrates/levels (tested for chromium/linux) but is cheap so do it anyway
export const AUDIO_BITRATES = [32_000, 48_000, 64_000, 96_000, 128_000, 192_000, 320_000];
export const VIDEO_BITRATES = [
210_000, 420_000, 720_000, 1_500_000, 3_000_000, 4_000_000, 6_000_000, 8_000_000, 10_000_000, 15_000_000,
20_000_000, 30_000_000, 40_000_000,
];
export enum FileDisplay {
FILENAME = 0,
TITLE = 1,
@@ -35,7 +49,6 @@ export enum FileDisplay {
export interface AppOptions {
enableTranscoding: boolean;
languages: string;
disableChunkedTranscoding: boolean;
maxTranscodes: number;
maxDownloads: number;
transcodeCovers: boolean;
@@ -49,11 +62,11 @@ export interface AppOptions {
fontResolution: number;
progressBarHeight: number;
useMSE: boolean;
rememberDir: boolean;
duplicateWarning: boolean;
videoSettings: {
streamingMode: StreamingMode;
container: MediaContainer;
videoCodec: VideoCodec;
videoEncodingSetting: VideoEncodingSetting;
@@ -62,6 +75,7 @@ export interface AppOptions {
audioBitrate: number;
};
audioSettings: {
streamingMode: StreamingMode;
container: MediaContainer;
audioCodec: AudioCodec;
audioBitrate: number;