Skip to main content
Version: next

Operating the Decoder

Definition​

export class WebCodecsVideoDecoder implements ScrcpyVideoDecoder {
    static get isSupported(): boolean;
    static readonly capabilities: Record<string, ScrcpyVideoDecoderCapability>;

    constructor(options: WebCodecsVideoDecoder.Options);

    // Properties
    get type(): "software" | "hardware";
    get codec(): ScrcpyVideoCodecId;
    get renderer(): VideoFrameRenderer;
    get rendererType(): string;
    get paused(): boolean;
    get writable(): WritableStream<Uint8Array>;
    get width(): number;
    get height(): number;
    get decodeQueueSize(): number;
    get onDequeue(): Event<void>;
    get framesDecoded(): number;
    get framesSkippedDecoding(): number;
    get framesRendered(): number;
    get framesDisplayed(): number;
    get framesSkippedRendering(): number;
    get totalDecodeTime(): number;
    sizeChanged: Event<ScrcpyVideoSize>;

    // Methods
    pause(): void;
    resume(): void;
    trackDocumentVisibility(document: Document): () => void;
    snapshot(options?: ImageEncodeOptions): Promise<Blob | undefined>;
    dispose(): void;
}

export namespace WebCodecsVideoDecoder {
    export interface Options extends Pick<
        VideoDecoderConfig,
        "hardwareAcceleration" | "optimizeForLatency"
    > {
        /**
         * The video codec to decode
         */
        codec: ScrcpyVideoCodecId;

        renderer?: VideoFrameRenderer | undefined;
    }
}

isSupported​

Check if the browser supports WebCodecs:

const isSupported = WebCodecsVideoDecoder.isSupported;

Example​

Here's an example of how to check for WebCodecs support before creating a decoder. If WebCodecs is not supported, you can fall back to the H264BSD decoder which works on most browsers but is slower and only supports H.264:

import { WebCodecsVideoDecoder } from "@yume-chan/scrcpy-decoder-webcodecs";
import { H264BsdDecoder } from "@yume-chan/scrcpy-decoder-h264bsd";

if (WebCodecsVideoDecoder.isSupported) {
    console.log("WebCodecs is supported, creating decoder...");
    // Proceed with creating the WebCodecs decoder
    const decoder = new WebCodecsVideoDecoder({
        codec: videoMetadata.codec,
        renderer: renderer,
    });
} else {
    console.log("WebCodecs is not supported, falling back to H264BSD decoder");
    // Create the H264BSD fallback decoder (see [H264BSD decoder](../tiny-h264.mdx) for more information)
    const decoder = new H264BsdDecoder();
}

capabilities​

Get the supported video codecs:

const capabilities = WebCodecsVideoDecoder.capabilities;

The capabilities property returns a record where the keys represent the supported video codecs (currently h264, h265, and av1). The empty object values indicate that the decoder supports all profiles and levels of each respective codec. Specifically, the implementation is:

{
    h264: {},
    h265: {},
    av1: {},
}

Note that this property can generally be ignored because WebCodecs support most video codecs and configurations. Unlike other decoders with limited codec support, WebCodecs provides broad compatibility with various video formats and settings.

Create a decoder​

import { ScrcpyVideoCodecId } from "@yume-chan/scrcpy";
import type { ScrcpyVideoDecoder } from "@yume-chan/scrcpy-decoder-shared";
import type { VideoFrameRenderer } from "@yume-chan/scrcpy-decoder-webcodecs";

export class WebCodecsVideoDecoder implements ScrcpyVideoDecoder {
  constructor(options: WebCodecsVideoDecoder.Options);
}

export namespace WebCodecsVideoDecoder {
  export interface Options extends Pick<
    VideoDecoderConfig,
    "hardwareAcceleration" | "optimizeForLatency"
  > {
    /**
     * The video codec to decode
     */
    codec: ScrcpyVideoCodecId;

    renderer?: VideoFrameRenderer | undefined;
  }
}

Parameters​

The constructor requires an options object with the following properties:

  • codec: the video codec to be decoded. It can be retrieved from the video stream metadata, or hard-coded if you only use a specific video codec.
  • renderer: a renderer created in the Renderers section (optional, defaults to AutoCanvasRenderer).
  • hardwareAcceleration: controls hardware acceleration preference ("no-preference", "require-hardware", "prefer-hardware", or "prefer-software"). Default is "no-preference".
  • optimizeForLatency: optimizes for latency when set to true. Default is true.

Example​

import { WebCodecsVideoDecoder } from "@yume-chan/scrcpy-decoder-webcodecs";

const decoder = new WebCodecsVideoDecoder({
  codec: videoMetadata.codec,
  renderer: renderer,
  hardwareAcceleration: "no-preference",
  optimizeForLatency: true,
});

After creating the decoder, append the renderer's canvas to the DOM:

document.body.appendChild(decoder.renderer.canvas);

Properties​

type​

Gets the decoder type ("software" or "hardware"), determined by the hardwareAcceleration option passed to the constructor:

const type = decoder.type;

codec​

Gets the video codec being decoded, as specified in the constructor options:

const codec = decoder.codec;

renderer​

Gets the active renderer instance that was passed to the constructor or created by default:

const renderer = decoder.renderer;

rendererType​

Gets the type of the active renderer, derived from the renderer's type property:

const rendererType = decoder.rendererType;

WebCodecs-specific metrics​

The WebCodecs decoder provides additional performance metrics beyond the common rendering metrics:

decodeQueueSize​

Gets the number of frames waiting to be decoded by the underlying VideoDecoder. A high queue size may indicate performance issues:

const queueSize = decoder.decodeQueueSize;

onDequeue​

An event that fires when a frame is dequeued (either decoded or discarded) by the underlying VideoDecoder:

decoder.onDequeue(() => {
    console.log('A frame was dequeued');
});

totalDecodeTime​

Gets the total time spent processing and decoding frames in milliseconds:

const totalDecodeTime = decoder.totalDecodeTime;

Methods​

snapshot()​

Take a screenshot of the current frame:

const blob: Blob | undefined = await decoder.snapshot();

If no frames have been rendered, the return value will be undefined. Otherwise it will be a Blob object with the PNG image data.

Common decoder operations​

The WebCodecs decoder implements the ScrcpyVideoDecoder interface. See the following pages for common decoder functionality: