Skip to main content
Version: next

WebCodecs decoder

Decodes and renders H.264, H.265 and AV1 streams in Web browsers using WebCodecs API, the new Web standard for hardware-accelerated video encoding and decoding.

It's fast, uses less hardware resources, and supports more profiles and levels.

npm install @yume-chan/scrcpy-decoder-webcodecs

Overview​

The WebCodecs decoder is more powerful and flexible than H264BSD decoder, but also slightly more difficult to use. Here's a flowchart showing the overall steps to create and use WebCodecs decoders:

note

Click underlined nodes to open their relevant documentations.

Check browser support​

Secure Context Required

WebCodecs API requires a Secure Context. Basically it means the page must be served over HTTPS or localhost.

  • If you have a domain name, you can easily get a free SSL certificate from Let's Encrypt.
  • If you only have an IP address, you can create a self-signed certificate and trust it in every device that will access your server (or ignore the warning every time).
  • For Chromium-based browser, you can also add your origin to chrome://flags/#unsafely-treat-insecure-origin-as-secure.

WebCodecs API is pretty well supported in modern browsers, but you should still check whether the browser supports it before using it. The WebCodecsVideoDecoder.isSupported property returns whether the browser supports basic WebCodecs API:

export declare class WebCodecsVideoDecoder {
  static get isSupported(): boolean;
}
import { WebCodecsVideoDecoder } from "@yume-chan/scrcpy-decoder-webcodecs";

console.log(WebCodecsVideoDecoder.isSupported);

Internally, it checks whether VideoDecoder is defined on the global object.

class WebCodecsVideoDecoder {
  static get isSupported() {
    return typeof globalThis.VideoDecoder !== "undefined";
  }
}

If WebCodecs API is not supported, you can fallback to the H264BSD decoder, but be aware only H.264 Baseline profile level 4 is supported, and the performance is much worse.

Check codec support​

Scrcpy v2.0 added the videoCodec option, to specify the video codec to use by the server. H.265 and AV1 codecs can provide better video quality at the same bitrate compared to H.264.

WebCodecs spec itself supports many codecs, including H.264, H.265 and AV1, but runtime support requires a joint effort from browsers, operating systems, graphics cards and drivers.

  • H.264: Supported by most browsers and operating systems. Except Chrome on older Android devices (like 10.0).
  • H.265 and AV1: Usually only hardware-accelerated decoding is supported, which requires compatible graphics cards and drivers. Software decoding is not supported in most browsers.

VideoDecoder.isConfigSupported static method can be used to check if a given codec is supported. It takes a codec parameter string, for example "hev1.1.60.L153.B0.0.0.0.0.0" for H.265 and "av01.0.05M.08" for AV1. Codec parameter string includes codec configurations like profile, level and/or color information. You should generate the codec parameter string based on your actual codec configuration, but some browsers like Chrome uses FFMpeg internally and does not care about the embedded configuration at all.

When ScrcpyWebCodecsDecoder initialize the underlying VideoDecoder, it will automatically generate the correct codec parameter string by parsing the video stream's configuration data.

const result = await VideoDecoder.isConfigSupported({
  codec: "hev1.1.60.L153.B0.0.0.0.0.0",
});
const isHevcSupported = result.supported === true;

Microsoft Edge on Windows​

By default, Chromium browsers uses FFMpeg internally for WebCodecs API. However, Microsoft Edge, when running on Windows, uses Media Foundation decoders instead.

Decoding H.265 requires the HEVC Video Extensions ($0.99) or HEVC Video Extensions from Device Manufacturer (free but not available anymore) app from Microsoft Store.

Decoding AV1 requires the AV1 Video Extension (free) app from Microsoft Store.

Firefox​

Firefox 133 supports playing H.265 videos, but does not support decoding H.265 streams using WebCodecs API yet.

Stream format​

For H.264 and H.265, WebCodecs's VideoDecoder supports both Annex B and AVC formats, through different parameters.

Scrcpy server produces an Annex B formatted stream, and WebCodecsVideoDecoder will feed it directly into VideoDecoder. There is no need to convert the stream to AVC format, except for muxing it into a video file.

Create decoder​

After checking browser and codec support, create a WebCodecsVideoDecoder instance. If no renderer option is specified, it automatically creates an AutoCanvasRenderer that selects the best available renderer: hardware-accelerated WebGLVideoFrameRenderer when available, falling back to software-based BitmapVideoFrameRenderer otherwise. See Renderers for advanced renderer configuration.

import { WebCodecsVideoDecoder } from "@yume-chan/scrcpy-decoder-webcodecs";
import type { ScrcpyVideoCodecId } from "@yume-chan/scrcpy";

declare const codec: ScrcpyVideoCodecId;

const decoder = new WebCodecsVideoDecoder({ codec });

The renderer property returns the AutoCanvasRenderer instance, which owns a <canvas> element for rendering. Append it to the DOM to display the video:

document.body.appendChild(decoder.renderer.canvas);

When the renderer switches between WebGL and Bitmap (e.g., after WebGL context loss), the canvas property may change to a new element. Listen to the onRendererTypeChange event to handle this:

decoder.onRendererTypeChange?.((type) => {
    console.log(`Renderer type changed to: ${type}`);
    // Re-append the (possibly new) canvas to the DOM
    document.body.appendChild(decoder.renderer.canvas);
});

For more in-depth usage, see:

  • Renderers - All renderer types, the onTypeChanged event, and the createCanvas option
  • Decoder operations - WebCodecs decoder constructor options, properties, and WebCodecs-specific metrics
  • Common decoder operations - Piping video streams, handling size changes, pausing, and rendering metrics