Everything you need to know about audio/video inside Chromium and Chromium OS! Whom To ContactIt's best to have discussions on chromium-dev@chromium.org or media-dev@chromium.org for media specific matters. We are component Internals>Media on the Chromium bug tracker. DocumentationSee media/README.md. For historical reference, here's the original design doc for HTML5 audio/video. Container formats
Codec formats (Decode Only) Audio
Video
Code LocationChromium media/ - Home to all things media!media/audio - OS audio input/output abstractions media/video/capture - OS camera input abstraction media/video - software/hardware video decoder interfaces + implementations third_party/ffmpeg - Chromium's copy of FFmpeg third_party/libvpx - Chromium's copy of libvpx Blink third_party/blink/renderer/core/html/media/html_media_element.{cpp,h,idl} - media element base class third_party/blink/renderer/core/html/media/html_audio_element.{cpp,h,idl} - audio element implementation third_party/blink/renderer/core/html/media/html_video_element.{cpp,h,idl} - video element implementation Particularly Interesting Bits media/base/mime_util.cc - defines canPlayType() behaviour and file extension mapping media/blink/buffered_data_source.{cc,h} - Chromium's main implementation of DataSource for the media pipeline media/blink/buffered_resource_loader.{cc,h} - Implements the sliding window buffering strategy (see below) third_party/blink/public/platform/web_media_player.h - Blink's media player interface for providing HTML5 audio/video functionality media/blink/webmediaplayer_impl.{cc,h} - Chromium's main implementation of WebMediaPlayer How does everything get instantiated?WebFrameClient::createMediaPlayer() is the Blink embedder API for creating a WebMediaPlayer and passing it back to Blink. Every HTML5 audio/video element will ask the embedder to create a WebMediaPlayer. For Chromium this is handled in RenderFrameImpl. GN FlagsThere are a few GN flags which can alter the behaviour of Chromium's HTML5 audio/video implementation.ffmpeg_branding Overrides which version of FFmpeg to use Default: $(branding) Values: Chrome - includes additional proprietary codecs (MP3, etc..) for use with Google Chrome Chromium - builds default set of codecs proprietary_codecs Alters the list of codecs Chromium claims to support, which affects <source> and canPlayType() behaviour Default: 0(gyp)/false(gn) Values: 0/false - <source> and canPlayType() assume the default set of codecs 1/true - <source> and canPlayType() assume they support additional proprietary codecs How the %#$& does buffering work?Chromium uses a combination of range requests and an in-memory sliding window to buffer media. We have a low and high watermark that is used to determine when to purposely stall the HTTP request and when to resume the HTTP request. It's complicated, so here's a picture: |