Yesterday I described audio, so today I'll describe video! Video, and graphics more generally, is defined by the sheer quantity of data that needs to be generated 30, 60+ times a second.

Even with lossy compression, a feature film takes about a gigabyte of storage. Without it, it'd take 4 bytes per pixel * 1920px wide * 1080px tall * 30fps * 60 s/min * 90min ~= 1.3e12 = 130Gb, excluding audio!

Timing appears to be less vital than for audio.

So how do we get this size down somewhat?

The first popular technique, employed even on early colour TV (with analog arithmatic, in part to be backwards to be compatible with black&white TV), is to take advantage of the fact we're more sensitive to brightness than how close to red, green, or blue the colour is.

So store the images in HSV (or HSL?) colourspace, reduce the bits dedicated to hue, & convert it to RGB for display.


Another technique is to have most frames encode movement over raw pixel values. (cont.)

Show thread

You can split the previous (and maybe next) frames up into blocks and encode how many pixels over they shift, filling in new gaps with a specified adjacent colour. The resulting image may still require minor corrections, but that doesn't require near as many bytes and can be more readily compressed.

The main drawbacks are that it can get in the way of seeking, doesn't play well with cuts/transitions, & can lead conspiracy theorists to see lizardpeople. Conference talks are tiny though!

Show thread

So don't apply movement-compression to every frame, and find more find details to discard where you can get away with it. Then ofcourse there's the lossless compression techniques to use as well!

Linux-based implementation-wise video files are usually decoded within the same GStreamer pipeline I described yesterday for audio, asking the window manager to use whatever hardware acceleration is at it's disposal to "composite" the video frame into the target window.


Show thread

@alcinnz lossy video compression is lizard people technology :P

Sign in to participate in the conversation

For people who care about, support, or build Free, Libre, and Open Source Software (FLOSS).