{"id":5513,"date":"2023-03-03T09:00:00","date_gmt":"2023-03-03T08:00:00","guid":{"rendered":"https:\/\/blog.besharp.it\/?p=5513"},"modified":"2023-03-02T17:55:43","modified_gmt":"2023-03-02T16:55:43","slug":"aws-elemental-mediaconvert-advanced-transcoding-for-streaming-platforms","status":"publish","type":"post","link":"https:\/\/blog.besharp.it\/aws-elemental-mediaconvert-advanced-transcoding-for-streaming-platforms\/","title":{"rendered":"AWS Elemental MediaConvert: advanced transcoding for streaming platforms"},"content":{"rendered":"\n

Introduction<\/h2>\n\n\n\n

Have you ever wondered what is meant by transcoding?<\/p>\n\n\n\n

Have you ever wondered how the big streaming platforms can provide you with such a complete and feature-rich service?<\/p>\n\n\n\n

In this article, we will understand what transcoding is, how it works, and why transcoding is used. We will first face a classic solution using FFmpeg and then we will combine it with the AWS Elemental MediaConvert service.<\/p>\n\n\n\n

What is transcoding and why is useful<\/h2>\n\n\n\n

We can define transcoding as the process that allows, given an audio\/video source, to obtain multiple versions with different formats and resolutions. This elaboration can be done, both in real-time and in post-production.<\/p>\n\n\n\n

Today there are many playback devices available and they differ from each other in resolution, size, and supported video codecs.<\/p>\n\n\n\n

The only way we have to reach all devices is to transcode the contents of our platform.<\/p>\n\n\n\n

Starting from content recorded in 4K we will create several versions with different resolutions, codecs, bitrates, etc.<\/p>\n\n\n\n

Nowadays the ability to set the video resolution is considered an almost mandatory feature, especially if you intend to compete with the most popular streaming websites. This brings incredible benefits both for the users and for those who have to manage the platform.<\/p>\n\n\n\n

If while playing a video the user decides to lower the resolution, this would reduce the consumption of his data network.<\/p>\n\n\n\n

Another crucial activity, carried out during transcoding, is fragmentation.<\/p>\n\n\n\n

This operation allows us to divide a single large video into sub-parts of the same duration. To organize the recomposition of the video, a specific file is used to orchestrate all the single fragments that will be transmitted by the media player.<\/p>\n\n\n\n

This operation allows the buffering of the video, thus preventing the user from downloading the entire content to the device before “live\/streaming” viewing. The most modern live streaming platforms use real-time fragmentation to allow users to have a fluid use of the broadcasted content.<\/p>\n\n\n\n

Galactic guide for transcodificators<\/h2>\n\n\n\n

Let’s consider the main video codecs according to the most popular formats and devices. Below is showed a table of the main video codecs used and their respective supported resolutions:<\/p>\n\n\n\n

\n
Codec<\/strong><\/td>Description<\/strong><\/td>Resolution<\/strong><\/td><\/tr>
H.264\/AVC<\/td>One of the most popular codecs, used for encoding high-definition video. It offers great video quality with low bandwidth.<\/td>HD – 720p con 1280x720px
FullHD – 1080p 1920x1080px<\/td><\/tr>
H.265\/HEVC<\/td>Successor of H.264, it offers even better video quality with lower bandwidth.<\/td>To 8k – 7680x4320px<\/td><\/tr>
VP9<\/td>Developed by Google, it is a free and open codec that offers video quality similar to H.265 (up to 4k) at a lower cost.<\/td>To 4k – 3840x2160px<\/td><\/tr>
AV1<\/td>Sviluppato da un consorzio di aziende tecnologiche, offre una qualit\u00e0 video simile a quella di H.265\/HEVC a un costo inferiore<\/td>To 8k – 7680x4320px<\/td><\/tr>
MPEG-2<\/td>Developed by a consortium of technology companies, it offers video quality similar to H.265\/HEVC at a lower cost<\/td>SD – 720x576px o 720x480px
HD – 720p con 1280x720px
FullHD – 1080p 1920x1080px<\/td><\/tr>
MPEG-4<\/td>Used for encoding video on the Internet and mobile devices, it offers good video quality with low bandwidth.<\/td>SD – 720x576px o 720x480px
HD – 720p con 1280x720px
FullHD – 1080p 1920x1080px<\/td><\/tr>
Theora<\/td>An open source video codec, it offers good video quality with low bandwidth.<\/td>SD – 720x576px o 720x480px
HD – 720p con 1280x720px<\/td><\/tr><\/tbody><\/table>
Table of the main video codecs used and their respective supported resolutions<\/figcaption><\/figure>\n<\/div><\/div>\n\n\n\n

To convert our content into these formats we can use two approaches:<\/p>\n\n\n\n

The “classic” approach<\/h2>\n\n\n\n

The term transcoding is associated by most users with the FFmpeg framework.<\/p>\n\n\n\n

Quoting from the official site:<\/p>\n\n\n\n

\n

FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter, and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards committee, the community, or a corporation. It is also highly portable: FFmpeg compiles, runs, and passes our testing infrastructure <\/em>FATE<\/em><\/a> across Linux, Mac OS X, Microsoft Windows, the BSDs, Solaris, etc. under a wide variety of build environments, machine architectures, and configurations.<\/em> <\/p>\n<\/blockquote>\n\n\n\n

Once installed on our machine we can use it to carry out transcoding operations locally.<\/p>\n\n\n\n

As an example, if we wanted to decrease (or increase) the video file\u2019s bitrate to 24 we would use the following command:<\/p>\n\n\n\n

ffmpeg -I input.avi -r 24 output.avi\n\n<\/code><\/pre>\n\n\n\n
If we find ourselves with the need to transcode many sources into multiple formats, we could adopt a scalable solution using EC2, SQS, and autoscaling groups.<\/p>\n\n\n\n
To process our videos we will use a battery of EC2 arranged within an autoscaling group; by doing so, our infrastructure will be capable of managing increases in traffic (and as a consequence of the necessary processing) by autonomously creating new EC2s and managing the possible failure of one of the AWS AZs.<\/p>\n\n\n\n
As a scaling metric, it will be enough for us to implement an SQS queue which will act as a buffer for the jobs to be performed, preventing us from excessive scaling when the requests are manageable with the current number of machines, and making the most of the computational resources that EC2 will make available. The SQS queue also provides other advantages such as the management of dead-letter queues, i.e. the possibility of recognizing jobs that have not been completed by our EC2 for any reason and then possibly retrying them.<\/p>\n\n\n\n
To use this union of the two services, however, it will be necessary to create a custom metric on AWS since the number of elements in the queue is not natively recognized by the autoscaling groups.<\/p>\n\n\n\n
<\/p>\n\n\n
\n
\"SQS
SQS queue for parallelized transcoding on an EC2 fleet<\/figcaption><\/figure><\/div>\n\n\n
Using the right types of EC2 we will certainly have the great computing power to complete our work, but we would run into several disadvantages:<\/p>\n\n\n\n