Thanks John, I’m Casey Wilms, a solutions architect with Brightcove focused on the Zencoder product line. That means I have the pleasure of travelling around to partners and customers to help them build the next generation of video services.
There are two converging trends that are converging to foment a truly momentous evolutionary phase in computing.
First of all, the availability of high throughput bandwidth, both underground and in the air, is increasing, and more people than ever have access to big pipes through their home or work Internet.
At the same time transistors amazingly continue to get smaller and cheaper, meaning that server farms are getting denser and cheaper. The convergence of these two trends means that it’s feasible to use the Internet to shuttle around massive files, such as data sets, and to manipulate those files using hardware that you rent from someone else.
Predicting climate trends is one such use.
And another is video transcoding. Basically every video that you watch on line has been transcoded at some point.
This is the old way to do it – a room full of servers in a closet at your office.
Zencoder is a cloud-based transcoding service, and represents a new way of approaching the challenge of processing files for delivery over the Internet.
It’s an API which allows you to access this.
-- Bare Metal is a Losing Proposition.When I was much younger, less disillusioned, but also less handsome, I built my first encoding array. And it was decent.
-- First time I built an encoding farm: -- Xserve RAID + Xserve + 2 8 core Opteron servers
-- First time I built an encoding farm: -- Xserve RAID + Xserve + 2 8 core Opteron servers -- Total cost $40,000 -- Could encode 8 simoultaneous high quality outputs at around 1/4 realtime
Could encode 8 simoultaneous high quality outputs at around 1/4 realtime.To push through a batch of just 100 movies, with both HD and SD versions, it would take 6 days and a breakfast.It ran fine, but it was slow and expensive.
After a few years it was time to upgrade-- Upgraded the system: -- Switched from Xserve RAID to a Dell Storage solution - 24 TB for around $12000 -- Purchased 2 4 node Dell servers for $10k each -- Total cost was $32000 -- Could encode 16 simoultaneous high quality outputs at around 1/2 realtime
-- Upgraded the system: -- Switched from Xserve RAID to a Dell Storage solution - 24 TB for around $12000 -- Purchased 2 4 node Dell servers for $10k each -- Total cost was $32000 -- Could encode 16 simoultaneous high quality outputs at around 1/2 realtime
Total cost was $3200
Could encode 16 simultaneous high quality outputs at around 1/2 real time.To push through a batch of just 100 movies, with both HD and SD versions it would take a couple days… Big improvement in terms of speed.
Around this time I started to hear about something called the cloud.
I had heard about Netflix’s cloud encoding operations and then I heard about Zencoder, but-- I was always was suspicious about encoding in the cloud... -- Would it perform like real hardware? -- How could we transfer files quickly? -- How could we customize the workflow?
Around that time I heard about Zencoder… I had heard about Netflix’s cloud encoding operations, but…-- Always was suspicious about encoding in the cloud... -- Would it perform like real hardware? -- How could we transfer files quickly? -- How could we customize the workflow?
Around that time I heard about Zencoder… I had heard about Netflix’s cloud encoding operations, but…-- Always was suspicious about encoding in the cloud... -- Would it perform like real hardware? -- How could we transfer files quickly? -- How could we customize the workflow?
Around that time I heard about Zencoder… I had heard about Netflix’s cloud encoding operations, but…-- Always was suspicious about encoding in the cloud... -- Would it perform like real hardware? -- How could we transfer files quickly? -- How could we customize the workflow?
-- The AHA moment: -- Ran a test file that was encoded in the cloud at 1.5x realtime as an HD, two pass high quality output. Even single job encodings mopped the floor with my homebrew solution -- $72,000 - At the best published pricing for Zencoder would have purchased 3.6 million minutes of encoding -- more than 3 years of encoding at the previous rates. -- $72,000 is just hardware cost. Also have to factor in maintanence, engineering, support... Creating and maintaining the system required around 2.5 full time resources. A good engineer in the valley these days costs a company around $150,000 per year, so you figure the total outlay for 3 years of development of a home grown solution has to be in the $1 million range. -- An in-house solution based on hardware will never be as fast, as scalable, or as flexible as a cloud based solution.
What does that mean? It means a few things. First, cloud encoding platforms enable cheap iteration. Iteration is the cornerstone of all great products, and with an encoding tool like Zencoder, it becomes extremely easy to iterate.
Let’s look at what it means to iterate for a range of outputs.
Usage based pricing is a revolution for encoding. In the real world, usage based pricing is synonymous with Paying More over the Long Term, but with cloud based encoding it actually means spending much, much less.
Building out a capable encoding infrastructure, even if it uses antiquated hardware, requires time and expertise, neither of which come cheaply. If you’re a start-up building the next great social video site, why waste money buying hardware when you have no realistic idea of how much traffic you will see. If you’re a media veteran working on a new product, why invest in encoding expertise that’s not core to your business?
It doesn’t matter if I have 1 output or 1,400 outputs, if I’m using Zencoder the price is the same, the speed is the same, and it’s all done better than if I spent the time to create my own solution from scratch. The cloud is helping companies get a sprinting headstart in front of the competition. Encoding companies like Zencoder are powering the infrastructure for the next generation of video services.
Beyond leveraging better pricing, faster iteration, quicker deployments, infinite scalability, and all of the other things that cloud based encoding offers, you’re also leveraging talent. Zencoder has a room full of video geeks that do nothing but obsess over quality, speed and reliability. If encoding is not your core competency as a business, and given that you can do it better and more cheaply in the cloud, it makes cloud based encoding a logical step.
So how do customers use the Zencoder service? Here are a couple of examples. The first is for converting large libraries, or batches of files. If you’ve got a large library of files, and you need to support a new device, such as iOS devices in adaptive bitrate, you’re looking at a huge job. The cloud is perfectly suited for jobs like this, and the Zencoder service has handled jobs with hundreds of thousands of output hours.
Content providers like PBS also use us to handle their day-to-day encoding needs, which also add up to massive numbers of files. PBS uses Zencoder to transcode all video for iOS distribution.
CASEY: As we’ve mentioned, a chief value of the instant scalability of the cloud is that it means that all of your output files can be done in parallel. This makes a big difference when you’re talking about supporting adaptive bitrate playback across a range of devices, and there are ways that you can structure output renditions for maximum efficiency . With modern formats like HLS and Smooth Streaming, it’s incredibly easy to provide a smooth playback experience that is always matching the quality of playback to the user’s device and network conditions. Let’s take a quick look at how simple the API is to create this.
.
The current live encoding looks something like this. An encoder takes in a raw feed at the venue or broadcast center and encodes streams in multiple formats and bitrates. The streams are then sent over a broadband connection to a CDN for delivery.
There are big challenges associated with this way of doing things. Encoding is a CPU bound process. This means that to add more bitrates, you have to add more encoders. This requires great upfront capex, and operational expertise, both of which make it difficult to scale, as indicated in the diagram.
Moreover, delivering multiple formats in multiple renditions requires a lot of bandwidth. This can be expensive, and can limit the venues from which live events are broadcast. It’s also essential to keep in mind that the encoder scalability and bandwidth problems are exacerbated by having multiple simultaneous events, or multiple camera angles from a single event, or both!
Users connect to the Zencoder Live Cloud Transcoding service with an application via the API. You also need a lightweight encoder on site that can output an RTMP feed. This could be as simple as the free Flash Media Live Encoder, or something with a little more heft, like WireCast. The application makes a call to the Zencoder service and the Zencoder service returns a URL, to which an RTMP feed will be published, along with a stream name.The URL and Stream Name are plugged into the encoder, and the RTMP feed is published to the Zencoder service. The Zencoder service then does the heavy lifting, transcoding a single RTMP feed into HLS and RTMP output formats, and in as many bitrates as desired.
Compared to the old way of doing things, with this architecture it’s cheap and easy to scale your on-site encoder.
And because you only need to publish one RTMP stream to the cloud there are no bandwidth bottlenecks at the point of origin. Together this means less capex, and more flexibility with your live events.
The service scales up and down seamlessly, so you never hit capacity or pay for idle processing power.
A centralized facility with established encoding infrastructure can realize bandwidth savings by encoding in the cloud, and quickly iterate applications and adapt to new formats and devices. Examples include a broadcast head end, a data center, a newsroom, etc.
A highly decentralized architecture is very challenging and expensive without cloud transcoding. Not only does it benefit from a significantly lighter technology stack, but also enables totally new set of live streaming use cases. New content services can aggregate live streams from disparate sources for playback within an application. Examples include live user generated content, services that aggregate sports programming from highschools and colleges, multi-stage concerts, and field reporters for news organizations.
