Content delivery networking crash course


Back in the past (I really need to stop this intro, I'm already sounding like I'm Vint Cerf...) delivering content over the internet to a non-local audience really was a pain. We couldn't trust Network Service Providers back then. The aforementioned not really being their fault as long haul communications were both technically complex and costly. Some clever guys from MIT, together with some startup named after a translation of Hawaian word "Clever" tought about working this around.

The initial idea is pretty simple. They would host the heavy content of their content provider customers, and deliver it closest to the audience.
Based on the source IP address of the user requesting the content, they would use some internal algorithm to select the most-likely-closest edge node to serve that content, based on two technologies:


First one to select the best serving node based on source IP AS info and GeoDNS mapping (deliberately simplifying things here). Second one for each edge-node to not have to keep all the content all the time. Content in the cache is served directly, content out of the cache either served by redirect to another node, either proxying http to another node.

This way, the hot content would always be present on the nodes, and cold content could be served from a somewhere more central location.

This is what would later be referred to as the short tail effect (a word from the search engine terminology) : when you are a content provider, you have very little of your content that is being viewed a lot (therefore pre-delivered on your edge nodes), and most of your content that can be cold stored for later delivery.

Flash video, the new stake


The mandatory use of CDN came quite naturally.
One day, some very smart person released some software named Flash, initially a vector motion scripted language that enabled the generation of rich animated, event-driven objects (pretty much programs) to be embeded within HTML pages. The day after, every single browser on the face of Internet had the flash player plugin installed.

Some time later, those same very smart folks out there added some video primitives within Flash, that brought a video envelope (FLV files, flash encapsulated video files) to be read while being downloaded from a Flash application on any web page. This technique was given the name of HTTP Progressive Download or Flash Pseudo Streaming. The key factor in all that was that you could pretty much 'stream' any video from a regular Web Server, just using plain TCP80/HTTP protocol. We very soon saw literally thousand video sites becoming the web's favorite destinations.

Problem is, when you're a geeky web developer, you don't really know anything about Network Bandwidth or Hosting, and honestly, you don't really care about it. When you wake up every day and the bandwidth you are consuming has doubled since the day before, you're kinda facing a decent amount of issues:

  • the hosting company you use delivers your content from one central location, therefore, you might upset some of your 'remote' fans and lock the growth of your audience
  • working around architecture and capacity issues is not what you want to do
  • your business model is yet undefined, so massive purchase of network infrastructure and bandwidth would not be that smart at that point of your startup's timeline...


This is when the $CDN sales rep comes knock on your startup garage door (in Califonia, so it definitely sounds like a cliche).
And he tells you some pretty appealing stuff: he will help you focus on your business:

  • expanding your audience worldwide (you know that monetizing chimera)
  • developing killer features, some even relying on his newest flash CDN offer.


He'll basically help you save time and money on not building and maintaining any network infrastructure... How nice is that ?

Epilogue: how to make this post sound not-serious-at-all-eventhough-it-is


After a nice and pricey meal in a trendy restaurant downtown, you'll soon figure out this sales rep is the "new guy in town". He knows personally all of your Web2.0 idols (some even are his customers), he's been in every single entrepreneur gathering on the west coast in the morning, drinking Nappa wine with Kevin Rose and/or Leah Culver in the evening ! He says he's even actually entered the googleplex ! He's standing right in the crossroad of Internet and Broadcast media, waiting for you

That's when you know you need CDN. Whenever this guy with the finest mix of ultra-bright smile and Italian pret-a-porter comes to knock on your garage door, that's when you are going to need CDN.
You better prep for this day so you don't sound like a perfect noob when he comes and tells you those tales of the new CDN era !

Until that blessed day comes, I'll try and keep you posted on this hectic industry in future posts, because ...oh, by the way, it is my new job.