In the next few posts, I’ll be analyzing the Apple CDN build-out, discussing the challenges they face, the cost for POP build-outs, strategy, talent gap, and so on. According to Rayburn, Apple has been busy building out its CDN infrastructure buying transit, wavelengths, servers, and colo from multiple providers. The caching software platform powering their CDN is the Apache Traffic Server (ATS). That’s all the info we need to do a deep dive analysis. First off, the Apple CDN is going to be orders of magnitude more complex than the Netflix CDN and Comcast CDN. The Netflix CDN and Comcast CDN are regional based CDNs delivering content almost exclusively to the US market, or parts of the US, like what Comcast is doing.
Next, Netflix and Comcast primarily deal with VOD content delivery. In the light of all things, VOD delivery is the simplest type of delivery. In contrast, Apple needs to build an extensive global CDN infrastructure with a presence in all major metropolitan areas, courtesy of the iPhone. In addition, Apple has to deal with small file delivery, and dynamic site acceleration over the wireless last mile, the two most complex types of delivery. Small file delivery over wireless is complex, especially when it is dynamic, because it requires bits and pieces of DSA and FEO like functionality, to make delivery work optimally. So not only does Apple need to build out a CDN platform, but they need to develop a robust feature set, something that is very difficult to do. There are entire CDNs that specialize in a particular CDN feature set, like Instart Logic, Incapsula and Yottaa. If Apple plans on building a self-sustaining CDN, they will need to build the most sophisticated CDN ever built, costing several hundred million dollars, that is on par with some parts of Akamai.
First Challenge – Talent Gap
Buying colo, transit, servers, storage and all other hardware pieces is the fun and easy part. Implementing the caching software on the servers, and setting up peering is a little more difficult. Creating the robust feature set is the most difficult part. Even some established CDNs are finding it difficult in building out robust feature sets, and they have been in business for many years. It’s difficult technically, but even more challenging is dealing with the supply and demand curve for this type of talent. The Http caching engineer are in very short supply. The savviest ones are what I refer to as the engineer’s engineer. When a smart engineer has a challenge they can’t solve, they typically go to this kind of engineer to solve the problem. These savvy engineers mindset are just “out there”, literally speaking. Also, the majority of these engineers happen to work for CDNs.
By default, Akamai has the largest staff of Http caching engineers of any CDN. And my guess is that Apple, having used Akamai, Limelight, and Level 3, have an agreement in place that restricts Apple from hiring away their CDN talent, especially their talented Http caching engineers. Even if Apple were to offer the CDN lots of money for their staff, my guess is the CDN would say no, because if that talent leaves, than whose going to build the next iteration of that CDN. Yes, Apple can use other extremely smart developers and train them on caching, however, experience is just as important as skill set. An experienced Http caching engineer can probably cut the development time in half, if not more. If it takes an extremely savvy non-experienced caching engineer 6 months to build a CDN feature, will it take an experienced caching engineer one month to build? Maybe. Continued in the next post.