Implementing Software Defined Infrastructure for Hyper-Scale

Earlier this summer, Intel announced our Cloud for All initiative signaling a deepening engagement with the cloud software industry on SDI delivery for mainstream data centers.  Today at IDF2015, I had my first opportunity post the announcement, to discuss why Cloud for All is such a critical focus for Intel, for the cloud industry, and for the enterprises and service providers that will benefit from enterprise feature rich cloud solutions. Delivering the agility and efficiency found today in the world’s largest data centers to broad enterprise and provider environments has the opportunity to transform the availability and economics of computing and reframe the role of technology in the way we do business and live our lives.

Why this focus? Building a hyperscale data center from the ground up to power applications written specifically for cloud is a very different challenge than migrating workloads designed for traditional infrastructure to a cloud environment.  In order to move traditional enterprise workloads to the cloud, either an app must be rewritten for native cloud optimization or the SDI stack must be optimized to support enterprise workload requirements.  This means supporting things like live workload migration, rolling software upgrades, and failover. Intel’s vision for pervasive cloud embraces both approaches, and while we expect applications to be optimized as cloud native over time, near term cloud adoption in the enterprise is hinged upon SDI stack optimization for support of both traditional applications and cloud native applications.

How does this influence our approach of industry engagement in Cloud for All?  It means that we need to enable a wide range of potential usage models while being pragmatic that a wide range of infrastructure solutions exists across the world today.  While many are still running traditional infrastructure without self-service, there is a growing trend towards enabling self-service on existing and new SDI infrastructure through solutions like OpenStack, providing the well-known “give me a server” or “give me storage” capabilities…  Cloud Type A – server focused.  Meanwhile SW developers over the last year have grown very fond of containers and are thinking not in terms of servers, but instead in terms of app containers and connections… a Cloud Type B – process focused.  If we look out into the future, we could assume that many new data centers will be built with this as the foundation, and will provide a portion of the capacity out to traditional apps.  Convergence of usage models while bringing the infrastructure solutions forward.

The enablement of choice and flexibility, the optimization of the underlying Intel architecture based infrastructure, and the delivery of easy to deploy solutions to market will help secure broad adoption.

So where are we with optimization of SDI stacks for underlying infrastructure? The good news is, we’ve made great progress with the industry on intelligent orchestration.  In my talk today, I shared a few examples of industry progress.

I walked the audience through one example with Apache Mesos detailing how hyper-scale orchestration is achieved through a dual level scheduler, and how frameworks can be built to handle complex use cases like even storage orchestration.  I also demonstrated a new technology for Mesos Oversubscription that we’re calling Serenity that helps drive maximum infrastructure utilization.  This has been a partnership with MesoSphere and Intel engineers in the community to help lower the TCO of data centers; something I care a lot about…. Real business results with technology.

I also shared how infrastructure telemetry and infrastructure analytics can deliver improved stack management. I shared an example of a power & thermal aware orchestration scheduler that has helped Baidu net a data center PUE of 1.21 with 24% of potential cooling energy savings.  Security is also a significant focus, and I walked through an approach of using Intel VT technology to improve container security isolation.  In fact, CoreOS announced today that its rkt 0.8 release has been optimized for Intel VT using the approach outlined in my talk, and we expect more work with the container industry towards delivery of like security capabilities present only in traditional hypervisor based environments.

But what about data center application optimization for SDI?  For that focus, I ended my talk with the announcement of the first Cloud for All Challenge, a competition for infrastructure SW application developers to rewrite for cloud native environments.  I’m excited to see developer response to our challenge simply because the opportunity is ripe for introduction of cloud native applications to the enterprise using container orchestration, and Intel wants to help accelerate the software industry towards delivery of cloud native solutions.  If you’re an app developer, I encourage you to engage in this Challenge!  The winning team will receive $5,000 of cold, hard cash and bragging rights at being at the forefront of your field.  Simply contact cloudforall@intel.com for information, and please see the preliminary entry form.