Enterprise grade cloud computing

Cloud computing and the as-a-service paradigm have gained a lot of interest recently. The separation of service provider from infrastructure provider has made it much easier for new services to be established online quickly and with low financial risk, and to scale those services as demand dictates. They can be built to run directly on top of infrastructure such as Amazon EC2 1, on application platforms such as Google App Engine 3, or within higher level platforms such as FaceBook 2 or force.com 6, with increasing levels of ease of development and task specialization.It is clear why a startup company might be attracted to the cloud computing model. Equipment is very costly to purchase and can only be amortized over a period of years. Using someone else's infrastructure on a pay-per-use basis converts this fixed cost into a variable cost based on actual consumption; reducing initial investment and risk. Also the demand for online services can be very variable and poor response due to overload can risk losing customers. So the ability to scale the use of cloud compute power also mitigates the risk of failure.The arguments for an established enterprise are not the same. Such a business would have a well understood compute capacity and multi-year investment lifecycles. As the financial risk becomes less significant other issues come into play such as security, legislation, and dependence on the provider. Exactly where data resides is important as it will be accountable to the local legal system, especially where the main line of business is concerned. Security requirements may not be compatible with those offered by existing infrastructure providers.In reality enterprises already use a mix of services, some in-house, some contracted out. They may use a pay-per-use model to access outsourced payroll, travel arrangement, or even legal services. These may be provided by a cloud computing platform and integrated with further services such as credit card payment and courier distributors.For in-house services there may still be a cost advantage in using someone else's infrastructure -- if they are big enough. Studies have shown that the proportional cost of building and running data centers with tens of thousands of machines is significantly lower than one with just a few hundred. Where an enterprise does retain its own internal systems for IT or its main line of business, it is likely to be interested in using a private cloud; its own internal infrastructure and services managed in just the same way as an external cloud provider would do. But they may also be interested expanding out to external clouds to accommodate peak demands. An enterprise is likely to straddle the line between self-owned, on-premises compute facilities and third party cloud infrastructure.The future will bring a world of services interacting securely and running across multiple infrastructures, scaling and distributing as required. These considerations provide drivers for cloud computing research, both at service level and infrastructure level.At HP Labs we have been investigating service provision in a shared compute infrastructure for more than a decade. Past prototypes include SoftUDC 5, Frame Factory/SE3D, and the HP Utility Rendering Service 4 used by DreamWorks to create the Shrek and Madagascar movies among others. During the course of this work we have addressed a variety of challenges. How can clients with low bandwidth connectivity interact effectively with a service that involves very large volumes of data? How can independent services operate, flex and scale within the same shared infrastructure yet achieve sufficient isolation at the data and service levels? How can data and services be managed across multiple geographically distributed data centers?In addition to the technical challenges we also experienced client behaviors during these trials that have both motivated the use of cloud computing for individual users, but at the same time challenged the economics of the whole paradigm.Our current research addresses the challenges of enterprise-grade cloud computing, starting with the question: what would cloud computing need to provide for enterprises and enterprise service providers? In this talk we discuss our view of cloud computing and what we are doing to address the challenges of this new paradigm.