Business Intelligence for Your Cloud

As cloud computing hits the initial incline of the maturity curve you begin to see a coupling of capabilities from a variety of disciplines, which may have previously been considered to be strange bedfellows. There are many examples of this such as security's impact on power usage efficiencies through the enabling of multi-tenancy. The one I want to focus on in this blog posting is that of business intelligence for your cloud operations. At the surface this sounds benign enough as we are often asked to produce business intelligence reports for measuring things that matter to our respective organization but this blog posting is looking further into the future. In many ways the future is available now and as such, should be factored into what is a relatively green field in the scope of IT operations. That green field is the journey from virtualization to cloud computing. Understanding the term ‘green field’ is bold in any context but with the direction to virtualize in general you generate some separation of concerns, lift and shift is a term I’ve often heard to describe this type of situation. In that respect it is an opportunity to rethink how your approach to something as complex as cloud computing might evolve over time and how to align management practices over this new paradigm to exercise proper controls.

At the top of that maturity curve you often hear the term ‘utility’ computing thrown around as well. In fact my ‘crystal ball’ is in part shaped by Telco experiences in days of yore supporting first a billing system that collected data from a 5ESS switch and subsequently managing a reporting system for FCC fairness in order for one of the baby Bells to enter the long distance market. Some anecdotal takeaways here are that the 5ESS switch ran the network carrying phone calls so capturing adequate data to convert for BSS purposes and other financial systems was a main focus of operations. Secondly, was the OSS data from such efforts was then 'rolled up' in many ways, not only for internal study of profitability, etc. but also for regulators and thereby competitors' scrutiny as well due to the open nature of those data sets. These business intelligence rollups are what is required for executives to discuss the state of the business in terms like compliance and profitability. There is opportunity for net new applications built on a modern Platform as a Service offering such as VMware vFabric and running in a modern cloud infrastructure like CloudFoundry, to exhibit the elastic operational capabilities along with the transparency necessary to achieve a true utility model. The amount of applications that will not appreciate these opportunities for a rewrite in the near term is vast and as such the target pattern for what will likely be the mainstay of cloud computing until such time as the scale tips towards those ‘utility’ capable applications is what is known as 'IT as a Service'. More succinctly this is a cloud that provides enough automation and manageability for consumers to request capabilities in the cloud. Also for IT supporting those existing applications that have been made cloud ready to understand how that cloud ecosystem can support the requirements driven by a concurrent diversity of consumers.

Perhaps the best analog I can think of to illustrate how these new breed of cloud management tools combine to form a gestalt is the way manufacturing, supply chain and logistics as well as inventory management and point of sale have become such a cohesive whole that enterprises controlling all of these facets, from creation of goods to their retail sale, for instance, can literally optimize their entire operation by simply funding more advertising or other campaigns. This capability is enabled since they have complete and near real time visibility into the discrete functional elements of their enterprise as well as how they can add resources to each element to support greater throughput while understanding the boundaries of how much capacity they have in total as well as the limits of individual facets and how they impact adjacent, likely tangible, dependencies. Those who can master the ability to procure and manage this pattern within a cloud infrastructure, it's cost and methods to leverage it in the most efficient ways, will experience the highest margins and have a customer base that, coming from on-premise solutions with perpetual licensing and other high initial capital outlays, will be more amenable to consuming the cloud based service at a price point that matches their business terms, e.g. per user/time period, etc.

So let’s talk for a minute about the title of this posting, Business Intelligence for Your Cloud. For VMware and our customers the simple message of 'Your Cloud' reflects our belief that your journey to the cloud begins with virtualizing your own systems in your own data center. One of the main reasons for this is the requirement for your organization to internalize what it means to ‘control’ a virtualized infrastructure. I use the term 'control' more so in the vein of IT Portfolio Management or Balanced Scorecard, understanding that that which can be measured can be verified. The main reason for establishing this baseline of control is that as you begin to move from virtualized enterprise to a private and eventually a hybrid or other cloud model, there will undoubtedly be third parties inserted into how your enterprise IT gets delivered. It will become critical to transpose these mechanisms of control to your cloud-hosting provider, as an example, so that contracts and SLAs can be more concretely negotiated then verified in an active, ongoing basis. At the end of the day this will involve collecting business intelligence just as it does for most efforts in other, more tangible areas of the enterprise.

To cover what business intelligence FOR your cloud means I’ll start by saying what it isn’t, business intelligence IN your cloud. That would be an interesting blog topic but likely to include Business Activity Monitoring, Complex Event Processing, Service Oriented Business Intelligence, Real Time Decisioning, etc. configured to study some other line of business within your enterprise. At VMware business intelligence comes in the form of capturing the ‘instrumentation’ of all layers of your virtualization and cloud infrastructure. The operational data needed to accurately monitor these applications and their supporting virtualization infrastructure is available via VMware vFabric Hyperic, which provides mechanisms to correlate all of this time series data in collections that provide meaningful event based insight. You may want to also automate the tracking of operational status of SLAs using VMware vCenter AppSpeed, achieve event-based notification of configuration compliance using VMware vCenter Configuration Manager or utilize VMware vCenter Chargeback to get clear usage translated into costs.

These VMware applications afford the appropriate perspectives into cloud operations as they happen but anyone who has been responsible for a business intelligence effort recently knows that the data is required to make better forward looking decisions. That’s where tools like VMware vCenter CapacityIQ, which leverages historical trend data to recommend resource planning for virtualized infrastructure, come into play. There are also applications that utilize more predictive methods to turn operations data into near real time events in the form of VMware vCenter Operations (which now includes CapacityIQ and Configuration Manager in Advanced and Enterprise editions), where ‘events’ are correlated across all applicable cloud layers, including hardware, network and OS, measured over time and notifications and/or actions generated as anomalies occur. Now you're slicing, dicing and provisioning capacity in the cloud with the measurements you need to manage your operations in flight as well as planning for the future. Leveraging the abstraction of virtualization as a vehicle to the cloud at the infrastructure layer and leveraging this homogeneous instrumentation capability is likely much easier than trying to wrestle your existing enterprise servers’ and application assets’ operational data into something like a SIEM or Governance, Risk and Compliance (GRC) solution.

Gaining the proper perspective to harness the data coming from your virtualized infrastructure and supporting applications is not unlike capturing data from any sort of group of sensors in the tangible world. And like any other entity, harnessing that data in order to measure key performance indicators over time is the way to assert control, even over something remote and intangible in certain aspects like cloud IT operations. It is the line of sight and transformation of the data captured from all layers of the cloud operation, IT and business, that will enable your cloud to become a strategic, agile appendage of what your business aspires to accomplish and allow the CIO to participate fully in delivering new offerings as strategic differentiators in the marketplace. It will be important not only to give the proper perspective to all stakeholders but also to look at the cloud as a portfolio of assets that should also accept input from those stakeholders in terms they speak, effectively letting business drive the evolutionary cloud configuration.

Looking at CIO priorities it's clear they want the agility of cloud computing to make the future of IT a driver for business strategy, however, most remain wary of how much control they may be required to give up as they move to cloud computing. In most cases these two are juxtaposed however the move to virtualization and the cloud brings an opportunity to automate not only for agility’s sake but also for capturing operational data needed for all types of control to be established. Ultimately what this will mean is the ability to avoid reinventing a very complex wheel over and over. To revisit my Telco analogy as an example, a complex QoS managed circuit, tailored to an individual customer profile that is difficult to support and even more difficult to price effectively for profit and loss. As this architecture evolves productizing added capabilities on the fly, e.g. more Internet throughput, more HD channels, more bundled long distance minutes or calling features at a competitive, market driven price point, becomes inherent to the culture. Having business intelligence for your virtualization infrastructure will elevate IT directly into a strategic line of conversation as an asset to the business instead of a cost center or liability while delivering the means to control the move to your cloud.