SDI Paves the Way for Analytics Workloads

In a series of earlier posts, we took a trip down the road to software-defined infrastructure (SDI). Now that we have established an understanding of SDI and where it is today, it’s a good time to talk about the workloads that will run on the SDI foundation. This is where SDI demonstrates it true value.

Much of this post will assume that your code is developed to be cloud-aware (and you understand what those changes are to help). Cloud-aware apps know what they need to do to fully leverage the automation and orchestration capabilities of an SDI platform. They are written to enable expansion and contraction automatically and to maintain optimal levels of performance, availability, and efficiency. (If you want some additional discussion around cloud-aware, just let me know. It’s another topic that’s close to my heart.)

With cloud aware taken care of, one key workloads targeting the SDI landing zone is business analytics, which is getting a lot of press today as it rises in importance to the enterprise. Analytics is the vehicle for turning mountains of raw data into meaningful business insights. It takes you from transitions to trends, from customer complaints to sentiment analysis, and from millions of rows of log data to hackers’ intent.

Analytics, of course, is not new. Virtually all IT shops have leveraged some form of analytics for years, from simple reporting presented in Excel spreadsheets to more complex data analysis and visualization. What is new is a whole set of technologies that allow for doing things differently, using new data and merging these capabilities. For example, we now have tools and environments, such as Hadoop, that make it possible to bring together structured and unstructured data in an automated manner, which was really difficult to do in an automated way. Over the next few blogs, I will talk about how analytics is changing and how companies might progress through the analytics world in a stepwise manner. For now, let’s begin with the current state of analytics.

Today, most organizations have a business intelligence environment. Typically, this is a very reactive and very batch dependent environment. In a common progression, organizations move data from online data sources into a data warehouse through various transformations, and then they run reports or create cubes to determine impact.

In these environments, latency between initial event and actual action tends to be very high. By the time data is extracted, transformed, loaded, and analyzed, its relevance has decreased and the associated costs continue to rise. In general, there is the cost of holding data and the cost of converting that data from data store to data warehouse, and these can be very high. It should be no surprise then that decisions on how far you can go back and how much additional data you can use are often are made based on the cost of the environment, rather than the value to the business.

The future of this environment is that new foundation technologies—such as Hadoop, in-memory databases NoSQL and graph databases with the use of advanced algorithms, and machine learning—will change the landscape of analytics dramatically.

These advances, which are now well under way, will allow us to get to a world in which analytics and orchestration tools do a lot of hard work for us. When an event happens, the analytics environment will determine what actions would best handle the issue and optimize the outcome. It will also trigger the change and lets someone know why something changed … all automatically and without human intervention.

While this might be scary for some, it is rapidly becoming a capability that can be leveraged. It is in use today on trading floors, for example, to determine if events are illegal or to trigger specific trades. The financial industry is where much of the innovation around these items is taking place.

It is only a matter of time where most companies will be able to figure out how to take advantage of these same fundamental technologies to change their businesses.

Another item to keep in mind is as organizations make greater use of analytics, visualization will become even more important. Why? Because a simple spreadsheet and graph will not be able to explain what is happening in a way humans will be able to understand. This is where we start to see the inclusion of what has existed in the high performance computing areas for years around modeling and simulation. These visualizations will help companies pick that needle out of a data haystack in a way that helps them optimize profits, go after new business, and win new customers.

In follow-on posts, I will explore the path forward in the journey to the widespread use of analytics in an SDI environment. This is a path that moves first from reactive to predictive analytics, and then from the predictive to the prescriptive.  I will also explore a great use case—security—for organizations getting started with analytics.