Mobility and Big Data - A Match Made in ... Cloud
Guest article contributed by Peter Rogers, Principal Architect at Cognizant
As a principal architect at Cognizant, I spend much of my time helping large enterprises with their mobility, cloud and social strategies. I also have a passion for anything related to Big Data, and I follow this evolving domain with great interest.
In fact at Cognizant, we refer to these collectively as the SMAC Stack. And so it was with great excitement that I learnt about Appcelerator's acquisition of Nodeable that brings together these enormously transformative areas.
The essence of Big Data is about dealing with enormous amounts of non-structured data (volume), coming in from multiple sources (variety) and at very high rates (velocity) – collectively known as the 3V’s. It’s about rapidly finding information within that data to provide business intelligence and help with informed decision-making.
Naturally, there are many potential applications of Big Data across a wide range of markets, including mobility. Popular customer-facing mobile apps can now reach tens or even hundreds of millions of people around the world in a matter of days (or even hours), so there’s plenty of real-time data to mine. That data can contain geographical elements, social interactions, device events as well as user and application behaviour.
Today, the most popular solution for the Big Data challenge is Apache’s Hadoop (along with the NoSQL databases), which offer a low cost and scalable approach to the distributed computing of very large amounts of data on commodity hardware. Hadoop, however, is better suited for batch processing rather than real-time systems. Storm was subsequently developed at Twitter, and offers true real-time stream processing. It’s also available as open source.
What Nodeable did was to take Storm and host it in the Cloud, offering it as StreamReduce to provide customers with a real-time analysis of their data streams via a Twitter-like interface. It’s also used as a complement to Hadoop, so that after the StreamReduce engine aggregates, normalizes and analyzes the data it can be served to Hadoop to significantly accelerate the batch processing under MapReduce.
Although the Nodeable solution was initially targeted for IT operations data in a very different use-case, I believe Appcelerator was the first vendor to not only see the potential of combining large scale, real-time data processing in the cloud for the mobility use-case, but also to act on it. The acquisition will help the company round out its enterprise mobile cloud offerings and provide customers with the ability to make sense of the enormous amount of mobile data being generated. And true to the Appcelerator philosophy, Nodeable’s StreamReduce will be open-sourced and posted on GitHub.
I’m sure there are many potential use-cases here, but getting a granular understanding of the patterns and trends within the mobile application traffic, seeing what features are popular, and how users interact with the apps will provide deep and valuable insight to any customer.
I think we’re still in the very early days of even knowing how to harvest such enormous amounts of mobile data, let alone turning it into something meaningful. But I see Appcelerator’s acquisition of Nodeable as a very important step on the path towards this. Mobility and Big Data really are a match made in heaven cloud.
Peter Rogers is a principal architect at Cognizant and can be reached at: Peter-2.Rogersfirstname.lastname@example.org