In the older days data was spoken about in terms of gigabytes, now we use petabytes of data. Obviously due to the evolving technologies, the amount of data available to us has gone up in volumes. The question here is what we do actually do with the data?
Google, Yahoo, Amazon, Facebook, Twitter are only some of the most popular companies that generate terabytes of data every second. Since these are customer driven, the most efficient use of the data generated is to improve user experience. In a telecom domain, the service providers want to keep track of customer behaviour on a day to day basis. This is so that they can introduce customised offers to prevent churning or attrition. A manufacturer would want to introduce a product only in the most profitable segment and optimise marketing spend. Big retailers may be interested in giving value added offers to the customers that will in turn help in retention and business profitability.
So while one can get engaged in the world of technology, from Analytics point of view, the most essential element of Big Data is to keep an overall eye on the business objective.
Essentials for Big Data include – data, technology, business solutions and analytics skills.
- Data – As the name Big Data implies, what we are talking about is datasets at a very large scale. From Analytics point of view, one needs to filter out the relevant data because such data can be quite unstructured. Data Visualization, integration, matching, merging, cleaning, scaling and standardising are aspects essential to preparing the Big Data for Analytics.
- Technology – The traditional technologies used for data analytics are not only incapable of handling/ storing big data but it can also be very costly affair. The Big Data technologies are still evolving. The most commonly used technologies are Hadoop, MapReduce and Nosql. Most of these are open source but require programming knowledge.The advantage of open source technologies is that they help in analysing Big Data cost effectively. Since the algorithms are still evolving, the pitfall at this point is reliability and accountability of technologies.Thus the technology essential for Big Data is twofold – Data warehousing/storage and algorithms to build solutions.
- Business Solutions – It is important to keep an eye on what is the ultimate goal of performing the analysis. Whether it’s providing value to the business and actionable items to improve profitability. The business intelligence goes hand in hand with technology while working on Big Data.
- Analytics Skills – While the other three points are essential to big data what is perhaps the key to the success of all of them is having the right people with the right skills. Though the field of Analytics itself is booming there is a big gap in supply versus demand. Big Data analysis naturally requires staffing, time and funds much more than conventional analytics. Big Data scientist needs both programming and analytical skills with relevant domain as well as business knowledge and great communication and persuasion skills. They need to be well versed in information/technology management, as well as have an analytic mindset.
In principle, the data analytics part remains the same as traditional analytics in terms of scope. What has changed or increased with Big Data is the volume and variety. It’s the scale of data that makes all the difference.