Big data has changed the world of business.
Companies are now able to organize, analyze and derive value from millions of rows of data that just a few years ago would have been meaningless.
To get some insights on where big data is headed in 2016 and beyond, we reached out to three leading experts for their predictions.
Big data gets cloudy, ETL gets personal and NOSQL is on the rise.
The cloud is everywhere, and we will continue to see adoption at extreme volumes. And big data is driving a lot of cloud growth: Revenues for the top 50 public cloud providers shot up 47 percent in Q4 of 2013 to $6.2 billion according to Technology Business Research. Amazon Redshift and Google Bigquery are growing dramatically. Database players like Teradata are also jumping in the game.
It has been suggested that 80 percent of an analyst’s time is spent on data prep, while only 20 percent is spent looking for insights. Enter the personal data cleansing tools focused on the analyst. Tools like Trifacta, Alteryx, Paxata and Informatica Rev are making data preparation easier to use with less technology and infrastructure required to support it.
Some may think that the jury is still deliberating, but NoSQL is making a mark in the industry. NoSQL was founded to provide scale, flexibility and the ability to leverage large sets of data faster. Companies like MarkLogic, Casandra, Couchbase and MongoDB are bringing new innovation to the SQL database market and are doing quite well with large production implementations in surprising places.
VP of Product Marketing
Big data will present significant challenges with regard to securing sensitive information and protecting individual privacy, and the only answer will be open technologies, standardized interconnectivity and collaboration on a global scale.
I think we can get a good sense of where big data is moving to from some of the efforts going on today in various industries and disciplines. A good example is the OpenPOWER Foundation effort, which is based on IBM’s POWER platform. IBM has opened up much of the platform in a way unheard of a couple years ago, enabling collaboration among various organizations in order to innovate along trajectories not always possible when operating behind closed doors.
When we start talking about the systems necessary for advanced scientific computing and analysis, as well as machine learning and real-time event streaming, we’re likely to conclude that it’s no longer possible to go it alone, that open platforms and standards will ultimately be the only way to meet the demands of the type of data processing that will be coming our way. By opening up the technology, we invite the innovation and collaboration needed to meet the needs of data head on.
But OpenPOWER is not the only movement worth noting to help gain an understanding into what might be coming. Microsoft efforts such as Event Hubs and Stream Analytics point to the plethora of data expected from the string of connected Internet of Things (IoT) devices. Event Hubs makes it possible to stream real-time data from a wide range of devices and then send it on to Stream Analytics to make sense of it. What makes such solutions possible is embracing open standards and cloud technology, two areas that Microsoft was relatively slow to warm up to. But Microsoft has no doubt been making up for lost time and has become a formidable force in the cloud frontier. Like IBM, they saw the writing on the wall and are responding accordingly.
Regardless of the technology or platform or type of solution, integration and interoperability, along with open collaboration, represent the foundations on which the big data solutions of the future will be built.
Robert H. Sheldon
Technical Consultant & Freelance Technology Writer
The adoption of the enterprise data hub concept will result in immediate enhancements to developer agility. Result: There will soon be two types of companies: those using big data to dominate in the market – and those being dominated by others.
Big data use cases continue to evolve, and in the coming months we expect to see an increasing number of business missions being served by new approaches to data. With more organizations adopting concepts of an enterprise data hub, developers are finding new agility to create services and applications that include increasingly elegant use cases.
Traditional missions for big data that include fraud detection, cyber security [and] customer 360-degree views [for] ecommerce are being joined by new use cases like corporate M&A decisions, critical employment decisions, targeted marketing/messaging and market influence.
By mid-2016 we expect to see corporations using enterprise data hub approaches to enhance agility and decision-making that leads to market dominating strategies. Across multiple sectors, we can expect a new corporate differentiator to be how the use of data leads to agility. By the end of 2016, there will be two types of companies: those using big data to dominate – and those being dominated.