Many professionals who use business intelligence (BI) tools agree that data discovery is crucial to an organization’s ability to derive true value from its enterprise information assets.
Data discovery, which is similar to data visualization but with an interactive component, can be supported by many software solutions on the market today, including advanced BI platforms with innovative data visualization capabilities. While enterprises have identified data discovery as a solution they desire for their business professionals on the front lines and IT supporting the business from the back end, many could benefit from envisioning what their true data visualization needs are.
Data discovery defined
Data discovery can be defined as the ability of end users to use a tool to access data from many sources, aggregate and combine subsets of data from these data sources, and analyze them to detect unknown patterns and trends. It’s also a way for an end user to find insights particular to his or her role, perhaps in a search for a pattern that a business intelligence analyst did not have in mind.
Data discovery also eliminates the need for end-users to learn complicated tools in order to interact with data, and enables real-time analysis without the need for cubes or pre-built data models.
For companies looking to analyze and integrate massive volumes of raw information generated from and residing in multiple disparate sources, data discovery can enhance the way enterprise data is accessed and exploited for strategic planning.
Data discovery alone is not enough
Data discovery and visualization tools appeal to enterprises because it can fulfill the need to integrate data from multiple data sources including enterprise data sources, external sources and even personal files and present it in a way that’s accessible to anyone in the business. As a unified data set for analysis, workers across the business can access, analyze and view the information.
Visualization makes information easier to interpret, understand and retain; when raw data is depicted through images and graphics, it becomes much easier to recognize patterns, dependencies, anomalies and other trends that could hold business value.
The allure of graphic displays of data can be misleading, however, because data discovery tools on their own will not meet the needs of most enterprises.
If the organization’s goal is to maximize the ROI from enterprise information by enabling more people to benefit from it, then data discovery tools may not be the right approach. This is because they have limited usability, do not scale easily, and cannot satisfy operational demands of real-time information.
How can enterprises get beyond these potential barriers while still achieving the benefits of data visualization?
To determine what type of BI solution can address data visualization needs, enterprises must address the five questions below.
Should the solution empower all decision-makers across the business?
Business professionals want to be empowered with tools, like BI apps, that allow them to make decisions quickly and easily.
Users of all types and skill levels should be able to explore and analyze data in the way they are most comfortable. The broader the user base, the more likely BI adoption will spread.
BI platforms that can diversify how users interact with data can rapidly advance user acceptance. For instance, information specialists and developers can work with easy-to-use tools to deliver dashboards and reports quickly for all users, non-technical business users can access simple BI apps to allow them to explore and analyze their data and analysts need easy-to-use and powerful visualization tools.
In all cases, if a business deploys a visualization tool without considering the required visualization expertise of different users, it risks creating graphics that don’t translate to real, actionable business intelligence.
Do rogue dashboards create more questions than answers?
Having an enterprise-wide data governance policy will help mitigate the risk of a data breach. This includes defining rules and processes related to dashboard creation, ownership, distribution and usage, and restrictions on who can access what data and ensuring that employees follow their organizations’ data usage policies. Many data discovery toolsets do not offer metadata management and data integrity solutions as part of the package – they often have to be bought separately, which complicates business processes and workflows unnecessarily.
Security also poses a problem with data discovery tools. IT staff typically have little or no control over these types of solutions, which means it can’t protect sensitive information. This can result in unencrypted data being cached locally and viewed by or shared with unauthorized users. The solution must offer adequate data management and security to be acceptable for both IT and business concerns.
How can companies preserve data integrity and prevent flawed insights?
Metadata matters. Many data discovery tools lack the control, reusability and analytical integrity of metadata management, which means users must truly know and understand the information with which they are working to effectively use the tools.
Data discovery tools expect relatively clean data in a tabular format—they do not provide profiling, cleansing, or standardizing of enterprise information. The point of effective visualization is to design and implement dashboards that communicate correctly. Companies that employ data discovery tools may generate beautiful graphics, but the information they depict may be misleading, inaccurate or invalid.
Are there costs associated with in-memory limitations?
Enterprises value BI because it helps to efficiently access and analyze large data volumes for a variety of critical purposes. One popular approach is to put all of the data into server RAM to take advantage of the inherent I/O rate improvements over disk, which has spawned a trend of using in-memory analytics for increased BI performance. But in-memory analytic solutions struggle to maintain performance as the size of the data goes beyond the fixed amount of server RAM.
For in-memory solutions to scale, they need a lot of hardware and a complex multi node sever deployment. That means either hiring an expert with the right technical skills to administer the environment, or purchase pre-built appliances that contain multiple nodes—both high-cost options.
Choosing the Right BI Platform
A large functionality gap exists between BI platforms and data discovery tools. Data discovery tools typically offer very narrow capabilities that are limited primarily to visualization and dashboarding. BI platforms can offer much more functionality, such as reporting, distribution and ad hoc query capability; predictive analytics; complex queries; and social media analytics.
By achieving visualization through a BI platform that is fast and flexible, organizations can go beyond the limited capabilities and data quality issues of a niche solution, and serve the diverse information needs of enterprise users, partners, suppliers, and customers. Data discovery alone is not a replacement for a BI platform that fits needs of different personas and departments across an enterprise.
About the Author: Dr. Rado Kotorov is Chief Innovation Officer for Information Builders.