How Data Virtualization Helps Data Discovery
Data virtualization, like its server, applications, and storage virtualization counterparts, is proving an excellent non-invasive method for delivering better agility and lower total cost of ownership to today’s business environment of extreme data complexity, endless disparate data silos, and exponentially growing data volumes. The combination of data virtualization with data discovery accelerates and expands the potential business benefits. Data virtualization solutions, often called virtual data federation or Enterprise Information Integration (EII) middleware, can be used to provide a wide range of complementary virtual data sources that abstract away complexity and federate disparate data to provide additional insight (see Figure 2). In the customer service example, a data virtualization solution would have provided a consistent set of customer, shipment, and delivery views or data services that shortcut the “data surfing” and “complete the picture” steps. Further, data virtualization is an ideal means for rapidly institutionalizing a newly discovered way of querying and consuming structured data. As a result, the new “undelivered shipments” view described would have been created and deployed in a data virtualization server in just minutes.

Getting Started with Data Discovery
The first step is to build the business case. This may be done by surveying 20 business professionals with questions such:

1. What kinds of questions need answering, problems solved, and decisions made and what is their current impact on revenues, costs, and business risks?
2. How many hours does finding the answer currently take every day? Is this number increasing or decreasing?
3. What percentage is necessary for making informed decisions? Is this number increasing or decreasing?
4. If answers and solutions and decision-making could be achieved 10% quicker, how much would this be worth to the enterprise? What about 20%?
5. If a well-informed answer, solution, or decision could be made 10% more frequently, how much would this be worth to the enterprise? What about 20%?

Extrapolating this data across all the business professionals who might benefit from a data discovery solution will typically provide an internal rate of return several times that of most projects in the IT queue.

With the justification in hand, it’s time to select a product that meets the enterprise’s specific structured data requirements. Remember the importance of relational inferencing and the Excel metaphor when dealing with tabular structured data. So don’t just pick hot products from a search vendor who made its name in text search. And regardless of the marketing glossies, right now no single vendor can do a great job with both structured and unstructured data, so it’s most prudent to choose a structured specialist.

However, as noted here, combining data discovery and data virtualization has significant benefits in terms of increased agility, better information, and decreased complexity. So a vendor who can provide both, in an intelligently integrated package, should get extra consideration. And since this is a newly emerging market, keep an open mind about considering start-ups and early stage companies. Often, these companies are the innovators best positioned to lead a new generation.

After the selection is completed, introduce the data discovery product to a select group of influential BI and Excel power users. In their hands, they will quickly build a treasure chest of success stories and best practices that will provide critical mass adoption momentum through organic adoption.

Best of Times with Data Discovery
In today’s business environment of “best of times/worst of times” when it comes to the exponential growth and complexity of data systems, enterprises are seeking better ways to leverage their massive investments in the IT systems that house this information and the IT professionals that manage these systems. The nascent technology, data discovery, is anticipated to deliver the next step in data integration by providing business users easier access to the business-critical information they need day-to-day, while at the same time freeing IT professionals from mundane report creation and generation to tackle alternative projects that more directly contribute to bottom-line profitability.

Resources

• Boris Evelson. “Data, Data Everywhere!” July 23, 2007. Forrester Research.
• “Needle, Meet Haystack: The Useless Hunt for Data Information.” InformationWeek. January 8, 2007, Page 19. .