Neda Abolhassani & Teresa Tung | Accelerating Industry Data Integration
KGC 2021 • 19m
A data supply chain is industry-specific, but many data prep tools are industry agnostic. As part doing this work, data engineers and domain experts apply their deep knowledge of how to transform raw data to a form that can address specific problems. In this way, the data supply chain is a domain like so many others to which AI is applied. It involves making decisions repeatedly informed by past experience.
Our work aims to leverage our own experience, to capture and apply industry and domain experience to accelerate industry data integration. To tackle this problem we’re taking an approach that layers a Knowledge Graph on top of the Data Mesh to capture domain knowledge and to connect the different data domains.
Like many others, we’re excited about Zhamak Dehghani’s new Data Mesh architecture paradigm. It’s distributed architecture follows many of the data integration strategies from the web, and includes mature rules around data products, quality, and governance.
This talk illustrates how some machine learning techniques can help in data profiling and mapping of siloed data sources for Enterprise Knowledge Graph construction. We present how this Knowledge Graph can be employed in a Data Mesh architecture for accelerating industry data integration for an Oil and Gas use case.
Up Next in KGC 2021
-
Julian Grummer | What Can We Learn Fr...
The Wirecard scandal was one of the most shocking economic events in Germany in 2020. The former DAX30 company collapsed on June 25, owing creditors more than €3.5 billion (almost $4 billion) after disclosing a gaping hole in its books that its auditor EY said was the result of a sophisticated gl...
-
Mohammed Aaser | Future Of Enterprise...
Many organizations have initiated data and analytics transformations with some success, however are beginning to face challenges in scaling efforts beyond a handful of applications/use cases. One of the major barriers remains around data management, including challenges with data transparency, i...
-
Zhamak Dehghani | Introduction To Dat...
For over half a century organizations have assumed that data is an asset to collect more of, and data must be centralized to be useful. These assumptions have led to centralized and monolithic architectures such as data warehousing and data lake, and neither of which have been able to enable data...
