Towards a Hybrid Imputation Approach Using Web Tables

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenBeitrag in KonferenzbandBeigetragenBegutachtung

Beitragende

Abstract

Data completeness is one of the most important data quality dimensions and an essential premise in data analytics. With new emerging Big Data trends such as the data lake concept, which provides a low cost data preparation repository instead of moving curated data into a data warehouse, the problem of data completeness is additionally reinforced. While traditionally the process of filling in missing values is addressed by the data imputation community using statistical techniques, we complement these approaches by using external data sources from the data lake or even the Web to lookup missing values. In this paper we propose a novel hybrid data imputation strategy that, takes into account the characteristics of an incomplete dataset and based on that chooses the best imputation approach, i.e. either a statistical approach such as regression analysis or a Web-based lookup or a combination of both. We formalize and implement both imputation approaches, including a Web table retrieval and matching system and evaluate them extensively using a corpus with 125M Web tables. We show that applying statistical techniques in conjunction with external data sources will lead to a imputation system which is robust, accurate, and has high coverage at the same time.

Details

OriginalspracheEnglisch
Titel2015 IEEE/ACM 2nd International Symposium on Big Data Computing (BDC)
Redakteure/-innenRajkumar Buyya, Ioan Raicu, Omer Rana
Herausgeber (Verlag)Institute of Electrical and Electronics Engineers (IEEE)
Seiten21-30
Seitenumfang10
ISBN (elektronisch)978-0-7695-5696-3
PublikationsstatusVeröffentlicht - 11 Feb. 2016
Peer-Review-StatusJa

Konferenz

Titel2nd IEEE/ACM International Symposium on Big Data Computing, BDC 2015
Dauer7 - 10 Dezember 2015
StadtLimassol
LandZypern

Externe IDs

ORCID /0000-0001-8107-2775/work/198592309

Schlagworte

Forschungsprofillinien der TU Dresden

Fächergruppen, Lehr- und Forschungsbereiche, Fachgebiete nach Destatis

Schlagwörter

  • Data preprocessing, Machine learning, Web mining