WeakAL: Combining Active Learning and Weak Supervision

Research output: Contribution to book/conference proceedings/anthology/reportConference contributionContributedpeer-review

Contributors

Abstract

Supervised Learning requires a huge amount of labeled data, making efficient labeling one of the most critical components for the success of Machine Learning (ML). One well-known method to gain labeled data efficiently is Active Learning (AL), where the learner interactively asks human experts to label the most informative data point. Nevertheless, even by applying AL in labeling tasks the amount of human effort is still too high and should be minimized further. In this paper therefore we propose WeakAL, which incorporates Weak Supervision (WS) techniques directly into the AL cycle. This allows us to reduce the number of annotations by human experts while keeping the same level of ML performance. We investigate different WS strategies as well as different parameter combinations for a wide range of real-world datasets. Our evaluation shows that for example in the context of Web table classification, 55% of otherwise manually retrieved labels can be generated by WS techniques with a negligible loss of test accuracy by 0.31% only. To further prove the general applicability of our approach we applied it to six datasets from the AL challenge from Guyon et al., where over 90% of the labels could be computed by the WS techniques, while still achieving competitive competition results.

Details

Original languageEnglish
Title of host publicationDiscovery Science
EditorsAnnalisa Appice, Grigorios Tsoumakas, Yannis Manolopoulos, Stan Matwin
PublisherSpringer, Berlin [u. a.]
Pages34-49
Number of pages16
ISBN (print)9783030615260
Publication statusPublished - 2020
Peer-reviewedYes

Publication series

SeriesLecture Notes in Computer Science, Volume 12323
ISSN0302-9743

Conference

Title23rd International Conference on Discovery Science, DS 2020
Duration19 - 21 October 2020
CityThessaloniki
CountryGreece

External IDs

Scopus 85094100383
ORCID /0000-0002-5985-4348/work/162348852

Keywords

Keywords

  • Active Learning, Classification, Information extraction, Machine Learning, Semi-supervised, Weak Supervision

Library keywords