ImitAL: Learned Active Learning Strategy on Synthetic Data

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

Abstract

Active Learning (AL) is a well-known standard method for efficiently obtaining annotated data by first labeling the samples that contain the most information based on a query strategy. In the past, a large variety of such query strategies has been proposed, with each generation of new strategies increasing the runtime and adding more complexity. However, to the best of our knowledge, none of these strategies excels consistently over a large number of datasets from different application domains. Basically, most of the existing AL strategies are a combination of the two simple heuristics informativeness and representativeness, and the big differences lie in the combination of the often conflicting heuristics. Within this paper, we propose ImitAL, a domain-independent novel query strategy, which encodes AL as a learning-to-rank problem and learns an optimal combination between both heuristics. We train ImitAL on large-scale simulated AL runs on purely synthetic datasets. To show that ImitAL was successfully trained, we perform an extensive evaluation comparing our strategy on 13 different datasets, from a wide range of domains, with 7 other query strategies.

Details

Original languageEnglish
Title of host publicationDiscovery Science
EditorsPoncelet Pascal, Dino Ienco
PublisherSpringer Science and Business Media B.V.
Pages47-56
Number of pages10
ISBN (print)978-3-031-18839-8
Publication statusPublished - 2022
Peer-reviewedYes

Publication series

SeriesLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13601 LNAI
ISSN0302-9743

Conference

Title25th International Conference on Discovery Science, DS 2022
Duration10 - 12 October 2022
CityMontpellier
CountryFrance

External IDs

ORCID /0000-0001-8107-2775/work/174431842
ORCID /0000-0002-5985-4348/work/174432435

Keywords

Keywords

  • Active learning, Annotation, Imitation learning, Learning to rank