To Measure or not to Measure? Adaptive Repetition Management in Parameter Tuning
Research output: Contribution to conferences › Paper › Contributed › peer-review
Contributors
Abstract
Variance is an inherent part of manifold real-world empirical studies. It can appear due to non-deterministic behavior, measurement or systematic errors, information bias, etc. The former two causes are especially important for parameter tuning, as they can lead to an inaccurate search space estimation and a subsequent decrease in quality of the best found solution. Under parameter tuning we understand an optimization problem of determining the best configuration setting, given a system under investigation and a set of its configurations. Repetitive configuration evaluation is a standard approach to tackle the problem of variance. However, a high number of repetitions can overwhelmingly bloat an effort spent on the search, motivating the development of repetition management strategies. Nevertheless, numerous parameter tuning approaches are either neglecting this problem or suggesting repetition management strategies that are based on a predefined number of repetitions, which can vary for different problem instances. In this paper, we present a repetition management strategy that is inspired by the notion of acceptable measurement error to adaptively determine the number of repetitions for each configuration. Moreover, we investigate the influence of the knowledge on already performed measurements as the additional feedback for the presented repetition management strategy. We evaluate the effectiveness of our approaches with a set of energy optimization problems. The presented strategies prove to be more scalable and robust to changes, while the utilization of experiment-related knowledge can even further increase the quality of the found solution.
Details
Original language | English |
---|---|
Pages | 1363-1370 |
Number of pages | 8 |
Publication status | Published - 4 Dec 2020 |
Peer-reviewed | Yes |
Conference
Title | 2020 IEEE Symposium Series on Computational Intelligence (SSCI) |
---|---|
Duration | 1 - 4 December 2020 |
Location | Canberra, ACT, Australia |
External IDs
Scopus | 85099716122 |
---|
Keywords
Keywords
- Tuning, Optimization, Predictive models, Adaptation models, Search problems, Time measurement, Systematics