Scalable high-quality 1D partitioning

Research output: Contribution to conferencesPaperContributedpeer-review

Abstract

The decomposition of one-dimensional workload arrays into consecutive partitions is a core problem of many load balancing methods, especially those based on space-filling curves. While previous work has shown that heuristics can be parallelized, only sequential algorithms exist for the optimal solution. However, centralized partitioning will become infeasible in the exascale era due to the vast amount of tasks to be mapped to millions of processors. In this work, we first introduce optimizations to a published exact algorithm. Further, we investigate a hierarchical approach which combines a parallel heuristic and an exact algorithm to form a scalable and high-quality 1D partitioning algorithm. We compare load balance, execution time, and task migration of the algorithms for up to 262 144 processes using real-life workload data. The results show a 300 times speed-up compared to an existing fast exact algorithm, while achieving nearly the optimal load balance.

Details

Original languageEnglish
Pages112-119
Number of pages8
Publication statusPublished - 2014
Peer-reviewedYes

Conference

TitleInternational Conference on High Performance Computing & Simulation (HPCS)
Duration21 - 25 July 2014
CityBologna
CountryItaly

External IDs

Scopus 84908655384
ORCID /0000-0003-3137-0648/work/142238862

Keywords

Keywords

  • parallel algorithms, resource allocation