Topology-aware optimization of big sparse matrices and matrix multiplications on main-memory systems
Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/Gutachten › Beitrag in Konferenzband › Beigetragen › Begutachtung
Beitragende
Abstract
Since data sizes of analytical applications are continuously growing, many data scientists are switching from customized micro-solutions to scalable alternatives, such as statistical and scientific databases. However, many algorithms in data mining and science are expressed in terms of linear algebra, which is barely supported by major database vendors and big data solutions. On the other side, conventional linear algebra algorithms and legacy matrix representations are often not suitable for very large matrices. We propose a strategy for large matrix processing on modern multicore systems that is based on a novel, adaptive tile matrix representation (AT MATRIX). Our solution utilizes multiple techniques inspired from database technology, such as multidimensional data partitioning, cardinality estimation, indexing, dynamic rewrites, and many more in order to optimize the execution time. Based thereon we present a matrix multiplication operator ATMULT, which outperforms alternative approaches. The aim of our solution is to overcome the burden for data scientists of selecting appropriate algorithms and matrix storage representations. We evaluated AT MATRIX together with ATMULT on several real-world and synthetic random matrices.
Details
Originalsprache | Englisch |
---|---|
Titel | 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016 |
Herausgeber (Verlag) | IEEE, New York [u. a.] |
Seiten | 823-834 |
Seitenumfang | 12 |
ISBN (elektronisch) | 9781509020195 |
Publikationsstatus | Veröffentlicht - 22 Juni 2016 |
Peer-Review-Status | Ja |
Extern publiziert | Ja |
Publikationsreihe
Reihe | International Conference on Data Engineering (ICDE) |
---|---|
ISSN | 1063-6382 |
Konferenz
Titel | 32nd IEEE International Conference on Data Engineering, ICDE 2016 |
---|---|
Dauer | 16 - 20 Mai 2016 |
Stadt | Helsinki |
Land | Finnland |
Externe IDs
ORCID | /0000-0001-8107-2775/work/142253545 |
---|