13.05 An Exascale Library for Numerically Inspired Machine Learning (ExaNIML)
Institute for Computational Engineering and Sciences (ICES) | University of Texas at Austin/ USA
There is a significant gap between algorithms and software in Data Analytics and those in Computational Science and Engineering (CSE) concerning their maturity on High-Performance Computing (HPC) systems. Given the fact that Data Analytics tasks show a rapidly growing share of supercomputer usage, this gap is a serious issue. This proposal aims to bridge this gap for a number of important tasks arising, e.g., in a Machine Learning (ML) context: density estimation, and high-dimensional approximation (for example (semi-supervised) classification).
To this end, we aim to (1) design and analyze novel algorithms that combine two powerful numerical methods: sparse grids and kernel methods; and to (2) design and implement an HPC library that provides an open-source implementation of these algorithms and supports heterogeneous distributed-memory architectures. The attractiveness of sparse grids is mainly due to their high-quality accuracy guarantees and their foundation on rigorous approximation theory. But their shortcoming is that they require (regular) Cartesian grids. Kernel methods do not require Cartesian grids but, first, their approximation properties can be suboptimal in a practice, and second, they require regularization whose parameters can be expensive to determine.
Our main idea is to use kernel methods for manifold learning and to combine them with the sparse grids to define approximations on the manifold. Such high-dimensional approximation problems find applications in model reduction, uncertainty quantification (UQ), and ML.
C. D. Yu, S. Reiz and G. Biros: Distributed-Memory Hierarchical Compression of Dense SPD Matrices [BibTeX][mediaTUM], In SC '18: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2018.
C. D. Yu, J. Levitt, S. Reiz and G. Biros: Geometry-Oblivious FMM for Compressing Dense SPD Matrices [BibTeX][mediaTUM], In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis SC17, November 2017