TY - GEN
T1 - A Multicore Path to Connectomics-on-Demand
AU - Matveev, Alexander
AU - Meirovitch, Yaron
AU - Saribekyan, Hayk
AU - Jakubiuk, Wiktor
AU - Kaler, Tim
AU - Odor, Gergely
AU - Budden, David
AU - Zlateski, Aleksandar
AU - Shavit, Nir
N1 - Publisher Copyright:
© 2017 ACM.
PY - 2017/1/26
Y1 - 2017/1/26
N2 - The current design trend in large scale machine learning is to use distributed clusters of CPUs and GPUs with MapReduce-style programming. Some have been led to believe that this type of horizontal scaling can reduce or even eliminate the need for traditional algorithm development, careful parallelization, and performance engineering. This paper is a case study showing the contrary: that the benefits of algorithms, parallelization, and performance engineering, can sometimes be so vast that it is possible to solve "cluster-scale" problems on a single commodity multicore machine. Connectomics is an emerging area of neurobiology that uses cutting edge machine learning and image processing to extract brain connectivity graphs from electron microscopy images. It has long been assumed that the processing of connectomics data will require mass storage, farms of CPU/GPUs, and will take months (if not years) of processing time. We present a high-throughput connectomics-on-demand system that runs on a multicore machine with less than 100 cores and extracts connectomes at the terabyte per hour pace of modern electron microscopes.
AB - The current design trend in large scale machine learning is to use distributed clusters of CPUs and GPUs with MapReduce-style programming. Some have been led to believe that this type of horizontal scaling can reduce or even eliminate the need for traditional algorithm development, careful parallelization, and performance engineering. This paper is a case study showing the contrary: that the benefits of algorithms, parallelization, and performance engineering, can sometimes be so vast that it is possible to solve "cluster-scale" problems on a single commodity multicore machine. Connectomics is an emerging area of neurobiology that uses cutting edge machine learning and image processing to extract brain connectivity graphs from electron microscopy images. It has long been assumed that the processing of connectomics data will require mass storage, farms of CPU/GPUs, and will take months (if not years) of processing time. We present a high-throughput connectomics-on-demand system that runs on a multicore machine with less than 100 cores and extracts connectomes at the terabyte per hour pace of modern electron microscopes.
KW - machine learning
KW - multicore programming
UR - http://www.scopus.com/inward/record.url?scp=85084179796&partnerID=8YFLogxK
U2 - 10.1145/3018743.3018766
DO - 10.1145/3018743.3018766
M3 - Conference contribution
AN - SCOPUS:85084179796
VL - 52
T3 - ACM SIGPLAN Notices
SP - 267
EP - 281
BT - PPoPP '17: Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
PB - Association for Computing Machinery
T2 - 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2017
Y2 - 4 February 2017 through 8 February 2017
ER -