A Multicore Path to Connectomics-on-Demand

Alexander Matveev, Yaron Meirovitch, Hayk Saribekyan, Wiktor Jakubiuk, Tim Kaler, Gergely Odor, David Budden, Aleksandar Zlateski, Nir Shavit

Research output: Contribution to Book/Report typesConference contributionpeer-review

Abstract (may include machine translation)

The current design trend in large scale machine learning is to use distributed clusters of CPUs and GPUs with MapReduce-style programming. Some have been led to believe that this type of horizontal scaling can reduce or even eliminate the need for traditional algorithm development, careful parallelization, and performance engineering. This paper is a case study showing the contrary: that the benefits of algorithms, parallelization, and performance engineering, can sometimes be so vast that it is possible to solve "cluster-scale" problems on a single commodity multicore machine. Connectomics is an emerging area of neurobiology that uses cutting edge machine learning and image processing to extract brain connectivity graphs from electron microscopy images. It has long been assumed that the processing of connectomics data will require mass storage, farms of CPU/GPUs, and will take months (if not years) of processing time. We present a high-throughput connectomics-on-demand system that runs on a multicore machine with less than 100 cores and extracts connectomes at the terabyte per hour pace of modern electron microscopes.

Original languageEnglish
Title of host publicationPPoPP '17: Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
PublisherAssociation for Computing Machinery
Pages267-281
Number of pages15
Volume52
Edition8
ISBN (Electronic)9781450344937
DOIs
StatePublished - 26 Jan 2017
Externally publishedYes
Event22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2017 - Austin, United States
Duration: 4 Feb 20178 Feb 2017

Publication series

NameACM SIGPLAN Notices
ISSN (Print)1523-2867

Conference

Conference22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2017
Country/TerritoryUnited States
CityAustin
Period4/02/178/02/17

Keywords

  • machine learning
  • multicore programming

Fingerprint

Dive into the research topics of 'A Multicore Path to Connectomics-on-Demand'. Together they form a unique fingerprint.

Cite this