PlinyCompute: A platform for high-performance, distributed, data-intensive tool development

Jia Zou, R. Matthew Barnett, Tania Lorido-Botran, Shangyu Luo, Carlos Monroy, Sourav Sikdar, Kia Teymourian, Binhang Yuan, Chris Jermaine

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

This paper describes PlinyCompute, a system for development of high-performance, data-intensive, distributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the "PC object model") and associated memory management system that has been designed from the ground-up for high performance, distributed, data-intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/-function dispatch to the JVM. This hybrid approach-declarative in the large, trusting the programmer's ability to utilize PC object model efficiently in the small-results in a system that is ideal for the development of reusable, data-intensive tools and libraries.

Original languageEnglish (US)
Title of host publicationSIGMOD 2018 - Proceedings of the 2018 International Conference on Management of Data
EditorsGautam Das, Christopher Jermaine, Ahmed Eldawy, Philip Bernstein
PublisherAssociation for Computing Machinery
Pages1189-1204
Number of pages16
ISBN (Electronic)9781450317436
DOIs
StatePublished - May 27 2018
Externally publishedYes
Event44th ACM SIGMOD International Conference on Management of Data, SIGMOD 2018 - Houston, United States
Duration: Jun 10 2018Jun 15 2018

Publication series

NameProceedings of the ACM SIGMOD International Conference on Management of Data

Conference

Conference44th ACM SIGMOD International Conference on Management of Data, SIGMOD 2018
Country/TerritoryUnited States
CityHouston
Period6/10/186/15/18

Keywords

  • Distributed computing
  • Object model
  • Query compilation

ASJC Scopus subject areas

  • Software
  • Information Systems

Fingerprint

Dive into the research topics of 'PlinyCompute: A platform for high-performance, distributed, data-intensive tool development'. Together they form a unique fingerprint.

Cite this