Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018

Önder Babur; Loek Cleophas; Mark van den Brand; Slimane Hammoudi; Luis Ferreira Pires; Bran Selic

doi:10.5220/0006735407670772

Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018

Önder Babur, Loek Cleophas, Mark van den Brand, Slimane Hammoudi (Editor), Luis Ferreira Pires (Editor), Bran Selic (Editor)

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › Academic › peer-review

3 Citations (Scopus)

Abstract

The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.

Original language	English
Title of host publication	Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development
Editors	Slimane Hammoudi, Luis Ferreira Pires, Bran Selic
Publisher	SCITEPRESS-Science and Technology Publications, Lda.
Number of pages	6
Volume	1
Edition	MOMA3N
ISBN (Print)	9789897582837
DOIs	https://doi.org/10.5220/0006735407670772
Publication status	Published - 2018
Externally published	Yes

Keywords

Apache Spark
Big Data
Distributed Computing
Model Analytics
Model-Driven Engineering
Scalability

Access to Document

10.5220/0006735407670772

Cite this

Babur, Ö., Cleophas, L., van den Brand, M., Hammoudi, S. (Ed.), Pires, L. F. (Ed.), & Selic, B. (Ed.) (2018). Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018. In S. Hammoudi, L. Ferreira Pires, & B. Selic (Eds.), Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development (MOMA3N ed., Vol. 1). SCITEPRESS-Science and Technology Publications, Lda.. https://doi.org/10.5220/0006735407670772

Babur, Önder ; Cleophas, Loek ; van den Brand, Mark et al. / Towards distributed model analytics with apache spark : 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018. Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development. editor / Slimane Hammoudi ; Luis Ferreira Pires ; Bran Selic. Vol. 1 MOMA3N. ed. SCITEPRESS-Science and Technology Publications, Lda., 2018.

@inproceedings{534d2dabaead48f2a5e0c7ba08038819,

title = "Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018",

abstract = "The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.",

keywords = "Apache Spark, Big Data, Distributed Computing, Model Analytics, Model-Driven Engineering, Scalability",

author = "{\"O}nder Babur and Loek Cleophas and \{van den Brand\}, Mark and Slimane Hammoudi and Pires, \{Luis Ferreira\} and Bran Selic",

year = "2018",

doi = "10.5220/0006735407670772",

language = "English",

isbn = "9789897582837",

volume = "1",

editor = "Slimane Hammoudi and \{Ferreira Pires\}, Luis and Bran Selic",

booktitle = "Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development",

publisher = "SCITEPRESS-Science and Technology Publications, Lda.",

edition = "MOMA3N",

}

Babur, Ö, Cleophas, L, van den Brand, M, Hammoudi, S (ed.), Pires, LF (ed.) & Selic, B (ed.) 2018, Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018. in S Hammoudi, L Ferreira Pires & B Selic (eds), Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development. MOMA3N edn, vol. 1, SCITEPRESS-Science and Technology Publications, Lda. https://doi.org/10.5220/0006735407670772

Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018. / Babur, Önder; Cleophas, Loek; van den Brand, Mark et al.
Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development. ed. / Slimane Hammoudi; Luis Ferreira Pires; Bran Selic. Vol. 1 MOMA3N. ed. SCITEPRESS-Science and Technology Publications, Lda., 2018.

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › Academic › peer-review

TY - GEN

T1 - Towards distributed model analytics with apache spark

T2 - 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018

AU - Babur, Önder

AU - Cleophas, Loek

AU - van den Brand, Mark

A2 - Hammoudi, Slimane

A2 - Pires, Luis Ferreira

A2 - Selic, Bran

A2 - Hammoudi, Slimane

A2 - Ferreira Pires, Luis

A2 - Selic, Bran

PY - 2018

Y1 - 2018

N2 - The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.

AB - The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.

KW - Apache Spark

KW - Big Data

KW - Distributed Computing

KW - Model Analytics

KW - Model-Driven Engineering

KW - Scalability

U2 - 10.5220/0006735407670772

DO - 10.5220/0006735407670772

M3 - Conference paper

SN - 9789897582837

VL - 1

BT - Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development

PB - SCITEPRESS-Science and Technology Publications, Lda.

ER -

Babur Ö, Cleophas L, van den Brand M, Hammoudi S, (ed.), Pires LF, (ed.), Selic B, (ed.). Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018. In Hammoudi S, Ferreira Pires L, Selic B, editors, Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development. MOMA3N ed. Vol. 1. SCITEPRESS-Science and Technology Publications, Lda. 2018 doi: 10.5220/0006735407670772

Towards distributed model analytics with apache spark: 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018

Abstract

Keywords

Access to Document

Fingerprint

Cite this