What do Practitioners Expect from the Meta-modeling Tools? A Survey

What Do Practitioners Expect fromthe Meta-modelling
Tools? A Survey
Mert Ozkaya
Assoc.Prof.of Computer Engineeringat YeditepeUniversity
ResearcherandConsultant onSoftwareEngineering
Web-site:http://cse.yeditepe.edu.tr/~mozkaya
E-mail:mozkaya@cse.yeditepe.edu.tr
Linkedin:www.linkedin.com/in/mert-ozkaya 1

Scope of Research
1. Practitioners use meta-modelling
tools to
1. design and develop modelling
languages
2. develop model transformation
tools
3. use existing modelling languages
2. Practitioners use modelling
languages to
1. specify models and
2. process models for, e.g., early
analysis and code generation
3. Practitioners use models for
1. abstraction and managing
complexity
2
Assoc.Prof.Dr.MertOzkaya WhatDoPractitionersExpectfromtheMeta-modellingTools?ASurvey

Scope of Research
• We are interested in understanding
practitioners’
• perspectives,
• expectations, and
• challenges
towards
meta-modelling tools,
meta-models & modelling languages,
and models
3

Practitioners – who are they?
• A practitioner is someone who
• uses/edits models
• uses/edits meta-models
• does not use/edit models and meta-models but
do feel interested
• Practitioners may hold diverse job
positions
• Including software developer, architect,
consultant, manager, system engineer, analyst,
etc.
• Practitioners may vary depending on their
• education level
• work industries
• experiences
4

Practitioners’
approach towards
modelling and
modelling languages
• Possible questions to be investigated
• Practitioners’ understanding towards modelling
• Practitioners’ expectations from modelling
• Practitioners’ challenges in modelling
• Practitioners’ perspectives towards different
modelling techniques
• E.g., software architectures, design patterns,
object-oriented modelling, etc.
types of modelling languages
• E.g., UML, formal languages, and ADLs
• Practitioners’ perspectives towards applying
models at particular domains
• E.g., embedded domain
• The good thing is: The literature already
includes several empirical researches
• Many survey papers with interesting results
5

Modelling Surveys -1
1. G. Liebel, N. Marko, M. Tichy, A. Leitner, J. Hansson, Model-based engineering in the embedded systems domain: an industrial
survey on the state-of-practice, Software and Systems Modeling 17 (1) (2018)
2. L. T. W. Agner, I. W. Soares, P. C. Stadzisz, J. M. Sim~aO, A brazilian survey on uml and model-driven practices for embedded software
development, J. Syst. Softw. 86 (4) (2013)
3. M. Torchiano, F. Tomassetti, F. Ricca, A. Tiso, G. Reggio, Preliminary findings from a survey on the md state of the practice, in: Proceedings of the
2011 International Symposium on Empirical Software Engineering and Measurement, ESEM '11, IEEE Computer Society, USA, 2011
4. J. Whittle, J. E. Hutchinson, M. Rounceeld, The state of practice in model-driven engineering, IEEE Softw. 31 (3) (2014)
5. P. Mohagheghi, W. Gilani, A. Stefanescu, M. A. Fernandez, An empirical study of the state of the practice and acceptance of model-driven
engineering in four industrial cases, Empir. Softw. Eng. 18 (1) (2013)
6. F. Tomassetti, M. Torchiano, A. Tiso, F. Ricca, G. Reggio, Maturity of software modelling and model driven engineering: A survey in the italian
industry, in: M. T. Baldassarre, M. Genero, E. Mendes, M. Piattini (Eds.), 16th International Conference on Evaluation & Assessment in Software
Engineering, EASE 2012, Ciudad Real, Spain, May 14-15, 2012. Proceedings, IET - The Institute of Engineering and Technology/ IEEE Xplore, 2012
7. D. Akdur, V. Garousi, O. Demir•ors, A survey on modeling and modeldriven engineering practices in the embedded software industry, Journal of
Systems Architecture - Embedded Systems Design 91 (2018)
8. D. Bork, D. Karagiannis, B. Pittl, A survey of modeling language specification techniques, Inf. Syst. 87 (2020)
9. J. Cabot, E. Teniente, Constraint support in MDA tools: A survey, in: A. Rensink, J. Warmer (Eds.), Model Driven Architecture - Foundations and
Applications, 2nd European Conference, ECMDAFA 2006, Bilbao, Spain, July 10-13, 2006, Proceedings, Vol. 4066 of Lecture Notes in Computer
Science, Springer, 2006
10. R. F. Paige, D. Varro, Lessons learned from building model-driven development tools, Softw. Syst. Model. 11 (4) (2012)
11. J. L. Perez-Medina, S. Dupuy-Chessa, A. Front, A survey of model driven engineering tools for user interface design, in: M. Winckler, H. Johnson,
P. A. Palanque (Eds.), Task Models and Diagrams for User Interface Design, 6th International Workshop, TAMODIA 2007, Toulouse, France,
November 7-9, 2007, Proceedings, Vol. 4849 of Lecture Notes in Computer Science, Springer, 2007
6

Modelling Surveys - 2
12. I. Malavolta, P. Lago, H. Muccini, P. Pelliccione, A. Tang, What industry needs from architectural languages: A survey, IEEE Trans. Software Eng. 39
(6) (2013)
13. M. Ozkaya, Do the informal & formal software modeling notations satisfy practitioners for software architecture modeling?, Information &
Software Technology 95 (2018)
14. P. C. Clements, A survey of architecture description languages, in: Proceedings of the 8th International Workshop on Software Specification and
Design, IWSSD '96, IEEE Computer Society, Washington, DC, USA, 1996
15. M. Ozkaya, What is software architecture to practitioners: A survey, in: S. Hammoudi, L. F. Pires, B. Selic, P. Desfray (Eds.), MODELSWARD 2016 -
Proceedings of the 4rd International Conference on Model-Driven Engineering and Software Development, Rome, Italy, 19-21 February, 2016., 35
SciTePress, 2016
16. A. Forward, T. C. Lethbridge, Perceptions of software modeling: A survey of software practitioners, Tech. Rep. TR-2008-07, School of Information
Technology and Engineering, University of Ottawa, Ottawa, Ontario, Canada K1N 6N5 (2008)
17. G. Liebel, N. Marko, M. Tichy, A. Leitner, J. Hansson, Model-based engineering in the embedded systems domain: an industrial survey on the
state-of-practice, Software & Systems Modeling 17 (1) (2018)
18. P. Mohagheghi, V. Dehlen, Where is the proof? - a review of experiences from applying mde in industry, in: I. Schieferdecker, A. Hartman (Eds.),
Model Driven Architecture Foundations and Applications, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008
19. T. Berger, R. Rublack, D. Nair, J. M. Atlee, M. Becker, K. Czarnecki, A. Wasowski, A survey of variability modeling in industrial practice, in:
Proceedings of the Seventh International Workshop on Variability Modelling of Software-intensive Systems, VaMoS '13, ACM, New York, NY, USA,
2013
20. C. F. J. Lange, M. R. V. Chaudron, J. Muskens, In practice: Uml software architecture and design description, IEEE Software 23 (2) (2006)
21. F. Tomassetti, M. Torchiano, A. Tiso, F. Ricca, G. Reggio, Maturity of software modelling and model driven engineering: A survey in the italian
industry, in: 16th International Conference on Evaluation Assessment in Software Engineering (EASE 2012), 2012
7

Practitioners’
approach towards
meta-modelling
• Many research questions again..
• Practitioners’ understanding towards meta-
modelling
• Practitioners’ expectations from meta-modelling
• Practitioners’ challenges in meta-modelling
meta-modelling tools
• Practitioners’ perspectives towards applying
meta-models at particular domains such as
embedded domain
• However, the literature does not really
consider answering those research questions
• Lack of survey studies on practitioners and meta-
modelling
• The current works focus more on
• analysing the existing meta-modelling tools for some
requirements and
• proposing a taxonomy of requirements for meta-
modelling tools
8

Our Goal
We aim at answering three important
research questions
1. What are the usage frequencies of the
existing meta-modelling tools in different
problem domains?
2. What do practitioners expect from the
meta-modelling tools?
3. What are the challenges that practitioners
face with on their meta-modelling
activities?
9
A survey has been designed and
executed on a number of
practitioners

Survey on Meta-Modelling Tool Requirements
• Question:
• How can we determine the meta-modelling tool
requirements that we focus on our survey?
• Answer:
• Erdweg S. et al. (2013) The State of the Art in
Language Workbenches. In: Erwig M., Paige R.F.,
Van Wyk E. (eds) Software Language Engineering.
SLE 2013. Lecture Notes in Computer Science, vol
8225. Springer, Cham.
• A feature model proposed out of the language
workbench workshops conducted in between 2011
and 2013
• Consisting of the meta-modelling tool features
10

Category of Requirements for the
Meta-modelling Tools
• Language definition
• Notation
• Textual, graphical, tabular, matrix, map
• Semantics
• Translational (model-to-text, model-to-
model) and interpretative
• Modelling editors
• Editing mode
• Free-form and projectional
• Syntactic services
• Highlighting, outline, folding, syntactic
completion, diff, auto formatting, etc.
• Semantic services
• Semantic completion, refactoring, error
marking, quick fixes, live translation, etc.
11
• Language validation
• Structural and semantic validations
• Language testing
• Testing the language definitions,
editors, validation rules, etc.
• Language composability
• Composing the language
definitions, editors, validation
rules, etc.

Survey Design
• We prepared 25 different questions
• 5 profile questions
• Countries, job positions, work industries,
domains, and experiences
• 1 question for understanding the tool usage
frequencies
• A set of questions for each category of
requirements
• Multiple-choice questions for
understanding participants’ thoughts on
the tool requirement
• A free-text question for understanding the
challenges
12
A pilot study conducted with a small set of
academics and practitioners
Many issues avoided:
• inconsistency and completeness of the
research questions wrt the survey
questions,
• any missing, ambiguous, or redundant
questions and answers,
• the coherence of the survey sections,
• the correctness of the answer formats,
• the appropriateness of the time
needed to complete the survey
New requirements added
• Model transformation/code-generation

Survey Execution
• The survey made available
online via google-forms
• The survey accessible in
between June 2020 and
August 2020
13
To spread the survey,
• Personal contacts
• Past R&D project members with many industrial
partners and past work experiences on software
and systems modelling
• Mailing lists
• Eclipse modelling platforms
• Netbeans mailing list
• IEEE architecture description mailing list
• AADL list
• Practitioners who contributed to the
conf./journal papers contacted by e-mail
• Determined by searching the well-cited papers on
google scholar
• Linkedin groups on modeling
• In total, 300 individuals
have been contacted
• We received 103 different
participants from diverse
countries and industries

Survey Results
14

Participant Profiles
• Participants from 24 different countries
• Countries include
• Austria, Belgium, Finland, France,
Germany, Hungary, Italy, Poland, Portugal,
Sweden, Turkey, the Netherlands, United
Kingdom., Argentina, Brasil, China,
Colombia, Japan, Iran, Israel, Taiwan, and
USA, Australia, South Africa
15

Participants' meta-modelling tool usages
• Most practitioners use either of the five
different meta-modeling tools
• Sirius,
• GEMS,
• Metaedit+,
• Xtext, and
• Microsoft DSL tools
• Among the top-five meta-modeling tools,
• Sirius, Xtext, and GEMS are Eclipse-based tools
• based on the Eclipse Modeling Framework (EMF)
• Xtext is the only meta-modeling tool that offers a
textual visualisation
• Some tools are rarely used
• ANTLR, ConceptBase, Melange, GEMOC,
Graphiti, WebGME, Cameo, EVA, and JastEMF
16

The correlations between the participants'
domains and meta-modelling tool choice
17
• Some meta-modelling tools used for a few
particular domains
• ANTLR, GEMOC, EVA, Cameo, JastEMF, and
Graphiti
• While GME, Melange, and MPS are not
among the frequently-used tools,
• those tools preferred for various domains
• In almost all the domains, the top-used
meta-modelling tool is Metaedit+
• Except the embedded domain where the top-
used meta-modelling tool is GEMS.

The correlations between the participants' meta-
modelling tool choice and their countries
18
• The tools that are used in the greatest
number of countries are GEMS, Metaedit+,
Sirius, and Xtext
• Some meta-modelling tools are each used in
one or two countries
• ANTLR, GEMOC, WebGME, Melange, JastEMF,
Graphiti
• Some countries use particular meta-modelling
tools more than the other tools
• USA's top used tool is Microsoft DSL tools
• Finland's is Metaedit+
• Many European countries’ top-used tools are
Eclipse-based tools (e.g., GEMS, Sirius, Xtext)
• The Netherland and Turkey's is GEMS
• Many countries include some practitioners
who prefer to use in-house solutions
• Belgium, Germany, Finland, France, Taiwan, The
Netherlands, Turkey and USA

Language
Definition
19
Picture taken from Prof. Dr. Knut Hinkelmann’s slides on “Meta-Modeling and Modeling Languages”

Language Definition – Types of notation sets
20
• Diagrammatic visualisation
• enables the model elements to be specified using
graphical symbols
• Textual visualisation
• enables the model elements to be specified in terms of
texts (e.g., writing code with the programming languages)
• Tabular visualisation
• enables the model elements to be specified using a table
editor that can be displayed as a table
• Matrix style visualisation
• enables the model elements to be specified in two axes
• each matrix cell is the relationship of the elements in
two axes
• Map visualisation
• enables the model elements to be specified with the
location and distances
• Hybrid visualisation
• supports multiple visualisations (e.g., textual, graphical,
and tabular) that can be used for editing the same model
in a synchronised way

Language Definition– Semantics definitions
21
• Interpretative language semantics
• promotes the model execution without performing
any translations into some intermediate formats
• Translational language semantics
• promotes the definition of the model translations
into an intermediate format that can be executed
• Translational semantics:
• Model-to-text translation
• defines the semantics in terms of rules for the translations
into some structured text notation such as source-code
• Model-to-model translation
• defines the semantics in terms of rules for the translation
into a model with a different notation set
• e.g., producing entity-relationship model from a UML
class diagram

The challenges on defining the languages
• The top challenges are
• meta-modelling tools' steep learning curve for defining the language syntax and
semantics
• One participant herein is especially concerned about learning the projectional editing with
MPS
• the lack of support for training
• e.g., tutorials, guidance, and examples on how to dene the language syntax and semantics
• A few other participants are concerned about the lack of support for
• the integration with the version management tools (e.g., GIT and SVN) for versioning
meta-models,
• the collaborative meta-modelling, and
• keeping the language syntax and semantics complete and consistent as the new
modelling elements are added
22

Editor Services
23

Editor Services – Editing mode
• Free-form editing
• Users edit a textual or graphical model that is stored
persistently
• A persistent model transformed into an abstract
representation that is transformed into an executable
representation
• Projectional editing
• Users edit projections of the model's abstract representation
that is stored persistently and transformed into an
executable representation
• Each projection may be in different formats
• e.g., graphical, textual, tabular, and matrix
• Unlike the free-form editing, the projections edited by the
users are not stored persistently
24

Editor Services – Syntactic editor services
• Model re-use
• The models created kept in a repository
• New models created by re-using the existing
models in the repository
• Model comparison (diff-like tool)
• enables to detect and review the differences
between models and even merge them
• Syntactic completion templates
• provide incomplete models to the users (e.g.,
design pattern models)
• Auto-formatting, restructuring, aligning
models’ presentation
25

Editor Services – Semantic editor services
• Error marker
• highlighting model elements and any associated error
messages for detecting the semantical errors
• e.g., the violation of well-formedness rules
• Automatic update of the models when
meta-model changes
• enables the modelers to detect any errors due to the
semantical changes automatically
• Semantic completions
• for receiving automatic suggestions at modelling time
• Refactoring of models
• the refactoring of models without changing semantics
• e.g., renaming and language-specific restructuring
• UML support
• Reusing and extending the UML language syntax and
semantics
• Live translation
• Displaying model and code side-by-side
26

Other Editor Services
• Document generation
• generating Word, PDF, or XML files from models
automatically
• Importing/exporting models
• for sharing models among collaborators
• Version control system integration
• Managing model versions
• Usability
• the minimum number of clicks for
modelling/meta-modelling
• Traceability
• checking the consistencies between structural
and behavioural models
• IDE integration
• the integration with development environments
such as Eclipse and Visual Studio
27

The challenges on developing/using editors
• The top concern is the usability of the editors
• Usability issues in using Eclipse Xtext/Xtend and MPS meta-modelling tools for developing editors
• Xtext’s support for scalability and traceability based on the OSLC* linking
• Also, some participants are concerned about editors' lack of support for
• model versioning
• training (i.e., any learning materials to learn how to produce modeling editors)
• integrating editors with other development technologies, such as .NET, to combine modelling and coding
• Lastly, one participant is concerned about the support for the web-based editors
• Another participant is concerned about the syntax colouring
28
*OSLC: https://open-services.net/

Model
Transformation/
Code-generation
29
Picture taken from https://www.metacase.com

Model transformation/code-generation
features
• Error detection
• Detecting errors during the code generator
development
• Syntax highlighting
• Colouring the code for transformation
• Facilitating the readability and the detection of
syntax errors
• Scalability
• Developing large transformation tools
• With hundreds of lines of code for
transforming large and complex models
30

The challenges on using the
model transformation/code-generation technologies
• A few of the participants stated some challenges on their use of the model
transformation technologies
• These challenges are to do with
• adapting the code generators with the end-user requirements
• the training support for using the transformation technologies
• integrating code generators with external Java programs, and
• the transformation technologies being too Java-centric and neglecting the needs of embedded domains.
31

Language
Validation
32

Validation Types
• Structural validation rules
• The structural aspects of the language
definitions
• E.g., the multiplicities of the language
elements and containment relationships
between them
• Semantic validation rules
• The semantical aspects of language
definitions
• E.g., Name/type analysis
33
Language validation is concerned with
meta-modelling tools' support for defining the language
validation rules that can be executed via the modelling editor
to validate models

The meta-modelling tool features for
language validations
• Some desired features are
• the integration with some external validation
tools
• formal verification tools
• theorem provers
• simulation tools
• testing tools
• automated model and meta-model
validations for
• user-defined rules
• pre-defined rules
34

The challenges on defining the language
validation rules
• While most participants did not state any challenges, a few participants pointed out
meta-modelling tools' lack of support for
• checking any model for the validation rules at modelling time and
• defining complex validation rules
35

Language Testing
36
Picture taken from https://www.scnsoft.com/software-testing/software-testing-trends

Testing different aspects of language
development
37
• The syntax and semantics testing is for checking if
• the language meta-model consists of the expected
modeling elements and relationships and
• the syntax and semantics rules have been defined
correctly and completely
• The editor testing is for checking if the editor
• enables creating models in accordance with the
syntax and semantics and
• meets such quality requirements as usability and
performance
• The code generator testing is for checking if the
code-generator
• performs the model transformation correctly and
meets the quality expectations
• The validation rules testing is for checking if
• the validation rules can be defined in accordance with
the language requirements and used for validating
models correctly
Language testing is concerned with testing the
• language definition (syntax and semantics)
• editor services
• code generator
• validation rules

The challenges on testing different
aspects of language development
• Only a few participants stated some challenges
• These challenges are to do with the lack of support for
• specifying and executing test cases on different aspects of languages,
• integrating any external testing tools that can aid in the language testing, and
• the model-based testing of different aspects.
38

Language
Composability
39
Picture taken from www.freepik.com

Different aspects of language composability
40
• The language syntax/semantics composed of
the syntax/semantics of any existing languages
• E.g., re-using the semantical definitions of the
existing languages such as UML
• A modelling editor developed by composing
multiple existing tools
• E.g., the model versioning tool, collaboration
tool, validation tool, and code-generation tool.
• The model transformation/code-generation tool
developed via reuse
• E.g., re-using transformation templates,
patterns, or the existing code
• The validation rules defined by re-using and
modifying the existing rules or composing
multiple rules
The language composability is concerned with
the meta-modelling tools’ support for extending an
existing language with some new features or unifying the
parts of multiple languages for developing a new language

The challenges on composing different
languages
• Just 3 participants stated some challenges, which are to do with
• composing different languages without changing them and
• the difficulties in performing composition with textual languages.
41

Lessons Learned
• While most practitioners prefer either diagrammatical or textual visualisations, hybrid
visualisation is neglected
• Practitioners are not familiar with hybrid visualisation
• This might be due to the fact that most meta-modelling tools ignore hybrid visualisation
• Practitioners define the language semantics by means of translations into text (e.g.,
source-code) or model (e.g., formal process algebra)
• Most practitioners ignore the interpretative semantics definition
• Collaborative meta-modelling is important for many practitioners
• Importing/exporting the language definitions and versioning the definitions in a repository for later re-use are
among practitioners’ top-interests
• Practitioners are not familiar with the projectional editing
• Indeed, hybrid modelling that could be possible with projectional editing is not preferred either
• Also, MPS meta-modeling tool that supports the projectional editing is rarely used
42

Lessons Learned
• Re-using models/meta-models are the top-concern
• Defining the language syntax and semantics by re-using the definitions of the existing languages
• Re-using the existing models by extending or modifying them
• Versioning models/meta-models is also highly important
• Practitioners wish to integrate with version control systems (e.g., GIT) and compare and merge model/meta-model
versions
• Defining the validation rules is so important for most practitioners
• Most practitioners are interested in validating both the structural and semantical aspects of languages
• Practitioners also wish the meta-modelling tools to be integrated with external
verification tools
• Formal verification tools, simulation tools, theorem provers, etc.
• Testing the language semantics definitions is the most desired testing facility
• Likewise, the compositional definitions of semantics is the most desired composition
facility
43

Thanks for Listening
44

What do Practitioners Expect from the Meta-modeling Tools? A Survey

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a What do Practitioners Expect from the Meta-modeling Tools? A Survey

Similar a What do Practitioners Expect from the Meta-modeling Tools? A Survey (20)

Más de Obeo

Más de Obeo (20)

Último

Último (20)

What do Practitioners Expect from the Meta-modeling Tools? A Survey