Presentation delivered at the 3rd IEEE Track on
Collaborative Modeling & Simulation - CoMetS'12.
Please see http://www.sel.uniroma2.it/comets12/ for further details.
A package system for maintaining large model distributions in vle software
1. A Package System for Maintaining Large Model
Distributions in VLE Software
WETICE/Comets 2012
Gauthier Quesnel, Ronan Tr´pos
e
INRA - French National Institute for Agricultural Research
MIA - The Division of Applied Mathematics and Computer Sciences
BIA - Biometrics and Artificial Intelligence Unit
Quesnel et al. (INRA) Package system for VLE 1 / 32
2. 1 Context
2 Problem
3 Package system
4 Results
5 Conclusions and perspectives
Quesnel et al. (INRA) Package system for VLE 2 / 32
3. VLE
Virtual Laboratory Environment
A multi-modeling and simulation environment:
A DEVS simulation engine
A command line interface : VLE
A modeling graphical user interface: GVLE
A python package to develop web services: PyVLE
A R package to analyze and visualize simulations results: RVLE
VLE is developed with the support of:
INRA, France
Computer Science Laboratory, Calais, France
CIRAD, Montpellier, France
Computer Science Laboratory, University of Mons-Hainaut, Belgium
Since 2007, VLE has more and more users and developers
Quesnel et al. (INRA) Package system for VLE 3 / 32
4. RECORD, an INRA project
An open platform for modeling and simulation of agro-ecosystems
Project initiated in 2006 which relies on VLE software
That makes
multi-disciplinary work easier
Capacity to take into
account all the different
elements of cropping
systems
Each element can be
expressed in the most suited
mathematical formalism
Target: all researchers and
engineers at INRA that use
dynamic systems
Quesnel et al. (INRA) Package system for VLE 4 / 32
5. Need to improve the collaboration
Needs for a collaboration system due to the heterogeneity of domains
involved: agronomy, economists, statistician, computer scientists, . . .
Needs in sharing models; eg. integration of models at the plot scale
to little territories scale (thousands of plots)
Needs in distributing existing models; eg. Stics (more than 25000
lines of fortran code)
Needs in helping the collaboration between modelers. More and more
people trained to the platform through 3 days practice sessions (2 per
year, about 200 persons trained since 2007) and access to an
E-learning web site
Quesnel et al. (INRA) Package system for VLE 5 / 32
6. 1 Context
2 Problem
3 Package system
4 Results
5 Conclusions and perspectives
Quesnel et al. (INRA) Package system for VLE 6 / 32
7. 1 Context
2 Problem
3 Package system
4 Results
5 Conclusions and perspectives
Quesnel et al. (INRA) Package system for VLE 7 / 32
8. How to develop models ?
DEVS
Initiated in 1976 by B. Zeigler to model and simulate dynamic systems
A discrete event formalism: the events drive the simulation
Modular and hierarchical approach: two entities atomic and coupled
models
Formal approach for model coupling:
The closed under coupling property: a coupled model is equivalent to
an atomic
Abstract simulators (algorithms) to develop simulation tools:
One simulator for driving one atomic model
One coordinator for driving one coupled model
Extensible:
DS-DEVS: dynamic-structure DEVS, to add or remove, models or
connections at run-time
Cell-DEVS, Fuzzy-DEVS, RT-DEVS etc.
Quesnel et al. (INRA) Package system for VLE 8 / 32
9. Atomic model definition (DEVS)
M = X , Y , S, δext , δint , δcon , ta, λ
Where:
δext , δint , δcon , ta, λ are functions, S the set of states
Coupled model definition (DEVS)
N = X , Y , D, EIC , EOC , IC
Executive model definition (DS-DEVS)
Mχ = Xχ , Yχ , Sχ , γ, Σ∗ , δintχ , δextχ , δconχ , λχ , τχ
Where:
γ and Σ∗ are structure change functions
Quesnel et al. (INRA) Package system for VLE 9 / 32
10. How to develop models ?
VLE
VLE provides C++ shared library which embeds the DEVS simulation
kernel, thus:
users have to develop C++ class to develop atomic (DEVS) or
executive models (DS-DEVS), or sub-formalisms
class Dynamics
{
public :
virtual Time timeAdvance () const ;
virtual void i n t e r n a l T r a n s i t i o n ( const Time & time );
virtual void e x t e r n a l T r a n s i t i o n ( const Time & time ,
const E x t e r n a l E v e n t L i s t & lst );
virtual void output ( const Time & time ,
E x t e r n a l E v e n t L i s t & out ) const ;
virtual void c o n f l u e n t T r a n s i t i o n ( const Time & time ,
const E x t e r n a l E v e n t L i s t & lst );
virtual Value * observation ( const O b s e r v a t i o n Ev e n t & event ) const ;
}
Quesnel et al. (INRA) Package system for VLE 10 / 32
11. How to develop models ?
Sub-formalisms
DEVS allows to develop sub-formalism of DEVS. With VLE, we provide:
dynamics which encapsulate mathematical formalisms such as
differential equations, difference equations, Petri nets, etc.
structurals which manipulates the structure of the model: cellular
automaton, graph of communications, etc.
Dynamic sub-formalisms are atomic models with a specified
behavior: we provide transition functions
Structural sub-formalisms are atomic models and executive models of
DS-DEVS
Quesnel et al. (INRA) Package system for VLE 11 / 32
12. How to develop models ?
For sub-formalisms
We provide simplified API (object oriented development)
We provide C++ generators
For example, with the difference equation sub-formalism (which inherits
the previous Dynamics class), only the compute function is available:
class MyModel : public vle :: extension :: D i f f e r e n c e E q u a t i o n
{
[...]
virtual double compute ( const Time & time )
{
x = y ( -1) + z ( -1);
y = z ( -1);
}
};
Quesnel et al. (INRA) Package system for VLE 12 / 32
13. How to develop models ?
GVLE: GUI of VLE, an IDE
To develop source code, the structure of the model, to assign initial
conditions, observations, to define experimental frames, etc.:
Quesnel et al. (INRA) Package system for VLE 13 / 32
14. The problem
Context
Modelers develop models in C++ and can reuse models from other
modelers
Models are stored into shared library. VLE loads and extracts atomic
model from shared library
Problems
We clearly identify a problem of sharing and reusing code:
VLE C++ API / ABI version
Sub-formalism C++ API / ABI version
Model change version (for example: the input or output DEVS ports)
Solution
To improve collaborative work under VLE we develop a package system
Quesnel et al. (INRA) Package system for VLE 14 / 32
15. 1 Context
2 Problem
3 Package system
4 Results
5 Conclusions and perspectives
Quesnel et al. (INRA) Package system for VLE 15 / 32
16. Package system
We identify the following parts to develop
Define the format of VLE’s package
Develop and upgrade the tools and libraries of VLE (VLE, GVLE,
PyVLE and RVLE) to take into account packages:
Open, install, read, compile, check, remove packages
Develop internet access from VLE to download and install packages
from distant distributions of packages
Add a checking consistency onto distributions and user’s installation
This work is inspired from the FOSS (Free Open Source Software)
community
Quesnel et al. (INRA) Package system for VLE 16 / 32
17. What’s a package in VLE
We need to share with the package system:
Data: eg. climate data files etc.
External shared library: eg. scientific libraries
Sub-formalism libraries: structurals and dynamics
Models: classes inherit from atomic, executive or sub-formalisms
classes
GUI plug-ins: to improve GVLE with new features
Output plug-ins: specific output simulation, database etc.
Documentation
Package must have a version number, an unique name and a list of
run-time and build dependencies
Quesnel et al. (INRA) Package system for VLE 17 / 32
18. What’s a package in VLE
Finite State automaton
The vle.extension.fsa package provides:
Harel’s state-chart, Mealy, Moore and FDDevs sub-formalisms
A graphical user interface to draw automaton an generate C++
For the user of the package:
Shared library which stores binary code of sub-formalisms
C++ Headers of the sub-formalisms
GUI plug-ins
The vle.extension.fsa package, only depends of VLE 1.1
Quesnel et al. (INRA) Package system for VLE 18 / 32
19. Example: FSA package
GUI to develop automaton
Quesnel et al. (INRA) Package system for VLE 19 / 32
21. What’s a package in VLE
The content and the structure of a package
package /
data / # data ( climate data file )
exp / # experiment file ( structure of DEVS model )
src / # source code
test / # unit test
lib / # shared sub - formalism libraries
plugins / # simulators , GUI plug - ins shared libraries
Authors . txt
Description . txt # the most important file
License . txt
A source package does not have the lib and plugins directories
A binary package does not have the src and test directories
Quesnel et al. (INRA) Package system for VLE 21 / 32
22. What’s a package in VLE
The description file
Source : weather - gen
Version : 1.0.0 -0
Section : agronomic
Maintainer : quesnel@users . sourceforge . net
Homepage : http :// www . vle - project . org
Build - Depends : vle ( >= 1.1) ,
differential - equation ( >= 1.0)
Depends : weather (= 1.2.3) , output - file ( >= 1.0)
Conflicts :
Description : A weather generator .
The build-dependencies are required to build and compile the package
The dependencies are required at run-time of the simulation
The conflicts package must be removed before the compilation of the
package or at run-time of the simulation
Quesnel et al. (INRA) Package system for VLE 22 / 32
23. What’s a package in VLE
An example of package dependencies with an agronomic model
DifferenceEquation,
Depends and build-depends of
FSA, Agent, PRNG
maize-optim package
are sub-formalisms:
indirect build-depends
packages for
maize-optim
Optim is
sub-formalism: direct
build-depend package
for maize-optim
Other packages are
run-time depends
Quesnel et al. (INRA) Package system for VLE 23 / 32
24. Distribution
Definition
The model developers build models or build sub-formalisms and make
upstream packages
The package maintainers get, check, fix upstream packages and pull
it into distribution
The distribution stores all packages. A consistency check is
automatically done before adding or removing a package.
Users of the distribution user community can download and install
distribution’s packages
Users can be connected to several distributions
Distributions are accessible via http or ftp protocols
Quesnel et al. (INRA) Package system for VLE 24 / 32
29. How to check package consistency ?
A distribution allows users to install and uninstall packages from a distant
repository
A distribution is a finite set of packages rules
Each package rule is a tuple of the form (p, B, D, C ): where:
p is a package,
B is a set a dependency build clauses for p,
D is a set of dependency clauses for p,
C is a set of conflict clauses for p
The dependency build clauses and dependency clauses must be present to
compile and use the package p
The dependency build packages can be removed after the installation
Quesnel et al. (INRA) Package system for VLE 26 / 32
30. How to check package consistency ?
Each dependency clause or dependency build clause is a disjunction of
packages p1 |...|pk
A dependency clause requires that some packages from the set
{p1 , ..., pk } to be present
Conflict clause requires that package p is not present in order to
install the package p
Thus, a valid installation is a subset of the packages in the
distribution
An installation profile is valid if dependency clauses and conflict
clauses are satisfied
A valid installation profile for a distribution should satisfy the
dependency and conflict clauses of each package rule of the
distribution
Quesnel et al. (INRA) Package system for VLE 27 / 32
31. How to check package consistency ?
The package system has two critical problems:
To install the new package p given a distribution R and an installation
profile P, the package system has to check if there is a profile P
containing p such that P ∪ P is a valid installation profile for R
It may happen that a new package p cannot be installed because it is
in conflict with some packages already installed on the machine. In
this case, we must first uninstall some packages before attempting to
install p. The task of the package system is then to identify a set of
packages P such that p can be installed on P − P
To address these two problems, we can use a satisfiability problem solver
(SAT) or a constraint satisfaction problem (CSP)
Quesnel et al. (INRA) Package system for VLE 28 / 32
32. 1 Context
2 Problem
3 Package system
4 Results
5 Conclusions and perspectives
Quesnel et al. (INRA) Package system for VLE 29 / 32
33. Results
The major parts of the package system has been developed
The major packages of VLE are converted into packages (near 20
packages)
RECORD project is converting its packages in new VLE packages
(near 100 packages)
Package system is working to download and install packages from a
server
All tools of VLE are converted to package systems: VLE, GVLE,
RVLE and PyVLE
However,
The package system does not embed a complete solver yet
The distribution package does not provide consistency checking
The package system of VLE is available. VLE and RECORD users use it !
Quesnel et al. (INRA) Package system for VLE 30 / 32
34. 1 Context
2 Problem
3 Package system
4 Results
5 Conclusions and perspectives
Quesnel et al. (INRA) Package system for VLE 31 / 32
35. Conclusions and perspectives
A success:
Users are happy to use the package system to share and reuse model,
sub-formalism and data
The use of package seems to simplify the construction of complex
models by aggregating atomic models and data from other packages
Users intensively use generic models in Weather or Glue packages:
these packages are now robust
Package system improves the global quality of the models and
simplify the maintenance of models
However,
The package system needs to have computer engineers:
To integrate and maintain packages of the community
To perform release of the distribution
Quesnel et al. (INRA) Package system for VLE 32 / 32