Nearly every Eclipse user is aware what Xtext is and what it is useful for. It is always a pleasure to see in which kind of areas it is used.
One reason is clearly, that it is really easy to build a first working example in no time. The grammar alone is sufficient and the rest is done by a code generator. Xtext will generate stub classes, that are ready to fill in individual logic for different areas.
By doing that most of the things just work, because Xtext comes with a lot of defaults that suits in most of the cases.
But as projects get bigger and the amount of models grow, the defaults might not scale and you might need to tweak your language to make working smooth again.
Here it becomes more complicated and you need to understand what’s going on under the covers.
In this session I will go through typical problems and solutions, that we have seen in customer projects. Sometimes it’s a tradeoff to tackle those problems, but it’s important to understand the implications.
For some typical examples I will go into details and explain what’s going on under the covers.
After the session attendees should have an idea how to tweak their own languages and why it might be worth the effort to migrate to a newer version of Xtext.
8. Xtext defaults are a good start!
But one size does not fit all!
Amount of DSLs increases
9. Xtext defaults are a good start!
But one size does not fit all!
Amount of DSLs increases
Files get bigger
10. Xtext defaults are a good start!
But one size does not fit all!
Amount of DSLs increases
Files get bigger
Amount of files increases
11. Xtext defaults are a good start!
But one size does not fit all!
Amount of DSLs increases
Files get bigger
Amount of files increases
Many cross references
12. Xtext defaults are a good start!
But one size does not fit all!
Amount of DSLs increases
Files get bigger
Amount of files increases
Many cross references
Maybe transitive cross references…
15. Look what we have build!
More than 20 DSLs…
and 200.000 lines in one file…
16. Look what we have build!
More than 20 DSLs…
and 200.000 lines in one file…
and we can check in broken models…
17. Look what we have build!
More than 20 DSLs…
and 200.000 lines in one file…
and we can check in broken models…
and we generate the missing stuff…
18. Look what we have build!
More than 20 DSLs…
and 200.000 lines in one file…
and we can check in broken models…
and we generate the missing stuff…
to make them valid again!
19. Look what we have build!
More than 20 DSLs…
and 200.000 lines in one file…
We are on Xtext 2.6.2 and can’t update!
and we can check in broken models…
and we generate the missing stuff…
to make them valid again!
20. Look what we have build!
More than 20 DSLs…
and 200.000 lines in one file…
We are on Xtext 2.6.2 and can’t update!
and we can check in broken models…
and we generate the missing stuff…
to make them valid again!
Could you please back-port the changes?
43. Scoping is sooo complicated…
… I heard that in nearly every project…
44. Scoping is sooo complicated…
… I heard that in nearly every project…
…and people stop improving things when
it work’s for the first time…
45. Scoping is sooo complicated…
… I heard that in nearly every project…
…and people stop improving things when
it work’s for the first time…
… code is not touched any more because
who knows…
46. Scoping is sooo complicated…
… I heard that in nearly every project…
…and people stop improving things when
it work’s for the first time…
… code is not touched any more because
who knows…
… it worked before you touched it!
55. Scoping API
scope_Element_feature(Element element ,EReference ref)
Called by the PolymorphicDispatcher…
and if grammar changed it might not get called any more…
…in a reflective way…
…BECAUSE IT’S SLOW!
56. Scoping API
scope_Element_feature(Element element ,EReference ref)
Called by the PolymorphicDispatcher…
and if grammar changed it might not get called any more…
…in a reflective way…
…BECAUSE IT’S SLOW!
Implement
getScope(EObject context, EReference ref)
if(context instanceof Element && ref ==
MyPackage.Literals.ELEMENT_FEATURE)
up to 10 %
57. Scoping API
scope_Element_feature(Element element ,EReference ref)
Called by the PolymorphicDispatcher…
and if grammar changed it might not get called any more…
…in a reflective way…
…BECAUSE IT’S SLOW!
Implement
getScope(EObject context, EReference ref)
if(context instanceof Element && ref ==
MyPackage.Literals.ELEMENT_FEATURE)
up to 10 %Will not work
with
Xbase
anyway…
59. Use caches in the ScopeProvider
For each cross reference the ScopeProvider gets asked…
60. Use caches in the ScopeProvider
…and it calculates the very same stuff
over and over again…
For each cross reference the ScopeProvider gets asked…
61. Use caches in the ScopeProvider
…and it calculates the very same stuff
over and over again…
For each cross reference the ScopeProvider gets asked…
…cache the calculated stuff and invalidate it when
necessary!
62. Use caches in the ScopeProvider
…and it calculates the very same stuff
over and over again…
For each cross reference the ScopeProvider gets asked…
…cache the calculated stuff and invalidate it when
necessary!
The cache of the GlobalScope should be the index and
nothing else!
64. Use the force of the Index
Holds all lightweight representations of
referenceable elements and their resources
65. Use the force of the Index
Holds all lightweight representations of
referenceable elements and their resources
It’s a HashMap…
It’s super fast since Xtext 2.x
and improved further
66. Holds all lightweight representations of
referenceable elements and their resources
Use the force of the Index
It’s a HashMap…
It’s super fast since Xtext 2.x
and improved further
It’s YOUR Index - you can put data in…!
68. Index - how to feed it
1. Builder picks up DSL files and loads them one by one
69. Index - how to feed it
1. Builder picks up DSL files and loads them one by one
2. Creates ResourceDescriptions through ResourceDescriptionManager
70. Index - how to feed it
1. Builder picks up DSL files and loads them one by one
2. Creates ResourceDescriptions through ResourceDescriptionManager
3. ResourceDescriptionManager calls ResourceDescriptionsStrategy to
create EObjectDescriptions
71. Index - how to feed it
1. Builder picks up DSL files and loads them one by one
2. Creates ResourceDescriptions through ResourceDescriptionManager
3. ResourceDescriptionManager calls ResourceDescriptionsStrategy to
create EObjectDescriptions
You have to bind a custom impl for the ResourceDescriptionsStrategy in
the RuntimeModule
72. Index - how to feed it
1. Builder picks up DSL files and loads them one by one
2. Creates ResourceDescriptions through ResourceDescriptionManager
3. ResourceDescriptionManager calls ResourceDescriptionsStrategy to
create EObjectDescriptions
You have to bind a custom impl for the ResourceDescriptionsStrategy in
the RuntimeModule
The default creates a EObjectDescription for
EVERY Element that has a name…
78. UserData - what for?
Validation
Don’t load the world to get informations - put
them in the index
79. UserData - what for?
Validation
Don’t load the world to get informations - put
them in the index
Scoping
Don’t resolve proxies for simple informations
81. Reference non Xtext models
We have our own GlobalScopeProvider that does the trick…
82. Reference non Xtext models
We have our own GlobalScopeProvider that does the trick…
…we load everything and the GlobalScope contains the
corresponding EObjectDescriptions…
83. Reference non Xtext models
We have our own GlobalScopeProvider that does the trick…
…we load everything and the GlobalScope contains the
corresponding EObjectDescriptions…
…but it’s slow and we cannot navigate those elements in the
editor…
You can do better…
85. Reuse the Xtext infrastructure
Bind a ResourceServiceProvider…
…so that you can put the elements in the index and make
them navigable!
86. Reuse the Xtext infrastructure
Bind a ResourceServiceProvider…
…so that you can put the elements in the index and make
them navigable!
GenericResourceServiceProvider
Use
It will put everything that has a name in the index…
… introduce your own ResourceDescriptionStrategy!
95. Index in Standalone
Fill the index and install it on the ResourceSet!
Load each resource and ask the ResourceDescriptionManager
to create a ResourceDescription…
96. Index in Standalone
Fill the index and install it on the ResourceSet!
Load each resource and ask the ResourceDescriptionManager
to create a ResourceDescription…
Install it on the ResourceSet
97. Make big things run again and save your ass!
Up to 80% more faster in 10 minutes!
103. Incremental generators
The builder triggers the generator for each resource…
…when a file changes the builder computes the impact and
retriggers the generator for each affected file!
104. Incremental generators
The builder triggers the generator for each resource…
…when a file changes the builder computes the impact and
retriggers the generator for each affected file!
What about m:n scenarios?
Solveable with a little effort - but it is worth the work!
105. Incremental generators
The builder triggers the generator for each resource…
…when a file changes the builder computes the impact and
retriggers the generator for each affected file!
What about m:n scenarios?
Solveable with a little effort - but it is worth the work!
There is a ParallelBuilderParticipant…
…you have to bind it for your language…
… make sure all proxies are already resolved!
109. The NodeModel is huge
Stores the syntax tree with text and positions…
110. The NodeModel is huge
Stores the syntax tree with text and positions…
…is used to get the position of a specific element or
feature…
111. The NodeModel is huge
Stores the syntax tree with text and positions…
…is used to get the position of a specific element or
feature…
…produces a lot of Strings in memory…
112. The NodeModel is huge
Stores the syntax tree with text and positions…
…is used to get the position of a specific element or
feature…
…produces a lot of Strings in memory…
…and during the builder runs you do not really need
it.
115. Switch off the NodeModel
The parser creates it …
…it is used to install proxies - so you have to do that
differently…
116. Switch off the NodeModel
The parser creates it …
…it is used to install proxies - so you have to do that
differently…
You want to have it when the editor is used…
117. Switch off the NodeModel
The parser creates it …
…it is used to install proxies - so you have to do that
differently…
You want to have it when the editor is used…
… the load options should be an indicator…
118. Switch off the NodeModel
The parser creates it …
…it is used to install proxies - so you have to do that
differently…
You want to have it when the editor is used…
… the load options should be an indicator…
…what are the implications?
120. Implications
You cannot use the NodeModel in validations and
scoping any more…
…Issues are marked at position 0,0…
…an EditorCallback should revalidate the content.
121. Implications
You cannot use the NodeModel in validations and
scoping any more…
…Issues are marked at position 0,0…
…an EditorCallback should revalidate the content.
THIS IS NOT A COMMON THING!
DO IT ONLY WHEN IT IS REALLY NECESSARY!
123. Order makes a difference
Files might belong together…
124. Order makes a difference
Files might belong together…
…and might reference each other.
125. Order makes a difference
Files might belong together…
…and might reference each other.
Putting them in an logical order for the builder…
126. Order makes a difference
Files might belong together…
…and might reference each other.
Putting them in an logical order for the builder…
… and they do not have be loaded more than once.
127. Order makes a difference
Files might belong together…
…and might reference each other.
Putting them in an logical order for the builder…
… and they do not have be loaded more than once.
Sometimes it makes sense to not unload them at all.