Scaling Xtext projects presented challenges for Sigasi's hardware design IDE due to large project sizes of over 250,000 lines of code. Initial performance of building and editing large files in Xtext was unacceptable, taking over 20 minutes to build and causing the editor to be unusable. Through performance analysis and optimizations including reducing workload, parallelizing builds, caching models, and implementing a lightweight editor, Sigasi was able to improve build times to under a minute and make editing large files usable. Continued work involves caching global index information and further linking and validation optimizations.
2. Sigasi
● IDE for Hardware Description Languages
○ VHDL, (System)Verilog
● Using Xtext for 4 years
● Large user base
○ (commercial, free, students)
3. Our company goal
● Assist hardware designer
● High quality interactive front-end compiler
○ Instant error reporting
■ parsing, semantic, linting, style checking
○ Visualisations, design exploration, ...
○ Integrate with ecosystem
■ other compilers, simulators, synthesizers
4. The challenge
● Large projects
○ > 250 KLOC is not uncommon
○ design + external libraries
○ big files
■ some libraries are distributed as 1 file
■ generated hardware cores
●
5. Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes
■ > 2 GB
○ Editing big files (> 1 MB)
■ unusable
6. Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes → < 1 min
■ > 2 GB → < 1 GB memory
○ Editing big files (> 1 MB)
■ unusable → usable with reduced editor
9. Analyzing builds: builder overview
Global
indexing
Linking
Validation
Custom
Validation
Global
index
Eclipse
resources
warnings
errors
resource
descriptions
Builder
Participants
resource
changes
?
10. Analyzing builds: metrics
● For each build
○ # of files being build
○ timing: Global index, Linking, Validation, Individual
builder participants
● Instrument by overriding
ClusteringBuilderState & XtextBuilder
● Example: Building 134 resources, timing: {
global index=1806,
linking=378,
validation=823,
totalLinkingAndValidation=1364
}
11. Analyzing builds: resource loads
● Observation:
○ Most time spent in resource loads
○ Certain files are loaded multiple times?!
Global
indexing
Linking Validation
Builder
Participants
resources
LOAD
POTENTIAL
RELOAD
POTENTIAL
RELOAD
12. Analyzing builds: Memory footprint
Global
index
Resource
Set
● Global index
○ always loaded →limit size
○ use hashing: name + hash of contents
● Reduce EMF size
derived model -> Xcore model
http://www.sigasi.com/content/view-complexity-your-xtext-ecore-model
13. Optimize loading
● What is resource load?
○ Parse
○ build EMF model
○ build Node model
● Parallelise
○ parse multiple files simultaneously
○ ~3 time faster loads on 4 core machine
○ EMF synchronization issues
● Cache
○ serialize EMF and Node model in a cache
○ 3-4 time faster loads
15. Linking
Pseudo-algorithm
● In your IScopePovider
○ Find a eContainer that can be linked eagerly
○ Link everything
■ unresolved links become UnlinkedDeclaration
● In your DefaultLinkingService
○ getLinkedObjects(...)
■ already resolved? → skip it
● Disable notifications
18. UI responsiveness
● Measuring: detect a blocked UI thread
○ initially Svelto https://github.com/dragos/svelto
○ now our own method & logging
○ Eclipse Mars
● Improvements
○ UI is for drawing only!
○ Make sure everything is cancellable
● Safeguards
○ certain services should never be executed on the UI
thread => check & log
21. Future work
● Continuous process
● Cache global index info per resource?
● Linking without node model?
● Avoid project locking build (Xtext 2.8?)
22. Come talk to us about...
● Documentation generation
● Fancy linking algorithms / type systems
● Graphical views
● Cross-language support
● Testing Xtext-plugins
● Lexical macros
● Manage large amount of validations
● ...
23. +1 0 -1
Sign in: www.eclipsecon.org
Evaluate the sessions