Headless and beyond by Pablo Tesone

1. Improving the Pharo VM
Headless and Beyond
Pablo Tesone
Pharo Consortium Engineer

2. Who I am!
!2
Pablo Tesone
Pharo Consortium
Engineer
• 20 years trying to code
• 10 years of experience in industrial
applications
• PhD in Dynamic Software Update
• Interested in improving development tools
and the daily development process.
• Enthusiast of the object oriented
programming and their tools.
Also, playing with me:
Guille PolitoCNRS Engineer
RMod Team
Esteban Lorenzano
Pharo Consortium
Engineer

3. Announcement
!3
Pharo 8 Headless VM is out!
Get it now from zero-conf
http://get.pharo.org
$ wget -O - get.pharo.org/64/80+vmHeadlessLatest
$ ./pharo Pharo.image
Still Alpha!
But im
proving
fast!

4. What's Headless?
• More than a VM not showing the GUI
• Remove window management
• Remove event handling
• Only running my program!
!4

5. Why Headless?
• Command-line tools
• Scripting
!5
UI:
- Command-line arguments
- Files
- Piping standard input / output

6. Why Headless? (II)
• Servers
!6
- No UI or Web UI
- Network communication
- Sockets (TCP/UDP/Unix)
- RPC

7. Why Headless? (III)
• Services
!7
- No UI
- Network communication
- Sockets (TCP/UDP/Unix)
- RPC
- External control of lifecycle
- Container dependent API
- E.g: Window Services / launchd /
xinetd / Lambda

8. Why Headless? (IV)
• Stand-alone Desktop Applications
!8
- Has GUI
- The GUI depends of the APP
- No Morphic
- E.g: GTK+3, OpenGL, WindowsForm,
Cocoa
- All events and windows handled in
the application (image side)

9. So…we want a headless
VM!!!
!9

10. Reality: a VM is Complex
!10
Platform
Code
Plugins
Interpreter
JIT
GC
Memory
Management
Pharo VM
We need tools to:
- Handle complexity
- Focus in the
important stuff
- No fear of change

11. Slang ported to Pharo
• Taking advantage of the tools.
• Tests for the translation
• Fix the incompatibilities of Slang with Pharo
• Ensure code-generation repeatability
• Generate the code in each build!!
!11
Eat our own food!

12. Improved Build Process
• Simpler declarative building process
• Multi-platform (Windows / OSX / Linux)
• IDE integration (XCode / Eclipse / Visual Studio / …)
• Faster (Better dependency)
!12
Easier to Understand
and to modify!

13. Continuously Integrated
• Automatic build on each commit
• Building and testing branches & pull-requests
• Testing in three platforms: Windows / OSX / Linux
• Testing with VM tests, FFI tests, Pharo tests &
benchmarks
!13
I want a lot of tests!

14. Version Controlled
• All the code in a single Git repository (Slang + C)
• No generated code
• Available and visible in GitHub
• Tagging / Branching policies
• All modifications through PRs
!14
We need to go back to
any point in the past.

15. As we won…
• Safety
• Ability to change
• Early detection of errors & performance regressions
• Easier contribution
!15
So,… without fear….
we touched the VM

16. Power to the Image
!16
• Fully Backward Compatible Image
• The image open and controls the UI & Events
• Customisation of Window and Menus
• Two backends: Gtk+3 / SDL2
• Fully implemented with UFFI

17. Version Controlled
!17
• Single GIT Repository
• VMMaker code in Tonel (Thanks !)
• VMMaker in Pharo 7 & 8
• Source code restructuring
• Binary Dependencies Control

18. Much Simpler VM
!18
• Removing Unused Plugin
• Cleanup of UUID, Socket & SSL plugins
• Source code restructuring
• Removing duplicated code
• Platform code minimisation

19. Towards Embeddings
!19
• All Plugins are External Dynamic Libraries
• The VM is a Dynamic Library
• The main executable is a thin customisable frontend
• Initial client API
• Executable Embeddable Image

20. Non Blocking FFI
!20
Queue Based FFI
(Pharo Threaded FFI Plugin)
Strategy #1: Thread per Call-out
Strategy #2: Worker Threads
Strategy #3: VM Thread Runner
Strategy #4: Main Thread Runner
Not all Libraries are designed the same
• Must run in the main thread (Cocoa)
• Must run in a single thread (Gtk+3)
• Runs on any thread but not concurrent
(libgit, sqlite)
• Is a Thread-safe Library
• ….

21. Decision Table
!21
Same Thread Worker Threads
Long Calls
Blocking Parallel
Short Calls
Little Overhead Lots of Overhead
OPTIMIZATIONS

22. Transparency through
UFFI
!22
Your Program
Unified FFI
Pharo Threaded FFI
Pharo Threaded FFI Plugin
S#1 S#2 S#3 S#4
Squeak FFI
Squeak FFI
Plugin
Library

23. Coming soon to Pharo 8
• Using a single Socket Implementation
• Implementing Async I/O in all platforms
• Implementing Interrupting Event Pooling (Idle VM)
!23
Codename:
Pharo Zero

24. Coming soon to Pharo 8
• Running the VM in a second Thread.
• Letting out the main thread to something else
• Having an idle VM with correct UI Event handling
• Interface to talk with the VM
!24
Codename:
Pharo Ogro

25. Mid / Long - Term
Work
• Multiples VMs in a Single Process:
• Do we share an image?
• How to communicate?
• Isolation / Modularisation
• Multi-core processing
!25

26. Mid / Long - Term
Work (2)
• New Smalltalk to C translation (Illicium)
• Better Type Inference
• A pipeline to different translators
• Better generated C abstractions
• Able to translate other programs to C
!26

27. Mid / Long - Term
Work (3)
• Bootstrapping Small / Handcrafted Kernels
• Tools to Analyse applications
• Select the requirements
• Debugging
• Generation
• Update
!27

28. Mid / Long - Term
Work (4)
• Embedded System / Embedded VM in applications
• Including ARM / ARM64 in the new process
• Communication
• Selecting Features to Include
• Remote Debugging
• Updating
!28

29. !29
pharo-project/opensmalltalk-vm
https://ci.inria.fr/pharo-ci-jenkins2/job/pharo-vm/
Try it!
Hack it! Improve it!
Report
Issues!
Break it!
Learn!
Thanks!!!
It’s yours!