Since Pharo 5 we have been developing a new framework for communicate with external libraries called UnifiedFFI (or UFFI) that has opened the doors to perform a huge step in our support for mainstream necessities. In this talk I will talk about the intrinsic problems of developing such frameworks and put it in practice through the example of the newest tool in the Pharo family: Iceberg (the git client).
2. About me
Pharo architect since 2012
Owned a company to develop in
Pharo back in 2008
Java senior architect for 7 years (and
15 years overall Java experience)
Web, microprocessors, etc., etc., etc.
JavaScript, C++, ObjC, C#, Delphi,
ASM and lots of languages no longer
exist or have been long-time
forgotten
24 years (!) programming experience
(yes… I’m becoming old)
3. Why FFI*?
Why we want it or even need it?
*FFI: Foreign Function Interface
4. –Dan Ingalls
“an operating system is a collection of things that
don't fit inside a language; there shouldn't be one”
7. FFI plugin
Compiler build (no extensions available).
Basic C types and structures
No callbacks
Not very easy to understand
void pango_cairo_update_layout (cairo_t *cr, PangoLayout *layout)
CairoCanvas >> pangoUpdateCairo: anAddress1 layout: anAddress2
<cdecl: void 'pango_cairo_update_layout' (void* void*) module: '/opt/
local/lib/libpangocairo-1.0.0.dylib'>
8. AlienFFI plugin
Object per method (“aliens”)
Very low level
Callbacks!
Not very easy to understand
void pango_cairo_update_layout (cairo_t *cr, PangoLayout *layout)
CairoCanvas >> pangoUpdateCairo: anAlien1 layout: anAlien2
^ (Alien
lookup: 'pango_cairo_update_layout'
inLibrary: '/opt/local/lib/libpangocairo-1.0.0.dylib')
primFFICallResult: (result := Alien newGC: 4)
with: anAlien1
with: anAlien2
9. NativeBoost FFI
Runtime build
Easy to create OO models with it (look Athens)
Very easy to understand
Very low level and hard to modify
void pango_cairo_update_layout (cairo_t *cr, PangoLayout *layout)
CairoCanvas >> pangoUpdateLayout: layout
^ self
nbCall: #(void pango_cairo_update_layout (self, PangoLayout *layout))
module: '/opt/local/lib/libpangocairo-1.0.0.dylib'
10. Why we removed
NativeBoost?
It was not working on Spur (or in 64bit)
It was hard to maintain (and we need a different
implementation for each architecture)
Since we were using NB exclusively for FFI, we decided
to replace it with a different one, using the VM plugin
11. What is UFFI?
A front-end to express FFI calls in Pharo.
Uses ThreadedFFIPlugin, but should be able to plug
others in the future.
It shares same philosophy as NativeBoost:
‣ keep as much as possible in the image
‣ no need to modify VM to add functionality
But it is not ASM: Just plain Smalltalk.
12. Goals
Keep NativeBoost syntax
‣ Because is cool :)
‣ Provide backward compatibility for most use-cases
Enhance documentation and self-documentation
Be the unified base for future FFI backends
implementations
13. Meta-goals
Expand the “universe of possible”:
- Be able to use existing libraries our there instead
needing to reproduce them (run away of the “not
invented here” syndrome).
- Do not need plugins to expand functionality
Provide an easy and efficient way to interact with the
outside world.
14. How does a call looks like?
char *getenv(const char *)
getEnv: variable
^ self
ffiCall: #( String getenv( String variable ))
module: LibC
(People who know NativeBoost will find this very familiar….)
15. How does a call looks like?
char *getenv(const char *)
getEnv: variable
^ self
ffiCall: #( String getenv( String variable ))
module: LibC
A regular Pharo method with one argument
16. How does a call looks like?
char *getenv(const char *)
getEnv: variable
^ self
ffiCall: #( String getenv( String variable ))
module: LibC
A literal array to represent C function
17. How does a call looks like?
char *getenv(const char *)
getEnv: variable
^ self
ffiCall: #( String getenv( String variable ))
module: LibC
Types annotation used to generate marshalling code
18. How does a call looks like?
char *getenv(const char *)
getEnv: variable
^ self
ffiCall: #( String getenv( String variable ))
module: LibC
The value to be passed when calling out
19. How does a call looks like?
char *getenv(const char *)
getEnv: variable
^ self
ffiCall: #( String getenv( String variable ))
module: LibC
The library to lookup C function
20. FFILibrary
A very simple abstraction to define module names that
can be different each platform.
Can be used also as a place to store C function
definitions (like a real library :) ).
22. How does a call looks like?
49 <20> pushConstant: <cdecl: void 'pango_cairo_update_layout' (void*
void*) module: '/opt/local/lib/libpangocairo-1.0.0.dylib'>
50 <05> pushRcvr: 5
51 <10> pushTemp: 0
52 <76> pushConstant: 1
53 <E1> send: instVarAt:
54 <8A 82> pop 2 into (Array new: 2)
56 <E2> send: invokeWithArguments:
57 <87> pop
58 <78> returnSelf
void pango_cairo_update_layout (cairo_t *cr, PangoLayout *layout)
23. Types
Support for standard C types: int, float, etc.
Support for type aliases (map a name to one of the defined types)
Complex types:
- FFIExternalObject: External addresses (objects)
- FFIOpaqueObject: Opaque C types/structures
- FFIExternalStructure
- FFIExternalArray, FFITypeArray
- FFIExternalEnumeration
- FFIExternalValueHolder: Buffers (to pass referenced data, e.g. “double *”)
- FFIConstantHandle: Windows HANDLE (constant addresses)