Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Ruby meets Go
Dec. 12, 2015
Masaki Matsushita
About Me
● Masaki Matsushita
● CRuby Committer
○ 138 Commits
■ Mainly for performance improvement
■ Marshal.load, Hash#fla...
Today’s Topic
● Go 1.5 Feature: buildmode “c-shared”
○ Cgo Basics
● Using Go Function from Ruby
○ FFI and Fiddle without r...
Buildmode “c-shared”
● Go 1.5 relased in August 2015
● Buildmode “c-shared” was introduced
○ go build -buildmode c-shared
...
Cgo Example: Say hello with puts() in C
package main
/*
#include <stdlib.h>
#include <stdio.h>
*/
import "C"
import "unsaf...
Cgo Example: define and use C function
package main
/*
char *hello(void) {
return "Hello, world!";
}
*/
import "C"
import ...
Try c-shared: add.go
package main
import "C"
//export add
func add(a C.int, b C.int) C.int {
return a + b
}
func main() {}...
Load c-shared Libraries
● ruby-ffi
○ https://github.com/ffi/ffi
○ gem install ffi
● fiddle
○ Standard ext library
● useful...
Call Go Function from Ruby: ruby-ffi
require "ffi"
module Int
extend FFI::Library
ffi_lib "int.so"
attach_function :add, [...
Call Go Function from Ruby: fiddle
require "fiddle/import"
module Int
extend Fiddle::Importer
dlload "int.so"
extern "int ...
Go String and C String: str.so
package main
import "C"
import "fmt"
//export hello
func hello(cstr *C.char) {
str := C.GoS...
Returning String: ruby-ffi
require "ffi"
module Hello
extend FFI::Library
ffi_lib "str.so"
attach_function :hello, [:strin...
Returning String: fiddle
require "fiddle/import"
module Hello
extend Fiddle::Importer
dlload "str.so"
extern "void hello(c...
Cgo Functions to Convert String
● C.CString(goString string) *C.char
○ copy Go String to C String
○ Users are responsible ...
Writing Extension Library with Go
● Naruse-san’s Amazing Talk:
“Writing extension libraries in Go”
at OedoRubyKaigi 05
htt...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
Minimal Go Ext Example?
//export rb_magic_num
func rb_magic_num(self C.VALUE) C.VALUE {
return INT2NUM(42)
}
//export Init...
Writing Extension Library with Go
● Wrapper Function equivalent to C Macro
○ C macros can’t be used by Cgo
● Convert Go St...
C Macros for Ruby Extention Libraries
● Useful C macros are defined in ruby.h
○ INT2NUM: C int to Ruby Numeric
○ NIL_P: tr...
Use Equivalent C Function
func LONG2NUM(n C.long) C.VALUE {
return C.rb_long2num_inline(n)
}
func NUM2LONG(n C.VALUE) C.lo...
Wrap C macros with C function
package main
/*
long rstring_len(VALUE str) {
return RSTRING_LEN(str);
}
*/
import "C"
func ...
Convert Go String into Ruby without Copy
● Go String -> C String -> Ruby String
● C.CString(goString string) *C.char
○ cop...
Basic Usage of C.CString()
// go/doc/progs/cgo4.go
func Print(s string) {
cs := C.CString(s)
defer C.free(unsafe.Pointer(c...
Basic Usage of C.CString()
str := "Hello, world!"
// Copy #1
cstr := C.CString(str) // will be discarded soon
// Copy #2
r...
Avoid Copy of Strings
● Get *C.char from Go String without Copy
func GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte...
Avoid Copy of Strings
● Technique to Get []byte from Go w/o Copy
http://qiita.com/mattn/items/176459728ff4f854b165
func GO...
Avoid Copy of Strings
● Get *C.char from []byte
func GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte)(unsafe.Pointer...
Example Usage of GOSTRING_PTR()
func RbString(str string) C.VALUE {
if len(str) == 0 { return C.rb_utf8_str_new(nil, C.lon...
Propagate Ruby Reference to Go
● Go’s GC doesn’t know refs from Ruby
● Go obj referenced from Ruby can be collected
● We h...
Propagate Ruby Reference to Go
var objects = make(map[interface{}]int)
//export goobj_retain
func goobj_retain(obj unsafe....
Propagate Ruby Reference to Go
static const rb_data_type_t go_type = {
"GoStruct",
{NULL, goobj_free, NULL},
0, 0, RUBY_TY...
Create Gem including Go code
● Directory Structure
● Rakefile
● extconf.rb
Directory Structure
● Use “bundle gem --ext foo”
├── ext
│ └── foo
│ ├── extconf.rb // configured to use go build
│ ├── fo...
Rakefile
require 'bundler'
Bundler::GemHelper.install_tasks
require 'rake/extensiontask'
task :default => [:compile]
spec ...
extconf.rb
require 'mkmf'
find_executable('go')
$objs = []
def $objs.empty?; false ;end
create_makefile("memberlist/member...
extconf.rb
File.open('Makefile', 'a') do |f|
f.write <<-EOS.gsub(/^ {8}/, "t")
$(DLLIB): Makefile $(srcdir)/memberlist.go ...
Ruby meets Go
● Buildmode “c-shared” and Cgo Basics
● Using Go Function from Ruby
○ FFI and Fiddle without ruby.h
● Writin...
Próxima SlideShare
Cargando en…5
×

Ruby meets Go

24.667 visualizaciones

Publicado el

Presentation in RubyKaigi 2015

Publicado en: Tecnología
  • Sé el primero en comentar

Ruby meets Go

  1. 1. Ruby meets Go Dec. 12, 2015 Masaki Matsushita
  2. 2. About Me ● Masaki Matsushita ● CRuby Committer ○ 138 Commits ■ Mainly for performance improvement ■ Marshal.load, Hash#flatten, etc. ● Software Engineer at NTT Communications ○ Contribution to OpenStack ○ Slide at OpenStack Summit Tokyo http://goo.gl/OXTYor ● Twitter: @_mmasaki Github: mmasaki
  3. 3. Today’s Topic ● Go 1.5 Feature: buildmode “c-shared” ○ Cgo Basics ● Using Go Function from Ruby ○ FFI and Fiddle without ruby.h ● Writing Extension Library with Go (and C) ○ Define Functions Equivalent to C Macros ○ Avoid Copy of Strings ○ Propagate Reference from Ruby to Go ○ Creating Gem including Go code
  4. 4. Buildmode “c-shared” ● Go 1.5 relased in August 2015 ● Buildmode “c-shared” was introduced ○ go build -buildmode c-shared ○ Build C shared library with cgo ● cgo enables: ○ Refer to C functions, types and variables ○ Export Go functions for use by C
  5. 5. Cgo Example: Say hello with puts() in C package main /* #include <stdlib.h> #include <stdio.h> */ import "C" import "unsafe" func main() { cstr := C.CString("Hello, world!") defer C.free(unsafe.Pointer(cstr)) C.puts(cstr) } Include C header file Convert Go string into C String
  6. 6. Cgo Example: define and use C function package main /* char *hello(void) { return "Hello, world!"; } */ import "C" import "fmt" func main() { cstr := C.hello() fmt.Println(C.GoString(cstr)) } Define C Function Convert into Go String Call C Function from Go
  7. 7. Try c-shared: add.go package main import "C" //export add func add(a C.int, b C.int) C.int { return a + b } func main() {} ● go build -buildmode c-shared -o add.so add.go Export Go Function for use by C
  8. 8. Load c-shared Libraries ● ruby-ffi ○ https://github.com/ffi/ffi ○ gem install ffi ● fiddle ○ Standard ext library ● useful to call Go functions simply (without ruby.h)
  9. 9. Call Go Function from Ruby: ruby-ffi require "ffi" module Int extend FFI::Library ffi_lib "int.so" attach_function :add, [:int, :int], :int end p Int.add(15, 27) #=> 42 Load c-shared library Add Go Function to Module
  10. 10. Call Go Function from Ruby: fiddle require "fiddle/import" module Int extend Fiddle::Importer dlload "int.so" extern "int add(int, int)" end p Int.add(15, 27) #=> 42
  11. 11. Go String and C String: str.so package main import "C" import "fmt" //export hello func hello(cstr *C.char) { str := C.GoString(cstr) fmt.Println("Hello, " + str) } func main() {} Receive C String Convert to Go String
  12. 12. Returning String: ruby-ffi require "ffi" module Hello extend FFI::Library ffi_lib "str.so" attach_function :hello, [:string], :void end Hello.hello("world") #=> "Hello, world" Ruby String can be passed
  13. 13. Returning String: fiddle require "fiddle/import" module Hello extend Fiddle::Importer dlload "str.so" extern "void hello(char *str)" end Hello.hello("world") #=> "Hello, world"
  14. 14. Cgo Functions to Convert String ● C.CString(goString string) *C.char ○ copy Go String to C String ○ Users are responsible to free C String ● C.GoString(cString *C.char) string ● C.GoStringN(cString *C.char, length C.int) string ○ copy C String to Go String
  15. 15. Writing Extension Library with Go ● Naruse-san’s Amazing Talk: “Writing extension libraries in Go” at OedoRubyKaigi 05 https://speakerdeck.com/naruse/writing-extension-libraries-in-go ● gohttp: https://github.com/nurse/gohttp ○ Implementation of extension library in Go
  16. 16. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); }
  17. 17. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); } Method Implementation
  18. 18. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); } Using C Macro
  19. 19. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); } Function Pointer
  20. 20. Minimal Go Ext Example? //export rb_magic_num func rb_magic_num(self C.VALUE) C.VALUE { return INT2NUM(42) } //export Init_foo func Init_foo() { rb_cFoo = rb_define_class("Foo", C.rb_cObject) rb_define_method(rb_cFoo, "magic_num", C.rb_magic_num, 0) }
  21. 21. Writing Extension Library with Go ● Wrapper Function equivalent to C Macro ○ C macros can’t be used by Cgo ● Convert Go String into Ruby without Copy ● Propagate Ruby Reference to Go ● Create gem including Go code ○ Modify Rakefile and extconf.rb
  22. 22. C Macros for Ruby Extention Libraries ● Useful C macros are defined in ruby.h ○ INT2NUM: C int to Ruby Numeric ○ NIL_P: true if obj is nil ○ RSTRING_PTR: pointer to buffer of String ○ RSTRING_LEN: lengh of String ● These macros can’t be used from Cgo… ● Define Go functions equivalent to C macros ○ Use equivalent C function ○ Wrap C macros with C function
  23. 23. Use Equivalent C Function func LONG2NUM(n C.long) C.VALUE { return C.rb_long2num_inline(n) } func NUM2LONG(n C.VALUE) C.long { return C.rb_num2long(n) }
  24. 24. Wrap C macros with C function package main /* long rstring_len(VALUE str) { return RSTRING_LEN(str); } */ import "C" func RSTRING_LEN(str C.VALUE) C.long { return C.rstring_len(str) }
  25. 25. Convert Go String into Ruby without Copy ● Go String -> C String -> Ruby String ● C.CString(goString string) *C.char ○ copy Go String to C String ○ Users are responsible to free C String ● VALUE rb_str_new(const char *ptr, long len) ○ copy C String to Ruby String
  26. 26. Basic Usage of C.CString() // go/doc/progs/cgo4.go func Print(s string) { cs := C.CString(s) defer C.free(unsafe.Pointer(cs)) C.fputs(cs, (*C.FILE)(C.stdout)) } ● Call C func and discard C str soon
  27. 27. Basic Usage of C.CString() str := "Hello, world!" // Copy #1 cstr := C.CString(str) // will be discarded soon // Copy #2 rbstr := C.rb_str_new(cstr, C.long(len(str))) ● Need to copy twice!
  28. 28. Avoid Copy of Strings ● Get *C.char from Go String without Copy func GOSTRING_PTR(str string) *C.char { bytes := *(*[]byte)(unsafe.Pointer(&str)) return (*C.char)(unsafe.Pointer(&bytes[0])) } // example of use cstr := GOSTRING_PTR(str) C.rb_utf8_str_new (cstr, C.long(len(str)))
  29. 29. Avoid Copy of Strings ● Technique to Get []byte from Go w/o Copy http://qiita.com/mattn/items/176459728ff4f854b165 func GOSTRING_PTR(str string) *C.char { bytes := *(*[]byte)(unsafe.Pointer(&str)) return (*C.char)(unsafe.Pointer(&bytes[0])) }
  30. 30. Avoid Copy of Strings ● Get *C.char from []byte func GOSTRING_PTR(str string) *C.char { bytes := *(*[]byte)(unsafe.Pointer(&str)) return (*C.char)(unsafe.Pointer(&bytes[0])) } Cast to char
  31. 31. Example Usage of GOSTRING_PTR() func RbString(str string) C.VALUE { if len(str) == 0 { return C.rb_utf8_str_new(nil, C.long(0)) } return C.rb_utf8_str_new(GOSTRING_PTR(str), GOSTRING_LEN(str)) } func rb_define_class(name string, parent C.VALUE) C.VALUE { return C.rb_define_class(GOSTRING_PTR(name), parent) } func rb_define_method(klass C.VALUE, name string, fun unsafe.Pointer, args int) { cname := GOSTRING_PTR(name) C.rb_define_method(klass, cname, (*[0]byte)(fun), C.int(args)) }
  32. 32. Propagate Ruby Reference to Go ● Go’s GC doesn’t know refs from Ruby ● Go obj referenced from Ruby can be collected ● We have to propagate Ruby Refs to Go ● Use Map to keep reference to Go Objects
  33. 33. Propagate Ruby Reference to Go var objects = make(map[interface{}]int) //export goobj_retain func goobj_retain(obj unsafe.Pointer) { objects[obj]++ // increment reference count } //export goobj_free func goobj_free(obj unsafe.Pointer) { objects[obj]-- // decrement reference count if objects[obj] <= 0 { delete(objects, obj) } }
  34. 34. Propagate Ruby Reference to Go static const rb_data_type_t go_type = { "GoStruct", {NULL, goobj_free, NULL}, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY|RUBY_TYPED_WB_PROTECTED }; VALUE NewGoStruct(VALUE klass, void *p) { goobj_retain(p); return TypedData_Wrap_Struct((klass), &go_type, p); } Increment Reference Count Decrement Reference Count
  35. 35. Create Gem including Go code ● Directory Structure ● Rakefile ● extconf.rb
  36. 36. Directory Structure ● Use “bundle gem --ext foo” ├── ext │ └── foo │ ├── extconf.rb // configured to use go build │ ├── foo.c // helper functions for use by Go │ └── foo.h // export helper functions │ ├── foo.go // created by hand │ └── wrapper.go // created by hand └── lib
  37. 37. Rakefile require 'bundler' Bundler::GemHelper.install_tasks require 'rake/extensiontask' task :default => [:compile] spec = eval File.read('foo.gemspec') Rake::ExtensionTask.new('foo', spec) do |ext| ext.lib_dir = File.join(*['lib', 'foo', ENV['FAT_DIR']].compact) ext.ext_dir = 'ext/foo' ext.source_pattern = "*.{c,cpp,go}" end ● Add .go into source_pattern
  38. 38. extconf.rb require 'mkmf' find_executable('go') $objs = [] def $objs.empty?; false ;end create_makefile("memberlist/memberlist") case `#{CONFIG['CC']} --version` when /Free Software Foundation/ ldflags = '-Wl,--unresolved-symbols=ignore-all' when /clang/ ldflags = '-undefined dynamic_lookup' end ● Some techniques to build successful
  39. 39. extconf.rb File.open('Makefile', 'a') do |f| f.write <<-EOS.gsub(/^ {8}/, "t") $(DLLIB): Makefile $(srcdir)/memberlist.go $(srcdir)/wrapper.go CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' go build -p 4 -buildmode=c-shared -o $(DLLIB) EOS end ● Modify Makefile to use go build
  40. 40. Ruby meets Go ● Buildmode “c-shared” and Cgo Basics ● Using Go Function from Ruby ○ FFI and Fiddle without ruby.h ● Writing Extension Library with Go ○ Define Functions Equivalent to C Macros ○ Avoid Copy of Strings ○ Propagate Reference from Ruby to Go ○ Creating Gem including Go code ● Let’s Hack Go for Ruby together!

×