This talk gives an introduction to the properties of the Ruby language and it presents the basics of the HornetsEye Ruby extension for doing machine vision. The end of the talk points out a few must-see videos and must-read books if you got excited about either Ruby or machine vision or both.
Ruby & Machine Vision - Talk at Sheffield Hallam University Feb 2009
1. Ruby & Machine Vision
Ruby & Machine Vision
Jan Wedekind
Wednesday, February 4th 2009
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 1/38
2. UK EPSRC Nanorobotics Project
Microscopy Software
• telemanipulation
• drift
compensation
• closed-loop
control
Machine Vision
• real-time software
• system integration
• theoretical
insights
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 2/38
3. Ruby Programming Language
Ruby
• created by Yukihiro Matsumoto
• released 1995 (free software(*) , Ruby license)
• inspired by Perl, Python, Smalltalk, Eiffel, Ada, Lisp
• “pseudo simplicity”: simple syntax ⇔ multi-paradigm
language
• highly portable
• Tiobe Programming Community Index #11
• 1.8.6 being superseded by 1.9.1
page url
Ruby Homepage http://www.ruby-lang.org/
Ruby Core-API http://www.ruby-doc.org/
RubyForge http://rubyforge.org/
Ruby Application Archive http://raa.ruby-lang.org/
(*) http://www.gnu.org/philosophy/free-sw.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 3/38
4. Dynamic Typing
#!/usr/bin/env ruby
def test( a, b )
a + b
end
x = test( 3, 5 ) # x -> 8
x = test( 'a', 'b' ) # x -> 'ab'
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 4/38
5. Garbage Collector
Mark and Sweep Garbage Collector
root
marked=true marked=true marked=true
marked=false marked=false marked=true
http://www.brpreiss.com/books/opus5/html/page424.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 5/38
6. (Pure) Object-Oriented, Single-Dispatch
#!/usr/bin/env ruby
class Numeric
def plus(x)
self.+(x)
end
end
y = 5.plus 6
# y is now equal to 11
http://www.ruby-lang.org/en/about/
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 6/38
7. Mixins
#!/usr/bin/env ruby
module TimesThree
def three_times
self + self + self
end
end
class String
include TimesThree
end
'abc'.three_times # -> 'abcabcabc'
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 7/38
8. Exception Handling
#!/usr/bin/env ruby
begin
print "Enter filename: "
STDOUT.flush
file_name = STDIN.readline.delete( "nr" )
file = File.new file_name, 'r'
# ...
rescue Exception => e
puts "Error opening file '#{file_name}': #{e.message}"
end
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 8/38
9. Closures
Unifying Concept for Iterators, Function Objects, and Loops
#!/usr/bin/env ruby
def inc( i )
lambda do |v|
v + i
end
end
t = inc( 5 )
t.call( 3 ) # -> 8
[ 1, 2, 3 ].each do |x|
puts x
end
[ 1, 2, 3 ].collect do |x|
x ** 2
end # -> [1, 4, 9]
[ 1, 2, 3 ].inject do |v,x|
v + x
end # -> 6
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 9/38
10. Continuations
#!/usr/bin/env ruby
def test( c2 )
callcc do |c1|
return c1
end
c2.call
end
callcc do |c2|
c1 = test( c2 )
c1.call
end
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 10/38
12. Metaprogramming
Interpreter modifies Program
#!/usr/bin/env ruby
eval 'x=5' # x -> 5
a = [ 1 ]
a.instance_eval do
push 2
end # a -> [ 1, 2 ]
a.send( 'push', 3 ) # a -> [ 1, 2, 3 ]
Object.const_get( 'String' ).class_eval do
define_method 'test' do
reverse
end
end
'abc'.reverse # -> 'cba'
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 12/38
13. Reification
Program modifies Interpreter
#!/usr/bin/env ruby
class Numeric
def method_missing( name, *args )
prefix = Regexp.new( "^#{name}" )
full_name = methods.find { |id| id =~ prefix }
if full_name
send( full_name, *args )
else
super
end
end
end
5.mod 2 # calls 5.modulo 2
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 13/38
14. Ruby Extensions
C Library
// gcc -shared -fPIC -I/usr/lib/ruby/1.8/x86_64-linux
// -o myextension.so myextension.c
#include <ruby.h>
#include <math.h>
VALUE wrap_logx( VALUE self, VALUE x )
{
return rb_float_new( log( NUM2DBL( self ) ) / log( NUM2DBL( x ) ) );
}
void Init_myextension(void) {
VALUE numeric = rb_const_get( rb_cObject, rb_intern( "Numeric" ) );
rb_define_method( numeric, "logx", RUBY_METHOD_FUNC( wrap_logx ), 1 );
}
Invoking Ruby Program
#!/usr/bin/env ruby
require 'myextension'
e = 1024.logx( 2 )
puts "2 ** #{e} = 1024"
http://www.rubyist.net/~nobu/ruby/Ruby_Extension_Manual.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 14/38
15. HornetsEye - Ruby Extension for Machine Vision
Free Software Project
• Real-Time Machine Vision
• Ruby Extension
• released under GNU General Public License
• 2 years development
• 22000 lines of code
http://www.wedesoft.demon.co.uk/hornetseye-api/
http://rubyforge.org/projects/hornetseye/
http://sourceforge.net/projects/hornetseye/
https://launchpad.net/hornetseye/
http://raa.ruby-lang.org/project/hornetseye/
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 15/38
16. Using existing Free Software
Libraries Integrated
C++ Boost
libdc1394
Tools in Use
DotGNU
Bazaar
FFTW
MPlayer GCC
OpenEXR make, automake, autoconf
NaturalDocs
Qt4-QtRuby
RMagick NSIS
NArray Ruby
libxine
Xorg, Mesa3D
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 16/38
21. Just-In-Time Compiler
#!/usr/bin/env
require 'hornetseye'
include Hornetseye
fun = JITFunction.compile( JITType::DFLOAT,
JITType::DFLOAT,
JITType::DFLOAT ) do |f,a,b|
Math.log( a ) / Math.log( b )
end
fun.call( 1024, 2 ) # -> 10.0
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 21/38
22. Some other Vision Libraries I/II
Framewave
EasyVision
CMVision
Camellia
Gamera
Gandalf
libCVD
Filters
Blepo
feature
Camera Input
Image Files
Video Files
Display
Scripting
Warps
Histograms
Custom Filters
Fourier Transforms
Feature Extraction
Feature Matching
GPL compatible ?
Also see http://www.wedesoft.demon.co.uk/hornetseye-api/files/Links-txt.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 22/38
23. Some other Vision Libraries II/II
NASA V. W.
HornetsEye
ITK/VTK
SceneLib
OpenCV
VIGRA
Mimas
LTIlib
Lush
IVT
feature
Camera Input
Image Files
Video Files
Display
Scripting
Warps
Histograms
Custom Filters
Fourier Transforms
Feature Extraction
Feature Matching
GPL compatible
Also see http://www.wedesoft.demon.co.uk/hornetseye-api/files/Links-txt.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 23/38
28. Capture Video
#!/usr/bin/env ruby
require 'hornetseye'
include Hornetseye
input = V4L2Input.new( '/dev/video0', 640, 480 )
output = MEncoderOutput.new( 'test.avi', 10,
'-ovc lavc -lavcopts vcodec=msmpeg4:vhq:vbitrate=4000' )
X11Display.show do
img = input.read
output.write( img )
img
end
output.close
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 28/38
29. Center of Gravity
#!/usr/bin/env ruby
require 'hornetseye'
include Hornetseye
input = V4L2Input.new '/dev/video0', 640, 480
idx = MultiArray.lint( input.width, input.height ).indgen!
x = idx % idx.shape[0]
y = idx / idx.shape[0]
img = nil
X11Display.show { img = input.read_rgb24 }
ref = img[ 0, 0 ]
X11Display.show do
img = input.read_rgb24.to_sintrgb
cdiff = img - ref
diff = cdiff.r.abs + cdiff.g.abs + cdiff.b.abs
mask = ( diff 40 ).to_ubyte
n = mask.sum
puts x = #{( mask * x ).sum / n}, y = #{(mask * y ).sum / n} if n 0
( img / 2 ) * ( mask + 1 )
end
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 29/38
30. David A. Forsyth, Jean Ponce - Computer Vision: A modern
Approach
Location: Adsetts Centre, Shelfmark: 006.37 FO (LEVEL 2)
http://catalogue.shu.ac.uk/search~S1/t?Computer%20vision:%20a%20modern%20approach
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 30/38
31. Hal Fulton - The Ruby Way
Location: Adsetts Centre, Shelfmark: 005.133 RUB FU (LEVEL 2)
http://catalogue.shu.ac.uk/search~S1/t?The%20Ruby%20way
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 31/38
32. Mark Pollefeys - Visual 3D modeling of real-world objects
and scenes from images
http://video.google.com/videoplay?docid=-1315387152400313941
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 32/38
33. Mark Pupilli - Particle Filtering for Real-time Camera
Localisation
http://www.cs.bris.ac.uk/home/pupilli/publications/thesis.pdf
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 33/38
34. Ben Bleything - Controlling Electronics with Ruby
http://rubyconf2007.confreaks.com/d1t2p1_ruby_and_electronics.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 34/38
35. Patrick Farley - Ruby Internals
http://mtnwestrubyconf2008.confreaks.com/11farley.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 35/38
36. Yukihiro Matsumoto - Does Language Matter?
http://rubyconf2007.confreaks.com/d2t1p8_keynote.html
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 36/38
37. Willow Garage, Inc.
http://www.willowgarage.com/
Wednesday, February 4th 2009 http://vision.eng.shu.ac.uk/jan/demfeb09.pdf 37/38
38. Thank You
This presentation was made with LT X,
A
E
TeXPower, InkScape, Ruby, and other
free software.
http://vision.eng.shu.ac.uk/jan/demfeb09.pdf