1. Small Eigen Collider
Creating random ruby programs
Andrew Grimm

2. Rationale
● Unit tests of existing projects failed on
Rubinius because of bugs in things like
String#split or StringIO#gets
● If I could generate random programs, I
could see whether less popular
implementations did the same thing as
more popular implementations

3. Producing random code
result = @receiver.send(@method,
@parameter_objects, &:inspect)
Where @receiver and
@parameter_objects are randomly chosen
objects, and @method is a randomly
chosen method name.

4. Logging the results
● Currently printing the receiver, method,
parameter objects and the result
Receiver object: “a”
Method: “casecmp”
Parameters: [“b”]
Result: -1

5. Comparing the results
● Run the script under different ruby versions
● Do a diff on the outputs

6. Example of a diff
● Lines above were running “a”.casecmp(“b”)
diff MRI_output.txt rubinius_output.txt
180c180
< Result: -1
---
> Result: 1

7. Contents of the two files
MRI_output.txt
Receiver object: "a"
Method: "casecmp"
Parameters: ["b"]
Result: -1
rubinius_output.txt
Receiver object: “a”
Method: “casecmp”
Parameters: [“b”]
Result: 1

8. Manual confirmation
rvm ruby -e 'puts [RUBY_ENGINE,
“a”.casecmp”b”].join(“t”)
jruby -1
macruby -1
ruby -1
rbx 1 <= spot the odd one out

9. Protip
● Test the !@#$ing happy path!
describe "String#casecmp" do
it "is a case-insensitive version of String#<=>" do
"abcdef".casecmp("abcde").should == 1
"aBcDeF".casecmp("abcdef").should == 0
"abcdef".casecmp("abcdefg").should == -1
"abcdef".casecmp("ABCDEF").should == 0
end
# Snipped I18N tests
end

10. Current tasks
● Make as safe as ruby allows (grain of salt
here)
● Rather than randomly generating tasks
each time, allow the tasks to be generated
once, and run many times

11. Making the program safe
Receiver object: nil
Method: “system”
Parameters: [“rm -rf /”]
Result: :(
● Applying $SAFE to try to prevent bad
things in a ruby-specific way
● Researching other sandboxing methods

12. Generate once, run many times
● Currently, I give all the rubies the same
random number generator seed and same
set of objects, and leave it up to them to
choose which objects are in each task
● JRuby gives different random values than
other platforms
● If the choice of tasks were made only once,
then I could compare JRuby with others

13. Questions?