1. OpenRacing
Is this a video game, or an AI research platform?
Cars driving themselves
(Note: this code already
exists)

2. About me
 Keith Curtis, keithcu@gmail.com
 Programmer at Microsoft for 11 years on
Windows, Office, MSN, Research and
mobility
 Randomly discovered Linux after I left
 Wrote a book describing why it is
superior, and remaining steps for World
Domination

3. My book gives fuller context
NEW YORK TIMES:
Keith Curtis, an 11-year veteran
of Microsoft, believes deeply that
open source is the future of
software.
He takes a programmer’s
approach in Software Wars,
attempting to systematically build
a case that software can help
pave the way for a 21st-century
renaissance in many fields
ranging from artificial
intelligence (cars that drive
themselves) to the human journey
into space (space elevators). For
Mr. Curtis, free software is all
about leveraging our collective
intelligence.

4. Book in 1 slide
 After thinking about it for 10 years
 And seeing the code inside and outside
Microsoft
 Free software wins in the end
 The lessons of Wikipedia, Linux et al can
be applied to many places
 FOSS movement should change how you
think about R&D in the 21st century
 The payoff is AI, cancer research

5. Free software
 If the code isn't out there, a community
of scientists can't work together on it
 As subtle a concept as free market
 Science is about making results available
 Better for the free market
 Cheaper hardware, richer services business
 Many problems too big for one company
 Coming goldrush of free software
 Google's Knol will lose to Wikipedia
 Book explains this

6. Just add water
 Create the conditions
 Create a vision that inspires
 Create a process that doesn't suck
 Codebase -> community -> success
 It isn't that hard
 Lots of things succeed by accident in the
Internet era
 Free software quietly taking an
increasing part
 Cheaper: share dev costs with others

7. State of free software
 Successful in many areas
 Esp. Servers, web and embedded
 Apple uses free software (Kernel, Safari,
Printing)
 Linux on the desktop is coming (Google
Chrome, Ubuntu on Dell)
 The desktop is the center of the IT universe
 Lucene is a free search engine used in many
enterprises
 Much left remaining, like AI

8. Driverless Cars
 Cars are the most widely used robot
today
 Billions of $$ from industry and government
 Optionally having a chaffeur is the
coolest feature ever
 Once we have cars that can see, we can
have personal butlers as well
 Has several Green applications
 Route around traffic to save gas

9. Why don't we have
driverless cars today?
 Airplanes land themselves
 Cars drive around in video games and
shoot and swear at you
 Video game cars aren't super smart yet
because crashing is part of the fun
 Code is custom and proprietary anyway
 The software doesn't exist
 Because there is no community
This is a social, not a technical problem

10. Darpa Grand Challenge
 Entrants were all proprietary
 A few produced papers, but no source
 The code is just being tinkered with or
abandoned
 Anyone who wants to work on this problem
today must mostly start from scratch
 Therefore, the contest was a failure

11. Importance of a simulator
 A driving simulator allows you to
simulate full loop
 Create sensors in the video game, and
feed them to vision engine
 Can compare results to what video game
says
 Create test scenarios and monitor it all
 You can cameras up to RC cars, but we
need to focus on the software now!

12. Importance of a simulator
 In an hour you can run 1,000s of
incredibly complicated tests
 A simulator is required to build
confidence in a system
 The idea of cars driving themselves down the
road at 150mph can involve death
 Researchers can test and improve this
code without access to a car
 You need the real world mostly to learn
how to make the simulator tests better

13. Rigs of Rods is best free sim

14. ●Internally, simulator will generate virtual
sensor input like Lidar

15. Rigs of Rods
 Created by just 3 people, but leverages a
lot of free software
 Community of 10 developers
 But young, codebase made free in early-2009
 Extensible with new cars and maps
 Can simulate dynamic objects like stoplights
 Soft-body physics engine
 Engine simulation needs work
 C++, with some Lua scripts
 We are trying to get Python

16. Driving is the Killer App of
vision
 Visicalc and Lotus 1-2-3 are the apps that built
the PC
 E-mail is what built the Internet
 Lots of specialized uses of computer vision
today such as in NFL TV's first down marker &
Project Natal.
 Researchers have been working for decades
but don't know why their code is still in
academia
 No other reason to dramatically improve
 No point building robot butlers w/o vision

17. Community needs kickstart
 Putting vision and driving researchers
together in one codebase can generate
lots of excitement
 Vision and driving logic can be built and
tested separately
 PhDs around the world can get to work on a
real problem
 Each new advancement will put new
requirements on the other subsystem
 When the vision system can recognize soft
curbs, the driving system must also treat
them as such

18. Vision code
 Vision sounds hard, but is the same
number of lines of code as a web browser
 Many vision PhDs write proprietary code
 Adopted Microsoft model
 Head of Oxford Vision lab told me all his
code is proprietary
 What free code out there is very
fragmented (http://www.cs.cmu.edu/~cil/v-source.html)
 204 codebases
 Everyone rolls their own

19. Vision code
 1000s of PhDs worldwide are working
 Image processing portion well understood
 Detecting shapes and motion becoming
mature
 Need to build an internal 3-d model, no
consensus on these details
 We have more than enough people, but
they are not working together yet

20. OpenCV
 Most popular free computer vision
codebase is OpenCV, created by Intel
 Intel created and abandoned, now sponsored
by Willow Garage
 But only a few full-time people working
on it
 E-mail traffic is low
 Codebase is C++ and big
 Intel cares more about perf than ease of use
 Hard to use bits and pieces of
 Vast majority of researchers don't use

21. OpenCV (2)
 Can't easily work with larger scientific
community
 Has lots of code that is not really vision
specific including machine learning, low-
level graphics code
 Should really move to Python and join
SciPy community
 Estimated < 1 man-year of work
 Much vision research uses random
images found on Internet or webcam

22. Driving
 In principle, not a hard problem
 Parking a car is easy
 Pathfinding is discussed in every AI
textbook, but not much progress in last
decades
 Simple proprietary code in industrial
scenarios
 Free code is fragmented and immature
 Without real scenarios, no reason to improve

23. Driving (2)
 Safety is the big challenge
 I'll bet a parallel parking car would run
over a foot today which is not acceptable
 The dynamic world is tricky
 Will require a number of different kinds
of AI
 But less than 1M lines of Python
 Google's OCR 100k lines of C++
 The code isn't big, but it has to be clever
in spots

24. Driving Logic
 Primitive automatic driving code in RoR
today, but several people working on it
 There is other free code out there to
leverage as well
 Torcs/SpeedDreams
 OpenSteer

25. Strategy 1: VDGC
 The Darpa Grand Challenge failed
 Efforts died out after contest over
 Code is locked up so may as well not exist
 Contestants didn't work together on any
code
 Lots of time spent writing device drivers for
all of the custom hardware before even
getting to the vision / driving code
 Create a smarter contest

26. One way to spend $100,000
 Goal of a contest should be to get the
maximum number of people working as
efficiently as possible.
 A contest around a simulator will create
a community and assets long after the
contest is over.
 We'd like to create a Virtual Darpa
Grand Challenge
 Raise money from investors or sponsors

27. Contest details
 Free software should be encouraged
 If you want the assets to live on.
 We sidestep driver junk
 But you can easily simulate hardware
 We have other ideas to have the
contestants work together for efficiency

28. Contests are complicated
 Many details to consider
 What are the test cases? They have to be
created
 What are the legal rules?
 How to hand out money in little chunks
based on quality of contributions (code, art,
test cases, infrastructure work)
 Timeline

29. What do we want?
 $300,000?
 $100,000 for a contest plus infrastructure
 Assume $100K is enough to generate
excitement
 If we get 100 people working together we
can claim a breakthrough
 The time is right
 Team today is just me + one part-time
programmer, plus FOSS people out there
 I want someone else to lead this

30. Future
 It could get on the cover of Wired
magazine!
 Can turn this into an engineering
services company
 There are enough people today working
on these problems
 They are not working together
 They need a codebase and a task to organize
them