Collected here are a few insights and lessons learned that I gathered during my recent time working with CloudMade in the realm of Machine Learning in automotive applications.

1. Machine Learning
for the Automotive World
February 25th, 2020 Jeff Rehm

2. Topics
Behavioral Learning
The Vision
Challenges and Pitfalls
Implications for UX
Design
Predictions are hard –
particularly when they’re
about the future.
--Yogi Bera (?)

3. First, a Misconception
Having a vision is important-- there should always
be a goal to strive for.
But… we should take a sober, rational approach.
“If it’s built in Python, it’s Machine Learning.
If it’s built in PowerPoint, it’s Artificial Intelligence”
-- CloudMade Data Scientist
The idea of “Artificial Intelligence” in an automobile.
It is still a long, long way in the future.

5. Making predictions based on
the behavioral patterns and
preferences of an individual
Making predictions based on
large-scale aggregation of
behavior (speed, feature use,
etc.)
Making predictions based on
similarities to others
Cohort Learning
Personal LearningFleet Learning
Behavioral Learning

6. The Vision,
the Promise
JLR Video
https://www.youtube.com/watch?v=F923EuB06CI

7. Applications Today
Led by Luxury, but moving mainstream
JLR Smart Settings
Audi MIB2+
BMW Connected+
Hyundai Smart Cruise
Ford Mach-E
Mercedes MBUX

8. Typical Use Cases
Focused on the Journey
• Destination
• Route
• Departure time
• Arrival time
What they do on the way
• Media preferences
• Climate preferences
Built around the driver’s profile

9. Why has it taken so
long?
Connectivity
In-vehicle hardware
Backend systems at carmakers
Readiness of users
But mostly: Because it’s hard.

10. Configuration
Data Collection
Feature Extraction
ML
Data
Verification
Machine
Resource
Management
Analysis Tools
Process Management
Tools
Serving
Infrastructure
Monitoring
“Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small
black box in the middle. The required surrounding infrastructure is vast and complex. “
Hidden Technical Debt in Machine Learning Systems
Google, Inc.
Why it’s so Hard

11. Challenges and Pitfalls
Going Rogue
Dangers: Feedback Loops, Non-
homogenous data, non-declared
consumers, etc.
à Technical debt. Potentially massive
amounts.
Privacy
• Trust – avoiding creepiness
• Data protection regulations
(GDPR, CCPA)
• Consent management
Disillusionment
• Expectations management
• Some people have predictable
behaviors. Others do not.
• UX Design is critical here
Coherency
ML must be managed across the entire
ecosystem (car, cloud, companion app)
• Consistency of predictions
• Holistic approach.

12. Big Challenge: The Business Case
What’s the ROI?
If there is a significant investment to
properly deploy ML, what’s the payback?
What is a prediction worth?
Valuing ML Solutions is not easy.
What did we learn?
• There are very few standalone “killer use-
cases”.
• High level “elegance” features are difficult
to value or monetize
• Instead it’s the accumulation of many
features that add up to provide synergy
Solve real problems
• Look further down Maslow’s Automaker’s
hierarchy of needs
à Focus on Safety, Emissions, Efficiency, Cost
of ownership, Retention, Costs of
development, manufacture, etc.

13. Deployment
Considerations
Architecture
• Where should the learning be done?
• Storage, memory, CPU resources, transmission,
cloud costs
• Where should the predictions be done?
• Latency? Context?
Managing data
• Validation
• Extraction
• Normalization across entire fleet
• Privacy

14. UX Design Paradigm Shift
Answer the ‘W’ questions:
• Where are they going? Who will go?
When will they leave? How will they
get there? What will they do on the
way?
Confidence? Predictions are
probabilities, not absolutes.
Integration: Non-deterministic
predictions must be integrated
into your deterministic UX logic.
This is where your skill enters in.
CONTEXT CONTEXT + INTENT

15. Example: 2 Variable Integration
Do Nothing
Confidence Level – Departure Time
ConfidenceLevel-Destination
100%0%
100%
Proactive push notification
Pre-populated suggestions
Proactive destination input

16. Future Applications
Adaptive UX
• Familiarity with area
• Cognitive workload
Feature on-boarding
• Cohort, Fleet, Personal
Preconditioning
• Climate
• EV batteries
Maintenance functions
• Performed at ideal time based on
predicted route, speed, etc.
Powertrain Efficiency
• Lean burn, ICE/BEV, Transmission
optimization
And on…

17. Take-Aways
Machine Learning is real.
Adds real value to the automaker and their customers.
Building a proper foundation is a must.
Maturity level is about to make a big jump.
It is truly a new frontier.

18. “To a man with a
hammer, everything
looks like a nail”
-- Mark Twain (?)
“AI is not magic pixie dust.
Predictions can provide value,
but there is no substitute for
good well-thought design.”
--Jeff

19. References
Yogi Berra quote
https://quoteinvestigator.com/2013/10/20/no-predict/
Land Rover's Self-Learning Intelligent Vehicle Video
https://www.youtube.com/watch?v=F923EuB06CI
Mercedes MBUX
https://www.mercedes-
benz.com/en/innovation/connected/mbux-mercedes-
benz-user-experience-revolution-in-the-cockpit/
Audi MIB2+ Infotainment
https://www.audi-mediacenter.com/en/the-new-audi-rs-
q8-the-most-sporty-q-12422/infotainment-and-audi-
connect-12432
Hyundai Smart Cruise (SCC-ML)
https://www.hyundainews.com/en-us/releases/2887
Ford Mustang Mach-E Launch
https://youtu.be/o0F9Uktpgtk?t=1309
Hidden Technical Debt in Machine Learning Systems
D. Sculley, Gary Holt, Daniel Golovin, et al , Google Inc.
https://papers.nips.cc/paper/5656-hidden-technical-debt-
in-machine-learning-systems.pdf
Mark Twain Quote
https://quoteinvestigator.com/2014/05/08/hammer-nail/