Jim Spohrer provides considerations for AI projects. He recommends performing an audit of existing AI projects and evolving evaluation criteria to include performance and trust. Spohrer also emphasizes the importance of celebrating victories, rewarding talent development through diversity and upskilling, and monitoring technology developments. He warns against underestimating ongoing costs and overestimating short-term impacts. Spohrer outlines timelines for AI progress based on compute costs and provides frameworks for benchmarking and evaluating AI capabilities.
1. AI: Do and Don’t Considerations
Jim Spohrer
Board of Directors, ISSIP.org
Questions: spohrer@gmail.com
Twitter: @JimSpohrer
LinkedIn: https://www.linkedin.com/in/spohrer/
Slack: https://slack.lfai.foundation
Presentations on line at: https://slideshare.net/spohrer
Thanks to Prof. Henry Chesbrough for the invitation to present
on September 7th, 2021
Garwood Center for Corporate Innovation
Haas School of Business at the University of California, Berkeley
2. AI Considerations: Do
• Do dashboard the opportunity investment
• Perform an audit of existing projects (machine learning, robotics) and opportunities
• Evolve evaluation criteria (e.g., performance, Trusted AI) and investment approach
• Celebrate victories (internally/externally)
• Do reward talent development
• Diversity: T-Shaped Upskilling (e.g., business, technical, systems, customer empathy)
• Open-Source Software: LF AI & Data, Github Rankings
• Open Data: Kaggle Rankings
• Do monitor technology developments
• AI is very hard and will take decades to solve
• LF AI & Data Landscape
• PapersWithCode.Com
9/3/2021 (c) IBM MAP COG .| 2
3. AI Considerations: Don’t
• Don’t under-estimate the on-going cost of data cleansing and architecture
• The “crude oil” refinery process metaphor is apt (many stages)
• Mitigating bias and ensuring security
• Synthetic data generation can be explored as well (e.g., autonomous vehicles)
• Don’t under-estimate the on-going cost of the full system
• The full system includes technology, people, governance, and other costs
• The full system includes AI ethics boards and monthly technical steering committee
• This is where being a good ecosystem player/partner can help stabilize costs
• Don’t over-estimate short-term impact/under-estimate long-term impact
• Rigorous business and technical evaluation criteria needed
• Evolve monthly (cross-functional steering committees to review cases)
• Open-source office coordinates business, development, research, ecosystem players
9/3/2021 (c) IBM MAP COG .| 3
4. Future of AI
• What is the timeline for solving AI and IA?
• TBD: When can a CEO/anyone buy AI capability <X> for price <Y>?
• Who are the leaders driving AI progress?
• What will the biggest benefits from AI be?
• What are the biggest risks associated with AI, and are they real?
• What other technologies may have a bigger impact than AI?
• What are the implications for stakeholders?
• How should we prepare to get the benefits and avoid the risks?
9/3/2021 (c) IBM 2020, Cognitive Opentech Group 4
5. Timeline: Every 20 years,
compute costs are down by 1000x
• Cost of Digital Workers
• Moore’s Law can be thought of as
lowering costs by a factor of a…
• Thousand times lower
in 20 years
• Million times lower
in 40 years
• Billion times lower
in 60 years
• Smarter Tools (Terascale)
• Terascale (2017) = $3K
• Terascale (2020) = ~$1K
• Narrow Worker (Petascale)
• Recognition (Fast)
• Petascale (2040) = ~$1K
• Broad Worker (Exascale)
• Reasoning (Slow)
• Exascale (2060) = ~$1K
5
9/3/2021 (c) IBM 2017, Cognitive Opentech Group
2080
2040
2000
1960
$1K
$1M
$1B
$1T
2060
2020
1980
+/- 10 years
$1
Person Average
Annual Salary
(Living Income)
Super Computer
Cost
Mainframe Cost
Smartphone Cost
T
P
E
T P E
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA
6. Timeline: GDP/Employee
9/3/2021 (c) IBM 2017, Cognitive Opentech Group 6
(Source)
Lower compute costs translate into increasing productivity and GDP/employees for nations
Increasing productivity and GDP/employees should translate into wealthier citizens
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA
7. Timeline: Leaderboards Framework
AI Progress on Open Leaderboards - Benchmark Roadmap
Perceive World Develop Cognition Build Relationships Fill Roles
Pattern
recognition
Video
understanding
Memory Reasoning Social
interactions
Fluent
conversation
Assistant &
Collaborator
Coach &
Mediator
Speech Actions Declarative Deduction Scripts Speech Acts Tasks Institutions
Chime Thumos SQuAD SAT ROC Story ConvAI
Images Context Episodic Induction Plans Intentions Summarization Values
ImageNet VQA DSTC RALI General-AI
Translation Narration Dynamic Abductive Goals Cultures Debate Negotiation
WMT DeepVideo Alexa Prize ICCMA AT
Learning from Labeled Training Data and Searching (Optimization)
Learning by Watching and Reading (Education)
Learning by Doing and being Responsible (Exploration)
2018 2021 2024 2027 2030 2033 2036 2039
9/3/2021 (c) IBM 2017, Cognitive Opentech Group 7
Which experts would be really surprised if it takes less time… and which experts really surprised if it takes longer?
Approx.
Year
Human
Level ->
+3
See: https://paperswithcode.com/sota
8. Who is winning
9/3/2021 (c) IBM 2017, Cognitive Opentech Group 8
https://www.technologyreview.com/s/608112/who-is-winning-the-ai-race/
9. Robots by Country
• Industrial robots per 10,000 people by country
9/3/2021 IBM #OpenTechAI 9
34
10. AI Benefits
• Access to expertise
• “Insanely great” labor productivity for trusted service providers
• Digital workers for healthcare, education, finance, etc.
• Better choices
• ”Insanely great” collaborations with others on what matters most
• AI for IA = Augmented Intelligence and higher value co-creation interactions
9/3/2021 (c) IBM 2017, Cognitive Opentech Group 10
11. AI Risks
• Job Loss
• Shorter term bigger risk
= de-skilling
• Super-intelligence
• Shorter term bigger risk
= bad actors
9/3/2021 (c) IBM 2017, Cognitive Opentech Group 11
12. Other Technologies: Bigger impact? Yes.
• Augmented Reality (AR)/
Virtual Reality (VR)
• Game worlds
grow-up
• Blockchain/
Security Systems
• Trust and security
immutable
• Advanced Materials/
Energy Systems
• Manufacturing as cheap,
local recycling service
(utility fog, artificial leaf, etc.)
9/3/2021 (c) IBM 2017, Cognitive Opentech Group 12
13. “The best way to predict the future is to inspire the
next generation of students to build it better”
Digital Natives Transportation Water Manufacturing
Energy Construction ICT Retail
Finance Healthcare Education Government
15. 9/3/2021 (c) IBM MAP COG .| 15
T-shaped Adaptive Innovator: Deep Problem-Solving and Broad Communication/Collaboration
Advanced Tech: AI to IoT to Quantum, GreenTech, RegTech, etc.
Work Practices: Agile, Service Design, Open Source
Mindset: Growth Mindset, Positive Mindset, Entrepreneurial
Many disciplines
Many sectors
Many regions/cultures
(understanding & communications)
Deep
in
one
sector
Deep
in
one
region/culture
Deep
in
one
discipline
21. 9/3/2021 (c) IBM MAP COG .| 21
Microsoft acquiring GitHub $7.5B
2018 John Marks on Open Source
Models will run the world
Why SW is eating the world
22. Step Comment
GitHub Get an account and read the guide
MAX CODAIT’s Model Asset Exchange
Learn 3 R's - Read, Redo, Report Read (Medium/arXiv), Redo (GitHub), Report (Jupyter Notebook)
PapersWithCode Stay on top of recent advances; Do 3 R’s.
Kaggle Compete in a Kaggle competition
Leaderboards Compete to advance AI progress
Linux Foundation AI Help end-to-end open source industry AI & Data infrastructure
Mozilla Common Voice Donate your speech; Label and verify data; Recruit others.
Figure Eight Generate a set of labeled data (also Mechanical Turk)
Design New Challenges Build for Call for Code/Code and Response; Build your AI Helper;
Build test-taker, that can switch to tutor-mode; Etc.
Open Source Guide Establish open source culture in your organization
9/3/2021 IBM Code #OpenTechAI 22
23. Is it fair?
Is it easy to
understand?
Is it accountable?
So what does it take to trust a decision made by
a machine?
(Other than that it is 99% accurate)?
Did anyone
tamper with it?
#21, #32, #93
#21, #32, #93
25. IfM, IBM (2010)
Succeeding through
service innovation:
a service perspective
for education, research,
business and government.
University of Cambridge
Institute for Manufacturing,
Cambridge UK
2010
26. What is a SIR?
• Service Innovation Roadmap (SIR) is a kind of Business Model Canvas
(BMC) that responsible entities create for themselves to describe
three types of investments in learning/upskilling activities:
• Run: BMC for optimize activities (e.g., agile improvement method)
• Transform: BMC for copy activities (e.g., find role models)
• Innovate: BMC for invent activities (e.g., research, pilot, prove, monetize)
• Based on March (1991)
• March JG (1991) Exploration and exploitation in organizational learning.
Organization science. 1991 Feb;2(1):71-87.
29. 29
How responsible entities (service systems) learn and change over time
History and future of Run-Transform-Innovate investment choices
• Diverse Types
• Persons (Individuals)
• Families
• Regional Entities
• Universities
• Hospitals
• Cities
• States/Provinces
• Nations
• Other Enterprises
• Businesses
• Non-profits
• Learning & Change
• Run = use existing knowledge
or standard practices (use)
• Transform = adopt a new best
practice (copy)
• Innovate = create a new best
practice (invent) Innovate
Invest in each
type of change
Spohrer J, Golinelli GM, Piciocchi P, Bassano C (2010) An integrated SS-VSA analysis of changing job roles. Service Science. 2010 Jun;2(1-2):1-20.
March JG (1991) Exploration and exploitation in organizational learning. Organization science. 1991 Feb;2(1):71-87. URL:
exploit
explore
30. 9/3/2021 (c) IBM MAP COG .| 30
Arthur, W.B. Foundations of complexity economics. Nat Rev Phys (2021). https://doi.org/10.1038/s42254-020-00273-3
31. In conclusion…
Situation
Competence
3 R’s
On Ramps
1. Platform & ecosystem competition for data and AI workloads
2. However, AI is hard; many capabilities 2-4 decades away
3. Industry in open source collaboration-competition mode
1. Read: Learn state-of-art
2. Redo: Apply and infuse in use cases/workloads
3. Report: Share back, others may improve
1. LF AI Landscape: Community projects
2. IBM CODAIT: Cloud Pak for Data (CPD), etc. – Enterprise workloads with Trusted AI
3. Red Hat ODH: OpenShift – Hybrid cloud platform and ecosystem
32. Jim Spohrer, Board of Directors, ISSIP.org
Jim Spohrer serves on the Board of Directors of the International
Society of Service Innovation Professionals, and as a contributor to the
Linux Foundation AI and Data Foundation. He is a retired IBM Executive
since July 2021, and previously directed IBM’s open-source Artificial
Intelligence developer ecosystem effort. After his MIT BS in Physics, he
developed speech recognition systems at Verbex (Exxon) before
receiving his Yale PhD in Computer Science/AI. In the 1990’s, he
attained Apple Computers’ Distinguished Engineer Scientist and
Technologist role for next generation learning platforms. He was CTO
IBM Venture Capital Group, co-founded IBM Almaden Service Research,
and led IBM Global University Programs. With over ninety publications
and nine patents, he received the Christopher Loverlock Career
Contributions to the Service Discipline award, Gummesson Service
Research award, Vargo and Lusch Service-Dominant Logic award, Daniel
Berg Service Systems award, and a PICMET Fellow for advancing service
science. Jim was elected and previously served as LF AI & Data
Technical Advisory Board Chairperson and ONNX Steering Committee
Member (2020-2021).
9/3/2021 (c) IBM 2020, Cognitive Opentech Group 32
In 2008, Jim co-founded and directed
IBM Almaden Service Research
helping to establish service science,
applying science, technology,
and T-shaped upskilling of people to
business and societal transformation.
33. The Three Stages of Systems Evolution
The Three Ages of Man (Giorgione)
Thanks to Alan Hartman for kind inspiration (slides) (recording) Service, the application of knowledge for mutual benefits
win-win/non-zero-sum games/value co-creation/capability co-elevation