The document discusses the concept of the technological singularity, which refers to the hypothetical future emergence of artificial superintelligence that greatly surpasses human intellectual abilities. It provides an overview of the main ideas, including the three schools of belief about the singularity (accelerating change, event horizon, intelligence explosion). It also discusses the four main paths to the singularity, including creating an AI that exceeds human intelligence. The document summarizes the history of ideas around the singularity concept from the 1800s onward and perspectives from thinkers like Kurzweil and Hutter on whether the singularity is negotiable or inevitable.
2. Objectives
Understand one should not fear complexity.
Do not seek the one great truth. There are
many truths and they can be true concurrently.
Understand the three main schools of belief
about the Singularity.
Understand the four main paths to the
Singularity.
Understand the history of the Singularity and
Marcus Hutter’s main ideas about it.
Understand the challenge of promoting the
Singularity and the idea behind Future Day.
3. Do Not Fear
Complexity!
There is beauty in complexity. The real world
is complex. Perfect “quadruple-think with
equipoise”. Seemingly contradictory ideas can
all be true, can find a balance between them.
The future of transplantation is promotion of
deceased donor donation until there are no
waiting lists, tolerance, tissue engineering
repair, and stem cell creation of new organs.
Peter Diamandis: “When faced with a choice
between two desirable goals, choose both!”
4. The Technological Singularity
The technological singularity occurs as artificial
intelligences surpass human beings as the smartest
and most capable life forms on the Earth.
Technological development is taken over by the
machines, who can think, act and communicate so
quickly that normal humans cannot even comprehend
what is going on. The machines enter into a "runaway
reaction" of self-improvement cycles, with each new
generation of A.I.s appearing faster and faster. From
this point onwards, technological advancement is
explosive, under the control of the machines, and
thus cannot be accurately predicted (hence the term
"Singularity"). – Ray Kurzweil
6. In Lewis Carrol’s Alice in
Wonderland a world is
confronted that is much
more organic than
expected.
“The main difficulty Alice had was
in managing her flamingo”
(Describing the croquet game)
.”
7. In the Technological Singularity
we face a world that is much less
organic than expected and could
develop without us!
8. There are three main
schools of belief about the
Singularity.
Accelerating Change
Event Horizon
Intelligence Explosion
9. There are Four main
paths to the Singularity.
1. Create an artificial intelligence that exceeds human
intelligence.
2. Build human-computer interfaces that allow humans to
go beyond their innate intelligence to a significant
extent. (‘cybernetic singularity’)
10. There are Four main paths
to the Singularity.
3. Find ways in biology to improve upon the natural human
intellect.
4. Build large computer networks in which‘beyond human
intelligence’ emerges.
11. All these different variations on the
belief in the Singularity are reflected in
the courses at Singularity University
The experience of attending Singularity
University is one that grows and grows
after completion of the course. The
associated memories become more vivid
rather than less vivid with time, they are
on an exponential curve of their own!
13. Singularity
Course
I have been arguing for new cross
disciplinary structures in Universities to
better prepare us for the future.
It became apparent that the best way to make this
happen was for me to create a novel course of
new design. Thus, this course.
Presently, we know of no similar courses being
presented elsewhere, except perhaps Bertalan
Mesko’s Social Media in Medicine course in
Budapest, Hungary.
Eventually it is our hope that hundreds of similar
courses will begin appearing at Universities all
over the world.
14. In a Post-Scarcity World of Abundance
Medicine Will Be About Enhancement of
Well People, Not About Disease
15. Ray Kurzweil’s views and
intellectual exploration are
as broad as that of the
University he founded.
So when you hear someone arguing with Ray
Kurzweil as if he held narrow rigid views, that
is a false, “straw man” argument.
16. Singularity
Course
Ancient: In 1847, R. Thornton, the editor of The
Expounder of Primitive Christianity, wrote about
the recent invention of a four function mechanical
calculator:
“...such machines, by which the scholar may, by
turning a crank, grind out the solution of a problem
without the fatigue of mental application, would by
its introduction into schools, do incalculable injury.
But who knows that such machines when brought
to greater perfection, may not think of a plan to
remedy all their own defects and then grind out
ideas beyond the ken of mortal mind!”
History
17. Singularity
Course
Ancient: In 1863, four years after Darwin published On
the Origin of Species Samuel Butler published a letter
captioned "Darwin among the Machines”. It compares
human evolution to machine evolution, prophesizing
(half in jest) that machines would eventually replace man
in the supremacy of the earth:
In the course of ages we shall find ourselves the inferior
race.
The letter raises many of the themes now being debated
by proponents of the Technological Singularity.
History
18. Singularity
Course
In Erewhon (1872) Butler argued that:
“There is no security against the ultimate development of
mechanical consciousness, in the fact of machines
possessing little consciousness now. A mollusc has not
much consciousness. Reflect upon the extraordinary
advance which machines have made during the last few
hundred years, and note how slowly the animal and
vegetable kingdoms are advancing. The more highly
organized machines are creatures not so much of
yesterday, as of the last five minutes, so to speak, in
comparison with past time.”
History
20. Singularity
Course
In science fiction / mathematicians
Stanislaw Ulam (1958)
I.J. Good (1965)
Ray Solomonoff (1985)
Vernor Vinge (1993)
Wide-spread popularization
Kurzweil Books (1999,2005,2012)Internet.
Events (Singularity Summit 2006+)
Organizations (Singularity Institute 2000+
& University)
Philosophers (David Chalmers 2010)
(Marcus Hutter, 2012)
History
(Next 28 Slides Modified from Marcus Hutter
http://www.hutter1.net/publ/ssingularity.ppsx )
21. Moore’s Law
(adapted from Moravec 1988 & Kurzweil 2005)
?
CalculationsperSecondper$1000
Year
1900 ‘20 ‘40 ‘60 ‘80 ‘20 ‘40 ‘60 ‘802000 2100
10-10
10-5
1
105
1010
1015
1020
1025
1030
Electro-
mechanical
Relay
Tube
Tran-
sistor
Integrated
Circuits
Manual
calculation
Bacterium
Worm
Spider
Lizard
Mouse
Monkey
Human brain
Parallel
Processors
Quantum
Comp.?
All
Human
brains
22. Singularity
Course
Moore's law: comp doubles every 1.5yrs. Now valid for
50yrs
As long as there is demand for more comp,
Moore's law could continue to hold
for many more decades before computronium is
reached.
in 20-30 years the raw computing power of a single
computer will reach 1015...1016 flop/s.
Computational capacity of a human brain: 1015...1016
flop/s
Some Conjecture: software will not lag far behind
(AGI or reverse engineer or simulate human brain)
Super-Intelligence by Moore's Law
Human-level AI in 20-30 years?
23. Acceleration of Doubling
Patterns
SizeofEconomy
time in years
Computer-dominated
Doublingevery1.5years
Superhumanintelligence
Dbl.Monthly(Hanson2008)
-106 -1/10-10-101-102-104
Hunter-gather-stone-ageera.
Doublingevery250’000yrs
Agriculturaleconomy,farming.
Doublingevery900years
Industrialrevolution
Doublingevery15years
2.5 mio BC 10’000 BC 1800AD 2025? 2040??
-103-105-10-7
2042???
25. Singularity
Course
Appearance of AI+ = ignition of the detonation cord towards the
Singularity = point of no return
Maybe Singularity already now unavoidable?
Politically it is very difficult (but not impossible) to resist technology
or market forces
it would be similarly difficult to prevent AGI research and even more
so to prevent the development of faster computers.
Whether we are before, at, or beyond the point of no return is also
philosophically intricate as it depends on how much free will one
attributes to people and society.
Analogy 1: politics & inevitability of global warming
Analogy 2: a spaceship close to the event
horizon might in principle escape a black hole
but is doomed in practice due to limited propulsion.
Is the Singularity Negotiable?
(Hutter)
26. Singularity
Course
Some Information Analogies
Inside process resembles a radiating
black hole observed from the outside.
Maximally compressed information
is indistinguishable from random noise.
Too much information collapses:
A library that contains all possible books has zero information
content.
Library of Babel: all information = no information
Maybe a society of increasing intelligence will become
increasingly indistinguishable from noise when viewed from
the outside.
…… …
27. Singularity
Course
Each way, outsiders cannot witness a true
intelligence singularity.
Expansion (inward outward) usually follows
the way of least resistance.
Outward explosion will stop when all accessible
convertible matter has been used up.
Historically, mankind was always outward
exploring
◦ just in recent times it has become more inward exploring
(miniaturization & virtual reality).
Comparison
28. Singularity
Course
Assume recording technology does not break
down:
then a singularity seems more interesting for
outsiders than for insiders.
On the other hand, insiders actively “live” potential
societal changes,
while outsiders only passively observe them.
Conclusion: Strict intelligence
singularity neither experienced by
insiders nor by outsiders
29. Singularity
Course
There have been numerous attempts to define
intelligence.
Legg & Hutter (2007) provide a collection of 70+
definitions
◦ by individual researchers as well as collective attempts
If/since intelligence is not (just) speed, what is it
then?
What will super-intelligences actually do?
What is Intelligence?
30. Singularity
Course
Evolution: Mutation, recombination, and selection
increases intelligence if useful for survival and procreation.
Animals: higher intelligence, via some correlated practical cognitive
capacity, increases the chance of survival and number of offspring.
Humans: intelligence is now positively correlated with power and/or
economic success (Geary 2007) and actually negatively with
number of children (Kanazawa 2007).
Memetics: Genetic evolution has been largely replaced by memetic
evolution (Dawkins 1976), the replication, variation, selection, and
spreading of ideas causing cultural evolution.
Evolving Intelligence
31. Singularity
Course
Self-preservation?
Self-replication?
Spreading? Colonizing the universe?
Creating faster/better/higher intelligences?
Learning as much as possible?
Understanding the universe?
Maximizing power over men and/or organizations?
Transformation of matter (into computronium?)?
Maximum self-sufficiency?
The search for the meaning of life?
What Activities are Intelligent?
Which Activities does Evolution
Select for?
32. Singularity
Course
More flexible notion: expected utility maximization
and cumulative life-time reward maximization
But who provides the rewards, and how?
◦ Animals: one can explain a lot of behavior as attempts
to maximize rewards=pleasure and minimize pain.
◦ Humans: seem to exhibit astonishing flexibility in choosing
their goals and passions, especially during childhood.
◦ Robots: reward by teacher or hard-wired.
Goal-oriented behavior often appears to be
at odds with long-term pleasure maximization.
Still, the evolved biological goals and
desires to survive, procreate, parent,
spread, dominate, etc. are seldom disowned.
Intelligence ≈ Rationality ≈
Reasoning Towards a Goal
33. Singularity
Course
Who sets the goal for super-intelligences and
how?
Anyway ultimately we will lose control,
and the AGIs themselves will build further AGIs (if
they were motivated to do so),
and this will gain its own dynamic.
Some aspects of this might be independent of the
initial goal structure and predictable.
Evolving Goals: Initialization
34. Singularity
Course
Assume the initial vorld is a society of cooperating
and competing agents.
There will be competition over limited
(computational) resources.
Those virtuals who have the goal to acquire them
will naturally be more successful in this endeavor
compared to those with different goals.
The successful virtuals will spread (in various
ways), the others perish.
Evolving Goals: Process
35. Singularity
Course
Soon their society will consist mainly of virtuals
whose goal is to compete over resources.
Hostility will only be limited if this is in the virtuals'
best interest.
For instance, current society has replaced war
mostly by economic competition,
since modern weaponry makes most wars a loss
for both sides, while economic competition in most
cases benefits at least the better.
Evolving Goals: End Result
36. Singularity
Course
Whatever amount of resources are available,
they will (quickly) be used up, and become scarce.
So in any world inhabited by multiple individuals,
evolutionary and/or economic-like forces will “breed”
virtuals with the goal to acquire as much (comp)
resources as possible.
Virtuals will “like” to fight over resources, and
the winners will “enjoy” it, while the losers will “hate” it.
In such evolutionary vorlds, the ability to survive and
replicate is a key trait of intelligence.
But this is not a sufficient characterization of intelligence:
E.g. bacteria are quite successful in this endeavor too,
but not very intelligent.
The Goal to Survive & Spread
37. Singularity
Course
Global collaboration, no hostile competition
likely requires
a powerful single (virtual) world government,
and to give up individual privacy,
and to severely limit individual freedom
(cf. ant hills or bee hives),
or requires
societal setup that can only produce conforming
individuals
might only be possible by severely limiting
individual's creativity (cf. flock of sheep or school
of fish).
Alternative Societies
38. Singularity
Course
Such well-regulated societies might better be
viewed as a single organism or collective mind.
Or maybe the vorld is inhabited from the outset by
a single individual.
Both vorlds could look quite different and more
peaceful (or dystopian) than the traditional ones
created by evolution.
Intelligence would have to be defined quite
differently in such vorlds.
Monistic Vorlds
39. Singularity
Course
Another important aspect of intelligence:
how flexible or adaptive an individual is.
Deep blue might be the best chess player on
Earth, but is unable to do anything else.
On the contrary, higher animals and humans have
remarkably broad capacities and can perform well
in a wide range of environments.
Adaptiveness of Intelligence
40. Singularity
Course
Informal definition:
Implicitly captures most, if not all traits of rational intelligence:
such as reasoning, creativity, generalization, pattern
recognition, problem solving, memorization, planning,
learning, self-preservation, and many others.
Has been rigorously formalized in mathematical terms.
Properties: Is non-anthropocentric, wide-ranging, general,
unbiased, fundamental, objective, complete, and universal.
Is the most comprehensive formal definition of intelligence so
far.
Formal Intelligence Measure
Intelligence is the ability to achieve goals in a wide range of
environments [LH07]
41. Singularity
Course
copying virtual structures should be
as cheap and effortless as it is for
software and data today.
The only cost is developing the
structures in the first place, and the
memory to store and the comp to run them.
Copying & Modifying Virtual
Structures
{hard}
{easy}
Cheap manipulation and experimentation and
copying of virtual life itself possible.
42. Singularity
Course
Virtual explosion with life becoming much more
diverse.
In addition, virtual lives could be simulated in
different speeds, with speeders experiencing
slower societal progress than laggards.
Designed intelligences will fill economic niches.
Our current society already relies on specialists
with many years of training.
So it is natural to go the next step to ease this
process by designing our descendants (cf.
designer babies).
Copying & Modifying Virtual Life
43. Singularity
Course
Another consequence should be that life becomes
less valuable.
Our society values life, since life is a valuable
commodity and expensive/laborious to
replace/produce/raise.
We value our own life, since evolution
selects only organisms that value their life.
Our human moral code mainly mimics this
(with cultural differences and some excesses)
If life becomes `cheap', motivation to value it will
decline.
The Value of Life
44. Singularity
Course
Cheap machines decreased value of physical labor.
Some Expert knowledge was replaced by hand-written
documents, then printed books, and finally electronic
files.
Each transition reduced the value of the same
information.
Digital computers made human computers obsolete.
In Games, we value our own virtual life
and that of our opponents less than real life,
because games can be reset and one can be
resurrected.
Abundance lowers Value
- Analogies -
45. Singularity
Course
Governments will stop paying my salary when
they can get the same research output from a
digital version of me, essentially for free.
And why not participate in a dangerous fun
activity if in the worst case I have to activate a
backup copy of myself from yesterday which just
missed out this one (anyway not too well-going)
day.
The belief in immortality can alter behavior
drastically.
Consequences of Cheap Life
46. Singularity
Course
Countless implications: ethical, political, economical, medical,
cultural, humanitarian, religious, in art, warfare, etc.
Much of our society is driven by the fact that we highly value
(human/individual) life.
If virtual life is/becomes cheap, these drives will ultimately vanish
and be replaced by other goals.
If AIs can be easily created, the value of an intelligent individual will
be much lower than the value of a human life today.
So it may be ethically acceptable to freeze, duplicate, slow-down,
modify (brain experiments), or even kill (oneself or other) AIs at will,
if they are abundant and/or backups are available, just what we are
used to doing with software.
So laws preventing experimentation with intelligences for moral
reasons may not emerge.
The Value of Virtual Life
With so little value assigned to an individual life, maybe it becomes
a disposable.
47. Singularity
Course
Are there any universal values or qualities
we want to see or that should survive?
What do we mean by we? All humans? Or the
dominant species or government at the time the
question is asked?
Could it be diversity?
Or friendly AI (Yudkowsky 200X)?
Could the long-term survival of at least one
conscious species that appreciates its surrounding
universe be a universal value?
Are there Universal Values
48. Trying to Raise Spirits and
Stimulate Interest in the
Singularity Through A New
Holiday - Future Day
50. First Future Day
March 1, 2012. Sixteen
celebrations Around The
World
Melbourne, Sydney, Hong Kong, Berkeley, Edmonton,
Houston, Hawaii, Sao Paulo, Thanksgiving Point, Utah,
Brussels, Paris, LA, Palo Alto, Washington, Carlton,
Australia, Wroclaw, Poland
51. Singularity
Course
Run like Autodesk Design Night.
Best. Salon. Ever. March 1, 2016.
Hosted by media professional
Dr. Julielynn Wong of Singularity U.
Analogous to Paris Salon of a
century ago which moved Western
thought and culture forward, music,
art, good conversation, something unique,
innovative, and memorable!
Julielynn Wong - Edmonton Salon Event
at Art Gallery of Alberta
Future Day March 1, 2016?
52. Singularity
Course
Mature youthful decision making!
You can do it!
You can help us figure out what our plans
should be for Future Day in 2015!
53. The Big Bang Theory is Watched
Regularly by 20 Million People in
the US. Singularity Episode on
October 1, 2010
63. Singularity
Course
Round
Like a circle in a spiral
Like a wheel within a wheel
Never ending or beginning
On an ever spinning reel
Like a snowball down a mountain
Or a carnival balloon
Like a carousel that’s turning
Running rings around the moon
The Windmills of Your Mind
64. Singularity
Course
Like a clock whose hands are sweeping
Past the minutes of it's face
And the world is like an apple
Whirling silently in space
Like the circles that you find
In the windmills of your mind !
The Windmills of Your Mind
(Continued)
67. Singularity
Course
How can we capture the imagination of the
public to start everyone thinking about these
matters?
We need the mainstream public to regard the
future technological Singularity as fact, not
fiction
We need to promote organized thinking about
the future in Universities and beyond
Please talk up our “Light the Bridge” application!
You never know what will make the difference!
Your Suggestions Greatly
Welcomed!
68. Singularity
Course
Marcus Hutter, Can Intelligence Explode?
http://www.hutter1.net/publ/sasingularity.pptx
Journal of Consciousness Studies, Vol 19, Issue
1-2 (2012) pages 143-166.
http://www.ingentaconnect.com/content/imp/jcs/20
12/00000019/F0020001/art00010
D. J. Chalmers. The Singularity: A philosophical
analysis. Journal of Consciousness Studies, 17:7–
65, 2010.
References