The Soul in The Machine - Developing for Humans
Chris Heilmann @codepo8, Confoo, Montréal - February 2017

CHRIS HEILMANN
@CODEPO8
HTTPS://CHRISTIANHEILMANN.COM

Today, I want us to think a bit about
what we are doing as developers.
🤔

Because, I think we’re wasting a lot of
energy on the wrong things…
🔋

I remember when writing code for the
web was mostly about building
interfaces for humans and making
sure they work.🤹

But with better computers, less
confusing browsers and fast
connections we lost that focus.🚀

And this is dangerous and not helpful.
🌵

We have become disconnected from
our users.
✂

Our computers work, our connections
are great, we’re always up-to-date and
often ahead of the curve.🏁

So we’ve become introspective. Most
of the headlines we read and get
excited about are about improving our
tools.🛠

We seem to be at a break-neck pace
to empower ourselves to create more
in a shorter amount of time.⏳

Creating more in a shorter amount of
time always comes with some cost.
🗑

We create for the sake of creation and
to cater to a culture of perpetual
growth…🎰

Where are we going with this?
🤔

For now, we’re good. Our jobs are
safe, we have more freedom than we
know what to do with and we work in
the world others are scared of.😎

Have we created machines that are too clever?

I do not fear computers.
I fear the lack of them.
— Isaac Asimov
http://www.zmescience.com/other/economics/china-factory-robots-03022017/

https://www.bloomberg.com/news/articles/2017-02-28/jpmorgan-
marshals-an-army-of-developers-to-automate-high-finance

http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
How safe is my job?
Abstract thinking
Working with humans
Images
Words
Numbers
Routine Variety
You’re out
Get nervous
You’re OK

This also applies to us, as
the more predictable we
get, the easier it is to
replace us.

https://www.technologyreview.com/s/603381/ai-software-learns-to-make-ai-software/

http://www.wired.co.uk/article/ai-write-code-microsoft

https://openreview.net/pdf?id=ByldLrqlx

We’re in a post-production
world. We use other
people’s products instead
of writing new code from
scratch…

Which is totally fine. Boring,
repetitive and tedious work
is where computers excel
and humans make
mistakes.

So here’s what I think we should be
considering…
🤔

The saddest aspect of
life right now is that
science gathers
knowledge faster than
society gathers wisdom.
— Isaac Asimov

All personal data on the web has
been harvested and analysed.

For years we used “free” systems that
spied on us - heck, we even bought
the devices that recorded our every
move.
-
📱

Our data and information has already
been recorded and filed. It is almost
impossible not to have left a track.
📷✏🎙➡🗄

Technological progress
has merely provided
us with more efficient
means for going
backwards.
— Aldous Huxley

The Power of Big Data and Psychographics
https://youtu.be/Yu3Vxy6lQwg

The Power of Big Data and Psychographics
https://youtu.be/Yu3Vxy6lQwg

The Power of Big Data and Psychographics
https://youtu.be/Yu3Vxy6lQwg

The future is already
here – it’s just not very
evenly distributed.
— William Gibson

Here is where we come in.
4

The interfaces we
build today condition
the users of now and
of tomorrow…

The interfaces we
build today condition
the users of now and
of tomorrow…

It is time for us to see machine
learning as a tool in our belt to built
great, human interfaces.🌈

There is no such thing as a perfect user - think inclusive.
https://www.microsoft.com/en-us/Design/inclusive
It isn’t about allowing access but about avoiding barriers.

Here are some great examples…
🦄

“Selfies” (Auto category) https://photos.google.com

“Katze” (cat) https://photos.google.com

“Essen” (food) https://photos.google.com

https://qz.com/920857/netflix-nflx-uses-ai-in-its-new-codec-to-compress-video-scene-by-scene/

http://www.theverge.com/2017/2/8/14549798/pinterest-lens-visual-discovery-shazam

https://www.youtube.com/watch?v=XMdct-5bERQ

aipoly.com

https://blogs.office.com/2016/12/02/accessibility-in-office-365-enabling-greater-digital-inclusion/
Cognitive services in Microsoft Office

Facebook’s alternatives…
Image may contain: dog,
outdoor and nature
https://code.facebook.com/posts/561187904071636

Adding automated content in context…
https://research.googleblog.com/2016/05/aw-so-cute-allo-helps-you-respond-to.html

https://www.videobreakdown.com/
Understanding and fixing video automatically…

https://www.videobreakdown.com/
Transcripts were just a
start…

https://www.videobreakdown.com/
People, topics, sentiments…

https://www.newscientist.com/article/
2113299-googles-deepmind-ai-can-lip-read-
tv-shows-better-than-a-pro/?sf43896303=1
https://arxiv.org/pdf/1611.05358v1.pdf
46.8% vs 12.4% success rate

It is time for us to go and play!
🕹

The question of
whether computers
can think is like the
question of whether
submarines can swim.
— Edsger W. Dijkstra

People are freaking out!

https://www.bleepingcomputer.com/news/security/researcher-
breaks-recaptcha-using-googles-speech-recognition-api/

Learning from data… http://image-net.org/

Understanding the human aspect…
https://github.com/
openimages/dataset

Refining the data with language
https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html

Compare and enhance…

Detect syntax, mix and match

Add visual information

http://captionbot.ai
More APIs and demos to use…

Verification Result:
JSON:
[
{
"isIdentical":false,
"confidence":0.01
}
]
Verify Faces

Cluster automatically…

Add quality by filtering with context…
https://www.microsoft.com/cognitive-services/en-us/custom-recognition-intelligent-service-cris

Our job right now is to
create interfaces that are
simple, human and fun
to use…

Machines and code are there to
help humans communicate better.
The more useful our “intelligent”
solutions are, the easier it will be for
people to stop fearing innovation…
🎙

We need to use our skills,
knowledge and products to give
people a sense of interaction and
data ownership.
🔬

It is up to us to make this revolution
one that improves all mankind, and
not only makes some quick money
and shows us more ads.
9

CHRIS HEILMANN
@CODEPO8
CHRISTIANHEILMANN.COM
THANKS!