SlideShare una empresa de Scribd logo
1 de 73
Descargar para leer sin conexión
Functions +
Messages + Concurrency
       = Erlang


    Joe Armstrong
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

      Fault                 Multicore
    tolerance
Problem domain

Highly concurrent (hundreds of thousands of parallel
activities)
Real time
Distributed
High Availability (down times of minutes/year – never down)
Complex software (million of lines of code)
Continuous operation (years)
Continuous evolution
In service upgrade
Erlang
Very light-weight processes
Very fast message passing
Total separation between processes
Automatic marshalling/demarshalling
Fast sequential code
Strict functional code
Dynamic typing
Transparent distribution
Compose sequential AND concurrent code
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

      Fault                 Multicore
    tolerance
2002
Fraction of Chip reachable in one clock cycle




 [source] Erik Hagersten http://www.sics.se/files/projects/
 multicore/day2007/ErikH-intro.pdf
Clock Frequency




Clock frequency trend for Intel Cpus (Linux Journal)


Read: Clock rate verses IPC. The end of the road for
Conventional Microarchitectures. Agarwal et.al 2000
Keynote joearmstrong
Due to hardware changes:


Each year your sequential
 programs will go slower

Each year your concurrent
 programs will go faster
2005 – 2015
Paradigm shift in
      CPU
 architectures
Three New
Architectures
ONE - Multi core
TWO -
GPUs

Cell Computers
THREE – network on
Intel Polaris – 2007   Chip (NOC)
1 Tflop at 24 Watts
ASCI RED- 1997   - 1997
                 - First machine over 1 Tera
                  Flop
                 - 2,500 sq ft floor space
                  104 cabinets
                 - 9326 pentium pro
                  processors
                 - 850 KW
2 cores won't hurt you
4 cores will hurt a little
8 cores will hurt a bit
16 will start hurting
32 cores will hurt a lot (2009)
...
1 M cores ouch (2019)
    (complete paradigm shift)

1997 1 Tflop = 850 KW
2007 1 Tflop = 24 W (factor 35,000)
2017 1 Tflop = ?
Goal
Make my program run N times faster on an
N core CPU with
 no changes to the program
 no pain and suffering

Can we do this?

Yes     Sometimes (often)
Due to hardware changes:

Each year your sequential
 programs will go slower

Each year your concurrent
 programs will go faster
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

     Fault                  Multicore
   tolerance
To make
a fault-tolerant system
   you need at least


      two
      computers
If one computer crashes
the other must take over




= No Shared data
= Distributed programming
= Pure Message passing
To do fault tolerant computing we
need at least two isolated computers




     = Concurrent programming
       with pure message passing
To do very fault tolerant computing
we need lots of isolated computers




       = Scalable
Fault tolerance
 Distribution
 Concurrency
  Scalability

are inseparable
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

      Fault                 Multicore
    tolerance
Two models of Concurrency
Shared Memory
 - mutexes
 - threads
 - locks

Message Passing
 - messages
 - processes
Shared
  Memory
Programming
Shared memory
Problem 1
            Your program
            crashes in
            the critical region
            having corrupted
            memory
Problem 2


Sweden           ?           Australia




 Where do we (physically) locate the
 shared memory?
 Impossible to get low-latency and make
 consistent (violates laws of physics)
Keynote joearmstrong
Keynote joearmstrong
Thread Safety
Erlang programs are
automatically thread
safe if they don't use
an external resource.
Sharing is the
 property that
   prevents
fault tolerance
      and
Thread safety
Message
  Passing
Concurrency
No sharing
Pure message passing
No locks
Lots of computers (= fault tolerant
scalable ...)
Functional programming (no side
effects)
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

      Fault                 Multicore
    tolerance
What is COP?

                                        Machine


                                        Process

                                        Message


               Large number of processes
               Complete isolation between processes
               Location transparency
               No Sharing of data
               Pure message passing systems
Why is COP nice?

We intuitively understand concurrency
The world is parallel
The world is distributed
Making a real-world application is based on
observation of the concurrency patterns and
message channels in the application
Easy to make scalable, distributed applications
Concurrency Oriented Programming

 A style of programming where
concurrency is used to structure the
application
                                  My first message is that
Large numbers of processes              concurrency
Complete isolation of          is best regarded as a program
 processes
                                    structuring principle”
No sharing of data
Location transparency
Pure message passing         Structured concurrent programming
                                          – Tony Hoare
Examples of COP architectures
remember – no shared memory
– pure message passing

Email
Google – map – reduce (450,000
machines)
People (no shared state, message
passing via voiceGrams, waving
arms, non-reliable etc.)
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

      Fault                 Multicore
    tolerance
Functional programming
Scary stuff
Or easy?


fac(0) -> 1;
fac(N) -> N*fac(N-1).
Why is FP good?
Side effects are strictly controlled


              If you call the
         same function twice with
           the same arguments
      it should return the same value
Referential transparency

  In    Out      In,S   Out,S'




 S     S'


OOP              FP
Functional programming languages
    FLPs carry state with them
   wherever the flow of control
   goes. Different FPLs provide
     different notations and        In,S   Out,S'
    mechanisms for hiding this
          from the user.

    In Erlang we hide the state
    in a process. In Haskell in a
               monad

  FLPs have are based on a formal   FP
        mathematical model
   Lambda calculus (Pi calc, CSP)
Why is this important?
Compositional properties
Output of one function must be input to next
f(g(h(i(k(X)))))
Echo “foo” | k | i | h | g | f
No mutable state means nothing to lock and
automatic thread safety when parallelised
Can reuse pure functions
FP is on the rise

 Haskell
 Erlang
 O'Caml, F#
BAD STUFF
                                           Very very bad
Threads

Sharing

Mutexes - Locks

Synchronized methods

Mutable state
                           Mutable state is the root of all evil


           FPLs have no mutable state
GOOD STUFF
Processes
Controlled side effects
Pure functions
Copying
Pure Message passing
Failure detection
Concurrent
   programming             Functional
                          programming


Concurrency
 Oriented
programming
                 Erlang

      Fault                 Multicore
    tolerance
Keynote joearmstrong
Erlang in 11 Minutes

Sequential Erlang 5 examples
Concurrent Erlang 2 examples
Distributed Erlang 1 example
Fault-tolerant Erlang 2 examples
Bit syntax 1 example
Sequential Erlang

                Factorial   -module(math).
                            -export([fac/1]).
  Dynamic types
  Pattern matching          fac(N) when N > 0 -> N*fac(N-1);
  No mutable data           fac(0) -> 1
  structures
                            > math:fac(25).
Binary Tree Search           15511210043330985984000000


  lookup(Key, {Key, Val,_,_}) -> {ok, Val};
  lookup(Key, {Key1,Val,S,B}) when Key < Key1 ->
     lookup(Key, S);
  lookup(Key, {Key1, Val, S, B})->
     lookup(Key, B);
  lookup(key, nil) ->
     not_found.
Sequential Erlang

append   append([H|T], L) -> [H|append(T, L)];
         append([],    L) -> L.

sort     sort([Pivot|T]) ->
            sort([X||X <- T, X < Pivot]) ++
            [Pivot] ++
             sort([X||X <- T, X >= Pivot]);
         sort([]) -> [].

         > Adder = fun(N) -> fun(X) -> X + N end end.
 adder   #Fun
         > G = Adder(10).
         #Fun
         > G(5).
         15
Concurrent Erlang

spawn      Pid = spawn(fun() -> loop(0) end)

 send      Pid ! Message,
           .....

 receive   receive
               Message1 ->
                  Actions1;
               Message2 ->
                  Actions2;
                  ...
               after Time ->
                  TimeOutActions
           end




           The concurrency is in the language NOT the OS
Distributed Erlang

Pid = spawn(Fun@Node)

alive(Node),
.....

not_alive(Node)
Fault-tolerant Erlang

     ...
     case (catch foo(A, B)) of
         {abnormal_case1, Y} ->
            ...
         {'EXIT', Opps} ->
            ...
         Val ->
            ...
     end,
     ...

     foo(A, B) ->
        ...
        throw({abnormal_case1, ...})
Monitor a process

   ...
   process_flag(trap_exit, true),
   Pid = spawn_link(fun() -> ... end),
   receive
         {'EXIT', Pid, Why} ->
         ...
   end
Bit Syntax - parsing IP datagrams

-define(IP_VERSION, 4).

-define(IP_MIN_HDR_LEN,5).
DgramSize = size(Dgram),
case Dgram of

 <<?IP_VERSION:4, HLen:4,
  SrvcType:8, TotLen:16, ID:16, Flgs:3,
                                                  This code parses the
  FragOff:13, TTL:8, Proto:8, HdrChkSum:16,
                                                  header and extracts
  SrcIP:32, DestIP:32, Body/binary>> when
                                                  the data from an IP
   HLen >= 5, 4*HLen =< DgramSize ->
    OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN),
                                                  protocol version 4

    <<Opts:OptsLen/binary,Data/binary>> = Body,   datagram
  ...
Bit syntax – unpacking MPEG data
Some code
loop() ->
   receive
      {email,From,Subject,Text} = Email ->
         {ok, S} = file:open("inbox",[append,write]),
         io:format(S, "~p.~n",[Email]),
         file:close(S);
      {msg, From, Message} ->
         io:format("msg (~s) ~s~n", [From, Message]);
      {From, get, File} ->
         From ! file:read_file(File)
   end,
   loop().

Mike ! {email, "joe", "dinner", "see you at 18.00"}.

Helen ! {msg, "joe", "Can you buy some milk on your way
home?"}
Programming Multicore computers is difficult
because of shared mutable state.


Functional programming languages have no shared
state and no mutable state


Erlang has the right intrinsic properties for
programming multicore computers (concurrency
maps to the multiple CPUs, non-mutability means we
don't get any problems with memory corruption)
Keynote joearmstrong
Keynote joearmstrong
Keynote joearmstrong
- Use “lots” of processes
- Avoid sequential bottlenecks
- Use “large computation”
  small data transfer (if
  possible)
- New abstractions (pmap,
  mapreduce)
Commercial projects
Ericsson AXD301 (part of “Engine”)
Ericsson GPRS system
Alteon (Nortel) SSL accelerator
Alteon (Nortel) SSL VPN
Teba Bank (credit card system – South Africa)
T-mobile SMS system (UK)
Kreditor (Sweden)
Synapse
Tail-f
jabber.org /uses ejabberd)
Twitter (uses ejabberd)
Lshift (RabbitMQ) AMQP (Advanced Message Queuing protocol)
Finally

  We've known how to program parallel
  computers for the last twenty years

We can make highly reliable fault tolerant
     distributed real-time systems

              ww.erlang.org
Questions?

Más contenido relacionado

La actualidad más candente

Platform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly languagePlatform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly languagezynamics GmbH
 
Everybody be cool, this is a roppery!
Everybody be cool, this is a roppery!Everybody be cool, this is a roppery!
Everybody be cool, this is a roppery!zynamics GmbH
 
Attention mechanisms with tensorflow
Attention mechanisms with tensorflowAttention mechanisms with tensorflow
Attention mechanisms with tensorflowKeon Kim
 
Comparing Cpp And Erlang For Motorola Telecoms Software
Comparing Cpp And Erlang For Motorola Telecoms SoftwareComparing Cpp And Erlang For Motorola Telecoms Software
Comparing Cpp And Erlang For Motorola Telecoms Softwarel xf
 
Real Time Filtering on Embedded ARM
Real Time Filtering on Embedded ARMReal Time Filtering on Embedded ARM
Real Time Filtering on Embedded ARMVincent Claes
 
エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~
エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~
エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~Yamagishi Laboratory, National Institute of Informatics, Japan
 
Sequence Learning with CTC technique
Sequence Learning with CTC techniqueSequence Learning with CTC technique
Sequence Learning with CTC techniqueChun Hao Wang
 
.Net Multithreading and Parallelization
.Net Multithreading and Parallelization.Net Multithreading and Parallelization
.Net Multithreading and ParallelizationDmitri Nesteruk
 
Lab6 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab6 of EE337 Microprocessors Lab, 2022, IIT BombayLab6 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab6 of EE337 Microprocessors Lab, 2022, IIT BombaySaravananVijayakumar4
 
Lab7 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab7 of EE337 Microprocessors Lab, 2022, IIT BombayLab7 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab7 of EE337 Microprocessors Lab, 2022, IIT BombaySaravananVijayakumar4
 
Functional Programming With Python (EuroPython 2008)
Functional Programming With Python (EuroPython 2008)Functional Programming With Python (EuroPython 2008)
Functional Programming With Python (EuroPython 2008)Adam Byrtek
 
CILK/CILK++ and Reducers
CILK/CILK++ and ReducersCILK/CILK++ and Reducers
CILK/CILK++ and ReducersYunming Zhang
 
ITFT_Semaphores and bounded buffer
ITFT_Semaphores and bounded bufferITFT_Semaphores and bounded buffer
ITFT_Semaphores and bounded bufferSneh Prabha
 
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural NetworksModeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural NetworksJosh Patterson
 
Why i need to learn so much math for my phd research
Why i need to learn so much math for my phd researchWhy i need to learn so much math for my phd research
Why i need to learn so much math for my phd researchCrypto Cg
 

La actualidad más candente (20)

Platform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly languagePlatform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly language
 
Everybody be cool, this is a roppery!
Everybody be cool, this is a roppery!Everybody be cool, this is a roppery!
Everybody be cool, this is a roppery!
 
Attention mechanisms with tensorflow
Attention mechanisms with tensorflowAttention mechanisms with tensorflow
Attention mechanisms with tensorflow
 
Slides
SlidesSlides
Slides
 
Comparing Cpp And Erlang For Motorola Telecoms Software
Comparing Cpp And Erlang For Motorola Telecoms SoftwareComparing Cpp And Erlang For Motorola Telecoms Software
Comparing Cpp And Erlang For Motorola Telecoms Software
 
Real Time Filtering on Embedded ARM
Real Time Filtering on Embedded ARMReal Time Filtering on Embedded ARM
Real Time Filtering on Embedded ARM
 
MPI - 4
MPI - 4MPI - 4
MPI - 4
 
エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~
エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~
エンドツーエンド音声合成に向けたNIIにおけるソフトウェア群 ~ TacotronとWaveNetのチュートリアル (Part 2)~
 
Sequence Learning with CTC technique
Sequence Learning with CTC techniqueSequence Learning with CTC technique
Sequence Learning with CTC technique
 
.Net Multithreading and Parallelization
.Net Multithreading and Parallelization.Net Multithreading and Parallelization
.Net Multithreading and Parallelization
 
Lab6 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab6 of EE337 Microprocessors Lab, 2022, IIT BombayLab6 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab6 of EE337 Microprocessors Lab, 2022, IIT Bombay
 
Mit cilk
Mit cilkMit cilk
Mit cilk
 
1 hour dive into erlang
1  hour dive into erlang1  hour dive into erlang
1 hour dive into erlang
 
Using Parallel Computing Platform - NHDNUG
Using Parallel Computing Platform - NHDNUGUsing Parallel Computing Platform - NHDNUG
Using Parallel Computing Platform - NHDNUG
 
Lab7 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab7 of EE337 Microprocessors Lab, 2022, IIT BombayLab7 of EE337 Microprocessors Lab, 2022, IIT Bombay
Lab7 of EE337 Microprocessors Lab, 2022, IIT Bombay
 
Functional Programming With Python (EuroPython 2008)
Functional Programming With Python (EuroPython 2008)Functional Programming With Python (EuroPython 2008)
Functional Programming With Python (EuroPython 2008)
 
CILK/CILK++ and Reducers
CILK/CILK++ and ReducersCILK/CILK++ and Reducers
CILK/CILK++ and Reducers
 
ITFT_Semaphores and bounded buffer
ITFT_Semaphores and bounded bufferITFT_Semaphores and bounded buffer
ITFT_Semaphores and bounded buffer
 
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural NetworksModeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural Networks
 
Why i need to learn so much math for my phd research
Why i need to learn so much math for my phd researchWhy i need to learn so much math for my phd research
Why i need to learn so much math for my phd research
 

Similar a Keynote joearmstrong

Erlang
ErlangErlang
ErlangESUG
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsSentifi
 
Erlang Message Passing Concurrency, For The Win
Erlang  Message  Passing  Concurrency,  For  The  WinErlang  Message  Passing  Concurrency,  For  The  Win
Erlang Message Passing Concurrency, For The Winl xf
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlangMirko Bonadei
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futureTakayuki Muranushi
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSkills Matter
 
Recursion & Erlang, FunctionalConf 14, Bangalore
Recursion & Erlang, FunctionalConf 14, BangaloreRecursion & Erlang, FunctionalConf 14, Bangalore
Recursion & Erlang, FunctionalConf 14, BangaloreBhasker Kode
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with ErlangMaxim Kharchenko
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with ErlangMaxim Kharchenko
 
Erlang plus BDB: Disrupting the Conventional Web Wisdom
Erlang plus BDB: Disrupting the Conventional Web WisdomErlang plus BDB: Disrupting the Conventional Web Wisdom
Erlang plus BDB: Disrupting the Conventional Web Wisdomguest3933de
 
BayFP: Concurrent and Multicore Haskell
BayFP: Concurrent and Multicore HaskellBayFP: Concurrent and Multicore Haskell
BayFP: Concurrent and Multicore HaskellBryan O'Sullivan
 
Erlang及其应用
Erlang及其应用Erlang及其应用
Erlang及其应用Feng Yu
 
The Erlang Programming Language
The Erlang Programming LanguageThe Erlang Programming Language
The Erlang Programming LanguageDennis Byrne
 

Similar a Keynote joearmstrong (20)

Erlang
ErlangErlang
Erlang
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
 
Erlang, an overview
Erlang, an overviewErlang, an overview
Erlang, an overview
 
Erlang OTP
Erlang OTPErlang OTP
Erlang OTP
 
Erlang Message Passing Concurrency, For The Win
Erlang  Message  Passing  Concurrency,  For  The  WinErlang  Message  Passing  Concurrency,  For  The  Win
Erlang Message Passing Concurrency, For The Win
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlang
 
Concurrent programming1
Concurrent programming1Concurrent programming1
Concurrent programming1
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_future
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelism
 
Recursion & Erlang, FunctionalConf 14, Bangalore
Recursion & Erlang, FunctionalConf 14, BangaloreRecursion & Erlang, FunctionalConf 14, Bangalore
Recursion & Erlang, FunctionalConf 14, Bangalore
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
Erlang plus BDB: Disrupting the Conventional Web Wisdom
Erlang plus BDB: Disrupting the Conventional Web WisdomErlang plus BDB: Disrupting the Conventional Web Wisdom
Erlang plus BDB: Disrupting the Conventional Web Wisdom
 
Disrupt
DisruptDisrupt
Disrupt
 
BayFP: Concurrent and Multicore Haskell
BayFP: Concurrent and Multicore HaskellBayFP: Concurrent and Multicore Haskell
BayFP: Concurrent and Multicore Haskell
 
Erlang os
Erlang osErlang os
Erlang os
 
Erlang in 10 minutes
Erlang in 10 minutesErlang in 10 minutes
Erlang in 10 minutes
 
Erlang及其应用
Erlang及其应用Erlang及其应用
Erlang及其应用
 
IN4308 1
IN4308 1IN4308 1
IN4308 1
 
The Erlang Programming Language
The Erlang Programming LanguageThe Erlang Programming Language
The Erlang Programming Language
 

Último

How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 

Último (20)

How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 

Keynote joearmstrong

  • 1. Functions + Messages + Concurrency = Erlang Joe Armstrong
  • 2. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 3. Problem domain Highly concurrent (hundreds of thousands of parallel activities) Real time Distributed High Availability (down times of minutes/year – never down) Complex software (million of lines of code) Continuous operation (years) Continuous evolution In service upgrade
  • 4. Erlang Very light-weight processes Very fast message passing Total separation between processes Automatic marshalling/demarshalling Fast sequential code Strict functional code Dynamic typing Transparent distribution Compose sequential AND concurrent code
  • 5. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 7. Fraction of Chip reachable in one clock cycle [source] Erik Hagersten http://www.sics.se/files/projects/ multicore/day2007/ErikH-intro.pdf
  • 8. Clock Frequency Clock frequency trend for Intel Cpus (Linux Journal) Read: Clock rate verses IPC. The end of the road for Conventional Microarchitectures. Agarwal et.al 2000
  • 10. Due to hardware changes: Each year your sequential programs will go slower Each year your concurrent programs will go faster
  • 11. 2005 – 2015 Paradigm shift in CPU architectures
  • 13. ONE - Multi core
  • 15. THREE – network on Intel Polaris – 2007 Chip (NOC) 1 Tflop at 24 Watts
  • 16. ASCI RED- 1997 - 1997 - First machine over 1 Tera Flop - 2,500 sq ft floor space 104 cabinets - 9326 pentium pro processors - 850 KW
  • 17. 2 cores won't hurt you 4 cores will hurt a little 8 cores will hurt a bit 16 will start hurting 32 cores will hurt a lot (2009) ... 1 M cores ouch (2019) (complete paradigm shift) 1997 1 Tflop = 850 KW 2007 1 Tflop = 24 W (factor 35,000) 2017 1 Tflop = ?
  • 18. Goal Make my program run N times faster on an N core CPU with no changes to the program no pain and suffering Can we do this? Yes Sometimes (often)
  • 19. Due to hardware changes: Each year your sequential programs will go slower Each year your concurrent programs will go faster
  • 20. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 21. To make a fault-tolerant system you need at least two computers
  • 22. If one computer crashes the other must take over = No Shared data = Distributed programming = Pure Message passing
  • 23. To do fault tolerant computing we need at least two isolated computers = Concurrent programming with pure message passing
  • 24. To do very fault tolerant computing we need lots of isolated computers = Scalable
  • 25. Fault tolerance Distribution Concurrency Scalability are inseparable
  • 26. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 27. Two models of Concurrency Shared Memory - mutexes - threads - locks Message Passing - messages - processes
  • 30. Problem 1 Your program crashes in the critical region having corrupted memory
  • 31. Problem 2 Sweden ? Australia Where do we (physically) locate the shared memory? Impossible to get low-latency and make consistent (violates laws of physics)
  • 34. Thread Safety Erlang programs are automatically thread safe if they don't use an external resource.
  • 35. Sharing is the property that prevents fault tolerance and Thread safety
  • 37. No sharing Pure message passing No locks Lots of computers (= fault tolerant scalable ...) Functional programming (no side effects)
  • 38. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 39. What is COP? Machine Process Message Large number of processes Complete isolation between processes Location transparency No Sharing of data Pure message passing systems
  • 40. Why is COP nice? We intuitively understand concurrency The world is parallel The world is distributed Making a real-world application is based on observation of the concurrency patterns and message channels in the application Easy to make scalable, distributed applications
  • 41. Concurrency Oriented Programming A style of programming where concurrency is used to structure the application My first message is that Large numbers of processes concurrency Complete isolation of is best regarded as a program processes structuring principle” No sharing of data Location transparency Pure message passing Structured concurrent programming – Tony Hoare
  • 42. Examples of COP architectures remember – no shared memory – pure message passing Email Google – map – reduce (450,000 machines) People (no shared state, message passing via voiceGrams, waving arms, non-reliable etc.)
  • 43. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 46. Or easy? fac(0) -> 1; fac(N) -> N*fac(N-1).
  • 47. Why is FP good? Side effects are strictly controlled If you call the same function twice with the same arguments it should return the same value
  • 48. Referential transparency In Out In,S Out,S' S S' OOP FP
  • 49. Functional programming languages FLPs carry state with them wherever the flow of control goes. Different FPLs provide different notations and In,S Out,S' mechanisms for hiding this from the user. In Erlang we hide the state in a process. In Haskell in a monad FLPs have are based on a formal FP mathematical model Lambda calculus (Pi calc, CSP)
  • 50. Why is this important? Compositional properties Output of one function must be input to next f(g(h(i(k(X))))) Echo “foo” | k | i | h | g | f No mutable state means nothing to lock and automatic thread safety when parallelised Can reuse pure functions
  • 51. FP is on the rise Haskell Erlang O'Caml, F#
  • 52. BAD STUFF Very very bad Threads Sharing Mutexes - Locks Synchronized methods Mutable state Mutable state is the root of all evil FPLs have no mutable state
  • 53. GOOD STUFF Processes Controlled side effects Pure functions Copying Pure Message passing Failure detection
  • 54. Concurrent programming Functional programming Concurrency Oriented programming Erlang Fault Multicore tolerance
  • 56. Erlang in 11 Minutes Sequential Erlang 5 examples Concurrent Erlang 2 examples Distributed Erlang 1 example Fault-tolerant Erlang 2 examples Bit syntax 1 example
  • 57. Sequential Erlang Factorial -module(math). -export([fac/1]). Dynamic types Pattern matching fac(N) when N > 0 -> N*fac(N-1); No mutable data fac(0) -> 1 structures > math:fac(25). Binary Tree Search 15511210043330985984000000 lookup(Key, {Key, Val,_,_}) -> {ok, Val}; lookup(Key, {Key1,Val,S,B}) when Key < Key1 -> lookup(Key, S); lookup(Key, {Key1, Val, S, B})-> lookup(Key, B); lookup(key, nil) -> not_found.
  • 58. Sequential Erlang append append([H|T], L) -> [H|append(T, L)]; append([], L) -> L. sort sort([Pivot|T]) -> sort([X||X <- T, X < Pivot]) ++ [Pivot] ++ sort([X||X <- T, X >= Pivot]); sort([]) -> []. > Adder = fun(N) -> fun(X) -> X + N end end. adder #Fun > G = Adder(10). #Fun > G(5). 15
  • 59. Concurrent Erlang spawn Pid = spawn(fun() -> loop(0) end) send Pid ! Message, ..... receive receive Message1 -> Actions1; Message2 -> Actions2; ... after Time -> TimeOutActions end The concurrency is in the language NOT the OS
  • 60. Distributed Erlang Pid = spawn(Fun@Node) alive(Node), ..... not_alive(Node)
  • 61. Fault-tolerant Erlang ... case (catch foo(A, B)) of {abnormal_case1, Y} -> ... {'EXIT', Opps} -> ... Val -> ... end, ... foo(A, B) -> ... throw({abnormal_case1, ...})
  • 62. Monitor a process ... process_flag(trap_exit, true), Pid = spawn_link(fun() -> ... end), receive {'EXIT', Pid, Why} -> ... end
  • 63. Bit Syntax - parsing IP datagrams -define(IP_VERSION, 4). -define(IP_MIN_HDR_LEN,5). DgramSize = size(Dgram), case Dgram of <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16, ID:16, Flgs:3, This code parses the FragOff:13, TTL:8, Proto:8, HdrChkSum:16, header and extracts SrcIP:32, DestIP:32, Body/binary>> when the data from an IP HLen >= 5, 4*HLen =< DgramSize -> OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN), protocol version 4 <<Opts:OptsLen/binary,Data/binary>> = Body, datagram ...
  • 64. Bit syntax – unpacking MPEG data
  • 65. Some code loop() -> receive {email,From,Subject,Text} = Email -> {ok, S} = file:open("inbox",[append,write]), io:format(S, "~p.~n",[Email]), file:close(S); {msg, From, Message} -> io:format("msg (~s) ~s~n", [From, Message]); {From, get, File} -> From ! file:read_file(File) end, loop(). Mike ! {email, "joe", "dinner", "see you at 18.00"}. Helen ! {msg, "joe", "Can you buy some milk on your way home?"}
  • 66. Programming Multicore computers is difficult because of shared mutable state. Functional programming languages have no shared state and no mutable state Erlang has the right intrinsic properties for programming multicore computers (concurrency maps to the multiple CPUs, non-mutability means we don't get any problems with memory corruption)
  • 70. - Use “lots” of processes - Avoid sequential bottlenecks - Use “large computation” small data transfer (if possible) - New abstractions (pmap, mapreduce)
  • 71. Commercial projects Ericsson AXD301 (part of “Engine”) Ericsson GPRS system Alteon (Nortel) SSL accelerator Alteon (Nortel) SSL VPN Teba Bank (credit card system – South Africa) T-mobile SMS system (UK) Kreditor (Sweden) Synapse Tail-f jabber.org /uses ejabberd) Twitter (uses ejabberd) Lshift (RabbitMQ) AMQP (Advanced Message Queuing protocol)
  • 72. Finally We've known how to program parallel computers for the last twenty years We can make highly reliable fault tolerant distributed real-time systems ww.erlang.org