Video recording of the presentation: http://bit.ly/1Lc2i6h This talk will discuss how your programming language’s ability to provide easy concurrency can have an impact on not just performance, but also your AWS bill. As a case study we will present an event-processing service we recently moved to Go from Python, which requires long-term storage for ~1K/sec data blobs, each on average ~70KB — and walk through a few approaches to meeting this demand cost-effectively using Amazon S3, SQS, Kinesis, and DynamoDB. After a very brief introduction to Go we will show how its concurrency features make it easier to achieve a lower-cost implementation than would be possible with Python, Ruby, or Node.js.

1. Python, Go, and the Cost of
Concurrency in the Cloud

2. • Quick discussion of moving from Python to Go
• Explaining key differences between Go and other “no
semicolons” languages
• Show an example application illustrating why those key
differences matter for your app’s bottom line
Goal of this talk

3. • Daily Python hacker
• Came from C/C++, UNIX systems hacking background
• PhD on P2P/crypto research, more C++ & Python
• Six months experience in Go
• Co-founder, Tracelytics; Chief Architect, AppNeta
About me
Chris Erway, AppNeta Chief Architect

4. • Fun to program — “Zen of Python”
• Built-in maps, sets, arrays, tuples
• Good library support
• Simple duck typing (as opposed to strict OO)
• A little code goes a long way
Things I like about Python

5. • Performance: not too slow, but not too fast either
• Dependencies can be a pain (virtualenv, pip, etc)
• The dreaded Global Interpreter Lock (GIL)
• Lack of typed function signatures can make reading code
difficult
Things I don’t like about Python

6. • Announced 2009
• Creators: Ken Thompson (B, Plan 9 from Bell Labs), Rob Pike
(Plan 9), Robert Griesemer (V8 engine)
• “all three of us had to be talked into every feature in the
language, so there was no extraneous garbage put into the
language for any reason”
• Statically typed, garbage-collected
• Fast compilation, static linking
Go

7. • Easy to read, no semicolons
• Built-in maps, arrays, strings
• Both support calling into C code when necessary
• Interfaces based on duck typing
• No virtual inheritance
• Statically typed, but automatic type inference and interfaces
give it a “dynamic feel”
Similarities between Go and Python

8. • Go is compiled to native machine code
• Fast compiler, single static binary
• Go is fast; memory usage depends on size of structs
• No per-object dictionaries, as in Python
• Go has concurrency features built into the language: goroutines,
channels, runtime scheduler
• Go has curly braces
Differences between Go and Python

9. Language comparison

10. • Lots of resources online for learning Go!
• Following 4 slides from “Go for Pythonistas” by Francesc
Campoy Flores, Google
• See also: The Go Programming Language Blog, blog.golang.org
For more on Go ...

11. • Goroutines: very light processing actors (the gophers)
• Channels: typed, synchronized, thread-safe pipes (the arrows)
Go concurrency
Based on goroutines and channels

12. Fibonacci with Python generators

13. “Generator” goroutines
A function that returns a channel:

14. Concurrency across languages
(From Brad Fitzpatrick’s Gocon Tokyo 2014 talk)

15. So who cares?
You do!
• You do — concurrency is very important in the modern
computing environment
• Programming for “the cloud” or for “SOA” or “microservices” is
fundamentally different than writing a LAMP/MEAN/Rails app
• Assumptions on latency, throughput, scale all change
• The language you pick can cost you time & money

16. Modern systems demand concurrency
• Pre-cloud systems and databases generally use pools of long-
living, low-latency connections
• Cloud & SOA/microservices often rely on HTTPS-based APIs
• “Infinite” scale, but with more latency
• For example: Amazon SQS vs RabbitMQ
• HTTP-based APIs have inherently higher latencies
• Amazon DynamoDB 5-10ms latency
• Amazon Kinesis PutRecords, S3, SQS 10-100ms latency
• Usage-based pricing
• Higher throughput = more concurrent HTTP connections

17. What about my async code for Python,
Ruby, Node?
• Async I/O makes network, disk reads & writes asynchronous
• Used by Python’s gevent, Tornado, Twisted
• Ruby EventMachine, Celluloid
• However Python, Ruby, Node.js
all still use a single-threaded
interpreter
• Interpreter can switch to another execution
context/greenlet/thread while I/O is pending
• Go: thousands of goroutines are mapped to all available cores

18. Cloud APIs require compute-heavy RPCs
• HTTP-based APIs with authenticated JSON/XML
• Encryption: TLS/SSL key exchange, negotiation
• Authentication: AWS, Google request signature schemes
• Serialization: Convert data to JSON, base64, etc
• Not as simple as binary data over raw sockets
• Not pure disk/network I/O — not as easy to use async I/O

19. Increasing prevalence of multi-core
architectures
• Quad-core, 8-core, 16-core, 20-core, 32-core, 40-core …
• How will you use all those CPUs?
• Strong opinion: Docker, containerization sometimes
used as a crutch for horizontally scaling services written
in single-threaded languages

20. Motivating example
• ~1000 items analyzed each second
• ~1000 Amazon S3 PUTs/sec, ~70KB each
• ~1000 Amazon DynamoDB item writes/sec

21. Cloud storage, queue, and log costs

22. Motivating example
• ~1000 items analyzed each second
• ~1000 Amazon S3 PUTs/sec, ~70KB each
• ~1000 Amazon DynamoDB item writes/sec

23. Why not queue the writes for later?
• ~1000 items analyzed each second
• ~1000 Amazon S3 PUTs/sec, ~70KB each
• ~1000 Amazon DynamoDB item writes/sec

24. Batch writes for fewer PUTs
• Read data objects from Amazon SQS
• Batch into larger files and store in Amazon S3

25. Baseline: Amazon SQS + S3 + DynamoDB
No batching

26. Amazon SQS + S3 + DynamoDB
BATCH_SZ=10

27. Amazon SQS + S3 + DynamoDB
BATCH_SZ=100

28. Amazon SQS + S3 + DynamoDB
BATCH_SZ=1000

29. Basic algorithm

30. Implementation difficulties

31. Single Python processes
~50 messages/sec

32. Multiple Python processes
4 procs = ~200 messages/sec

33. Process-based scaling leads to
suboptimal cost performance
• Impossible to scale number of
Amazon SQS pollers and S3 writers
independently
• One batch buffers per process:
smaller batches than optimal, hard
to “max out” S3 batch size before
timeout
• Hard to “max out” 10 messages each
SQS read
• Hard to detect when system is falling
behind, problematic if write latency >
read latency

34. Go implementation

35. Concurrency costs money
• The concurrency model your language provides is very important
when your code combines lots of high-latency API calls / RPCs
• Ruby, Python, Node.js all require lots of concurrent processes to
achieve good concurrency
• Result: over-provisioning, over-polling, IPC when you don’t need to
• Result: suboptimal cost when using usage-priced APIs

36. Couldn’t I just use {C, C++, Java, Scala,
Clojure, Erlang, Haskell} for concurrency?
• Yes, but —
• it may still be such a pain to spawn & use threads in your
language that you don’t do it enough (e.g. in Java, C/C++ vs.
just typing “go func()”)
• Lock-based synchronized memory access more
complicated than channels
• C/C++ and Java have pretty heavyweight thread sizes, typically
can only support 1K-10K threads
• Go (and Erlang) have very lightweight threads and can support
millions of goroutines

37. Does this apply to me?
• Increasingly, yes
• More cores, more cloud, all the time
• SOA, “microservices”
• Do you have code that calls multiple independent services
serially?
• Why?

38. Thank you!
• Hope this was useful and interesting!
• Win a BB-8 droid at our booth #131!
• Next drawing is at 3:15PM today
• AppNeta is hiring! Engineering roles in Providence, Boston,
Vancouver
• http://www.appneta.com/about/careers/

39. Backup slides (if potential Q&A question comes up)

40. Amazon Kinesis + S3 + DynamoDB
Provisioned throughput per shard + PUT units

41. RabbitMQ + Amazon S3 + DynamoDB
self-managed queue instances