This document provides guidance on designing microservices using the Go programming language. It begins with an introduction to Go's core concepts like packages, functions, methods, structs, interfaces, errors, goroutines, and what Go does not include. It then discusses when Go is well-suited and not well-suited through examples. The document concludes with tips for designing Go microservices, including leveraging existing frameworks, using interfaces, ORM for entities, centralizing configurations, and making errors meaningful. The overall message is to understand where Go works best and mix technologies as needed while avoiding unnecessary complexity.
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
Hurry Up and Go Microservice Design
1. Hurry Up and Go
(But Not Too Fast)
Stuart Fettinger
sfettinger@nvisia.com
2. About Me
Project Architect at NVISIA
10+ years of experience designing, developing, and delivering solutions
Recent co-founder of a tech startup
Passionate about limiting tech debt through clean and efficient design
3. Topic Breakdown
Brief Introduction to GO
Guiding Principles and Core Concepts
Go’s strong (and not so strong) suits
Highlighting when GO is a preferred choice, and when it’s not,
through real-world examples
Designing a GO Microservice
Tips for your design to take advantage of Go to its fullest and avoid
common pitfalls
5. • Developed by Google in 2007
• Designed to overcome criticism
of other languages
•Statically typed and compiled
• Syntactically similar to C
6. Go’s Guiding
Principles
Simplicity
Type inference
Declare-and-initialize operations
No generics
Composition rather than inheritance
Readability
As little “magic” as possible
Explicit and lean error mechanisms
Orthogonality
Do one thing well, keep things simple, standardize communication
Elimination of cross-linking complexity
Performance
Built in garbage collection
Synchronization, channels and Go routines
Pointers
7. A Basic Example
Main package – compiled binary
Short, concise imports
Function keywords
Multiple return values
Meaningful spaces – no semicolons
9. Packages provide logical groupings and reusable functionality
Every Go program starts in the main package’s main() function
Main package is compiled to binary
Functions beginning with a capital letter are exported outside their package
Functions beginning with a lowercase letter are essentially private
Packages
10.
11. Functions are stand-alone pieces of functionality
Methods are functions with receivers - tied to a type
Methods should be used when state is important and with interfaces
Rule of thumb – prefer methods over functions
Methods vs. Functions
12.
13. Variables that store the memory address of other variables
Zero type of nil – popularly used where nil is valid and relied upon
Memory address available via &
Pointer dereferenced via *
Pointers
14.
15.
16. Structs
Named collections of fields
When initialized, all fields are set to their zero values
Fields of a struct are accessed via “.” operator
Composition over inheritance – one struct cannot be the child of another
Structs can contain other structs
17.
18. Named collections of method signatures
No “Implements” keyword – can be implemented directly by implementing all methods
Interfaces can be empty – automatically satisfied by any type
Go’s answer to polymorphism
Interfaces
19.
20. Errors
Errors are treated as values
Stack traces don’t come for free with errors
Idiomatic to check for errors frequently via “if” operations
No try/catch framework – error handling up to developers
21.
22. Go Routines
Lightweight threads managed by the Go runtime
Spawned by simply using “go” keyword
Initially small stack size with ability to scale
Leverage channels to pass messages, block, and sync between Go routines
Performant – can run thousands of Go routines at a time with little drag
23.
24. What Go Doesn’t Have
• Inheritance
• Classes
• Exceptions
• Generics
26. Go Is Not One-Size Fits All
• Go works well in some cases, not so great in others
• Some scenarios to consider Go
• Small, predictable routines
• Where performance margins are razor thin (ie: trading)
• Quick start-up with limited resources
• Scaling is a prime factor or an unknown
• Concurrency and communication is a prime use-case
• Scenarios where Go may fall short
• Where complexity requires extensive third-party libraries
• Where OO reduces complexity
• Front-end MVC
• Team makeup is less experienced
• Remember – design decisions should be case-by-case – try not to promote blanket use of technology
27. Considerations
• What are the requirements for the problem I am trying to solve?
• Can I solve the problem another way, and will doing so lose me any benefits?
• Consider technical pros and cons, but also intangible factors
• Developer happiness
• Community support
• Framework maturity
• Current trends
• Consider supporting a solution after it is live
• Scaling needs
• Technical debt and BAU maintenance
28. Use Case #1
Requirements
• Microservice with capabilities to
• Store high resolution images on the cloud
• Resize images without damaging resolution
• Be available on-demand to fetch data
• Data throughput may be larger than other streams (ie: text)
As a user, I want to be able to upload a high-resolution image, and retrieve it later in
whatever dimensions I want, so that I can use the image for different situations
29. Use Case #1
Performance exceeds
that of an interpreted
language
Package manager and
libraries are more
diverse
Higher scalability to
meet unknown data
demands
Ease of using docker to
spin up containers
Serverless architecture
support
Further developed
community
Smaller learning curve
for front-end
developers
Channels and
GoRoutines to spread
out workloads
30. Use Case #2
Requirements
• Several microservices connecting with different resources and providing different outputs
• Similarities in data may be a high percentage of make-up
• Data has potential to be large – multi-threading may be important
As a user, I want to be able to call APIs for different but similar data, and aggregate them
together, so that I can streamline my usage of that data
31. Use Case #2
Lightweight syntax –
avoid getting lost in
complex data
manipulation
Inheritance and
genericsHigh Performance
Mature libraries for
persistence and data
combination
Go Channels and Go
Routines – automate
polling data
Extensive API strategy
support
32. Use Case #3
Requirements
• Many file types may exist in many different formats
• Files may be made available at different times
• Must store data, potentially in a variety of formats
As a user, I want to be able to ingest and persist files from many different sources, so that I
can scrape the data later via my own processes
33. Use Case #3
Significant performance
benefits
Mature ORM libraries
Low learning curve
Wide database support
Higher developer
popularity
Multithreading
capabilities
34. Use Case #4
Requirements
• Application available on the web, potentially mobile as well
• Rich Interface and modern design
• Communicates with various microservices
• Possibilities are endless – login, registration, alerts, etc…
As a developer, I want to create a front-end web application for my users to interact with,
so that I can expand my customer base
35. Use Case #4
Lower learning curve Mature MVVM patterns
and support
Templating and
rendering engines
High number of built-in
must-haves for front-
end development
Third-party routing
support
Seamless integration
with a back-end Go
stack (Revel)
38. Plenty of third-party
frameworks exist to
ease microservice
development
GoMicro
• Foundation for RPC and event-driven communication with reasonable defaults built in
• Out-of-the-box:
• Service discovery
• Auto registration and name resolution
• Load Balancing
• Even distribution and node-retry
• Message Encoding
• Proto-RPC and Json-RPC by default
• Asynchronous
• Event-driven architecture
• “Pluggable” interfaces
• Platform agnostic implementations
GoKit
• Toolkit for Go – designed to be imported into a binary package
• Fills the gaps left by the base package(s)
• Security
• Monitoring
• Service Discovery
• Can be thought of as similar to
• Spring Boot
• Eureka
• Ribbon
• Integrates well with Docker, Kubernetes, and Heroku
• Different from GoMicro in that it’s more of a toolkit, less of a platform
42. Make your entities
more meaningful by
tying them to an ORM
implementation
Go’s lean syntax drives you to use straight SQL prepared statements
• Gets messy fast with many parameters and outputs
• Nil considerations are difficult – Go values not easily nillable
• Compiler will not catch issues with prepared statements, field names, etc...
GoRM
• Provides associations between entities – Has One, Has Many, Many to
Many, Belongs To, etc…
• Uses any field with ID as the primary key – no need to annotate
• Auto support for creation, update, and soft-delete timestamps
• Auto-update capabilities – No need to explicitly call update vs insert
• Provides hooks to execute functionality pre or post running SQL
• API for transaction management and rollback
• Eager loading vs lazy loading (default) capabilities
• Can run raw SQL if needed
44. Go’s binary nature may drive you to place configs in files within the application, or
on the server in a central location
• It works – but you’re missing out
Create a central location where configurations live – ideally another microservice
serving up files
• Place security on top of configs
• Integrate with CI/CD pipelines
• Inject configs via use of Docker secrets, etc…
• Leverage caching for failure scenarios
Read configurations into your services by leveraging Go-based configuration
solutions
Viper
• API to find and load config files of varying formats
• Also supports writing configuration files
• Provides defaults and gracefully handles overrides
• Live-watching of configs, and hot-reloading
• Alias-based naming, allowing non-breaking renames
• Uses environment variables to manage different configurations
• Remote key/value store support
Centralized
configuration keeps
things DRY
46. • Errors are values so it’s easy to treat them as strings and just pass them up the
chain
• There is no stacktrace included with an error by default
• There is no concept of an exception in Go – up to the developer to handle
• Implement the error interface to construct meaningful errors of different types
• Add stack trace information by using go-errors package
• Add important logging info using go log flags
Supplement your error
framework with
additional logging and
packages
47. When Designing Your Microservices…
• Understand the cases where your chosen technology works best
• Do POCs, reach out to the community, benchmark test – make informed decisions based on use cases
• Don’t be afraid to mix and match to get your desired result
• i.e – Go, third-party Go packages, Spring, Docker, etc…
• Always look for opportunities to improve, but not to the point of development paralysis
• Go is meant to be simple – don’t head down the rabbit hole of over-inventiveness
• Don’t invite unnecessary technical debt
• Consider your long-term goals and scaling needs when deciding how to implement functionality
• Design for technical soundness but also developer satisfaction
• Go is so popular, in part, because it’s fun