Developer experience is about reducing friction between creating an idea and delivering business value in production. It includes factors like lead time, deployment frequency, and monitoring. DevEx has three components: workflow, platforms, and the developer experience itself. The ideal workflow follows progressive delivery principles. Teams should focus first on automating the inner development loop and CI/CD processes, then on observability and best practices. Questions around local vs cluster development, verification approaches, and whether to provide guardrails can help guide platform design decisions.

1. Creating an Effective Developer
Experience for Cloud-Native Apps
Daniel Bryant
@danielbryantuk | @datawireio

2. “Developer Experience”

3. @danielbryantuk
Independent Technical Consultant, Product Architect at Datawire
Previously: Academic, software developer (from startups to gov),
architect, consultant, CTO, trainer, conference tourist…
Leading change through technology and teams

4. DevEx 101

5. Developer Experience (DevEx) is about...
“...reducing engineering friction between creating a hypothesis, to
delivering an observable experiment (or business value) in production”
- Adrian Trenaman (SVP Engineering, HBC)
https://www.infoq.com/news/2017/07/remove-friction-dev-ex

6. DevEx isn’t new, but it is important
● Lead time
● Deployment frequency
● Mean time to restore (MTTR)
● Change fail percentage
● Rapid provisioning
● Basic monitoring
● Rapid app deployment
https://martinfowler.com/bliki/MicroservicePrerequisites.html

7. DevEx: Three Components

8. DevEx,
Workflow,
Platforms

9. The Ideal Workflow

10. “Progressive Delivery”
https://redmonk.com/jgovernor/2018/08/06/towards-progressive-delivery/ https://launchdarkly.com/blog/progressive-delivery-a-history-condensed/

11. https://speakerdeck.com/stilkov/microservices-patterns-and-antipatterns-1

12. Decentralised Biz/Product Teams;
Centralised Specialists and Platform
Team Z:
Prototyping
Team A: Mission-
Critical
Team T:
Production Phase

13. https://twitter.com/kelseyhightower/status/851935087532945409

14. https://www.infoq.com/news/2017/06/paved-paas-netflix
https://www.infoq.com/news/2018/07/shopify-kubernetes-paas

15. Should I Build a
PaaS on k8s?

16. Fundamental questions
Do you understand your domain?
Is your problem domain complex?
Do you have product/market fit?
Question
Is your solution event-driven (and simple)?
Should you be adding value elsewhere?

17. SOLID K8s: Open for Extension...
● Kubernetes becoming de facto CoaaS (the new cloud broker?)
○ Lots of hosted options
● Know the extension points
○ Custom Resources & Controllers
○ Operators
○ operatorhub.io (kudos to Red Hat)
● Extension enables custom workflow
○ “Kubernetes Custom Resource, Controller and Operator Development Tools”

18. https://github.com/weaveworks/flux

19. How quick do you need feedback?
Question
https://mitchdenny.com/the-inner-loop/
https://bit.ly/2RXfokz

20. Automate Inner Dev Loop
https://blog.hasura.io/draft-vs-gitkube-vs-helm-vs-
ksonnet-vs-metaparticle-vs-skaffold-f5aa9561f948
https://codeengineered.com/blog/2018/kubernet
es-helm-related-tools/

21. ● Draft
○ Automates “inner loop” build-push-deploy
○ Utilises Helm
● Gitkube
○ Automates build-push-deploy
○ Provides heroku / CF like experience
● Skaffold
○ Automates build-push-deploy
○ Watches source code
○ Provides “dev” and “run” (CD) modes
● Tilt
○ Automates “inner loop” build-test-deploy
● Garden
○ Automates local build-push-test-deploy
Automate Inner Dev Loop
● Helm (*)
○ Package manager for k8s
○ Deploy and manage (ready-made) charts
● Ksonnet
○ Define k8s manifests in jsonnet
○ Create composable config/services
● Telepresence (*)
○ Enables local-to-prod development
(*) CNCF projects

22. Develop and test services locally, or
within the cluster (or both)?
● Working locally has many advantages
○ Reduce ops cost of multi-cluster
● However, some systems are simply too
large to run locally (for integration tests)
● Local/remote container dev tools like
Telepresence and Squash allow hybrid
Question

23. Develop and test services locally, or
within the cluster (or both)?
● Working locally has many advantages
○ Reduce ops cost of multi-cluster
● However, some systems are simply too
large to run locally (for integration tests)
● Local/remote container dev tools like
Telepresence and Squash allow hybrid
Question

24. “Bring the cloud to you” “Put you in the cloud”

25. How do want to verify your system?
● Pre-prod testing in distributed systems
○ Dealing with complex adaptive systems
○ Probabilistic guarantee of “correctness”
https://medium.com/@copyconstruct/testing-microservices-
the-sane-way-9bb31d158c16
Question

26. https://skillsmatter.com/skillscasts/13773-london-java-community-april

27. How do want to verify your system?
● Pre-prod testing in distributed systems
○ Dealing with complex adaptive systems
○ Probabilistic guarantee of “correctness”
https://medium.com/@copyconstruct/testing-microservices-
the-sane-way-9bb31d158c16
Question
● Traffic shaping/splitting is powerful
○ Canarying
○ Shadowing

28. https://github.com/weaveworks/flagger

29. The Importance of L7 (and Envoy)
● “Service-mesh all the things”?
● Old pattern, new technology
○ Allows fine-grained release
● Many control planes for Envoy
○ Ambassador
○ Gloo
○ Istio
○ Consul Connect
https://www.infoq.com/articles/ambassador-api-gateway-kubernetes

30. https://www.youtube.com/watch?v=o1MJi54_R4o&list=PLj6h78yzYM2PpmMAnvpvsnR4c27wJePh3&index=179
https://www.infoq.com/articles/api-gateway-service-mesh-app-modernisation/

31. Canary gotchas (and mitigations)
● Observability is a prerequisite
○ Service Level Indicators (SLIs)
○ Service Level Objectives (SLOs)
○ Key Performance Indicators (KPIs)
● Needs high volume of diverse
(representative) traffic
● Take care with side effects
● Focus on “golden signals”
○ Latency, traffic, errors, saturation
○ Okay to initially “eyeball” data
○ Create actionable alerts
● Load test (with flag header)
● Run synthetic transactions
● Service virtualisation (Hoverfly)

32. Observability UX

33. Do you want to implement “guard rails”
for your development teams?
● Larger teams often want to provide
comprehensive guard rails
● Startups and SMEs may instead value
team independence
● Hybrid? Offer platform, but allow service
teams freedom and responsibility
https://blog.openshift.com/multiple-deployment-methods-openshift/
Question

34. Build vs buy
https://www.infoq.com/news/2019/03/airbnb-kubernetes-workflow
https://docs.openshift.com/container-platform/3.9/dev_guide/openshift_pipeline.html

35. Security guard rails

36. Getting Started:
Where to Focus

37. Decentralised Biz/Product Teams;
Centralised Specialists and Platform
Team Z:
Prototyping
Team A: Mission-
Critical
Team T:
Production Phase

38. Some thoughts on where to focus...
Prototype Production Mission Critical
Dev and test Local / hybrid Hybrid / local / staged Local / (hybrid) staged
Release Canary
(synthetic shadow)
Canary / pre-prod test Pre-prod test / Canary
Guide rails “YOLO” Limited Strong
Where to focus? Inner development
loop & CI/CD
Observability and
scaffolding (codifying
best practices)
Observability, debugging
and “recreatability”
(environment & data)

39. Conclusion

40. In Summary
The developer experience is primarily about minimising the friction between having
an idea, to dev/test, to release, to delivering observable business value
How you construct your ‘platform’ impacts the developer experience greatly
You must intentionally curate the experience of: local development, continuous
delivery, release control, observability, debuggability, and more...

41. Thanks for Listening!
Questions, comments, thoughts…
db@datawire.io
@danielbryantuk