2. DELIVERING NEW FEATURES TO OVER
30,000 CUSTOMERS DAILY
INSIDE THE DEVELOPMENT TEAM
RYAN THOMAS
DEVELOPMENT TEAM LEAD
DEVELOPER ACCELERATION GROUP, ATLASSIAN
rthomas@atlassian.com

3. AGENDA
!
Atlassian OnDemand
!
Releasing with Confidence
!
Mechanics of Deploying
!
The Feedback Loop

5. “
“
Atlassian OnDemand takes our awardwinning software for development &
collaboration into the cloud, and makes
getting started easier than ever.

7. 35,535

8. UNICORN

9. OPERATIONS
INFRASTRUCTURE
SOFTWARE

10. INFRASTRUCTURE
•
•
•
•
Atlassian owned hardware
Purpose built for OnDemand
3 Datacenters across US & Australia
• Ashburn
• Santa Clara
• Sydney
~57Tb of RAM in our Ashburn DC

11. OPERATIONS
•
•
•
Dedicated Operations team around the globe
• Australia
• US
• Europe
24x7 Monitoring and support of the platform
and applications
99.972% availability over the last 31 days

12. SOFTWARE
•
•
•
Traditionally self-hosted applications
We are SaaS’ing them into a true SaaS
solution
• Shared services
• Multi-tenanted applications
The same JIRA, Confluence & Bamboo that
you have already used
• But with more frequent features delivered!

13. RELEASING WITH CONFIDENCE

14. ?
?
?
?
?
What is a release process?
?
?

15. UNIT
INTEGRATION
ACCEPTANCE
RELEASE

19. 350
COMMITS A WEEK
TO MASTER

20. TESTING
REVIEW
! AUTOMATION

21. TESTING
•
•
•
Keep master releasable
Automatically build on feature branches
We do acceptance testing of:
• The change against current Production
versions
• The change against current Development
versions
!

22. REVIEW
• Every. Single. Change. Is reviewed
• Code review is part of our definition of done
• Peer review by team / relevant people
• QA comes in once we have reviewed our
own changes
• All stories have QA demos
• All stories have QA testing notes
!

23. AUTOMATION
!
• We are in the process of automating our
•
•
release process end-to-end
This removes the risk of human error
Allows us to move quicker, with a higher
confidence

24. #summit13
- Eric
Raymond
“
“
Release early. Release often. And listen to
your customers.

25. MECHANICS OF DEPLOYING
!

27. Development
Smoke Tests
Dogfooding
Smoke Tests
Production

28. • Contains all of the releases that go to
Development
•
•
Dogfooding
Production
OnDemand
Developers instances can opt-in to
receiving updates of new plugins &
products
Most likely to encounter bugs here

29. Development
• We “Eat our own dogfood”
• Contains business-critical systems we
•
Dogfooding
•
•
Production
use every day
All instances updated
• New plugins when available
• Products updates every night
A sub-set of releases in the DEV
environment
Bugs should be found before here :)

30. Development
Dogfooding
Production
• What our customers use every day
• A sub-set of releases in the DOG
•
environment
Less likely to encounter bugs here

31. Development
Dogfooding
Production

32. #summit13
“
“
The environments you deliver to should
continually be refining their output. The aim is
to deliver high quality software.

33. POWER

34. POWER
• Large increase is ~ 600W of
•
•
power usage - per rack
We have 37 racks in our
Ashburn DC
This is ~ 22.2kW of power for a
standard US & EMEA upgrade

35. POWER
Enough to power my
MacBook Air for 29.3
weeks straight
More than 6 months!

37. THE FEEDBACK LOOP

39. POST DEPLOYMENT VERIFICATION
• Ensuring what we’ve deployed is actually
•
•
working
Allows quick pickup of defects in
production
We use the atlassian-healthcheck
plugin for this

40. INFRASTRUCTURE ANALYTICS
• A deeper level of feedback for our
•
•
operations and infrastructure teams
Check CPU time, IO, JVM statistics,
power usage
Helps us diagnose platform related usage
and problems

41. PRODUCT ANALYTICS
• Statistics of new feature discovery and
•
•
usage
Consumed by product and plugin teams,
and their product managers
Helps us experiment and A/B test new
features in production

42. BUG REPORTS & SUPPORT CASES
• The last resort - we want to minimize this
•
kind of feedback
If an issue slips through to production,
and is not picked up by our automated
checking it will generally result in a
support case

43. • These four types of feedback
•
•
are all critical in pushing
information back to the
developers and PMs
Closes the delivery loop
An awesome delivery pipeline
is next to useless if you can’t
see what’s going on

44. #summit13
“
“
There are many types of feedback, these four
work for Atlassian. You’ll need to find out
what works for your team and processes remember: having feedback is crucial to
continual improvement.

45. The environments you deliver to should
continually be refining their output. The aim is
to deliver high quality software.
There are many types of feedback, these four
work for Atlassian. You’ll need to find out
what works for your team and processes remember: having feedback is crucial to
continual improvement.
#summit13
“
“
Release early. Release often. And listen to your
customers.

46. TYING IT ALL TOGETHER

47. Enables us to deliver new features to
35,535
customers every day, with confidence

48. Thank you!
RYAN THOMAS
•
DEVELOPMENT TEAM LEAD
@RYAN_ATLASSIAN •
•
ATLASSIAN
RTHOMAS@ATLASSIAN.COM

49. Rate this Talk
Delivering New Features to Over 30,000 Customers - Daily
Text code below to 22333
or visit http://bit.ly/19dYF9v
MEH = 25
NO T BA D = 26
P R ET T Y GO O D = 27
A WES O ME = 28
To join this session, send text 136888 to