A Tale of Two Pipelines: To DevOps or Not to DevOps Trying to move to a DevOps methodology, or improve your current DevOps methods? Need to increase innovation and the speed of your software delivery pipeline? In this webinar, Brett Hofer, Global DevOps Practice Lead at Dynatrace, will show you how. He’s led large-scale enterprise DevOps adoption projects at top organizations worldwide. Now he’ll give you a tour of two pipelines from his years in the field to illustrate how you can incorporate DevOps practices and rapid innovation into your everyday activities. Join us to see a pipeline in an environment where people are running around putting out fires, pointing fingers, operating in siloes and losing sleep. Contrast that view with another pipeline where teams have highly optimized communications, well-defined roles and unified, understood goals. Learn how to identify blinking spots of inefficiency in communication, architecture, and processes so you can escape the fire drills. Discover where to build in unity, better tools and automation. See how to embrace points of failure to increase success. Find out how top performers build and analyze the viability of their apps by improving the visibility of their app lifecycles. You’ll get insights on key pipeline factors that plague traditional delivery pipelines and how to overcome them with practices that establish a true DevOps culture for a faster and less troublesome continuous delivery pipeline. Brett Hofer Global DevOps Practice Lead at Dynatrace Brett Hofer is as passionate about DevOps and specializes in delivering complex mission-critical software under methodologies such as agile, lean and waterfall (to name a few). Brett has a strong record of success in managing and delivering projects with complex technical and political challenges. With more than twenty years of broad experience—from product designer and solution architect to senior management—he has a unique 360° perspective on IT and the business it supports.

1. A Tale of Two Pipelines:
To DevOps or Not To DevOps

2. Brett Hofer
Global DevOps
Practice Lead
25+ years of IT experience
Dynatrace
• Recognized thought leader DevOps & Big Data
• Featured speaker, presenter, writer - DevOps.com,
DevOpsSummit Journal, InfoWorld and many more
• Author eBook – “The Art of DevOps”
Prior to Dynatrace
• Senior app manager for 80M+ budget, staff of 70+
including onshore/offshore
• Lead designer and developer of multi-platform
apps, multi-channel intranet/internet web
• Senior Solutions Architect on 60M+ initiative(s)

3. Welcome to
“A Tale of Two
(very different)
Pipelines”

4. It’s about people,
processes and
technologies!

5. Are you sinking,
surfing, or just
surviving?

6. Our idea here will be to help identify
bad situations by relating to all or part of
the two stories …

7. We’ll aim to ignite ideas of
positive change by sharing
experiences through these
stories.

9. Which pipeline will look like yours?
Maybe somewhere in the middle?

10. What can you do to
influence change?

11. What exactly are we moving through the pipeline?

12. Servicing requests from business ideas, non
functional needs, fixes, functionality adjustments

13. Servicing requests from business ideas, non
functional needs, fixes, functionality adjustments

14. Servicing requests from business ideas, non
functional needs, fixes, functionality adjustments
Not every change is
created equal!

15. Success equals the RIGHT change at the RIGHT time
• Precision in priority ensures
all efforts are focused on
what team has agreed is
success!
• Getting people to agree can
be tough.
• Key: invest in “best-of-breed”
technology to track,
collaborate, and continually
monitor FACTS versus
OPINIONS when prioritizing

16. Typical top priority factors
• Value
• Cost
• Risk
• Impact
• Time

17. Add continuous monitoring to deliver
continuous improvements
• Find opportunities and
improve value
• Identify / manage cost
factors
• Reduce and mitigate
risk
• Identify and manage
impacts
• Find problems fast and
reduce time, costs and
risk

18. All the right people know the health status of a
change and build as it moves through the pipe
• Test and monitor early and often
• Find and fix problems fast and early

19. To DevOps or not to DevOps
Fingers
crossed…

20. Welcome to easyTravel

21. Debute latest online
booking system
• Development over a year
• Lots of internal jokes
“neverTravel”
• Ops “hope they load tested”
• Late and over budget
• App manager changed twice
• Tension between business
and dev stalled budget twice

22. Mark Smolten
Chief Marketing Officer
• 2 years at easyTravel
• Other successful roles driving biz up
30%
• Challenged by
• channels against fierce competition
• convincing exec management to
invest in new, unproven outlets
• not enough info on current
platform use and state
• not being tied into new system in
dev

23. • Mark contacts TravelVision and
lines up a spot on their wildly
popular live TV Show “Book Me
Now!”
• Mark offers viewers an amazing
deal on a live TV promo code in
exchange for debuting the new
easyTravel site
• Live show November 7th, 8:00
p.m.
• Yay, Mark!
TravelVision’s BookMeNow!

24. Tina Anton
VP, eCommerce
• 8 years at easyTravel, 15 years industry
• Promoted to head of eCommerce
two years ago
• More experienced in the travel
business than eCommerce
• Oversees the overall vision of every
easyTravel eBusiness function
• Maintains online budgeting decisions
• Very friendly, but long history of
battling it out with IT

25. Mark: Enthusiastic | Tina: Concerned

Mark speaks with Tina
about his Live Promo Deal with TravelVision.
He explains the code should affect pricing only for the
duration of the show and only for those who provide
the code issued during live broadcasting.


26. Mark: Enthusiastic | Tina: Concerned

Mark speaks with Tina
about his Live Promo Deal with TravelVision.
He explains the code should affect pricing only for the
duration of the show and only for those who provide
the code issued during live broadcasting.


Tina mentions that they don’t have such a feature on
the site and wishes he had consulted with her first.
She asks him “when is this?”


27. Mark: Enthusiastic | Tina: Concerned

Mark speaks with Tina
about his Live Promo Deal with TravelVision.
He explains the code should affect pricing only for the
duration of the show and only for those who provide
the code issued during live broadcasting.


Tina mentions that they don’t have such a feature on
the site and wishes he had consulted with her first.
She asks him “when is this?”


“We air on live TV to show off the new site and the
promo on November 7th at 8pm! It’s the only time
until next fall that they will have this event.”


28. Mark: Enthusiastic | Tina: Concerned

Mark speaks with Tina
about his Live Promo Deal with TravelVision.
He explains the code should affect pricing only for the
duration of the show and only for those who provide
the code issued during live broadcasting.


Tina mentions that they don’t have such a feature on
the site and wishes he had consulted with her first.
She asks him “when is this?”


“We air on live TV to show off the new site and the
promo on November 7th at 8pm! It’s the only time
until next fall that they will have this event.”


Tina, very apprehensive, explains “but that’s less than
two months away! We don’t even have a promo
feature on the site since all our travel packages are
already discounted.”


29. Mark: Enthusiastic | Tina: Concerned

Mark states “This is the business nowadays. We have
to adapt quickly to capture market opportunities. The
discounts will come from the marketing budget. We
need to make this happen!”


30. Mark: Enthusiastic | Tina: Concerned

Mark states “This is the business nowadays. We have
to adapt quickly to capture market opportunities. The
discounts will come from the marketing budget. We
need to make this happen!”


Tina, continues to voice her concerns trying to explain
that there is a big change process for something like
this.


31. Mark: Enthusiastic | Tina: Concerned

Mark states “This is the business nowadays. We have
to adapt quickly to capture market opportunities. The
discounts will come from the marketing budget. We
need to make this happen!”


Tina, continues to voice her concerns trying to explain
that there is a big change process for something like
this.


“Mark, things are also more complicated these days
due to all the channels we support. This would need
to be available on Web, Web Mobile and in our native
Mobile apps.”


32. Rick Trenton
Senior Project Manager
• Consultant with the company one year
• Ex-military very structured and organized
• Assigned to the project management of
easyTravel Web and mobile
• Brokers budgets and costs between
business and IT for major deliverables
• Challenged with producing weekly
project statuses to management in
PowerPoint
• Regularly maintains high level tasks in his
Microsoft Project Plan

33. Tim Folk
Director of Digital Services
• 5 years at easyTravel
• Former Senior Solutions Architect
• Inherited responsibility 4 months ago
• Not a fan of his Management Admin
Tasks
• Manages all the Development of Web
& mobile channels for easyTravel
• Challenged with lack of time, too many
meetings, putting out fires
• He manages a local smaller IT staff but
required to use at least 70% offshore
staffing for his dev to keep costs down

34. Mark: Enthusiastic | Tina: Concerned

Tina now contacts both Rick and Tim to explain this
new Feature that must be ready and tested in all
channels by November 7th. Less than two months
away.


35. Mark: Enthusiastic | Tina: Concerned

Tina now contacts both Rick and Tim to explain this
new Feature that must be ready and tested in all
channels by November 7th. Less than two months
away.


Rick’s PM contract is funded by the business so he is
very appeasing to Tina while Tim is drowning in
numerous problems. Tim explains “I’m worried about
the viability of the entire new site by November never
mind this new feature.”


36. Mark: Enthusiastic | Tina: Concerned

Tina now contacts both Rick and Tim to explain this
new Feature that must be ready and tested in all
channels by November 7th. Less than two months
away.


Rick’s PM contract is funded by the business so he is
very appeasing to Tina while Tim is drowning in
numerous problems. Tim explains “I’m worried about
the viability of the entire new site by November never
mind this new feature.”


“Well you guys know the marching orders. I have
confidence you will get this done. Keep me posted on
the progress and take care.”


38. Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
Tim contacts
three key members
critical to
getting this
done.

39. Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
Digital Services Operations Lead
easyTravel
Kelly explains
her understanding
of the feature
and it’s urgency.

40. Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
Digital Services Operations Lead
easyTravel
“Ok Kelly, we are
going to need to
get this fully
documented and
exactly how it
should work.”

41. Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Hey Tim, I’m
going to need to
put this on a
completely
separate code
branch and we
will need another
environment. It’s
going to impact
too many things
to put in the Oct
release.”

42. To understand Tim’s concerns,
let’s look at the app pipeline he owns …
Fingers
crossed…

43. Tale of Two Pipelines
by Brett Hofer
Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Hey Tim, I’m
going to need to
put this on a
completely
separate branch
and we will need
another
environment. It’s
going to impact
too many things
Scott’s Traditional Reasoning on incorporating this changePROMO
FEATURE
CHANGE
easyTravel Master CODE Branch v2.0
easyTravel CODE Branch v2.2 (Oct)
easyTravel CODE Branch v2.5 (Jan)
easyTravel CODE Branch v3.0 (Mar)
DEV1
vm
DEV2
vm
DEV3
vm
DEV4 (ER)
vm
QA1
vm
QA2
vm
QA3
vm
QA4 (ER)
vm
UAT
vm
PERF
vm
PROD A
vms
PROD B
vms
merge
Hosts 2.2 build
Hosts 2.5 build
Hosts 3.0 build
Hosts Fix Builds
Hosts 2.2 build
Hosts 2.5 build
Hosts 3.0 build
Hosts Fix BuildseasyTravel CODE Branch v2.3 (Nov)
promotions
PROMO Only Dev related Code

44. Tale of Two Pipelines
by Brett Hofer
FEATURE
CHANGE
PROMO Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Haha, we have
about as much
chance of getting
new machines
provisioned in
that timeframe as
I have in getting
a decent build
from Scott.”

45. Tale of Two Pipelines
by Brett Hofer
FEATURE
CHANGE
PROMO Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Hey Frank, it
might be easier
if the machines
you get me
weren’t as slow
as you. I’m
tempted to have
our guys bring in
their home
machines and
deploy to those.”

46. Tale of Two Pipelines
by Brett Hofer
FEATURE
CHANGE
PROMO Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Honestly, if I
put in a ticket
to provision new
VMs with the
infrastructure
department it
will have mold on
it by November.”

47. Tale of Two Pipelines
by Brett Hofer
Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Hey Tim, I’m
going to need to
put this on a
completely
separate branch
and we will need
another
environment. It’s
going to impact
too many things
Frank’s Traditional Ops Reasoning on procuring new VMsPROMO
FEATURE
CHANGE
DEV1
vm
DEV2
vm
DEV3
vm
DEV4 (ER)
vm
QA1
vm
QA2
vm
QA3
vm
QA4 (ER)
vm
UAT
vm
PERF
vm
PROD A
vms
PROD B
vms
Hosts 2.2 build
Hosts 2.5 build
Hosts 3.0 build
Hosts Fix Builds
Hosts 2.2 build
Hosts 2.5 build
Hosts 3.0 build
Hosts Fix Builds
promotions
PROMO
Enter a Dev 5
Provisioning Ticket
Enter a QA 5
Provisioning Ticket
Outsourced Company Managing
Environments locates frame & Builds Host
according to security, OS, CPU, Memory.
Outsourced Company Managing
Environments locates frame & Builds Host
according to security, OS, CPU, Memory.
Franks staff will
manually setup
the DEV environment
for deploying Nov
Franks staff will
manually setup
the QA environment
for deploying Nov
DEV5
vm
Hosts Fix Builds
QA5
vm
Hosts 2.3 PromoLead Time to identify, provision and get cost contract – 30 to 60 days

48. Tale of Two Pipelines
by Brett Hofer
FEATURE
CHANGE
PROMO Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Guys, maybe we
should just
concentrate on
finding a
solution that
will get us
there. This needs
to get done.
Kelly let us know
when you have our
requirements.”

49. Tale of Two Pipelines
by Brett Hofer
FEATURE
CHANGE
PROMO Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Wow, it sounds
like you guys
have your work
cut out and on
that note, thanks
Tim, we will have
your requirements
in a few days”

50. Now let’s
evaluate this
feature as it
moves through
their pipeline.

51. The good news – the pipeline is
somewhat agile
Plan
Design
DevelopTest
Operate

52. The bad news – all the cycles are long!
3 month cycles now require an interim release
Plan
Design
DevelopTest
Operate

53. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking

54. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking
CMO Recognizes Major Opportunity Initiates Need
FEATURE
CHANGE
• Industry is fast past and
always fluid.

55. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking
High Level Business Functionality
• All Channel must change
• Actually 3 Pipelines affected
• Mechanics of how it will work

56. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking
Separately Feature tracking ++Risk, +Time, +Cost
• Dev uses tracking software, but no
other groups do
• Frank adds the task to his tracking
spreadsheet

57. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Design
Testing
Strategy
Gather
Time &
Cost
Track on
Overall
Plan

58. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Design
Testing
Strategy
Gather
Time &
Cost
Track on
Overall
Plan
Senior Quality Assurance Lead
easyTravel Contractor
- His company contracted by QA department
- Follows his QA dept and corporate guidelines
- Needs very specific detailed technical specs
to put his team in motion
- Runs a large team of offshore manual testers

59. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Design
Testing
Strategy
Gather
Time &
Cost
Track on
Overall
Plan
Design Feature Impact Assessment - +++RISK, +++TIME, +++COST
• Done conversationally among
a few technical resources
• Frequent inaccuracy and guessing.
Has major cascading effects on
capacity, architecture,

60. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Design
Testing
Strategy
Gather
Time &
Cost
Track on
Overall
Plan
Architecture of Feature - +++RISK, +++TIME, +++COST
• Done conversationally among
architects
• Many know the general architecture
but aren’t up to speed on all of the
detailed changes over each release.

61. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Story
Board
Identify
Feature
Impacts
Design
Testing
Strategy
Gather
Time &
Cost
Track on
Overall
Plan
Capacity Planning - +++RISK, +++TIME, +++COST
• Lacks comprehensive understanding
of site traffic and visits
• Lacks comprehensive understanding
of scalability concerns
Capacity
Design
Plan

62. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
PROMO
Test

63. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
Using Offshore Development - +RISK, ++TIME, +COST
• Timezone diff often = long response
times for fix & feedback cycles
• Lack of Backend System Support

64. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
Developers on varying Platforms - +RISK, +TIME
• Not developing on the same
technical stack as intended deploy
targets - introduces unnecessary
risks

65. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
Overuse of Merging - +RISK, +TIME, +COST
• Increased manual effort on changes

66. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
Too many premature Check-Ins - ++RISK, ++TIME, ++COST
• No static policy checking on dev code
• No runtime policy checking on dev code
• No code review on critical functions
• No architectural validation

67. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
Manual Run Builds - +RISK, +TIME
• Often requires contacting one person
• Limited Visibility and Feedback of
build results

68. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
Slow Build Time - +TIME
• Build is taking over 50 minutes

69. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Diagnose
Dev4
9% Low and Limited Unit Coverage - ++RISK, -VALUE
• Limited Visibility and Feedback
• Places high/all reliance on functional
testing – Which is manual!!!

70. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
PROMO
Dev4
Limited Result Visibility - +RISK, +TIME
• Only one or two people
• Manual skimming of build logs
• Often too much promoted “thrown
over the wall”
Diagnose

71. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
Manual Changes to Environments - +RISK, -VALUE
• Prone to cross environment errors
• Manual adjustments forgotten and
often not repeated as necessary
• Static environments get dirty
• Kills innovation speed
PROMO
Check
Results
Dev4
Diagnose

72. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
Test
Manual Deployments in Development - +TIME, -VALUE
• More manual delays in promotion
PROMO
Dev4
Check
Results
Diagnose

73. Tale of Two Pipelines
by Brett Hofer
Dev
Communication spreadsheets/meetings/phone/emails
Dev
DevOps
FEATURE
CHANGE
Code Check-In Run
Build
Manual
Dev Env
Config
Manual
Deploy
Check
Results
Manual
Merge
Unit Test
Test Prod
Testers
Manually
Prepare
Test Data
Document
Test Scripts
Manual
Stage Env
Config
Manual
Prod Env
Config
PROMO
Test
Dev4Dev4
Diagnose
Scott’s Dev Visibility & Status Sharing
• Few actually know the build health in Dev
• Few actually know common Dev problems
• Too little screened
• Lacks macro level metrics

74. Tale of Two Pipelines
by Brett Hofer
Dev
spreadsheets/meetings/phone/emails
Dev
DevOps
Test Prod
Testers
Determine
Test
Suite
Manual
Deploy
QA 5
Prod
FEATURE
CHANGE
PROMO
Manual
Tests
Enter
Problems
Manual
Deploy
Perform
Manual
Tests
Report
Problems
Validate
Build
Validate
Build
Diagnose
Root
Cause
Check
Infrastruct
ure
Manual
Deploy
UAT

75. Tale of Two Pipelines
by Brett Hofer
Dev
spreadsheets/meetings/phone/emails
Dev
DevOps
Test Prod
Testers
Determine
Test
Suite
Manual
Deploy
QA 4
Prod
FEATURE
CHANGE
PROMO
Manual
Tests
Enter
Problems
Manual
Deploy
Perform
Manual
Tests
Report
Problems
Validate
Build
Validate
Build
Diagnose
Root
Cause
Check
Infrastruct
ure
Manual
Deploy
UAT
Manual Test Selection - ++TIME, --VALUE, ++COST
• Selecting a lot more manual testing
than what the change requires stifles
new changes and costs time
• Manual Testing Greatly slows
feedback time.

76. Tale of Two Pipelines
by Brett Hofer
Dev
spreadsheets/meetings/phone/emails
Dev
DevOps
Test Prod
Testers
Determine
Test
Suite
Manual
Deploy
QA 4
Prod
FEATURE
CHANGE
PROMO
Manual
Tests
Enter
Problems
Manual
Deploy
Perform
Manual
Tests
Report
Problems
Validate
Build
Diagnose
Root
Cause
Check
Infrastruct
ure
Manual
Deploy
UAT
MTTR SLOW - ++TIME, --VALUE, ++COST
• Test usually reported as
just failed with confusing
description
• Finding exact issue is hard
• Wasted Time - Finger
Pointing
• Unnecessary people
involved in failure
Validate
Build

77. Almost 2
months later
. . .

78. Now it’s prime
time November 7
at 8:10 p.m.

79. Tale of Two Pipelines
by Brett Hofer
Hi everyone, I’m Lisa
Goodheart, your host of
Book Me Now!

80. Tale of Two Pipelines
by Brett Hofer
Today is our Big Deal day
where once a year we
showcase a select travel
service live allowing them to
put their best foot forward
demonstrating great deals to
our viewers.

81. Tale of Two Pipelines
by Brett Hofer
Here with me is Eric
Wallace the VP of Sales
from easyTravel. He has a
special treat for you
today.

82. Tale of Two Pipelines
by Brett Hofer
Thanks Lisa, I’m honored
to be here and to show
your audience what we
believe is the most
exciting travel site on the
market.

83. Tale of Two Pipelines
by Brett Hofer
Our business negotiates
some of the most
incredible packages for
the cheapest prices and
then bundles them.

84. Tale of Two Pipelines
by Brett Hofer
Today we will demonstrate
an absolute exclusive for this
show. If your audience
jumps on our site either web
or on their phone and uses
this one time PROMO code.
Your Book Me Bo

85. Tale of Two Pipelines
by Brett Hofer
Your Book Me Now Exclusive Live Promo
It will randomly discount
our packages between 5%
to 95% upon searching and
if you begin to book it
upon that search you can
book that deal.

86. Tale of Two Pipelines
by Brett Hofer
Your Book Me Now Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Bo
Thank you Eric. This is
really exciting. Our
audience will now notice
the Promo Code scrolling
on the bottom of their
screen.

87. Tale of Two Pipelines
by Brett Hofer
Let’s demonstrate
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi

88. Tale of Two Pipelines
by Brett Hofer
We put in the
Promo
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE

89. Tale of Two Pipelines
by Brett Hofer
Where we want to
take our Journey
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize

90. Tale of Two Pipelines
by Brett Hofer
When we want to
go…
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize
12/18/2016
1/5/2017

91. Tale of Two Pipelines
by Brett Hofer
Now click search…
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize
12/18/2016
1/5/2017

92. 20 seconds
later . . .

93. Tale of Two Pipelines
by Brett Hofer
Um this is
unusually long…
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize
12/18/2016
1/5/2017

94. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize
12/18/2016
1/5/2017
Wow, your site must
be working pretty
hard on those deals.

95. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize
12/18/2016
1/5/2017
Yes, it’s chugging
away to find your
viewers the
deals… is it
possible it’s your
internet here?

96. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE

97. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
And… that doesn’t
look good. Maybe a
commercial?

98. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Well folks we are
sorry but it looks like
easyTravel is running
into a technical
difficulty. We will be
back after these
messages…

99. Tale of Two Pipelines
by Brett Hofer
ROLLTHEDICE
Belize
12/18/2016
1/5/2017

100. Tale of Two Pipelines
by Brett Hofer
ROLLTHEDICE
Belize
12/18/2016
1/5/2017
Well that was a
complete waste of
time and money. You
people need to get
your act together.

101. Tale of Two Pipelines
by Brett Hofer
ROLLTHEDICE
Belize
12/18/2016
1/5/2017
You’re not kidding, heads
will roll for this one. That
was embarrassing and
expensive.

102. Tale of Two Pipelines
by Brett Hofer
ROLLTHEDICE
Belize
12/18/2016
1/5/2017
And that had nothing
to do with our
internet.

103. Tina’s house . . .
Are you
kidding me!

104. And the War Room begins . . .
• Not only was all the money spent
on the feature a loss but it
turned out the 60% booking
growth was a long term growth
strategy for Mark the CMO.
• The actual event traffic was
1600% of peak.
• An estimated $4M in new net
bookings was lost.
• Losses due to brand damage
unquantifiable.

105. Now let’s walk through this same scenario in a DevOps
culture . . .

106. Let’s see the story
when the promo
feature is sent
through their new
DevOps pipeline.

107. The good news – the pipeline IS Agile
2 Week Sprints
Plan
Design
DevelopTest
Operate

108. The new Digital Services Environment
Pipeline Transparency, Proximity & Availability Matters
Test

109. Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Hey Tim, I’m
going to need to
put this on a
completely
separate branch
and we will need
another
environment. It’s
going to impact
too many things
The DevOps Reasoning on incorporating this change (Use a feature toggle)
easyTravel Release
Ops Infrastructure As Code
Performance and Dashboards
PROMO
Dev Source Code
Test Scripts
CI_CD Configs
CLOUDAPI
LOAD BALANCE POLICY
easyTravel
Instance
Tech Stack
Image for 2.0
Tech Stack
Image for 3.0
Tech Stack
Image for 3.5
PROMO
FEATURE
CHANGE
Cloud Image Library

110. Senior Business Analyst
easyTravel
- Works for Tina
Digital Services Dev Lead
easyTravel
- Works for Tim
Digital Services Operations Lead
easyTravel
- Works for Tim
“Hey Tim, I’m
going to need to
put this on a
completely
separate branch
and we will need
another
environment. It’s
going to impact
too many things
The DevOps Reasoning on incorporating this change (Use a feature toggle)
easyTravel Release
Ops Infrastructure As Code
Performance and Dashboards
PROMO
Dev Source Code
Test Scripts
CI_CD Configs
CLOUDAPI
LOAD BALANCE POLICY
easyTravel
Instance
Tech Stack
Image for 2.0
Tech Stack
Image for 3.0
Tech Stack
Image for 3.5
PROMO
FEATURE
CHANGE
Cloud Image Library
Aligning All Release Source Code,
Automation Scripts, Database Scripting,
Performance and Operational Assets in Source
Control Is critical.
It is the PIPE!
Build Once, Deploy and
Continuously Monitor in any environment

111. Senior Business Analyst
easyTravel
- Works for Tina
“Hey Tim, I’m
going to need to
put this on a
completely
separate branch
and we will need
another
environment. It’s
going to impact
too many things
The DevOps General FlowPROMOPROMO
FEATURE
CHANGE
Test EnvironmentDev Environment
Developer
Local
Environments
Source Control
Production Environment
Dynatrace in Dev Dynatrace in Stage Dynatrace Prod
Load, Int, Func Tests
Report Results
Env Automation
Deploy Automation
Build Server
Perform Build
Run Unit Tests
Report Results
Environment Setup
Deploy Automation

112. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking

113. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking
CMO Recognizes Major Opportunity Initiates Need
FEATURE
CHANGE
• Industry is fast past and
always fluid.

114. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking
High Level Business Functionality
• All Channel must change
• Actually 3 Pipelines affected
• Mechanics of how it will work

115. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITMarket
Plan DesignFEATURE
CHANGE
PROMO
PMO
Design
Creates
Need
Validates
Sets
Urgent
Identifies
Biz Details
High Level
Estimate
Initiates
Tracking
Initiates
Tracking
Aligned all Release Tracks and Sprint Tasks
• BA
• Dev
• Ops
• Test
• Data
• Security
• Automation
• XA

116. Tale of Two Pipelines
by Brett Hofer
DevBizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Autom
Architect
Design
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Design

117. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Autom
Architect
Design
Design Feature Impact Assessment
• Leveraging Dynatrace runtime Visualizations on current
relationships creates high accuracy on Impact decisions.

118. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Autom
Architect
Design
Design - Feature Impact Assessment
• Leveraging Dynatrace Actions provides high accuracy on page
Impact decisions.

119. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Autom
Architect
Design
Architecture of Feature
• Leveraging Dynatrace runtime Visualizations on
current relationships creates high accuracy on
architectural decisions.

120. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Autom
Architect
Design
Architecture of Feature
• Leveraging Dynatrace end-to-end transactional
visibility creates high accuracy on architectural
decisions.

121. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
DevBizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Automation
Architect
Design
eXperience Assurance Decisions
• Leveraging all of the Dynatrace User Experience
Management Data

122. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Automation
Architect
Design
Automation Architecture - ++Time, ++Cost, ++Risk
• Role in designing and managing all
automation to move the feature
through the pipeline.
• Leveraging Technologies to Automate
Processes & Orchestration in CI & CD

123. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
BizITQA
Plan Design FEATURE
CHANGE
PROMO
PMO
Design
Identify
Feature
Impacts
App
Architect
Capacity
Design
Plan
Story
Board
Identify
Feature
Impacts
Gather
Time &
Cost
Track on
Overall
Plan
Test
Design
Plan
XA
Design
Perf
Architect
Review
Identify
Feature
Impacts
Automation
Architect
Design
Performance Architecture
• Performance Architect Reviews
Performance & Scalability
Concerns of the Feature
• Designs Dashboarding Strategy for
Monitoring Feature & Health

124. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
PerfEng
Create
Perf
Dashboard
Instrument
App
Unit TestCI
Server
Check
Results

125. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
PerfEng
Create
Perf
Dashboard
Instrument
App
Unit TestCI
Server
Check
Results
BIG GOALS
• How fast can we determine a check-in will fail and get feedback to the right resource
• How far can we progress a healthy build/app without manually touching it
• How fast can we identify the problem and turnaround the fix

126. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Using Onsite & Localized Development - -RISK, --TIME, -COST
• Collaborate Often
• No Black Box Development
• Culture
Check
Results
PerfEng

127. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Developers use Cloud Instances - -RISK, -TIME
• Developer Instances match target
deployment stack.
• Easy to onboard other developers
• Allows switching technology stacks
Check
Results
PerfEng

128. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Check
Results
Architectural Validation - --RISK, --TIME, +Value
• Developers, Leads and Solution Architects use Dynatrace to
check transaction validity prior to check-in. Great on critical
transactions and faster than code reviews.
PerfEng

129. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
Create
Perf
Dashboard
Create
Perf
Dashboard
Check
Results
Unit TestCI
Server
Using a feature toggle - -TIME, -COST
• Increased manual effort on changes
PerfEng

130. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit
Test
CI
Server
Check
Results
PerfEng
Code Coverage Static Code Checking - -RISK, -TIME, -COST
• Manage Testing Code Coverage
• Govern Coding Practices
• Code Coverage is on Main
Dashboards
JaCoCo
Test

131. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit
Test
CI
Server
Source Control Respository - --RISK, -TIME, -COST
• Aligning ALL Release assets
is critical
Check
Results
Test

132. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Build Automation - -RISK, -TIME
• Comprehensive Build Scripts
• Include Dependency Resources
• Leverage Profiles (Build script
is host aware)
• Includes Monitoring of Tests
Check
Results
Ant
Test

133. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Automated Builds- -RISK, -TIME
• Scheduled off a check-in trigger
• Orchestrate tests and deployments
• Steps Exec Build Automation Goal
Check
Results
Test

134. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Fast Build Time - -TIME
• Build takes only 15 minutes
Check
Results
Test

135. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CI-CD
Prov Env
Check
Results
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit TestCI
Server
Unit & Unit Integration Testing - -RISK, -TIME
• 78% Coverage
• All Unit Tests are monitored by
Dynatrace with Results maintained
by Build
Check
Results
Test

136. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
FEATURE
CHANGE
Code Check-In CI
Server
Setup
Release
Blueprint
CI-CD
Prov Env
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Unit Test
Check
Results
• dynaTrace Monitored Unit Results - --RISK, --TIME
•
• Regression Violations generate change actions in tracking software

137. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Release
Cloud
Blueprint
CI-CD
Prov Env
Unit Test
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Developer
Cloud
Blueprint
Manual
Merge
Code
Config
Mgmnt
PerfEng
Create
Perf
Dashboard
Create
Perf
Dashboard
CI
Server
Check
Results
Configuration Management --RISK, --TIME
• All ops/infrastructure configs are represented in code and
aligned with the release
• EnvForRelease = Cloud/Container Blueprint + ConfigCode

138. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CI-CD
Prov Env
Unit Test
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
TestEng
Create
Perf
Dashboard
Create
Perf
Dashboard
Automated Cloud/Container Provisioning - --RISK, --TIME
• Establish Instance to Deploy
CI
Server
Check
Results

139. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CI-CD
Prov Env
Unit Test
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
PerfEng
Create
Perf
Dashboard
Create
Perf
Dashboard
CI
Server
Check
Results
Automated Test Development- --RISK, --TIME
• All testers are test automation engineers.
• All tests are integrated with Dynatrace
APM
Silk Performer
Selenium

140. Tale of Two Pipelines
by Brett Hofer
Dev
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
DevOps
FEATURE
CHANGE
Code Check-In
Setup
Release
Blueprint
CI-CD
Prov Env
Unit Test
Test Prod
TestEng
Manually
Prepare
Test Data
Create
Automation
Scripts
PROMO
Test
Diagnose
Dev4Setup
Developer
Blueprint
Manual
Merge
Code
Config
Mgmnt
PerfEng
Create
Perf
Dashboard
Create
Perf
Dashboard
CI
Server
Check
Results
Performance Dashboards are Deliverables --RISK, ++VALUE
• Major Features need Visibility
• The dashboard indicates if the PROMO feature
is being used, response times, inter-dependency
health or any errors related. Watched throughout
lifecycle.

141. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
Test Prod
TestEng
Auto Run
Full
Test Suite
Env
Ready
Prod
FEATURE
CHANGE
PROMO
Manual
Tests
Report
Problems
Validate
Build
Validate
Results
Diagnose
Root
Cause
Check
Infrastruct
ure
Selective
Load
Test

142. Tale of Two Pipelines
by Brett Hofer
FEEDBACK LOOP dashboards/emails/tracking technology/group chat
Dev
Dev
DevOps
Test Prod
TestEng
Auto Run
Full
Test Suite
Env
Ready
Prod
FEATURE
CHANGE
PROMO
Manual
Tests
Report
Problems
Validate
Build
Validate
Results
Diagnose
Root
Cause
Check
Infrastruct
ure
Selective
Load
Test
Finding Root Cause –TIME, ++VALUE

143. The Team under DevOps was able
to do four iterations and a full load test
three weeks ahead of schedule.
The feature has been in.

144. Tale of Two Pipelines
by Brett Hofer
Now click search…
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
Belize
12/18/2016
1/5/2017

145. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
11/29/2016
12/5/2016
And there you go
an 80% discount
on a trip to
Belize…

146. Tale of Two Pipelines
by Brett Hofer
Exclusive Live Promo Code is ROLLTHEDICE for cheap travel.. Your Book Me Now Exclusi
ROLLTHEDICE
11/29/2016
12/5/2017
Wow, that is fast and
incredible. I’m sure our
viewers want to thank
easyTravel. I’m being
told this is our highest
viewed show to date!

147. The DevOps
Xcelerator
Confidential, Dynatrace LLC
Master
Performance
Architect

148. Confidential, Dynatrace LLC
1
2
3
What We Do

149. Confidential, Dynatrace, LLC
Questions or to Inquire about the DevOps Engagement email:
brett.hofer@dynatrace.com
Please put TALES WEBINAR in the Subject Line