Using Metrics in Value Streams to Align Business and IT

EGUIDE: USING METRICS IN VALUE STREAMS TO ALIGN BUSINESS AND IT 1
Using Metrics in Value Streams
to Align Business and IT
E
G
U
I
D
E

This guide looks at how to achieve that alignment
at each step from idea to realization. Taking a
value stream-centric approach, this guide provides
a framework for establishing metrics that drive
continuous improvement throughout value streams.
Read on to learn:
Agile and DevOps have both had meaningful impacts on technology teams.
However, in order to better realize business objectives, their next evolution
must renew their focus on the alignment of business and technology.
INTRODUCTION
To identify and organize metrics in your value streams
Why risk, velocity, quality, and performance metrics help IT
align with business outcomes
How to conduct your value stream metrics program to
drive success year after year

Traditional business operations
are based heavily on a top-down
structure that relies on managers
exerting considerable control over
processes to drive compliance.
METRICS IN VALUE STREAMS
In this structure, organizational objectives, including IT’s corresponding key
results or key performance indicators (KPIs), are defined by leadership.
Each IT process owner then monitors and manages their processes to
improve their isolated metrics. The expectation behind this approach is that
a company will achieve superior organizational performance when owners
optimize each process for its associated KPIs.
Modern software development challenges this traditional mindset. It
is inherently an innovative and collaborative process that continuously
produces new products and services. As a result, optimizing IT processes for
these top-down defined KPIs has not always yielded the intended results.
At the same time, more companies need their internal software
development teams to drive customer value. In order to support this every
team needs to be working toward customer-centric business outcomes.
To achieve these outcomes while driving collaboration and innovation,
software development teams are beginning to look to value streams -
everything needed to deliver a product or service - to break down silos
between teams and align to business goals.
Value stream-centric thinking enables organization-wide business objectives
to meet with bottom-up KPIs. This ensures alignment between the two while
improving software development and delivery.
Value Stream Centric Design
Plan Design Code Test Deploy Monitor
VALUE STREAM
AGILE DEVOPS

The DevOps Culture
as a Foundation
In the traditional model, each function in IT - from
planning to development to operations - had their
own goals, though they were often directly at odds
with each other. Development teams sought to
accelerate releases while operations focused
on stability.
The DevOps movement, in contrast, seeks to break
down these silos between teams. It establishes
a culture where disparate groups, collaborating
together, can drive superior value to customers.
DevOps is primarily about culture. It’s not about
tools, technologies, or frameworks. However,
simply knowing that fact isn’t enough to set up
most companies for success.
Creating openness and
transparency in and
among teams so they
stay aligned on goals.
SHARING
Requiring the will to
automate as many
manual tasks as
possible to save time
and reduce errors.
AUTOMATION
Steering the organization
to collect data on its
technology processes,
including development,
testing, deployment,
change management,
and operations.
MEASUREMENT
Calling for management
support combined
with cross-functional
teams committed to a
shared goal.
CULTURE
Bringing in the principles
of lean software flow to
manage work in progress,
eliminate waste, and
minimize handoffs.
LEAN
The CALMS model
was created to
help companies
with successful
DevOps adoption
and application.
It outlines the
main DevOps
principles as:

Using the CALMS model can guide organizations
toward a successful DevOps adoption, but it
leaves out a critical piece — it says nothing in
particular about business alignment.
Ultimately, any technology initiative that lacks business support will not
achieve any meaningful results. This isn’t a new challenge. But through
a strong DevOps culture and the correct use of metrics companies can
finally achieve this alignment.

How are any of those changes in IT going to help me reach
my business goals?
How can I optimize my investments in certain products
or teams?
How can I know if my investments are paying off or are
in trouble?
Business Outcome and IT Metric Alignment
Business leaders, no matter how much
they might be interested in technology,
are focused on business outcomes. Most
often, we think about outcomes in terms
of revenue, but there can be others
depending on the company’s strategies
and short-term goals.
Business leaders may be happy to hear
that IT plans to collaborate internally more
effectively leveraging DevOps principles
and that doing so will lead to certain
improvements. But what they really want
to understand are these points:
DevOps has little to say about these points without a considerable
amount of interpretation by technology leaders.
This gap has led to value streams gaining
traction in Agile and DevOps teams.
A company may have dozens of value
streams supporting their products or
services. For example a healthcare
company may have one value stream
A
B
C
supporting its mobile app, which is connected to
another value stream supporting its customer
database. Regardless of the number of value
streams, the performance of each value stream is
defined by its ability to achieve the target outcomes
the business assigned to it.

Technology doesn’t exist for its
own ends, but to enable the
business to deliver customer
value.
IDENTIFYING AND ORGANIZING YOUR METRICS
Technology doesn’t exist for its own ends, but to enable
the business to deliver customer value. For this reason,
technology can’t operate within a silo where IT and product
leaders evaluate its performance (based on technology
metrics) separately from the overall business outcomes.
Value streams, as end-to-end cross-functional systems,
bring business and IT objectives together. And value stream
metrics are how business and IT ensure the value stream is
delivering effectively.
As a business leader, they may not have a deep
knowledge of technology, but they absolutely
must understand the levers that can help them
reach their goals. And in the truest spirit of DevOps,
this has to take place in a culture that drives the
business and technology toward more collaboration
and interaction.
Value streams are necessarily customer-centric and cross-functional.
Because of this, they open up even more opportunities for collaboration, but
now between business and IT.
However, still the problem remains: how can business leaders know that the
value stream is delivering successfully?

We suggest a three-level framework for organizing your value stream metrics:
Value Metrics
Define the objectives of value
delivery for the value stream set by
leadership and product teams.
Pipeline Metrics
Establish four categories of software
delivery that support a breadth of
value stream activities set by product
and software development teams.
Tactical Metrics
Lay out how the Pipeline metrics will be
achieved. Each Pipeline metric has two
tactical metrics defined by software
development teams.
RISK
BUSINESS
OUTCOME
#1
BUSINESS
OUTCOME
#2
BUSINESS
OUTCOME
#3
B B B B
A A A A
VELOCITY QUALITY PERFORMANCE
1
2
3
Top Down
Bottom Up

Value Metrics
A Value metric is a business-level outcome for the value
stream to achieve. These customer-centric goals focus on
understanding the value delivered to customers and guide
Pipeline and Tactical metrics.
There should ideally be three Value metrics for any value stream. The
three metrics should be set to balance each other and work synergistically.
This is because if your goal is to optimize a single metric, you’ll be
incentivized to improve that one number leaving initiatives outside that goal
to be ignored - no matter how much value they could deliver to customers.
For example, think about driving your IT teams to improve availability
above all other metrics. It would certainly lead to the introduction of
additional control processes, but these new processes inevitably would slow
down the flow of value to the customer, resulting in worsened business
outcomes overall.
By using three Value metrics, you can establish
the necessary tension among each metric. Now
attempts to maximize (“game”) any one of them
will be offset by losses in the others unless your
teams are collaborating effectively to find solutions
that keep all three in balance.
It’s important for business leaders and product managers to set Value
metrics, with the business really taking the lead. The value delivered to
customers should reinforce the business’s goals and be reflected in these
three metrics.
1

Pipeline Metrics
Tactical Metrics
In this framework, four Pipeline metric
categories should be part of any value stream:
IT teams will then define two tactical
metrics that are directly actionable to
support each of the Pipeline metrics.
In our experience, these four categories best capture the principles that
underlie successful Agile and DevOps value stream delivery. Like the Value
metrics before, these four categories ensure coverage across functions and
introduce the necessary tension to balance initiatives.
Here product managers work with IT team leaders to identify a metric in
each Pipeline category that will together support the three Value metrics.
The power of using this framework is to show the business exactly how
aspects of the software align with their intended outcomes. This alignment
begins to build the connection between business objectives, customer value,
and IT initiatives to actually move the value stream in the right direction.
It’s these metrics that IT managers will be most focused on monitoring
and improving. When used as part of this framework, as IT teams
determine the Tactical metrics, they will drive collaboration instead of
pitting groups against one another.
2
3
Risk Quality
Velocity Performance

Let’s take a detailed look at the categories of metrics within the
framework to bring out specific metrics that organizations can use.
METRIC CATEGORIES
Some examples include customer usage measures, revenue targets, and
cost-saving objectives.
Different industries tend to have particular metrics that would also fit
here if the value stream owner is committed to them for the period. For
example, customer lifetime value or NPS are used by many software-as-a-
service (SaaS) companies.
Again, the key is to establish three Value metrics for each value stream
that represent different categories of business and product goals to
ensure a balance among them.
Since Value metrics are really business outcomes, they need to
be guided by business leaders’ as well as Product’s goals.
Value Metrics
Customers are able to see value in multiple products
through cross-selling
Customers are continually delivered value in existing
products to minimize churn
All resources are not spent on any one product initiative
by increasing cost efficiency
For example combining cross-selling, customer churn, and
cost efficiency initiatives ensure that:

Risk is everywhere. It’s part of every business, every process, and every
decision. Organizations seek to control risk to ensure they keep their customers’
trust and stay competitive.
But risk treatments have associated costs, meaning organizations must
analyze these costs and keep them in balance. Not every risk can or should be
addressed as a top priority. Risk metrics allow you to monitor and mitigate the
greatest threats to your organization.
The risk metric ensures you focus on the largest impact to your value stream.
For example, you can focus on increasing your compliance rate in releases,
but selecting that risk metric doesn’t mean it’s the only risk IT will address. It is
simply the risk objective you have lifted up in order to ensure it is achieved.
As mentioned, the four pipeline metrics that make up the
framework are risk, velocity, quality, and performance.
Pipeline Metrics
Risk
Within a value stream, technology risks take on
several potential forms:
Compliance risk where you must demonstrate you comply
with internal or external controls (including regulatory or
governmental ones). For example, you may need to provide a
change log documenting all system changes that have occurred.
In that case, tracking compliance with that control becomes a key
tactical measure for you.
Security risk in how you build, deploy, and operate the
applications and systems that compose your service. For
example, you may want to monitor supply chain risk in the
use of open-source components, code analysis for security
vulnerabilities in the source code, or penetration testing to find
real application vulnerabilities. In that case, you’d use these
measures as your tactical metrics.
Operational risk in your software delivery processes, which can
lead to tactical metrics such as software defects, technical debt,
or code churn.

Velocity refers to those metrics that measure the speed of delivery
within the value stream, or the rate at which new value is delivered to
customers. Velocity matters because it’s a direct reflection of the pace
with which the organization converts customer needs into actual
technology changes.
Most product managers are aware of velocity because they can, at least
in their own minds, observe how long it seems to take to deliver value to
customers. And they know if that’s too slow for their liking!
Because product managers look at velocity from the customer perspective,
they tend to evaluate it based on how much value is being delivered to the
customers over time. This approach may lead to Deployment Frequency as a
Pipeline metric.
However, as with all Pipeline metrics, it’s important to note that IT will
need to make it actionable through the supporting Tactical metrics.
While Deployment Frequency is a valid Pipeline metric, it may not give IT
teams enough guidance for how they will focus on improving it. After all,
Quality is a challenging aspect of a value stream to measure. Customers
will complain (or leave) if they encounter too many defects. So, that’s
often the main lens used to assess quality.
But quality also affects the value stream’s ability to deliver over time or
to operate its services economically due to an increase in rework. This, in
turn can negatively impact the overall set of Value metrics.
Metrics related to quality must be used carefully. Why? Because it’s easy
to over-optimize for what can be measured (defects) and try to minimize
Velocity
Quality
Deployment Frequency is a consequence of many interactions within the
value stream.
Tactical metrics provide more clarity into how the Pipeline metrics can be
achieved by including more granular internal value stream metrics. For
example, this could include cycle time and lead time to create more focus on
how a team will improve Deployment Frequency.

them at the expense of other elements (such as adding manual testing
processes to improve quality, thereby reducing velocity).
Business leaders care about quality to the extent it affects customers and
business outcomes. It’s important for them to understand what quality
means and to have a common understanding with you of “how much”
quality the organization really needs.
For instance, is the cost of the occasional defect slipping into production
worth slowing down velocity by 25%? In some companies, industries, or
value streams, the answer is yes, and in others, no.
Use quality metrics to have an effective conversation
with the value stream owner about what it means in
light of the Value metrics. That way you can ensure that
quality stays in balance with the other Pipeline metrics.
However, when used properly quality metrics can be very
useful in supporting the other Pipeline metrics. For example:
Escape ratio will tell you the ratio of defects found relative
to the number of overall work items. You can track this
either before reaching customers, after, or both.
Test coverage allows you to understand the percentage
of a code base that’s covered by tests. It can be useful in
two ways. One, it serves as a broader reflection on change
risk, as less test coverage tends to make changes riskier.
Two, monitoring the test coverage trend over time will
inform you if the development teams are adding tests
simultaneously with adding new code.

Performance reflects the operation of your product and service.
It tends to include many of the classic “Ops” metrics, such as latency,
load time, and mean time to restore service (MTTR).
Most performance measures are directly quantifiable. That said, because
performance measures are relatively easy to understand, it’s common
to overuse them. This can be harmful in that it’s easy to get lost in the
“noise” of a lot of metrics in this space. Instead, focus on a handful of
high-impact ones.
Performance
Failure Occurs Repairs Start Resume Operations
Time to Repair
Repairing Testing Back to Normal
Failure
Notification
Time
Diagnose
Issue
Time to Recovery
Latency measures the delay caused across a network. It is
represented by the amount of time it takes a system to process
a request.
Load time measures the time it takes to load content when the
user performs an action, such as clicking a link.
Mean Time to Restore Service (MTTR) calculates the time passed
from the beginning of an incident until the moment it’s solved.

Each of the four pipeline metrics should be associated with
two Tactical metrics that support it.
Tactical metrics should be determined by software development teams
in order to create buy-in and empower teams to:
There are a number of metrics used across the industry in the four
Pipeline metrics for you to choose from. However, the exact Tactical
metrics you choose must reflect your clear-eyed understanding of which
areas correlate most closely with reaching the associated Pipeline goals.
Tactical Metrics
Innovate to find ways to improve.
Establish achievable goals for software development
teams to reach.
Detect issues and course-correct as early as possible.

Value stream metrics aren’t a one-time, set-it-and-forget-it trick.
USING THE FRAMEWORK IN PRACTICE
First, develop Value metrics each year. It’s common for most businesses
to lay out one-year goals as part of a three-year (or even five-year) plan. It’s
within this typical planning process that you should develop Value metrics.
If your company doesn’t have such planning now, initiating an annual value
stream metrics review is an excellent place to start.
Next, select Pipeline and Tactical metrics that align with the Value metrics at
the start of the year. Then review them frequently during the year.
For example, let’s say you decide at the beginning of the year that reducing
customer churn is a critical business outcome. Service availability could
become the Performance metric with the supporting Tactical metrics of
MTTR and latency.
After the first quarter, review how MTTR and latency have changed. Then
consider if you should keep focusing on latency for more improvements or if
it makes more sense to select a new measure to further the Pipeline metric
of service availability.
In the same manner, after two quarters you may review the Pipeline metrics.
How has service availability changed? Is it still the best performance metric
to support the Value metrics?
The key isn’t that you change your metrics on
some cadence. You simply commit to reviewing
them periodically.
We suggest doing so quarterly for Tactical metrics and bi-annually for
Pipeline metrics, though more often is also acceptable.
Having said that, it’s important to give yourself enough time between
applying a treatment and measuring it. You need to ensure the metrics
have caught up. Otherwise, you run the risk of taking too many changes
steps to make long-term impacts.

This process applies to both Value, Pipeline, and Tactical metrics. The only
difference is the cadence. For Value metrics, work through the process over the
course of a year, using the “assess” step at the end of the year to inform the next
year’s “identify” and “select” steps. For Pipeline and Tactical metrics, perform this
process more frequently, such as quarterly.
Metric Refinement Process
Any metrics process must be iterative. Our suggested process consists of six steps:
Identify multiple
potential metrics
relevant to your goals.
Evaluate progress as
the metrics change
over time.
Of those identified,
select the metrics
that best align with
your goals.
Adjust your
strategies and
tactics based on
actual performance.
Create a dashboard
or scorecard to
disseminate the metrics
on a recurring basis.
Assess whether the
metrics still align with
your outcomes, and
iterate from Step 1.
1 2 3 5
4 6
PROCESS CADENCE
Value Metrics: 1x per year
Pipeline Metrics: Quarterly
Tactial Metrics: Quarterly

In this section, we’ll cover a
representative list of potential
Pipeline metrics that can serve
as input for identifying useful
tactical measures. But keep in
mind, no matter how useful any
of these sound, you must work
through the process to align
them with Value metrics.
SAMPLE VALUE STREAM METRICS
Gartner’s list of value stream metrics is a good starting place.
These include the following:
Gartner Value Stream Metrics
Lead time is the amount of time from a customer requesting a
capability until it’s released. Note there are also other potential
definitions of lead time, particularly ones that emanate from an
internal idea and not directly from the customer. Further, Gartner
differentiates deployment (code is moved to an environment) from
release (code is actually available to a customer) as part of
this definition.
Cycle time is the amount of time from when work is started until it’s
completed. There are also several potential formulations for cycle
times depending on the workstream you’re examining.
Throughput is the number of work items completed over some
period. This works best if work item sizes are standardized, often
through story points.
Work in progress are the number of items that have been started
and are still active or are waiting. In general, the larger the number of
works in progress, the lower the throughput.
Work profile is the balance of work items across various categories
(such as feature and defect) over time.

DORA published an annual State of DevOps report that
analyzed industry DevOps practices from 2014 to 2019.
The research has over the years established four bedrock
DevOps metrics (and have accompanying thresholds on
high vs. low performers):
DevOps Metrics
Lead time for change is the amount of time from a code commit
until that change is running in production.
Deployment frequency is the number of deployments a team
does in a day. High performers deploy more often, with world-class
firms doing so multiple times per day.
Mean time to restore is the amount of time from service
interruption to restoration.
Change failure rate is the percentage of changes causing service
degradation that has to be remediated.
Despite having listed DORA’s thresholds for top performers, it’s critical to assess your company’s
outcomes and decide what makes sense for you. If you’re starting from a lead time measured in hours
or change failure rate over 50%, any investments you make to improve will pay off, regardless of the
absolute value of the number. But this is only worthwhile if it clearly aligns with your value metrics.

The metrics listed above are a great starting point. In most
cases, lead time and cycle time are the most effective velocity
measures available because a commitment to reducing and
stabilizing them can drive significant behavioral changes. There
are some additional metrics that, while very situational, are
worth knowing about. We talked about these briefly earlier,
but let’s explore them in a little more depth.
Other Potential Metrics
Code churn is how often the same line or lines of code change within a
period of time. This is a velocity measure that hints at excessive waste.
Test environment utilization is a measure of efficiency in test
environments. You can use it as a quality metric associated with
cost efficiency initiatives or as a velocity measure that focuses on
eliminating waste.
Build time is the amount of time it takes to actually create a new
version of the software. If this is too long, developers may take
shortcuts to save time that negatively affect quality.

22
EGUIDE: USING METRICS IN VALUE STREAMS TO ALIGN BUSINESS AND IT
EXAMPLE
Value Metrics
EXAMPLE
Pipeline Metrics
EXAMPLE
Tactical Metrics
NPS Score
QUALITY:
Reduce
Rework %
QUALITY:
Test Coverage,
Escape Defect
Ratio
PERFORMANCE:
Service
Availability
PERFORMANCE:
MTTR,
Latency
RISK:
Adhere to
CCPA & GDPR
RISK:
Compliance to Release
Processes, Change
Failure Rate
SPEED:
Deployment
Frequency
SPEED:
Value Stream Lead
Time, DevOps
Cycle Time
Revenue Cost

Plutora, the market leader of value stream management solutions for
enterprise IT, improves the speed and quality of software creation by
capturing, visualizing and analyzing critical indicators of every aspect
of the delivery process. Plutora orchestrates release pipelines across a
diverse ecosystem of development methodologies, manages hybrid test
environments, correlates data from existing toolchains, and incorporates
test metrics gathered at every step. The Plutora Platform ensures
organizational alignment of software development with business strategy
and provides visibility, analytics and a system of insights into the entire
value stream, guiding continuous improvement through the measured
outcomes of each effort.
Learn more: www.plutora.com
Email: contact@plutora.com
The biggest opportunity facing
Agile and DevOps today is the
same that has faced technology
for most of its history: the effective
alignment of business goals with
technology operations.
CONCLUSION
This guide outlined a six-step, three-layer framework to
help you succeed in drawing the business and technology
closer together through the use of metrics. By starting
from your business goals and working through the Pipeline
and tactical metrics layers, you can identify, select, and
monitor the technology measures that matter most for
your business success.

Using Metrics in Value Streams to Align Business and IT

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Using Metrics in Value Streams to Align Business and IT

Similar a Using Metrics in Value Streams to Align Business and IT (20)

Más de Plutora

Más de Plutora (7)

Último

Último (20)

Using Metrics in Value Streams to Align Business and IT