Broadband is a relatively new technology, and its underlying science is still being developed. We have long understood the 'right' units in other engineering disciplines: mass, length, hardness, etc. What is the 'right' unit for supply and demand for broadband? This presentation discusses the need for having the right metric. This means solving two problems: the 'abstraction' gap, and the 'inference' gap. ∆Q is the ideal metric because it fills both gaps.

1. Why ΔQ is
the ideal
network metric
© Predictable Network Solutions 2015

2. Dr Neil Davies Co-founder and Chief Scientist
Ex: University of Bristol (23 years).
Former technical head of joint university/research institute (SRF/PACT).
The only network performance science company in the world.
• New mathematical performance measurement and analysis techniques.
• Performance assessment methodology.
• World’s first packet network quality assurance solution.
PREDICTABLE
NETWORK
SOLUTIONS
Peter Thompson CTO
Ex: GoS Networks, U4EA, SGS-Thomson, INMOS & Universities of Bristol, Warwick and Cambridge
and Oxford .
Authority on technical and commercial issues of converged networking.
Martin Geddes Associate Director of Business Development
Ex: BT, Telco 2.0, Sprint, Oracle, Oxford University.
Thought leader on the future of the telecommunications industry.

3. Customer Experience and Service Quality
Millions of users
1001 1110 1011 0001 1011
Billions of packets 3
How are customer
experience and
service quality
related?

4. SQM
Customer Experience and Service Quality
Millions of users
1001 1110 1011 0001 1011
Billions of packets
CEM
4
We have to link
Customer Experience Management (CEM)
to Service Quality Management (SQM).
But how?

5. Customer Experience and Service Quality
Millions of users
1001 1110 1011 0001 1011
Billions of packets
+
We want to offer
the best collective
experience
-
We also want the
lowest capital cost
5

6. Customer Experience and Service Quality
Millions of users
1001 1110 1011 0001 1011
Billions of packets
+
We want to offer
the best collective
experience
-We also want the
lowest capital cost
We make
trade-offs
(at all timescales)
of QoE and cost based
on metrics
6

7. Net promoter
There are many QoE & network metrics
Jitter
Millions of users
1001 1110 1011 0001 1011
Billions of packets
MOS
Average link use
Effective
bandwidth
User-centric
metrics
Network-centric
metrics
Current network analytic
approaches use
correlation to imply
causality to predict how
to control the trade-offs.
They typically lack a
model to inform model
users of the accuracy of
the prediction.RTT
Churn
7

8. What distinguishes stronger metrics
of QoE and cost from weaker ones?
Millions of users
1001 1110 1011 0001 1011
Billions of packets
Strong QoE proxy
Network measure
?
8
The ideal metric is one
that simultaneously is
a network measure
and a strong proxy for
the delivered QoE.
Today we face an
endemic capability
gap, as metrics fall
short of this ideal.

9. Metrics differ in their ability
to capture what really matters
Millions of users
1001 1110 1011 0001 1011
Billions of packets
These metrics
maintain the
needed fidelity
These metrics
lack the needed
fidelity
?
9

10. Trade-offs of QoE and cost
are always required
Millions of users
1001 1110 1011 0001 1011
Billions of packets 10
1001 1110 1011 0001 10111001 1110 1011 0001 1011
We can’t support
an unbounded
load or quality of
experience
We don’t have
access to
unbounded
free capital to
create network
resources

11. Making trade-offs requires a model
Millions of users
1001 1110 1011 0001 1011
Billions of packets 11
What is the likely
effect of my
intervention?

12. What distinguishes stronger models
of QoE and cost from weaker ones?
Millions of users
1001 1110 1011 0001 1011
Billions of packets
Abstractive
Extracts insight
Predictive
Exploits insight
12
The ideal model captures
only what is relevant, and
makes accurate predictions
of QoE and/or cost from
that information. Today’s
inference models are
typically weak or invalid.

13. Metrics help us to abstract & predict
QoE and cost relationships
Millions of users
1001 1110 1011 0001 1011
Billions of packets
Abstract
Predict
Abstract
Predict

14. Issue: ‘abstraction gap’
Millions of users
1001 1110 1011 0001 1011
Billions of packets
The abstractive
power of any metric
is constrained by the
fidelity of
measurement
14

15. Issue: ‘prediction gap’
Millions of users
1001 1110 1011 0001 1011
Billions of packets
The predictive power
of any metric is
constrained by the
robustness of its
inference model
15

16. So why do we have these gaps?
Experience
without theory
teaches nothing

— W Edwards Deming
(and we, as an industry, are lacking sufficient theory)
16

17. Good abstraction
hides irrelevant variation
Source: http://xkcd.com/676/
17
Computers work because
we have many layers of
good abstraction.

18. Is a metric suitably abstractive?
Millions of users
1001 1110 1011 0001 1011
Billions of packets
Is this metric
capturing the
right network
information?
Is this metric a
strong proxy for
QoE?
18

19. Without a model you
have no abstractive power
© Predictable Network Solutions 2013 19
1:1 scale map of London
(not very useful as lacks abstraction)

20. Prediction needs a
robust inference model
20
Source: http://xkcd.com/552/
The joke is about the robustness of the inference model being used.
(In this case, the false presumption of correlation being causation.)

21. Is a metric suitably predictive?
Millions of users
1001 1110 1011 0001 1011
Billions of packets
Can we correctly infer
what to do with the
network to fix our
QoE problem?
Can we correctly infer
what the QoE effect
of our network
change will be?
21

22. No model = no predictive power
Source: http://www.venganza.org/about/open-letter/
Global average temperature vs number of pirates
22
Correlation really isn’t causation!

23. ΔQ measures fill the ‘abstraction gap’
Millions of users
1001 1110 1011 0001 1011
Billions of packets
A general measure that
is both a network
performance metric and
a strong QoE proxy.
Furthermore,
mathematics implies it is
the only measure
needed – as it is both
necessary and sufficient.
ΔQ
QoE
Network
Performance
23

24. ΔQ models fill the ‘prediction gap’
Millions of users
1001 1110 1011 0001 1011
Billions of packets
ΔQ
A predictive network
performance calculus:
robustly models cause and
effect at all levels of
abstraction
QoE
Network
Performance
24

25. Right link speed?
New
segmented
product?
Video buffering
problem?
Which
direction?
Architecture
problem?
Scheduling
issue?
Over-demand or
under-supply?
Which element(s)?
Slow page
load times?
Need a new
low-cost offer?
ΔQ enables ‘network science’ by strongly
relating application and network performance
QoE
Network
Performance
Millions of users
1001 1110 1011 0001 1011
Billions of packets
ΔQ
25

26. Summary:
∆Q is the ideal network metric
∆Q framework is the ‘ideal’ performance engineering system
The prior assumptions of the ∆Q framework are clear
Metrics have practical interest and value
Captures how much trust should be given to metrics (due to error propagation)
The framework offers a robust language in which to reason about performance
∆Q metrics have the ‘ideal’ abstraction properties
∆Q metrics capture everything that is relevant (and nothing that is not)
∆Q is a universal strong QoE proxy – and no others are known
The algebra of ∆Q is mathematically well grounded, so it can be (de)composed in space and time
∆Q appropriately relates performance between levels of abstraction
∆Q models have the ‘ideal’ inference properties
∆Q closely aligns to reality, and differences between the model and reality are understood
∆Q can be composed and decomposed along supply chains, so performance can be ‘budgeted’
∆Q models allow the root causes of issues to be identified with high certainty
∆Q strongly relates resource costs to QoE, facilitating rational network economics
26

27. We can help you!
Measure the true customer experience
with high fidelity metrics
Isolate the root cause of QoE issues
in your supply chain with scientific accuracy
Safely optimise
the trade-off of QoE and cost
Get in touch! sales@pnsol.com
27