1. introduction simulation result analysis conclusions and future work conlcusions
EMPLOYING LOCAL AND GLOBAL SENSITIVITY ANALYSIS
TECHNIQUES TO GUIDE USER INTERFACE DEVELOPMENT
OF ENERGY CERTIFICATION AND COMPLIANCE SOFTWARE
TOOLS
Filippo Monari
filippo.monari@strath.ac.uk
Department of Mechanical and Aerospace Engineering
University of Strathclyde
September 2012
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

2. introduction simulation result analysis conclusions and future work conlcusions
Abstract
This work reports on how sensitivity analysis techniques, applied to
the inputs of calculation engines for energy certification and
regulation compliance purposes, can provide guidance for simplifying
their user interfaces and simplify model imput.
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

3. introduction simulation result analysis conclusions and future work conlcusions
SBEM
The focus of the research is SBEM (Simplified Building Energy
Model) which is the standard software used in the UK for
energy certification and regulation compliance of non-domestic
buildings. It was developed by BRE (Building Research
Establishment), based on the BS EN ISO 13790 Standard.
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

4. introduction simulation result analysis conclusions and future work conlcusions
analysed cases
Two building models from the iSBEM’s installation package
have been considered:
Approval Case 1 (case 1)
Example Building Complete (case 2)
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

5. introduction simulation result analysis conclusions and future work conlcusions
analysed cases
Two building models from the iSBEM’s installation package
have been considered:
Approval Case 1 (case 1)
Example Building Complete (case 2)
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

6. introduction simulation result analysis conclusions and future work conlcusions
analysed cases
Two building models from the iSBEM’s installation package
have been considered:
Approval Case 1 (case 1)
Example Building Complete (case 2)
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

7. introduction simulation result analysis conclusions and future work conlcusions
analysed cases
Two building models from the iSBEM’s installation package
have been considered:
Approval Case 1 (case 1)
Example Building Complete (case 2)
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

8. introduction simulation result analysis conclusions and future work conlcusions
analysed cases
case2
it is developed on two levels:
ground floor: supermarket and coffee shops
first floor: offices
it is composed of 19 thermal zones
total area 2900 square metres
it is provided with a solar energy system
it is served by an HWS and HVAC (heating, cooling and
heat recovery)
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

9. introduction simulation result analysis conclusions and future work conlcusions
employed methods
Two different sensitivity techniques were applied to the input
data required:
the Morris Method which is used to screen the input factors
the Monte Carlo Analysis which is used to assess the
effects of groups of parameters
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

10. introduction simulation result analysis conclusions and future work conlcusions
employed methods
Two different sensitivity techniques were applied to the input
data required:
the Morris Method which is used to screen the input factors
the Monte Carlo Analysis which is used to assess the
effects of groups of parameters
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

11. introduction simulation result analysis conclusions and future work conlcusions
employed methods
Two different sensitivity techniques were applied to the input
data required:
the Morris Method which is used to screen the input factors
the Monte Carlo Analysis which is used to assess the
effects of groups of parameters
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

12. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
elementary effects
The Morris Method characterizes the sensitivity of a model
respect to its inputs through the concept of elementary effects
(EE)
Definition
the elementary effects (EE) can be defined as approximations
of the partial derivatives of the model
EEi =
y(¯x + ei ∗ ∆i) − y(¯x)
∆i
where:
ei : is a zero vector wherein only the position i is in equal to 1
y: is the fucntion representing the model to assess
x: a vector of variables
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

13. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
calculating elementary effects
the EE are estimated along traictories of points, randomly
selected on an adequately discretized space but each one
differing from the preious just in one coordinate.
Definition
the discretized space is represented by p-level k-dimensional
grid, where:
k: number of input variables of the model
p: assumed number of steps defining the values of the
variables
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

14. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
calculating elementary effects
For each parameter, a finite distribution (Fi) of r EE (r within [10,
50]) is estimated
Then for each Fi are calculated:
absolute mean: µ∗
i = 1
r
r
t=1 |EEit |, indicator of the
magnitude of the effect
standard deviation: σi = 1
r−1 ∗ r
t=1(EEit − µi)2,
indicator of the typology of the effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

15. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
calculating elementary effects
For each parameter, a finite distribution (Fi) of r EE (r within [10,
50]) is estimated
Then for each Fi are calculated:
absolute mean: µ∗
i = 1
r
r
t=1 |EEit |, indicator of the
magnitude of the effect
standard deviation: σi = 1
r−1 ∗ r
t=1(EEit − µi)2,
indicator of the typology of the effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

16. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
calculating elementary effects
For each parameter, a finite distribution (Fi) of r EE (r within [10,
50]) is estimated
Then for each Fi are calculated:
absolute mean: µ∗
i = 1
r
r
t=1 |EEit |, indicator of the
magnitude of the effect
standard deviation: σi = 1
r−1 ∗ r
t=1(EEit − µi)2,
indicator of the typology of the effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

17. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
effect typology
σi
µi
≤ 0.1 ⇒ xi has an almost linear effect
0.1 ≤ σi
µi
≤ 0.5 ⇒ xi has a monotonic effect
0.5 ≤ σi
µi
≤ 1 ⇒ xi has a quasi-monotonic effect
σi
µi
≥ 1 ⇒ xi has a non-linear non-monotocnic effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

18. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
effect typology
σi
µi
≤ 0.1 ⇒ xi has an almost linear effect
0.1 ≤ σi
µi
≤ 0.5 ⇒ xi has a monotonic effect
0.5 ≤ σi
µi
≤ 1 ⇒ xi has a quasi-monotonic effect
σi
µi
≥ 1 ⇒ xi has a non-linear non-monotocnic effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

19. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
effect typology
σi
µi
≤ 0.1 ⇒ xi has an almost linear effect
0.1 ≤ σi
µi
≤ 0.5 ⇒ xi has a monotonic effect
0.5 ≤ σi
µi
≤ 1 ⇒ xi has a quasi-monotonic effect
σi
µi
≥ 1 ⇒ xi has a non-linear non-monotocnic effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

20. introduction simulation result analysis conclusions and future work conlcusions
Morris Method
effect typology
σi
µi
≤ 0.1 ⇒ xi has an almost linear effect
0.1 ≤ σi
µi
≤ 0.5 ⇒ xi has a monotonic effect
0.5 ≤ σi
µi
≤ 1 ⇒ xi has a quasi-monotonic effect
σi
µi
≥ 1 ⇒ xi has a non-linear non-monotocnic effect
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

21. introduction simulation result analysis conclusions and future work conlcusions
uncertainty analysis
macro-parameters
the parameters for both the cases have been collected and
grouped in order to create comparable macro-parameters; then
for each one of them it has been attributed:
a probability distribution
and suitable uncertainty factors (standard deviation (σ) or
Delta (∆) depending on the distribution) based on a
literature review
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

22. introduction simulation result analysis conclusions and future work conlcusions
uncertainty analysis
macro-parameters
the parameters for both the cases have been collected and
grouped in order to create comparable macro-parameters; then
for each one of them it has been attributed:
a probability distribution
and suitable uncertainty factors (standard deviation (σ) or
Delta (∆) depending on the distribution) based on a
literature review
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

23. introduction simulation result analysis conclusions and future work conlcusions
uncertainty analysis
macro-parameters
the parameters for both the cases have been collected and
grouped in order to create comparable macro-parameters; then
for each one of them it has been attributed:
a probability distribution
and suitable uncertainty factors (standard deviation (σ) or
Delta (∆) depending on the distribution) based on a
literature review
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

24. introduction simulation result analysis conclusions and future work conlcusions
uncertainty analysis
distributions and uncertainties
macro-parameter id distribution uncertainty set 0, 1, 2 class
factor(%)
ext wall U 1 normal σ 15, 15, 15 MIP
inf 50 Pa 20 normal σ 30, 30, 30
lighting Wattage 22 uniform ±∆ 10, 10, 10
zone area 14 log-normal σ 2, 2, 2
ext wall area 38 log-normal σ 2, 2, 2
hot water generator sesonal efficiency 7 uniform ±∆ 3, 3, 3
effective thermal mass 2 normal σ 7, 7, 7
HVAC cooling sesonal efficiecny 11 uniform ±∆ 3, 3, 3
SFP air distribution system 13 uniform ±∆ 3, 3, 3
SFP zone thermal units 19 uniform ±∆ 3, 3, 3 FIXED
HVAC heating sesonal efficiency 12 uniform ±∆ 3, 3, 3 LIP
heat recovery sesonal efficiency 10 uniform ±∆ 3, 3, 3
lighting control parasitic power 23 uniform ±∆ 3, 3, 3
Air flow rate MEV 16 uniform ±∆ 3, 3, 3
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

25. introduction simulation result analysis conclusions and future work conlcusions
uncertainty analysis
distributions and uncertainties
macro-parameter id distribution uncertainty set 0, 1, 2 class
factor(%)
SFP MEV 17 uniform σ 3, 3, 3 FIXED
window frame factors 40 log-normal σ 2, 2, 2 LIP
window aspect ratios 41 log-normal σ 4, 4, 4
window areas 39 log-normal σ 2, 10, 20 APPROX
thermal bridges Psi − values 24-36 uniform ±∆ 10, 15, 20 LIP
glazing U 4 normal σ 5, 10, 15
glazing solar transmission 5 uniform ±∆ 5, 10, 15
glazing light transmission 6 uniform ±∆ 5, 10, 15
SES storage volumes 9 log-normal σ 3, 15, 30
SES panels areas 8 log-normal σ 2, 10, 20
ext wall length 37 normal σ 1, 5, 10
ext door areas 42 log-normal σ 2, 10, 20
int wall U 3 normal σ 15, 20, 25
length hot water pipework in zones 21 normal σ 1, 5, 10
zone height 15 normal σ 1, 5, 10
int wall length 43 normal σ 1, 5, 10
int wall areas 44 log-normal σ 2, 10, 20
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

26. introduction simulation result analysis conclusions and future work conlcusions
simulation process
work flow
step 1
the Morris Method has been run according to the defined
uncertainties
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

27. introduction simulation result analysis conclusions and future work conlcusions
simulation process
work flow
step 2
for each output the variables have been classified in most
important (MIP) and least important (LIP) parameters
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

28. introduction simulation result analysis conclusions and future work conlcusions
simulation process
work flow
step 3
LIP have been divided in:
FIXED LIP: coefficients mainly relative to the building
services, for which the uncertainties are low and suitable
approximated values could be easily found through
technical specification or literature.
APPROX LIP: physical properties and dimensions of
secondary importance for the models, which could be
defined within certain ranges
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

29. introduction simulation result analysis conclusions and future work conlcusions
simulation process
work flow
step 3
LIP have been divided in:
FIXED LIP: coefficients mainly relative to the building
services, for which the uncertainties are low and suitable
approximated values could be easily found through
technical specification or literature.
APPROX LIP: physical properties and dimensions of
secondary importance for the models, which could be
defined within certain ranges
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

30. introduction simulation result analysis conclusions and future work conlcusions
simulation process
work flow
step 3
LIP have been divided in:
FIXED LIP: coefficients mainly relative to the building
services, for which the uncertainties are low and suitable
approximated values could be easily found through
technical specification or literature.
APPROX LIP: physical properties and dimensions of
secondary importance for the models, which could be
defined within certain ranges
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

31. introduction simulation result analysis conclusions and future work conlcusions
simulation process
work flow
step 4
the possibility of use approximated values has been
investigated by running Monte Carlo simulations for increasing
APPOX LIP’s uncertainties
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

32. introduction simulation result analysis conclusions and future work conlcusions
Morris method
Morris Method - energy demand
The total energy demand
showed linear and
monotonic effects for
most of the MIP and LIP
with the majority of them
having a monotonic
influence. Non-linear
effects are caused by
glass transmittances,
internal wall areas, zone
areas (ids: 4, 3 and 14).
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

33. introduction simulation result analysis conclusions and future work conlcusions
Morris method
Morris Method - energy consumption
All the MIP variables
have linear and
monotonic effect. Only
the U of the external
envelope has a
non-linear influence.
Considering the LIP
irregular influences are
shown by effective
thermal mass, inifltration
at 50 Pa, heat recovery
efficiency, glazing U,
envelop area, int wall
areas and U (ids: 2, 20,
4, 38, 44, 3).
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

34. introduction simulation result analysis conclusions and future work conlcusions
Morris method
Morris Method - asset rating
The number of
non-linearities and
non-monotonic effects
increases for the building
asset rating. All the
parameters have at least
a non-monotonic effect.
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

35. introduction simulation result analysis conclusions and future work conlcusions
Monte Carlo
Monte Carlo - increased uncertainties
output index set 0 set 1 set 2
energy demand s(MJ/m2
) 3.758 3.856 4.737
s/¯x 0.016 0.016 0.020
energy consumption s(MJ/m2
) 4.678 4.59 4.798
s/¯x 0.013 0.013 0.013
asset rating s(MJ/m2
) 0.581 0.594 0.629
s/¯x 0.016 0.016 0.017
The incremented uncertainties for the APPROX-LIP, do not lead to
any relevant growth of the global uncertainties. Comparing the
different values of standard deviation, increments are always less
than or equal to the 1.5% of the mean
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

36. introduction simulation result analysis conclusions and future work conlcusions
final results
quantifying the error increments
the previous result show that it should be possible to replace
the "most exact" set of input data (i.e. in these example SET-0),
with an "approximated" one (i.e. in these examples SET-1 and
SET-2), without sensibly affecting the result of the calculation
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

37. introduction simulation result analysis conclusions and future work conlcusions
final results
quantifying the error increments
The possible increment in the percentage errors produced
could be calculated as follow:
increased error
IEi,n = 2(σ%i,n − σ%i,0)
where:
σ%i,0: standard deviation as % of the mean, relative to the probability distribution of the i − th SBEM’s
output produced by the "most exact" set of data available. It represents the unavoidable amount of
uncertainty
σ%i,n: standard deviation as % of the mean, relative to the probability distribution of the i − th SBEM’s
output produced by the "approximated" set of data. It represents the sum of the unavoidable amount and
the increment in the uncertainty
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

38. introduction simulation result analysis conclusions and future work conlcusions
final results
quantifying the error increments
error increments for case 1 and case 2
case output set 1 set 2
case 1 energy demand 0.02 0.05
energy consumption 001 0.02
asset rating 0.01 0.03
case 2 energy demand 0.00 0.01
energy consumption 0.00 0.00
asset rating 0.00 0.01
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

39. introduction simulation result analysis conclusions and future work conlcusions
main findings
[1]
At a general level the calculation method showed an almost
linear character. In particular, the most influencing factors have
linear and monotonic influences on SBEM’s outputs.
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

40. introduction simulation result analysis conclusions and future work conlcusions
main findings
[2]
The opportunity to approximate the two main models as
meta-models depending only upon the MIP has been
demonstrated, as well as the possibility of considering the least
important ones in a simplified way. LIP have been divided,
depending on the kind of possible approximations:
FIXED-LIP: parameters that can be fixed to reasonable
values
APPROX-LIP: parameters which can be defined within
rabges
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

41. introduction simulation result analysis conclusions and future work conlcusions
main findings
[3]
A criterion to quantify the error incremnt caused by the possible
approximation has been proposed:
IEi,n = 2(σ%i,n − σ%i,0)
where:
σ%i,0: standard deviation as % of the mean, relative to the probability distribution of the i − th SBEM’s
output produced by the "most exact" set of data available. It represents the unavoidable amount of
uncertainty
σ%i,n: standard deviation as % of the mean, relative to the probability distribution of the i − th SBEM’s
output produced by the "approximated" set of data. It represents the sum of the unavoidable amount and
the increment in the uncertainty
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

42. introduction simulation result analysis conclusions and future work conlcusions
applications
The method described is flexible and not software dependent. It
can help in:
guiding the design of user interfaces
developing guide lines for all the data input and collection
processes
structuring the assessors’ training, so that the focus would
be proportionally distributed depending on the influence
and importance of each input parameter
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

43. introduction simulation result analysis conclusions and future work conlcusions
future developments
[1]
the design and definition of procedures and tools involved in the
analysis of a multitude of buildings should be based on relevant
statistically results. Thus the methodology in this paper should
be applied to a statistically relevant sample of buildings to
confirm the results presented.
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

44. introduction simulation result analysis conclusions and future work conlcusions
future developments
[2]
there is a significant gap between predicted and real data. In
future developments a similar approach could be adopted in
calibration studies employing metered data in order to see how
and to what extent different parameters contribute to the
mismatch between predictions and reality.
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde

45. introduction simulation result analysis conclusions and future work conlcusions
thank you for your interest and attention
A Sensitivity Analysis on the SBEM’s inputs University of Strathclyde