Urjan Jacobs fPET-2010 presentation

1. Quantitative Design Tools
Decision Matrices in Engineering Design of Innovative Technology
Option 1 Option 2 Option 3 … Weight
Criterion A ++ - 0 … …
Criterion B 1 5 3 … …
Criterion C 0.1 m/s 0.4 m/s 0.03 m/s … …
… … … … … …
Score … … … …
ir Urjan Jacobs
10 May 2010
1
Biotechnology and Society - TNW & Philosophy - TPM

2. Contents
Quantitative Design Tools
• Innovative conceptual design
• Case study & matrix methods
• Methodological problems
• Examples of issues
• A way forwards
May 20, 2010 2

3. Innovative technology
Engineering design of a
system with a new concept
Nanotechnology
Biotechnology
Chemical technology
May 20, 2010 3

4. The conceptual design phase
Problem definition
Concept generation
Evaluation & selection
Detailed design
May 20, 2010 4

5. Case studies
Conceptual Process/Product Design
(bio)chemical
engineering MSc students
PDEng trainees
10-12 working weeks
May 20, 2010 5

6. Case studies
Research methods
Observations of design team
Following meetings
Analysing design documents
Semi-structured interview
May 20, 2010 6

7. Quantitative design tools
Decision matrix methods
Quality function deployment
Pair-wise comparison charts
Analytic Hierarchy Process
May 20, 2010 7

8. Matrix methods
Multi-criteria decision analysis
Decision matrix
Solution m
rid atrix
lect ion g
Se
Decision grid
May 20, 2010 8
Note: Multi-attribute value theory (MAVT) and multi-attribute utility theory (MAUT)
have a very different starting point.

9. Arrowian impossibility theorem
Considering a finite number of evaluation criteria and at
least three alternative design concepts, no method can
simultaneously satisfy:
Global rationality theory
Voting
•
• Unrestricted scope
• Independence of irrelevant concepts
• Weak pareto principle
• Non-dominance
Social choice theory
May 20, 2010 9
K.J. Arrow, Journal of Political Economy 58, 1950, 328-346
A. Hylland, Econometrica 48, 1980, 539-542

10. Source of the issues
Commensurability of criteria
• Measurability
(scale of measurement)
• Comparability
(relation between measures)
May 20, 2010 10

11. Measurability
Scale Type Admissible Transformation Example
Nominal One to one Labels
Ordinal Monotonic increasing Mohs scale
Interval Positive linear Celsius scale
Ratio Positive similarities Miles scale
Unknown to Engineers
May 20, 2010 11
S.S. Stevens, Science 103, 1946, 677-680

12. Comparability
Trade-off relation between measures
• Value comparability
Revenues
• Technical comparability
e
tu r
Safety p era
Pr o d t em
uctio tor
n vol
ume R eac
Reliability
Sustainability
May 20, 2010 12

13. Other issues
Uncertainty
• Setting up of full set criteria.
• Independent criteria.
• Assigning performance ratings.
Design concepts not at same level of abstraction
Weights dependant on concept performance
May 20, 2010 13

14. Example: Weighted objectives
Convincing the design engineers
Option 1 Option 2 … Option n Weight
Criterion 1 Performance11 Performance12 … Performance1n w1
w2
Criterion 2 Performance21 Performance22 … Performance2n
… … … … … …
Criterion m Performancem1 Performancem2 … Performancemn wm
Score S1 S2 … Sn
m
Sj = ∑w
i =1
i ⋅ Pij
May 20, 2010 14

15. Grading issue
Criteria Weight Option 1 Option 2 Option 3
Yield 1 2 3 1
By-products 1 3 1 2
Safety 1 2 3 1
Controllabity 1 2 3 1
Revenues 1 3 1 2
Score 12 11 7
Grade: 1=worst, 2=neutral, 3=best.
Criteria Weight Option 1 Option 2 Option 3
Yield 1 2 5 1
By-products 1 5 1 2
Change grading Safety
Controllabity
1
1
2
2
5
5
1
1
(best 3 5) Revenues 1 5 1 2
Score 16 17 7
Grade: 1=worst, 2=neutral, 5=best.
May 20, 2010 15

16. Weighting issue
Criteria Weight Option 1 Option 2 Option 3
Yield 0.1 3 2 1
By-products 0.3 1 3 2
Safety 0.2 3 1 2
Controllabity 0.3 3 2 1
Revenues 0.1 1 2 3
Score 2.2 2.1 1.7
Grade: 1=worst, 2=neutral, 3=best.
Criteria Weight Option 1 Option 2 Option 3
Yield 0.07 3 2 1
By-products 0.36 1 3 2
Change weighting Safety
Controllabity
0.14
0.36
3
3
1
2
2
1
(0.07; 0.14; 0.36) Revenues 0.07 1 2 3
Score 2.14 2.22 1.64
Grade: 1=worst, 2=neutral, 3=best.
May 20, 2010 16

17. Buridan's paradox
Criteria Weight Option 1 Option 2 Option 3
Yield 0.1 3 2 1
By-products 0.3 2 3 1
Safety 0.2 1 2 3
Controllabity 0.3 2 1 3
Revenues 0.1 3 2 1
Score 2 2 2
Grade: 1=worst, 2=neutral, 3=best.
No rational choice …
May 20, 2010 17
Aristotle, De Caelo II (On the Heavens), 350 BC

18. Irrelevant alternative issue
Criteria Weight Option 1 Option 2 Option 3 Option 4
Yield 1 4 3 2 1
By-products 1 2 4 3 1
Safety 1 4 2 1 3
Controllabity 1 4 2 1 3
Revenues 1 2 4 3 1
Score 16 15 10 9
Grade: 1=worst, 2=poor, 3=fine, 4=best.
Criteria Weight Option 1 Option 2 Option 3
Yield 1 3 2 1
By-products 1 1 3 2
Safety 1 3 2 1
Controllabity 1 3 2 1
Revenues 1 1 3 2
Remove/not consider Score 11 12 7
poor option Grade: 1=worst, 2=neutral, 3=best.
May 20, 2010 18

19. Traded-away criteria
Criteria Weight Option 1 Option 2 Option 3
Yield 1 1 2 3
By-products 1 3 2 1
Safety
Controllabity
1
1
1
3
2
1
3
2 Biased on
Revenues 1 2 3 1
Sustainability
Score
1 3
13
2
12
1
11
sustainability
Grade: 1=worst, 2=neutral, 3=best.
criterion.
Condorcet distortion
May 20, 2010 19
M. J.A.N. de Caritat Condorcet, Essai sur l'application de l'analyse à la probabilité
des décisions rendues à la pluralité de voix, Paris 1785.

20. How to proceed?
Many designers utilize
decision matrices.
What is their use
if not to find
the best option?
May 20, 2010 20

21. Assessment of design tools
Theories of truth
Consistent
• Coherence
with rules
Checked
• Correspondence by facts
• Pragmatic
• …
Facilitate obtaining goals
May 20, 2010 21

22. Pragmatic goals in design practice
Goals of matrix methods
• Structuring problem
• Supports communication
• Enhance creativity
May 20, 2010 22

23. Problem structuring
Ill-structured design problem
• No criterion to decide the best solution
• Not well defined solution space
• No normative framework available
Co-evolution of
problem & solution
May 20, 2010 23

24. Facilitating communication
Visual summary
Show alternative concepts
Converting requirements
Judgement on performances
Supports debate on the choice
CSTR Feb-batch Batch
Yield - + ++
By-products + 0 --
Safety + ++ -
Revenues + - 0
May 20, 2010 24

25. Creativity enhancement
Option 1 Option 2 Option 3 Option 4 Option 5 Option 6
Criterion A + D + ++ + -
Criterion B ++ A ++ + - --
Criterion C + T 0 + 0 +
Criterion D 0 U - -- ++ 0
Criterion E + M + - -- --
May 20, 2010 25
Controlled convergence method
S. Pugh, Total Design, Harlow 1991

26. Conclusion
Option 1 Option 2 Option 3 Option 4 …
Criterion A + - + 0 …
Criterion B ++ ++ + - …
Criterion C 0 - -- ++ …
Criterion D + + - -- …
… … … … … …
Keep using the matrix
Hold all options & criteria
Never calculate a decision
May 20, 2010 26

27. Further research
Midstream modulation
• Collaboration with designers
• Stimulate awareness
• Motivate to discuss ‘soft’ issues
• Safety, sustainability, robustness
May 20, 2010 27

28. Many thanks!
PDEng trainees
MSc students
Supervisors & Clients
May 20, 2010 28

29. Quantitative Design Tools
Decision Matrices in Engineering Design of Innovative Technology
Option 1 Option 2 Option 3 Option 4 Option 5
Criterion A + D ++ 0 --
Criterion B ++ A - + -
Criterion C + T 0 + 0
Criterion D 0 U - -- ++
Criterion E + M + - -
ir. Urjan Jacobs
t: +31 (0)15 278 6626
e: j.f.jacobs@tudelft.nl
29
Biotechnology and Society - TNW & Philosophy - TPM