SlideShare una empresa de Scribd logo
1 de 33
Descargar para leer sin conexión
Detec%ng	
  Performance	
  An%-­‐pa2erns	
  for	
  
Applica%ons	
  Developed	
  Using	
  
Object-­‐Rela%onal	
  Mapping	
  
1	
  
Mohamed	
  Nasser,	
  Parminder	
  Flora	
  
Tse-­‐Hsun(Peter)	
  Chen	
   Ahmed	
  E.	
  Hassan	
  Weiyi	
  Shang	
   Zhen	
  Ming	
  Jiang	
  
Databases	
  are	
  essen%al	
  in	
  large-­‐scale	
  
soBware	
  systems	
  
2	
  
Database	
  
3	
  
Applica%on	
  developers	
  work	
  with	
  objects	
  
More	
  intui*ve	
  if	
  we	
  can	
  
map	
  objects	
  directly	
  to	
  DB	
  
4	
  
Object-­‐Rela%onal	
  Mapping	
  eliminates	
  
the	
  gap	
  between	
  objects	
  and	
  SQL	
  
Database	
  
•  	
  Lots	
  of	
  boilerplate	
  code	
  	
  
•  	
  Need	
  to	
  manage	
  object-­‐DB	
  
transla%ons	
  manually	
  
	
  
Object	
  	
  
Classes	
  
Problem	
  of	
  using	
  raw	
  SQLs	
  
ORM	
  
Much	
  less	
  code	
  and	
  shorter	
  
development	
  %me	
  
ORM	
  is	
  widely	
  used	
  in	
  prac%ce	
  
5	
  
•  Java	
  Hibernate	
  has	
  more	
  than	
  8	
  
million	
  downloads	
  
•  In	
  2013,	
  15%	
  of	
  the	
  17,000	
  Java	
  
developer	
  jobs	
  require	
  ORM	
  
experience	
  (dice.com)	
  
Different	
  ORM	
  technologies	
  
An	
  example	
  class	
  with	
  ORM	
  code	
  
6	
  
@En%ty	
  
@Table(name	
  =	
  “user”)	
  
public	
  class	
  User{	
  
	
  @Column(name=“id”)	
  
	
  private	
  int	
  id;	
  
	
  
	
  @Column(name=“name”)	
  
	
  String	
  userName;	
  
	
  
@OneToMany(fetch=FetchType.EAGER)	
  
	
  List<Team>	
  teams;	
  
	
  
	
  public	
  void	
  setName(String	
  n){	
  
	
   	
  userName	
  =	
  n	
  
	
  }	
  
…	
  other	
  ge]er	
  and	
  se]er	
  methods	
  
User.java	
  User	
  class	
  is	
  
mapped	
  to	
  “user”	
  
table	
  in	
  DB	
  
id	
  is	
  mapped	
  to	
  the	
  
column	
  “id”	
  in	
  the	
  
user	
  table	
  
A	
  user	
  can	
  belong	
  
to	
  mul%ple	
  teams	
  
Eagerly	
  retrieve	
  
associated	
  teams	
  
when	
  retrieving	
  a	
  
user	
  object	
  
Accessing	
  the	
  database	
  using	
  ORM	
  
7	
  
User	
  u	
  =	
  findUserByID(1);	
  
ORM	
  
database	
  
select	
  u	
  from	
  user	
  
where	
  u.id	
  =	
  1;	
  
u.setName(“Peter”);	
  
update	
  user	
  set	
  
name=“Peter”	
  
where	
  user.id	
  =	
  1;	
  
Objects	
   SQLs	
  
Developers	
  may	
  not	
  be	
  aware	
  of	
  
database	
  access	
  
Wow!	
  I	
  don’t	
  
need	
  to	
  worry	
  
about	
  DB	
  code!	
  
ORM	
  code	
  with	
  	
  
performance	
  ang-­‐pa]erns	
  
8	
  
Bad	
  system	
  	
  
performance	
  
The	
  performance	
  difference	
  can	
  be	
  LARGE!	
  
Performance	
  an%-­‐pa2ern	
  
detec%on	
  framework	
  
Performance	
  ang-­‐pa]ern	
  detecgon	
  and	
  	
  
ranking	
  framework	
  	
  
Ranked	
  according	
  to	
  	
  
performance	
  impact	
  
Ranked	
  
Performance	
  
ang-­‐pa]erns	
  
Source	
  	
  
Code	
  
detecgon	
  
ranking	
  
9	
  
Performance	
  an%-­‐pa2ern	
  
detec%on	
  framework	
  
Performance	
  ang-­‐pa]ern	
  detecgon	
  and	
  	
  
ranking	
  framework	
  	
  
Ranked	
  according	
  to	
  	
  
performance	
  impact	
  
Ranked	
  
Performance	
  
ang-­‐pa]erns	
  
Source	
  	
  
Code	
  
detecgon	
  
ranking	
  
10	
  
Discuss	
  only	
  one	
  an%-­‐pa2ern	
  
in	
  the	
  presenta%on	
  
	
  (more	
  pa2erns	
  in	
  the	
  paper)	
  
ORM	
  excessive	
  data	
  an%-­‐pa2ern	
  
Class	
  User{	
  
	
   	
  @EAGER	
  
	
   	
  List<Team>	
  teams;	
  
}	
  
User	
  u	
  =	
  findUserById(1);	
  
u.getName();	
  
EOF	
  
11	
  
Objects	
  
SQL	
  
Eagerly	
  retrieve	
  
teams	
  from	
  DB	
  
User	
  Table	
   Team	
  Table	
  
join	
   Team	
  data	
  is	
  never	
  
used!	
  
Detec%ng	
  excessive	
  data	
  
using	
  sta%c	
  analysis	
  
12	
  
First	
  find	
  all	
  the	
  objects	
  that	
  
eagerly	
  retrieve	
  data	
  from	
  DB	
  
	
  	
  
Class	
  User{	
  
	
   	
  @EAGER	
  
	
   	
  List<Team>	
  teams;	
  
}	
  
Idengfy	
  all	
  the	
  data	
  usages	
  of	
  
objects	
  	
  
User	
  user	
  =	
  findUserByID(1);	
  
Check	
  if	
  the	
  retrieved	
  data	
  is	
  ever	
  
used	
  
user.getName();	
  
user	
   team	
  
user	
   team	
  
Performance	
  an%-­‐pa2ern	
  
detec%on	
  framework	
  
Performance	
  ang-­‐pa]ern	
  detecgon	
  and	
  	
  
ranking	
  framework	
  	
  
Ranked	
  according	
  to	
  	
  
performance	
  impact	
  
Ranked	
  
Performance	
  
ang-­‐pa]erns	
  
Source	
  	
  
Code	
  
detecgon	
  
ranking	
  
13	
  
Performance	
  an%-­‐pa2ern	
  
detec%on	
  framework	
  
Performance	
  ang-­‐pa]ern	
  detecgon	
  and	
  	
  
ranking	
  framework	
  	
  
Ranked	
  according	
  to	
  	
  
performance	
  impact	
  
Ranked	
  
Performance	
  
ang-­‐pa]erns	
  
Source	
  	
  
Code	
  
detecgon	
  
ranking	
  
14	
  
Performance	
  an%-­‐pa2erns	
  have	
  
different	
  impacts	
  
15	
  
User	
  user_in_1_team	
  =	
  findUserByID(1);	
  
Retrieving	
  1	
  user	
  and	
  1	
  team	
  
User	
  user_in_100_teams	
  =	
  findUserByID(100);	
  
Retrieving	
  1	
  user	
  and	
  100	
  teams!	
  
One	
  can	
  only	
  reveal	
  performance	
  
impact	
  by	
  execu%on	
  
Measuring	
  the	
  impact	
  using	
  repeated	
  
measurements	
  and	
  effect	
  sizes	
  
16	
  
We	
  use	
  effect	
  sizes	
  (Cohen’s	
  D)	
  to	
  measure	
  
the	
  performance	
  impact	
  
Effect	
  sizes	
  =	
  	
  
We	
  repeat	
  each	
  test	
  30	
  gmes	
  to	
  obtain	
  stable	
  
measurement	
  results	
  
Size	
  of	
  performance	
  impact	
  is	
  not	
  defined:	
  
Performance	
  measurements	
  are	
  unstable:	
  
Studied	
  systems	
  and	
  detec%on	
  results	
  
Large	
  open-­‐source	
  	
  
e-­‐commence	
  system	
  
>	
  1,700	
  files	
  
>	
  206K	
  LOC	
  
Enterprise	
  system	
  
>	
  3,000	
  files	
  
>	
  300K	
  LOC	
  
Spring	
  open-­‐source	
  system	
  
Online	
  system	
  for	
  a	
  pet	
  clinic	
  
51	
  files	
  
	
  	
  	
  	
  3.3K	
  LOC	
  
482	
  excessive	
  data	
   >	
  10	
  excessive	
  data	
  
	
  
10	
  excessive	
  data	
  
17	
  
Performance	
  impact	
  
Research	
  ques%ons	
  
Ranks	
  of	
  the	
  ang-­‐pa]erns	
  at	
  
different	
  scales	
  
18	
  
Performance	
  impact	
  
Research	
  ques%ons	
  
Ranks	
  of	
  the	
  ang-­‐pa]erns	
  at	
  
different	
  scales	
  
19	
  
Assessing	
  an%-­‐pa2ern	
  impact	
  by	
  
fixing	
  the	
  an%-­‐pa2erns	
  
Execugon	
  
Response	
  %me	
  user.getName()	
  
Code	
  with	
  	
  
ang-­‐pa]erns	
  
fetchType.set(LAZY)	
  
user.getName()	
  
Code	
  without	
  	
  
ang-­‐pa]erns	
  
20	
  
Execute	
  test	
  
suite	
  30	
  gmes	
  
Response	
  %me	
  
aOer	
  fixing	
  the	
  
anS-­‐paTerns	
  
Avg.	
  %	
  
improvement	
  and	
  
effect	
  sizes	
  
Execugon	
  
Execute	
  test	
  
suite	
  30	
  gmes	
  
Performance	
  an%-­‐pa2erns	
  have	
  
medium	
  to	
  large	
  effect	
  sizes	
  
0%	
  
20%	
  
40%	
  
60%	
  
80%	
  
100%	
  
Excessive	
  Data	
  
BL	
  
EA	
  
PC	
  
21	
  
%	
  improvement	
  in	
  response	
  gme	
  
large	
  	
  
effect	
  size	
  
large	
  	
  
effect	
  size	
   medium	
  
effect	
  size	
  
Performance	
  impact	
  
Research	
  ques%ons	
  
Ranks	
  of	
  the	
  ang-­‐pa]erns	
  at	
  
different	
  scales	
  
22	
  
Removing	
  an%-­‐pa2ern	
  	
  
improves	
  response	
  by	
  ~35%	
  
Performance	
  impact	
  
Research	
  ques%ons	
  
Ranks	
  of	
  the	
  ang-­‐pa]erns	
  at	
  
different	
  scales	
  
23	
  
Removing	
  an%-­‐pa2ern	
  	
  
improves	
  response	
  by	
  ~35%	
  
Performance	
  problems	
  usually	
  arise	
  
under	
  large	
  load	
  
24	
  
Performance	
  problems	
  revealed	
  at	
  
small	
  scales	
  may	
  be	
  more	
  serious	
  
25	
  
We	
  should	
  first	
  fix	
  the	
  an%-­‐pa2erns	
  that	
  
have	
  larger	
  effects	
  at	
  smaller	
  scales	
  
Input	
  scales	
  may	
  have	
  
exponenSal	
  effects	
  on	
  
performance	
  
Different	
  input	
  scales	
   Performance	
  at	
  
different	
  input	
  scales	
  
Comparing	
  ranked	
  an%-­‐pa2erns	
  at	
  
different	
  data	
  scales	
  
26	
  
Ranked	
  
Performance	
  
ang-­‐pa]erns	
  
from	
  small	
  data	
  
detecgon	
  
ranking	
  
Ranked	
  
Performance	
  
ang-­‐pa]erns	
  
from	
  large	
  data	
  
?	
  
Small	
  size	
  input	
  
Large	
  size	
  input	
  
An%-­‐pa2erns	
  have	
  large	
  effects	
  on	
  
performance	
  even	
  at	
  smaller	
  data	
  scales	
  
27	
  
0	
  
10	
  
20	
  
30	
  
40	
  
50	
  
0	
  
1	
  
2	
  
3	
  
4	
  
5	
  
Effect	
  size	
  
Effect	
  sizes	
  and	
  the	
  ranks	
  of	
  the	
  an%-­‐pa2erns	
  
are	
  consistent	
  in	
  different	
  data	
  scales	
  
Performance	
  impact	
  
Research	
  ques%ons	
  
Ranks	
  of	
  the	
  ang-­‐pa]erns	
  at	
  
different	
  scales	
  
28	
  
Removing	
  an%-­‐pa2ern	
  	
  
improves	
  response	
  by	
  ~35%	
  
Ranks	
  of	
  the	
  an%-­‐pa2erns	
  
are	
  consistent	
  in	
  different	
  
data	
  scales	
  
29	
  
30	
  
31	
  
32	
  
33	
  
tsehsun@cs.queensu.ca	
  

Más contenido relacionado

La actualidad más candente

Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...
Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...
Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...Work-Bench
 
Mining and Untangling Change Genealogies (PhD Defense Talk)
Mining and Untangling Change Genealogies (PhD Defense Talk)Mining and Untangling Change Genealogies (PhD Defense Talk)
Mining and Untangling Change Genealogies (PhD Defense Talk)Kim Herzig
 
Runtime Behavior of JavaScript Programs
Runtime Behavior of JavaScript ProgramsRuntime Behavior of JavaScript Programs
Runtime Behavior of JavaScript ProgramsIRJET Journal
 
Using Control Charts for Detecting and Understanding Performance Regressions ...
Using Control Charts for Detecting and Understanding Performance Regressions ...Using Control Charts for Detecting and Understanding Performance Regressions ...
Using Control Charts for Detecting and Understanding Performance Regressions ...SAIL_QU
 
Requirements driven Model-based Testing
Requirements driven Model-based TestingRequirements driven Model-based Testing
Requirements driven Model-based TestingDharmalingam Ganesan
 
Reproducibility with Checkpoint & RRO
Reproducibility with Checkpoint & RROReproducibility with Checkpoint & RRO
Reproducibility with Checkpoint & RROWork-Bench
 
An Industrial Case Study on the Automated Detection of Performance Regression...
An Industrial Case Study on the Automated Detection of Performance Regression...An Industrial Case Study on the Automated Detection of Performance Regression...
An Industrial Case Study on the Automated Detection of Performance Regression...SAIL_QU
 
QUICKAR-ASE2016-Singapore
QUICKAR-ASE2016-SingaporeQUICKAR-ASE2016-Singapore
QUICKAR-ASE2016-SingaporeMasud Rahman
 
Ase2010 shang
Ase2010 shangAse2010 shang
Ase2010 shangSAIL_QU
 
An Industrial Case Study of Automatically Identifying Performance Regression-...
An Industrial Case Study of Automatically Identifying Performance Regression-...An Industrial Case Study of Automatically Identifying Performance Regression-...
An Industrial Case Study of Automatically Identifying Performance Regression-...SAIL_QU
 
An Empirical Study on the Adequacy of Testing in Open Source Projects
An Empirical Study on the Adequacy of Testing in Open Source ProjectsAn Empirical Study on the Adequacy of Testing in Open Source Projects
An Empirical Study on the Adequacy of Testing in Open Source ProjectsPavneet Singh Kochhar
 
What We Learned Building an R-Python Hybrid Predictive Analytics Pipeline
What We Learned Building an R-Python Hybrid Predictive Analytics PipelineWhat We Learned Building an R-Python Hybrid Predictive Analytics Pipeline
What We Learned Building an R-Python Hybrid Predictive Analytics PipelineWork-Bench
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programsgreenwop
 
Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...
Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...
Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...Priyanka Aash
 
It’s Not a Bug, It’s a Feature: Does Misclassification Affect Bug Localization?
It’s Not a Bug, It’s a Feature:Does Misclassification Affect Bug Localization?It’s Not a Bug, It’s a Feature:Does Misclassification Affect Bug Localization?
It’s Not a Bug, It’s a Feature: Does Misclassification Affect Bug Localization?Pavneet Singh Kochhar
 
Mining Performance Regression Testing Repositories for Automated Performance ...
Mining Performance Regression Testing Repositories for Automated Performance ...Mining Performance Regression Testing Repositories for Automated Performance ...
Mining Performance Regression Testing Repositories for Automated Performance ...SAIL_QU
 
Quality Control of Sequencing Data
Quality Control of Sequencing Data Quality Control of Sequencing Data
Quality Control of Sequencing Data Surya Saha
 
PhD public defense: A Measurement Framework for Analyzing Technical Lag in ...
PhD public defense: A Measurement Framework for  Analyzing Technical Lag in  ...PhD public defense: A Measurement Framework for  Analyzing Technical Lag in  ...
PhD public defense: A Measurement Framework for Analyzing Technical Lag in ...Ahmed Zerouali
 

La actualidad más candente (20)

Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...
Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...
Nobody Knows What It’s Like To Be the Bad Man: The Development Process for th...
 
Mining and Untangling Change Genealogies (PhD Defense Talk)
Mining and Untangling Change Genealogies (PhD Defense Talk)Mining and Untangling Change Genealogies (PhD Defense Talk)
Mining and Untangling Change Genealogies (PhD Defense Talk)
 
Runtime Behavior of JavaScript Programs
Runtime Behavior of JavaScript ProgramsRuntime Behavior of JavaScript Programs
Runtime Behavior of JavaScript Programs
 
Using Control Charts for Detecting and Understanding Performance Regressions ...
Using Control Charts for Detecting and Understanding Performance Regressions ...Using Control Charts for Detecting and Understanding Performance Regressions ...
Using Control Charts for Detecting and Understanding Performance Regressions ...
 
Requirements driven Model-based Testing
Requirements driven Model-based TestingRequirements driven Model-based Testing
Requirements driven Model-based Testing
 
Reproducibility with Checkpoint & RRO
Reproducibility with Checkpoint & RROReproducibility with Checkpoint & RRO
Reproducibility with Checkpoint & RRO
 
An Industrial Case Study on the Automated Detection of Performance Regression...
An Industrial Case Study on the Automated Detection of Performance Regression...An Industrial Case Study on the Automated Detection of Performance Regression...
An Industrial Case Study on the Automated Detection of Performance Regression...
 
QUICKAR-ASE2016-Singapore
QUICKAR-ASE2016-SingaporeQUICKAR-ASE2016-Singapore
QUICKAR-ASE2016-Singapore
 
Ase2010 shang
Ase2010 shangAse2010 shang
Ase2010 shang
 
An Industrial Case Study of Automatically Identifying Performance Regression-...
An Industrial Case Study of Automatically Identifying Performance Regression-...An Industrial Case Study of Automatically Identifying Performance Regression-...
An Industrial Case Study of Automatically Identifying Performance Regression-...
 
An Empirical Study on the Adequacy of Testing in Open Source Projects
An Empirical Study on the Adequacy of Testing in Open Source ProjectsAn Empirical Study on the Adequacy of Testing in Open Source Projects
An Empirical Study on the Adequacy of Testing in Open Source Projects
 
What We Learned Building an R-Python Hybrid Predictive Analytics Pipeline
What We Learned Building an R-Python Hybrid Predictive Analytics PipelineWhat We Learned Building an R-Python Hybrid Predictive Analytics Pipeline
What We Learned Building an R-Python Hybrid Predictive Analytics Pipeline
 
2014 toronto-torbug
2014 toronto-torbug2014 toronto-torbug
2014 toronto-torbug
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programs
 
Icse2018 autonomic
Icse2018 autonomicIcse2018 autonomic
Icse2018 autonomic
 
Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...
Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...
Protecting the Protector, Hardening Machine Learning Defenses Against Adversa...
 
It’s Not a Bug, It’s a Feature: Does Misclassification Affect Bug Localization?
It’s Not a Bug, It’s a Feature:Does Misclassification Affect Bug Localization?It’s Not a Bug, It’s a Feature:Does Misclassification Affect Bug Localization?
It’s Not a Bug, It’s a Feature: Does Misclassification Affect Bug Localization?
 
Mining Performance Regression Testing Repositories for Automated Performance ...
Mining Performance Regression Testing Repositories for Automated Performance ...Mining Performance Regression Testing Repositories for Automated Performance ...
Mining Performance Regression Testing Repositories for Automated Performance ...
 
Quality Control of Sequencing Data
Quality Control of Sequencing Data Quality Control of Sequencing Data
Quality Control of Sequencing Data
 
PhD public defense: A Measurement Framework for Analyzing Technical Lag in ...
PhD public defense: A Measurement Framework for  Analyzing Technical Lag in  ...PhD public defense: A Measurement Framework for  Analyzing Technical Lag in  ...
PhD public defense: A Measurement Framework for Analyzing Technical Lag in ...
 

Similar a ICSE2014

ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...
ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...
ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...Concordia University
 
Icse2014 v3
Icse2014 v3Icse2014 v3
Icse2014 v3SAIL_QU
 
Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...
 Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ... Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...
Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...Big Data Spain
 
Automated Discovery of Performance Regressions in Enterprise Applications
Automated Discovery of Performance Regressions in Enterprise ApplicationsAutomated Discovery of Performance Regressions in Enterprise Applications
Automated Discovery of Performance Regressions in Enterprise ApplicationsSAIL_QU
 
Sfeldman performance bb_worldemea07
Sfeldman performance bb_worldemea07Sfeldman performance bb_worldemea07
Sfeldman performance bb_worldemea07Steve Feldman
 
What are Software Testing Methodologies | Software Testing Techniques | Edureka
What are Software Testing Methodologies | Software Testing Techniques | EdurekaWhat are Software Testing Methodologies | Software Testing Techniques | Edureka
What are Software Testing Methodologies | Software Testing Techniques | EdurekaEdureka!
 
Mechanisms for Database Intrusion Detection and Response
Mechanisms for Database Intrusion Detection and ResponseMechanisms for Database Intrusion Detection and Response
Mechanisms for Database Intrusion Detection and ResponseAshish Kamra
 
Aspiring Minds | Automata
Aspiring Minds | Automata Aspiring Minds | Automata
Aspiring Minds | Automata Aspiring Minds
 
SE2018_Lec 19_ Software Testing
SE2018_Lec 19_ Software TestingSE2018_Lec 19_ Software Testing
SE2018_Lec 19_ Software TestingAmr E. Mohamed
 
Crating a Robust Performance Strategy
Crating a Robust Performance StrategyCrating a Robust Performance Strategy
Crating a Robust Performance StrategyGuatemala User Group
 
Performance testing : An Overview
Performance testing : An OverviewPerformance testing : An Overview
Performance testing : An Overviewsharadkjain
 
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...RTTS
 
Владимир Гулин, Mail.Ru Group, Learning to rank using clickthrough data
Владимир Гулин, Mail.Ru Group, Learning to rank using clickthrough dataВладимир Гулин, Mail.Ru Group, Learning to rank using clickthrough data
Владимир Гулин, Mail.Ru Group, Learning to rank using clickthrough dataMail.ru Group
 
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Chakkrit (Kla) Tantithamthavorn
 
Can ML help software developers? (TEQnation 2022)
Can ML help software developers? (TEQnation 2022)Can ML help software developers? (TEQnation 2022)
Can ML help software developers? (TEQnation 2022)Maurício Aniche
 
B2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draftB2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draftSteve Feldman
 
Dataiku at SF DataMining Meetup - Kaggle Yandex Challenge
Dataiku at SF DataMining Meetup - Kaggle Yandex ChallengeDataiku at SF DataMining Meetup - Kaggle Yandex Challenge
Dataiku at SF DataMining Meetup - Kaggle Yandex ChallengeDataiku
 

Similar a ICSE2014 (20)

ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...
ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...
ICSE2014 - Detecting Performance Anti-patterns for Applications Developed usi...
 
Icse2014 v3
Icse2014 v3Icse2014 v3
Icse2014 v3
 
Software metrics
Software metricsSoftware metrics
Software metrics
 
Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...
 Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ... Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...
Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...
 
Automated Discovery of Performance Regressions in Enterprise Applications
Automated Discovery of Performance Regressions in Enterprise ApplicationsAutomated Discovery of Performance Regressions in Enterprise Applications
Automated Discovery of Performance Regressions in Enterprise Applications
 
Sfeldman performance bb_worldemea07
Sfeldman performance bb_worldemea07Sfeldman performance bb_worldemea07
Sfeldman performance bb_worldemea07
 
What are Software Testing Methodologies | Software Testing Techniques | Edureka
What are Software Testing Methodologies | Software Testing Techniques | EdurekaWhat are Software Testing Methodologies | Software Testing Techniques | Edureka
What are Software Testing Methodologies | Software Testing Techniques | Edureka
 
Mechanisms for Database Intrusion Detection and Response
Mechanisms for Database Intrusion Detection and ResponseMechanisms for Database Intrusion Detection and Response
Mechanisms for Database Intrusion Detection and Response
 
Aspiring Minds | Automata
Aspiring Minds | Automata Aspiring Minds | Automata
Aspiring Minds | Automata
 
SE2018_Lec 19_ Software Testing
SE2018_Lec 19_ Software TestingSE2018_Lec 19_ Software Testing
SE2018_Lec 19_ Software Testing
 
Performance testing and rpt
Performance testing and rptPerformance testing and rpt
Performance testing and rpt
 
Crating a Robust Performance Strategy
Crating a Robust Performance StrategyCrating a Robust Performance Strategy
Crating a Robust Performance Strategy
 
Performance testing : An Overview
Performance testing : An OverviewPerformance testing : An Overview
Performance testing : An Overview
 
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
 
Владимир Гулин, Mail.Ru Group, Learning to rank using clickthrough data
Владимир Гулин, Mail.Ru Group, Learning to rank using clickthrough dataВладимир Гулин, Mail.Ru Group, Learning to rank using clickthrough data
Владимир Гулин, Mail.Ru Group, Learning to rank using clickthrough data
 
Web 2.0 Development with IBM DB2
Web 2.0 Development with IBM DB2Web 2.0 Development with IBM DB2
Web 2.0 Development with IBM DB2
 
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
 
Can ML help software developers? (TEQnation 2022)
Can ML help software developers? (TEQnation 2022)Can ML help software developers? (TEQnation 2022)
Can ML help software developers? (TEQnation 2022)
 
B2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draftB2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draft
 
Dataiku at SF DataMining Meetup - Kaggle Yandex Challenge
Dataiku at SF DataMining Meetup - Kaggle Yandex ChallengeDataiku at SF DataMining Meetup - Kaggle Yandex Challenge
Dataiku at SF DataMining Meetup - Kaggle Yandex Challenge
 

Último

Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxRustici Software
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024The Digital Insurer
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...apidays
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 

Último (20)

Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 

ICSE2014

  • 1. Detec%ng  Performance  An%-­‐pa2erns  for   Applica%ons  Developed  Using   Object-­‐Rela%onal  Mapping   1   Mohamed  Nasser,  Parminder  Flora   Tse-­‐Hsun(Peter)  Chen   Ahmed  E.  Hassan  Weiyi  Shang   Zhen  Ming  Jiang  
  • 2. Databases  are  essen%al  in  large-­‐scale   soBware  systems   2   Database  
  • 3. 3   Applica%on  developers  work  with  objects   More  intui*ve  if  we  can   map  objects  directly  to  DB  
  • 4. 4   Object-­‐Rela%onal  Mapping  eliminates   the  gap  between  objects  and  SQL   Database   •   Lots  of  boilerplate  code     •   Need  to  manage  object-­‐DB   transla%ons  manually     Object     Classes   Problem  of  using  raw  SQLs   ORM   Much  less  code  and  shorter   development  %me  
  • 5. ORM  is  widely  used  in  prac%ce   5   •  Java  Hibernate  has  more  than  8   million  downloads   •  In  2013,  15%  of  the  17,000  Java   developer  jobs  require  ORM   experience  (dice.com)   Different  ORM  technologies  
  • 6. An  example  class  with  ORM  code   6   @En%ty   @Table(name  =  “user”)   public  class  User{    @Column(name=“id”)    private  int  id;      @Column(name=“name”)    String  userName;     @OneToMany(fetch=FetchType.EAGER)    List<Team>  teams;      public  void  setName(String  n){      userName  =  n    }   …  other  ge]er  and  se]er  methods   User.java  User  class  is   mapped  to  “user”   table  in  DB   id  is  mapped  to  the   column  “id”  in  the   user  table   A  user  can  belong   to  mul%ple  teams   Eagerly  retrieve   associated  teams   when  retrieving  a   user  object  
  • 7. Accessing  the  database  using  ORM   7   User  u  =  findUserByID(1);   ORM   database   select  u  from  user   where  u.id  =  1;   u.setName(“Peter”);   update  user  set   name=“Peter”   where  user.id  =  1;   Objects   SQLs  
  • 8. Developers  may  not  be  aware  of   database  access   Wow!  I  don’t   need  to  worry   about  DB  code!   ORM  code  with     performance  ang-­‐pa]erns   8   Bad  system     performance   The  performance  difference  can  be  LARGE!  
  • 9. Performance  an%-­‐pa2ern   detec%on  framework   Performance  ang-­‐pa]ern  detecgon  and     ranking  framework     Ranked  according  to     performance  impact   Ranked   Performance   ang-­‐pa]erns   Source     Code   detecgon   ranking   9  
  • 10. Performance  an%-­‐pa2ern   detec%on  framework   Performance  ang-­‐pa]ern  detecgon  and     ranking  framework     Ranked  according  to     performance  impact   Ranked   Performance   ang-­‐pa]erns   Source     Code   detecgon   ranking   10   Discuss  only  one  an%-­‐pa2ern   in  the  presenta%on    (more  pa2erns  in  the  paper)  
  • 11. ORM  excessive  data  an%-­‐pa2ern   Class  User{      @EAGER      List<Team>  teams;   }   User  u  =  findUserById(1);   u.getName();   EOF   11   Objects   SQL   Eagerly  retrieve   teams  from  DB   User  Table   Team  Table   join   Team  data  is  never   used!  
  • 12. Detec%ng  excessive  data   using  sta%c  analysis   12   First  find  all  the  objects  that   eagerly  retrieve  data  from  DB       Class  User{      @EAGER      List<Team>  teams;   }   Idengfy  all  the  data  usages  of   objects     User  user  =  findUserByID(1);   Check  if  the  retrieved  data  is  ever   used   user.getName();   user   team   user   team  
  • 13. Performance  an%-­‐pa2ern   detec%on  framework   Performance  ang-­‐pa]ern  detecgon  and     ranking  framework     Ranked  according  to     performance  impact   Ranked   Performance   ang-­‐pa]erns   Source     Code   detecgon   ranking   13  
  • 14. Performance  an%-­‐pa2ern   detec%on  framework   Performance  ang-­‐pa]ern  detecgon  and     ranking  framework     Ranked  according  to     performance  impact   Ranked   Performance   ang-­‐pa]erns   Source     Code   detecgon   ranking   14  
  • 15. Performance  an%-­‐pa2erns  have   different  impacts   15   User  user_in_1_team  =  findUserByID(1);   Retrieving  1  user  and  1  team   User  user_in_100_teams  =  findUserByID(100);   Retrieving  1  user  and  100  teams!   One  can  only  reveal  performance   impact  by  execu%on  
  • 16. Measuring  the  impact  using  repeated   measurements  and  effect  sizes   16   We  use  effect  sizes  (Cohen’s  D)  to  measure   the  performance  impact   Effect  sizes  =     We  repeat  each  test  30  gmes  to  obtain  stable   measurement  results   Size  of  performance  impact  is  not  defined:   Performance  measurements  are  unstable:  
  • 17. Studied  systems  and  detec%on  results   Large  open-­‐source     e-­‐commence  system   >  1,700  files   >  206K  LOC   Enterprise  system   >  3,000  files   >  300K  LOC   Spring  open-­‐source  system   Online  system  for  a  pet  clinic   51  files          3.3K  LOC   482  excessive  data   >  10  excessive  data     10  excessive  data   17  
  • 18. Performance  impact   Research  ques%ons   Ranks  of  the  ang-­‐pa]erns  at   different  scales   18  
  • 19. Performance  impact   Research  ques%ons   Ranks  of  the  ang-­‐pa]erns  at   different  scales   19  
  • 20. Assessing  an%-­‐pa2ern  impact  by   fixing  the  an%-­‐pa2erns   Execugon   Response  %me  user.getName()   Code  with     ang-­‐pa]erns   fetchType.set(LAZY)   user.getName()   Code  without     ang-­‐pa]erns   20   Execute  test   suite  30  gmes   Response  %me   aOer  fixing  the   anS-­‐paTerns   Avg.  %   improvement  and   effect  sizes   Execugon   Execute  test   suite  30  gmes  
  • 21. Performance  an%-­‐pa2erns  have   medium  to  large  effect  sizes   0%   20%   40%   60%   80%   100%   Excessive  Data   BL   EA   PC   21   %  improvement  in  response  gme   large     effect  size   large     effect  size   medium   effect  size  
  • 22. Performance  impact   Research  ques%ons   Ranks  of  the  ang-­‐pa]erns  at   different  scales   22   Removing  an%-­‐pa2ern     improves  response  by  ~35%  
  • 23. Performance  impact   Research  ques%ons   Ranks  of  the  ang-­‐pa]erns  at   different  scales   23   Removing  an%-­‐pa2ern     improves  response  by  ~35%  
  • 24. Performance  problems  usually  arise   under  large  load   24  
  • 25. Performance  problems  revealed  at   small  scales  may  be  more  serious   25   We  should  first  fix  the  an%-­‐pa2erns  that   have  larger  effects  at  smaller  scales   Input  scales  may  have   exponenSal  effects  on   performance   Different  input  scales   Performance  at   different  input  scales  
  • 26. Comparing  ranked  an%-­‐pa2erns  at   different  data  scales   26   Ranked   Performance   ang-­‐pa]erns   from  small  data   detecgon   ranking   Ranked   Performance   ang-­‐pa]erns   from  large  data   ?   Small  size  input   Large  size  input  
  • 27. An%-­‐pa2erns  have  large  effects  on   performance  even  at  smaller  data  scales   27   0   10   20   30   40   50   0   1   2   3   4   5   Effect  size   Effect  sizes  and  the  ranks  of  the  an%-­‐pa2erns   are  consistent  in  different  data  scales  
  • 28. Performance  impact   Research  ques%ons   Ranks  of  the  ang-­‐pa]erns  at   different  scales   28   Removing  an%-­‐pa2ern     improves  response  by  ~35%   Ranks  of  the  an%-­‐pa2erns   are  consistent  in  different   data  scales  
  • 29. 29  
  • 30. 30  
  • 31. 31  
  • 32. 32