1.
Experiments on Design Pattern Discovery Jing Dong and Yajing Zhao Department of Computer Science The University of Texas at Dallas {jdong, yxz045100}@utdallas.edu

2.
Outline <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Our Approach </li></ul><ul><li>Experiment Data </li></ul><ul><li>Result and Discussion </li></ul><ul><li>Precision and Recall </li></ul><ul><li>Benchmark </li></ul><ul><li>Conclusion and Future Work </li></ul>

3.
Introduction <ul><li>Difficult to understand large computer-based systems </li></ul><ul><li>No original architecture and design </li></ul><ul><li>Patterns help on understanding systems </li></ul><ul><li>Patterns embed future change of systems </li></ul><ul><li>Need of design pattern discovery since patterns are lost in source code </li></ul>

4.
Introduction (Cont’d) <ul><li>DP-Miner </li></ul><ul><li>Experiment data: System source code </li></ul><ul><ul><li>JUnit </li></ul></ul><ul><ul><li>JEdit </li></ul></ul><ul><ul><li>JHotDraw </li></ul></ul><ul><ul><li>Java.AWT </li></ul></ul><ul><li>Such experiments lead to benchmarks </li></ul>

5.
Related Work X Flyweight X Facade X X Chain of Responsibility X X Builder X X X Abstract Factory X X X X X X X X X X Balanyi 2003 X X Command X X X Proxy X X X X X X Composite X X X X X Decorator X X X X Factory Method X X X X Observer X X Prototype X X X X Singleton X X X X Strategy X X X Template Method X X X Visitor X X X X Bridge X X X X Adapter Gueheneuc 2006 Shi 2006 Blewitt 2001 Niere 2002 Heuzeroth 2003 Antoniol 1998 Tsantalis 2006 Authors Tools

6.
Our Approach – DP-Miner <ul><li>Matrix and Weight </li></ul><ul><li>Product of Prime Numbers </li></ul><ul><li>Structural, Behavioral, and Semantic Analysis </li></ul>

7.
Experiment Data <ul><li>JUnit: A regression test framework that helps developers to implement unit tests in Java. </li></ul><ul><li>JEdit: A mature and easy-to-use text editor. </li></ul><ul><li>JHotDraw: A two dimensional graphics frameworks for technical and structured drawing editors written in Java. </li></ul><ul><li>Java.AWT: A library for developing graphical user interfaces for Java programs. </li></ul>

8.
Why Those Four Systems as Data? <ul><li>At the time they were developed, the idea of design patterns is already mature and widely applied. </li></ul><ul><li>Other works on design pattern discovery use one or more of these systems to evaluate their approaches, which allows us to compare and evaluate our experiment results. </li></ul>

9.
Experiment Data <ul><li>System information </li></ul><ul><ul><li>Version </li></ul></ul><ul><ul><li>Number of Classes </li></ul></ul><ul><ul><li>Number of Files </li></ul></ul>484 530 6.0 beta 1 JHotDraw 394 1001 4.2 JEdit 93 126 3.8.2 JUnit 345 570 JDK1.4.2 Java.AWT File # Class # Version Systems

10.
Experiment Result 64 24 6 76 Strategy 0 58 4 JHotDraw 0 24 17 JEdit 3 6 3 JUnit 3 65 21 Java.AWT Composite Bridge Adapter Systems

11.
Experiment Result – Java.AWT Results for Each Analysis Phase N/A 76 65 N/A Semantic Analysis 3 3 92 Composite 76 76 100 Strategy 65 76 100 Bridge 21 21 57 Adapter Final Result Behavioral Analysis Structural Analysis Systems Java.AWT

12.
Experiment Result – JUnit Results for Each Analysis Phase N/A 6 6 N/A Semantic Analysis 3 6 6 3 Final Result 3 9 Composite 6 6 Strategy 6 6 Bridge 3 15 Adapter Behavioral Analysis Structural Analysis Systems JUnit

13.
Experiment Result – JEdit Results for Each Analysis Phase N/A 24 24 N/A Semantic Analysis 0 0 0 Composite 24 24 33 Strategy 24 24 33 Bridge 17 17 80 Adapter Final Result Behavioral Analysis Structural Analysis Systems JEdit

14.
Experiment Result – JHotDraw Results for Each Analysis Phase N/A 64 58 N/A Semantic Analysis 0 0 0 Composite 64 64 74 Strategy 58 64 74 Bridge 4 4 27 Adapter Final Result Behavioral Analysis Structural Analysis Systems JHotDraw

15.
Observations <ul><li>Behavioral analysis phase only deals with a small number of classes, since it’s based on the result of structural analysis phase. </li></ul><ul><li>Significant reduction of the candidates between structural and behavioral analysis of Adapter and Composite pattern than those of Bridge and Strategy patterns, since the former have more behavioral characteristics than the latter do. </li></ul><ul><li>Result of structural and behavioral analysis of the Bridge and Strategy patterns are the same, since they have same structural and behavioral characteristics and only differ in their intents. </li></ul>

16.
Recovery Precisions for JHotDraw <ul><li>Manually checked the results generated by our tool and see whether they are real pattern instances. </li></ul><ul><li>TP: True Positive </li></ul><ul><li>FP: False Positive </li></ul>100% 0 0 Composite 90.63% 6 58 Strategy 91.38% 5 53 Bridge 100% 0 4 Adapter Precision FP TP JHotDraw

17.
Instances Missed <ul><li>Instances manually found but missed by DP-Miner </li></ul><ul><li>Recall: 89.23% </li></ul>DrawingView SelectionTool [7] Locator LocatorConnector [6] Locator LocatorHandle [5] Locator PolygonHandle [4] Connector ConnectionTool [3] Connector ChangeConnectionHandle [2] Connector LineConnection [1] STRATEGY CONTEXT

18.
Reasons of Discrepancy <ul><li>Flexibility of design pattern </li></ul><ul><ul><li>Composite pattern: Collapse both the Component and Composite classes into a single class or not. </li></ul></ul><ul><ul><li>Adapter pattern: Full matching all three roles, i.e. Target, Adapter, and Adaptee, or partial matching of only Adapter and Adaptee. </li></ul></ul><ul><ul><li>Object-oriented programming languages provide special language constructs that greatly simplify the implementation of a design pattern. E.g., Java provides LinkedList, ArrayList, HashMap, and Hashtable, which make the implementation of aggregate elements in patterns easy, but the detection harder. </li></ul></ul>

19.
Benchmark <ul><li>Correct number of pattern instances and their locations are generally not available. </li></ul><ul><li>Lacking benchmarks is the main impediment </li></ul><ul><ul><li>Hard to evaluate and compare design pattern discovery techniques. </li></ul></ul><ul><ul><li>Hard to judge whether an approach discovers all instances and whether the discovered ones are correct. </li></ul></ul><ul><li>To calculate the precision and recall of a pattern matching result, it’s essential to know </li></ul><ul><ul><li>The number of correct pattern instances </li></ul></ul><ul><ul><li>The locations of correct pattern instances </li></ul></ul><ul><li>JHotDraw pattern benchmark </li></ul>

20.
Conclusion <ul><li>We did a series of experiments on design pattern discovery from open-source systems using DP-Miner </li></ul><ul><li>Patterns concerned: Adapter, Bridge, Strategy, Composite </li></ul><ul><li>Experiment data: Java.AWT, JUnit, JEdit, JHotDraw </li></ul><ul><li>We compared our experiment results with others and found several discrepancies. </li></ul><ul><li>We analyzed the issues and discussed possible reasons. </li></ul><ul><li>We argue for benchmarks for design pattern discovery. </li></ul>

21.
Future Work <ul><li>Improve our tool. </li></ul><ul><li>Experiment on other systems </li></ul><ul><li>Manually check results of other systems and perfect the benchmark </li></ul><ul><li>Research on other object-oriented languages, such as C++. </li></ul>

22.
Thank You!

23.
Questions?

24.
Overall Architecture of Our Approach System  Design Pattern match Source Code Intermediate Representation

25.
Structural Analysis – Matrix

26.
1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button

27.
1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button 1 35 1 Composite 1 1 1 Component 1 7 1 Leaf Composite Component Leaf