John's talk at CSMR 2013

1. Analyzing the Eclipse API
Usage: Putting the Developer
in the Loop
John Businge, Alexander Serebrenik,
Mark van den Brand

2. Introduction
Software Engineering and Technology (SET) PAGE 19-6-2014
Software engineering researchers
obtaining convincing evidence

3. Introduction
Software Engineering and Technology (SET) PAGE 29-6-2014
Software engineering researchers
obtaining convincing evidence
Few examples of convincing
evidence

4. Introduction
Software Engineering and Technology (SET) PAGE 39-6-2014
Software engineering researchers
obtaining convincing evidence
Few examples of convincing
evidence
Researchers generate evidence
about A, C, D and E yet developers
care about B

5. Introduction
Software Engineering and Technology (SET) PAGE 49-6-2014
Software engineering researchers
obtaining convincing evidence
Few examples of convincing
evidence
Researchers generate evidence
about A, C, D and E yet developers
care about B
Include developer in the loop

6. Terms used
Software Engineering and Technology (SET) PAGE 59-6-2014
P1 P4P3P2
Eclipse
Framework
Eclipse Third-
party Plug-ins
(ETPs)

7. Terms used
Software Engineering and Technology (SET) PAGE 69-6-2014
P1 P4P3P2
Eclipse
Framework
Eclipse Third-
party Plug-ins
(ETPs)
Eclipse APIs(“good”)
• no “internal”
• stable,
• supported
Eclipse non-APIs (“bad”)
• “internal”
• unstable,
• discouraged,
• unsupported
 P3 – good ETPs
 P1, P2 and P4 – bad ETPs

8. Terms used
• Eclipse provisional API guidelines – document
provided by Eclipse describing the different interfaces
Software Engineering and Technology (SET) PAGE 79-6-2014
Embryonic interfaces

9. Terms used
• Eclipse provisional API guidelines – document
provided by Eclipse describing the different interfaces
Software Engineering and Technology (SET) PAGE 89-6-2014
Embryonic interfaces Battle-headed interfaces

10. Motivation of our study
• Our previous studies about Eclipse API
Usage were based on source code analysis.
• Our observations:
1. 44% of ETPs on SourceForge use bad interfaces.
2. bad ETPs are larger than good ETPs.
3. Good interfaces are indeed very stable in new
framework releases.
4. Bad interfaces are indeed unstable in new
framework releases.
Software Engineering and Technology (SET) PAGE 99-6-2014

11. Motivation of our study
• Omitting developer in the loop in previous
studies
• Why bad ETPs are larger than good
ETPs????
• Why developers use bad interfaces????
Software Engineering and Technology (SET) PAGE 109-6-2014
Include developer in the loop

12. Survey to include developer in loop
• Goal – obtain the state-of-practice Eclipse interface
usage
1. Reasons for use of bad interfaces
2. Difference in characteristics of bad ETPs and good
ETPs
• Research questions
RQ1: Can we observe some relationships in factors related
to Eclipse interface usage in the survey?
RQ2: What are the differences in characteristics between
bad and good ETPs?
RQ3: What are the differences in characteristics between
proprietary and open-source ETPs?
Software Engineering and Technology (SET) PAGE 119-6-2014

13. Survey to include developer in loop
• Goal – obtain the state-of-practice Eclipse interface
usage
1. Reasons for use of bad interfaces
2. Difference in characteristics of bad ETPs and good
ETPs
• Research questions
RQ1: Can we observe some relationships in factors related
to Eclipse interface usage in the survey?
RQ2: What are the differences in characteristics between
bad and good ETPs?
RQ3: What are the differences in characteristics between
proprietary and open-source ETPs?
Software Engineering and Technology (SET) PAGE 129-6-2014

14. Survey factors of investigation
Software Engineering and Technology (SET) PAGE 139-6-2014
No Factors
1 Years of Education
2 Experience as a software engineer
3 Experience as a Java developer
4 Experience as an ETP developer
5 ETP development team size
6 Importance of updating ETP with new Eclipse releases
7 Number of versions of the ETP
8 Number of files of the ETP
9 Awareness of Eclipse guidelines
10 Following Eclipse guidelines

15. Sample Questions in the survey
i. How many years of experience as an Eclipse
product/solution developer do you have?
ii. Are you aware of the “Eclipse Provisional API
Guidelines”?
iii. Do you use non-APIs?
iv. If answer was “NO” in (iii). Do you deliberately
avoid using Eclipse non-APIs?
• Survey Response
− 30 ETP developers fully answered the questionnaire.
Software Engineering and Technology (SET) PAGE 149-6-2014

16. RQ2 and RQ3 - groups
• RQ2 – Difference between good and bad ETPs
Software Engineering and Technology (SET) PAGE 159-6-2014
Use of bad interfaces
Group Respondents
Developers who use bad interfaces 21
Developers who do not use bad interfaces 9
Open-source vs proprietary ETPs
Group Respondents
Developers who develop open-source ETPs 16
Developers who develop proprietary ETPs 14
• RQ3 – Difference between proprietary and open-
source ETPs

17. RQ2 and RQ3 - groups
• The two groups of RQ2 and RQ3 can be compared using
the 10 identified factors.
Software Engineering and Technology (SET) PAGE 169-6-2014

18. Differences between bad and good ETPs –
RQ2
• Hypotheses:
• H0: Developers who use bad interfaces and
those who do not use bad interfaces have the
same average values on each of the identified
factors.
• Ha: There is a difference on the average values
of the identified factors between the developers
that use and do not use bad interfaces.
Software Engineering and Technology (SET) PAGE 179-6-2014

19. Differences between developers that use
and those that do not use bad interfaces
Factors Difference
Years of Education NO
Experience as a software engineer YES
Experience as a Java developer YES
Experience as an Eclipse plug-in developer YES
ETP development team size YES
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files YES
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO
Software Engineering and Technology (SET) PAGE 189-6-2014

20. Differences between developers that use
and those that do not use bad interfaces
Factors Difference
Years of Education NO
Experience as a software engineer YES
Experience as a Java developer YES
Experience as an Eclipse plug-in developer YES
ETP development team size YES
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files YES
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO
Software Engineering and Technology (SET) PAGE 199-6-2014
We reject H0 that Developers who use bad interfaces and
those who do not use bad interfaces have the same
average values on each of the identified factors

21. How different are the two groups of
developers?
• We discovered that developers that use bad interfaces
have higher values on these factors compared to those
that do not use bad interfaces
Software Engineering and Technology (SET) PAGE 209-6-2014
Factors
Experience as a software engineer
Experience as a Java developer
Experience as an Eclipse plug-in developer
ETP development team size
Number of files

22. How different are the two groups of
developers?
• We discovered that developers that use bad interfaces
have higher values on these factors compared to those
that do not use bad interfaces
Software Engineering and Technology (SET) PAGE 219-6-2014
Factors
Experience as a software engineer
Experience as a Java developer
Experience as an Eclipse plug-in developer
ETP development team size
Number of files
1. More experiences
developers
2. Larger ETP
development teams
Previously unanswered
question “Why bad ETPs are
larger than good ETPs?”
Human factors
Developers of bad ETPs

23. Differences between open-source and
proprietary ETPs
• Motivation:
• Previous studies we only considered open-source
software potentially putting generalizability to
proprietary software in jeopardy.
• Hypotheses:
• H0: Developers of open-source and proprietary ETPs
have the same average values on each of the identified
factors.
• Ha: There is a difference on the average values of the
identified factors between the developers of open-
source and proprietary ETPs.
Software Engineering and Technology (SET) PAGE 229-6-2014

24. How different are the two groups of open-
source and proprietary developers?
Software Engineering and Technology (SET) PAGE 239-6-2014
Factors Difference
Years of Education NO
Experience as a software engineer NO
Experience as a Java developer NO
Experience as an Eclipse plug-in developer NO
ETP development team size NO
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files NO
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO

25. How different are the two groups of open-
source and proprietary developers?
• We accept H0 that there is no difference between open-
source and proprietary ETPs in terms of the factors
investigated
Software Engineering and Technology (SET) PAGE 249-6-2014
Factors Difference
Years of Education NO
Experience as a software engineer NO
Experience as a Java developer NO
Experience as an Eclipse plug-in developer NO
ETP development team size NO
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files NO
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO

26. How different are the two groups of open-
source and proprietary developers?
• Current findings clear our doubts on generalizability of
our previous studies.
Software Engineering and Technology (SET) PAGE 259-6-2014

27. Use of bad interfaces
• Asked why developers deliberately avoid using bad
interfaces.
− Answerers were related to, knowing/assuming that the
bad interfaces are unstable.
• Asked why developers deliberately use bad interfaces
− No good interface with necessary functionality.
− Avoid reinventing the wheel.
− To expose these bad interfaces.
− Old bad interfaces are unlikely to disappear
Software Engineering and Technology (SET) PAGE 269-6-2014

28. Use of bad interfaces
• Asked why developers deliberately avoid using bad
interfaces.
− Answerers were related to, knowing/assuming that the
bad interfaces are unstable.
Less experienced developers
• Asked why developers deliberately use bad interfaces
− No good interface with necessary functionality.
− Avoid reinventing the wheel.
− To expose these bad interfaces.
− Old bad interfaces are unlikely to disappear
More experienced developers
Software Engineering and Technology (SET) PAGE 279-6-2014

29. Conclusion
Software Engineering and Technology (SET) PAGE 289-6-2014
Survey was conducted to understand why developers
use bad interfaces.

30. Conclusion
Software Engineering and Technology (SET) PAGE 299-6-2014
Less experienced developers
Survey was conducted to understand why developers
use bad interfaces.
Instability overshadows
benefits

31. Conclusion
Software Engineering and Technology (SET) PAGE 309-6-2014
Less experienced developers
More experienced developers
Survey was conducted to understand why developers
use bad interfaces.
Instability overshadows
benefits
Enjoy benefits despite the
instability

32. Conclusion
Software Engineering and Technology (SET) PAGE 319-6-2014
Less experienced developers
More experienced developers
Survey was conducted to understand why developers
use bad interfaces.
Instability overshadows
benefits
Enjoy benefits despite the
instability
Discovered that there is no difference between
proprietary and open-source ETPs based on the
factors we studied.

33. Thank you for listening
Software Engineering and Technology (SET) PAGE 329-6-2014
• http://www.win.tue.nl/~jbusinge/CSMR13/
Researchers should generate
evidence about topics developers
care about
Include developer in the loop
by