1. An Exploratory Study of
Backtracking Strategies
Used by Developers
YoungSeok Yoon (youngseok@cs.cmu.edu)
Institute for Software Research (ISR)
Brad A. Myers (bam@cs.cmu.edu)
Human-Computer Interaction Institute (HCII)
CHASE 2012 School of Computer Science
June 2nd, 2012. Carnegie Mellon University

2. What is Backtracking?
• When developers write source code, they often need
to step back when something goes wrong
• We define backtracking as:
going back to an earlier state of
at least some part of the code,
either by removing inserted code
or by restoring removed code
CHASE 2012 2

3. Examples of Backtracking
Correcting minor mistakes
Tuning parameters
Trying out methods
while learning an unfamiliar API
Experimenting with different classes
(e.g. UI layout classes)
CHASE 2012 3

4. Existing Support for Backtracking
CHASE 2012 4

5. Existing Support for Backtracking
Conventional Undo Command
• Only works for the most recent changes
• Have to undo all changes in order
• Undone changes are thrown away after making a new edit
CHASE 2012 5

6. Existing Support for Backtracking
Version Control
• The desired version needs to be committed in advance
• Unstable code should not be committed to the repository
• Inappropriate when wanted and unwanted code is mixed
CHASE 2012 6

7. Existing Support for Backtracking
Manual Edit / Commenting Out
• Have to manually find all the relevant code
• Redo the work by hand
CHASE 2012 7

8. Research Question
As a first step towards
supporting more robust backtracking tools…
How can we better understand
developers' backtracking behaviors?
CHASE 2012 8

9. STUDY #1.
LAB STUDY
CHASE 2012 9

10. Study Settings
• 12 participants, all MS/PhD students at CMU
• 2 hrs, THINK ALOUD
• Code base: Paint program [Ko 2005][Fogarty 2005]
CHASE 2012 10

11. Tasks
• Two sets of features each done in two different ways,
▪ intended to lead participants to backtrack
Feature 1 (F1) Feature 2 (F2)
“Thickness Control” “X, Y coordinates indicator”
F1-1 F2-1
F1-2 F2-2
CHASE 2012 11

12. Commenting Out
• 7 out of 12 participants preferred to comment out
rather than to delete code
• 3 main reasons for commenting out
Reuse Good Bad
later example example
e.g. activating one leave the code as a remind themselves
alternative at a time good example that the code was
not good
CHASE 2012 12

13. Problems while Backtracking
• Difficulties in finding relevant code fragments that
have to be reverted when:
▪ there are two or more similar looking code fragments
▪ relevant code fragments are scattered across multiple
locations
• Forgetting to remove temporary code
▪ e.g., Debug output statements such as:
• System.out.println
• JOptionPane.showMessageDialog
CHASE 2012 13

14. Backtracking as Restoring Code
• 2 participants accidentally deleted some code
• One participant failed to restore the code from
memory after spending about 10 mins.
• The other succeeded using the undo command
because he knew:
▪ what the code looked like
▪ that it was deleted very recently
CHASE 2012 14

15. Backtracking as Restoring Code
• When trying to restore deleted code, participants often
remembered some features about the desired code
▪ the original location of the code
▪ the surrounding code
▪ the names of some code elements (variable, method, …)
▪ what the code looked like
• Participants also reproduced the same code fragments from
memory repeatedly
▪ e.g. complex formatted string looking like
“(X, Y) = ” + x + “, ” + y + “)”
CHASE 2012 15

16. STUDY #2.
ONLINE SURVEY
CHASE 2012 16

17. Demographics
Female • 48 professional
8%
programmers
Male • 13 yrs exp. on average
92%
• recruited from online
developer communities
CHASE 2012 17

18. Flexibility of their work
Highly specified before I
The results that my code start developing
achieves
The architecture of the code Specified, but some
itself opportunity to negotiate
changes
Which APIs/libraries are used Somewhat flexible, within
broad constraints
Which elements of the API
are used Highly flexible
The details of the
implementation code Completely unspecified; I
can do whatever I want
0% 20% 40% 60% 80% 100%
CHASE 2012 18

19. How often do they have to backtrack?
Tuning parameters
All the time
Figuring out how to use an API
correctly
Frequently
Fixing code just added, because it is
not working Sometimes
Trying out various user interface
designs Rarely
Trying to find an appropriate algorithm Never
0% 20% 40% 60% 80% 100%
CHASE 2012 19

20. Backtracking Strategies
• For each backtracking situation, only a few strategies
were primarily used
▪ e.g. “select & overtype” when tuning parameters
▪ This hints at how to detect a certain backtracking situation
• Other strategies in the open-ended responses
▪ using Boolean variables to temporarily turn on/off code
▪ writing a small code snippet (outside the main project)
▪ moving parameters out of the code
▪ writing unit tests to see how the API works
CHASE 2012 20

21. Future Work
• Conducting an extended version of this study
▪ A field study is ongoing, using our Eclipse logging tool
called FLUORITE [Yoon 2011]
▪ If you are interested in participating, please let me know!
• Designing new backtracking tools, based on the
needs identified in these studies
CHASE 2012 21

22. Conclusion
• Backtracking happens a lot while coding
• The backtracking strategies used by developers are
still manual and error-prone
• Better backtracking tools would help developers
write code more correctly and effectively
CHASE 2012 22

23. Questions?
Contact Info: YoungSeok Yoon
E-mail: youngseok@cs.cmu.edu
Web: http://www.cs.cmu.edu/~yyoon1/
FLUORITE logging tool can be found at:
http://www.cs.cmu.edu/~fluorite/
Thanks for funding from:
CHASE 2012 23