Clone Digger- Tool to discovers duplicate in Python.

1. Clone Digger - discovers
duplicate code in Python.
Sangharsh Agarwal

2. What is Clone Digger?
• Clone Digger aimed to detect similar code in
Python and Java programs. The synonyms for the
term "similar code" are "clone" and "duplicate
code".
• What is software clone?
Two continuous fragments of code form clone if
they are similar enough.
• Why is it important to detect clones?
The presence of clones can increase maintenance
cost of the code. Detected clones can be
refactored or just kept in mind.

3. Why is it important to detect
code clones?
• 5% - 20% of code in software systems are
clones.
• Why do programmers produce clones?
– Development strategy
– Maintenance benefits
– Overcoming underlying limitations
– Cloning by accident
• Why is the presence of code clones bad?
– Errors in the original must be fixed in every clone.

4. Why to use Clone Digger?
• Why should I use Clone Digger to detect clones?
There are several clone detection tools, they are
listed here.
The benefits of Clone Digger are:
1. Variety of handled clone types. Strictly speaking, a
pair of sequences of statements is considered a
clone if one sequence can be obtained from the
other by replacing some small sub expressions.
Particularly, changes of variable and function names
and constants are allowed.
2. It's free (provided under the GPL license).

5. Definition of clone
• Different clone definitions can be classified according
to the level of granularity:
– List of strings
– Sequence of tokens
– Abstract syntax trees (AST)
– Semantic information
• Clone Digger work on the AST level
• Clone Digger consider two sequences of statements
as a clone if one of them can be obtained from the
other by replacing some sub trees.

6. Quick Start
• $ easy_install clonedigger
• $ clonedigger –o output.html <source_path>
• $ firefox output.html
• Additional parameters such as thresholds can be
also set (use --help to know more)

7. Clone Digger – Run from command
prompt -1 (pytests/xdcr)
[root@cen-1413 testrunner]# sudo /root/.local/bin/clonedigger -o output.html pytests/xdcr/
Parsing pytests/xdcr/upgradeXDCR.py ... done
Parsing pytests/xdcr/xdcrbasetests.py ... done
Parsing pytests/xdcr/biXDCR.py ... done
Parsing pytests/xdcr/esXDCR.py ... done
Parsing pytests/xdcr/esbasetests.py ... done
Parsing pytests/xdcr/uniXDCR.py ... done
Parsing pytests/xdcr/checkpointXDCR.py ... done
Parsing pytests/xdcr/observexdcrtest.py ... done
Parsing pytests/xdcr/__init__.py ... done
Parsing pytests/xdcr/xdcrMiscTests.py ... done
Parsing pytests/xdcr/rebalanceXDCR.py ... done
Parsing pytests/xdcr/pauseResumeXDCR.py ... done
376 sequences
average sequence length: 5.752660
maximum sequence length: 69
Number of statements: 2163
Calculating size for each statement... done
Building statement hash... done
Number of different hash values: 284
Building patterns... 1000, 2000, 601 patterns were discovered
Choosing pattern for each statement... 1000, 2000, done
Finding similar sequences of statements...
-----------------------------------------
Warning: sequence of statements starting at pytests/xdcr/xdcrbasetests.py:33
consists of many similar statements.
It will be ignored. Use --force to override this restriction.
Please refer to http://clonedigger.sourceforge.net/documentation.html
-----------------------------------------
369 sequences were found
Refining candidates... 167 clones were found
Removing dominated clones... -36 clones were removed

8. Plugins for Clone Digger
• Integration with Hudson
See a short manual by Pavol Zibrita to learn how to get Clone
Digger and Hudson (continuous integration tool) to work
together.
• Eclipse Plugin
Eclipse Plugin for Clone Digger has been implemented during
Google Summer of Code 2008. See the manual to learn how
install it and how to work with it.

9. What to do with found clones?
• Remove clones by refactoring. Extract method
and Pull Up method can be used.
• Detect library candidates.
• Search for bugs.

10. References
• http://clonedigger.sourceforge.net/index.html
• http://clonedigger.sourceforge.net/download.html
• http://wiki.hudson-ci.
org/display/HUDSON/Meet+Hudson