1.
Discount Expertise Metrics for
Augmenting Community
Interaction
Pei-Yao Hung1
, Mark S. Ackerman1, 2
School of Information1
and Dept. of EECS 2
, University of Michigan, USA
1

2.
2

3.
3

4.
4
He has done almost nothing…

5.
Estimating Expertise
• Profile: curation => needs maintenance
!
(Farrell, Lau, Nusser, Wilcox, & Muller, 2007)
• Artifact: production => needs contribution
!
(Nam & Ackerman, 2007)
• Interaction: participation => needs contribution
!
(Hanrahan, Convertino, & Nelson, 2012; Zhang & Ackerman, 2005; Zhang,
Ackerman, Adamic, & Nam, 2007)
5

6.
Estimating Expertise
• Profile: curation => needs maintenance
!
(Farrell, Lau, Nusser, Wilcox, & Muller, 2007)
• Artifact: production => needs contribution
!
(Nam & Ackerman, 2007)
• Interaction: participation => needs contribution
!
(Hanrahan, Convertino, & Nelson, 2012; Zhang & Ackerman, 2005; Zhang,
Ackerman, Adamic, & Nam, 2007)
6
You need to do/contribute a lot
of work before we can estimate
your expertise!

7.
!
A lot of people consume, but do not contribute.
7

8.
Q: Can we use the browsing history to
estimate levels of technical expertise?
8
docs.python.org
tutorialspoint.com
docs.ggplot2.org
cyclismo.org/tutorial/R
...
github.com
stackoverflow.com
pypi.python.org
ruby-doc.org
...

9.
How do we analyze browsing history?
• Intuition: programmers at different levels might visit
different type of web pages.
• Library/Repository: https://github.com/
• Tutorial: http://www.tutorialspoint.com/python/
• Q&A: http://stackoverflow.com/
• …
9

10.
Q: Can we use the browsing history to
estimate levels of technical expertise?
10
Document
Tutorial
Document
Tutorial
...
Library/Repository
Q&A
Library/Repository
Document
...

11.
Recruiting Participants
• Recruiting: presentations, email lists, and snowball
referrals
• 26 participants who are ‘actively’ programming
• 11 male and 15 female.
• 24 students (undergrad ~ Ph.D.)
• Diverse majors (e.g., Russian, economics, to
computer science)
11

12.
12

13.
Rating Scheme for Expertise
!
!
!
!
!
!
Level Experience
+1 Learning programming for the first time/year
+2
Electrical Engineering (EE) training, 1 year professional
programming experience, or 3 - 4 years assistant/part-
time programming experience
+3
Computer Science (CS) training, or 2 - 3 years
professional programming experience
+4 4+ year professional programming experience
+5 6+ year professional programming experience
13

14.
Analysis
Programming
Relevant Web
Page Visits
(distribution)
Level of
Expertise
Level 1 ~ 5Tutorial, Library/Repository,
Q&A, Document, Blog,
Forum, Search, Other
Logistic
Regression
InterviewBrowsing History
14

15.
If everything goes as expected…
15

16.
16
Tutorial

17.
17
Library/Repository

18.
Logistic Regression (N=26): the relationship between
page visits and expertise isn’t that straight forward.
18

19.
Conservative Classifiers using
Heuristics
• Beginner (Lv 1 or 2) <- over 80% of programming relevant
visits on “Tutorial”
• Expert (Lv 4 or 5) <- over 80% of programming relevant
visits on “Library/Repository and Q&A”
!
!
!
19

20.
20
Well, but he is probably an
expert pythonist…

21.
How can this discount expertise measure
augment community interaction?
• “Inclusive”
• Provides initial expertise estimation to smooth the
process of “blending in” a new community.
• Tracks expertise development in a learning
community (e.g, MOOC).
• Allows ad-hoc network formation.
21

22.
Future Work
• Distinguishes expertise development for different
programming languages.
• Monitors changes of expertise through a
longitudinal study (e.g., 6-12 months).
!
!
22

23.
Takeaways
• Browsing history could be a source for a discount
expertise metric.
• Discount expertise metric has the potential to
argument community interaction.
!
!
23
Contact: Pei-Yao Hung, peiyaoh@umich.edu