These are the slides of my talk at international Labour Processes Conference (ILPC) on 4 April 2016 in Berlin, Germany.

1. Understanding crowdworkers’
learning practices
Anoush Margaryan
Caledonian Academy
Glasgow Caledonian University, UK
@anoush

2. Vakharia, D. & Lease, M. (2015)Beyond Mechanical Turk: An analysis of paid crowdwork
platforms.

3. Types of crowdwork
• Microwork (AMT, CrowdFlower):
– microtasks (automated and routine)
– short timeframe (seconds or minutes)
– algorithm-governed
– anonymous
– small pay
– lower barriers to entry
• Online freelancing (Upwork – formerly oDesk,
Freelancer):
– simple or complex projects
– longer timeframe (hours, days or months)
– worker and employer identities more transparent
– professional skills often required
Kuek, S. C., et al. (2015). The global opportunity in online outsourcing. Washington, DC:
World Bank.

4. Examples of crowdwork tasks
• Data entry
• Graphic design (logos, brochures, web
products and mobile apps, drawing and
illustration)
• Video production
• Transcription
• Writing texts
• Rating sentiment about a product
• Screening and tagging images
• Searching and extracting information from
websites
• Completing surveys

5. Microwork vs online freelancing
TASKS Microwork Online
freelancing
High complexity
Medium complexity
Low complexity

6. Economic significance
• Estimated gross revenue (in 2013) -
$2 billion
• Forecast for 2016 - $4.8 billion ($4.4
billion for online freelancing and $0.4
billion for microwork)
• Estimated number of crowdworkers
worldwide – 48 million
Kuek, S. C., et al. (2015). The global opportunity in online outsourcing. Washington, DC:
World Bank.

7. Demographics
• 2/3 of crowdworkers come from:
– USA
– India
– Philippines
• Predominately male, below 35 y.o.
• Education - university degree:
– 75% of online freelancers
– 33% of microworkers
• Hours worked: majority work for fewer than 20 hrs/w
• Motivation -income generation
– For microworkers – supplementary income
– For online freelancers – the only means of income
Kuek, S. C., et al. (2015). The global opportunity in online outsourcing. Washington, DC:
World Bank.

8. Why study crowdworker learning?
• Worker self-organisation to learn when there is
no access to traditional workplace learning support
(eg training, access to experienced colleagues)
• Social dimension of workplace learning in
crowdwork
• Learning-intensity of crowdwork
• Enhancing the learning potential of crowdwork

9. Research design
1. Self-regulated Learning at Work Questionnaire,
SRLWQ (Fontana et al, 2015):
– Personal details (education, profession, employment
status, country, experience level, etc)
– SRL strategies: planning, performance and reflection
– Workplace Learning Activities
– Nature of work tasks (routinisation, decision-making,
expertise & collaboration required, etc)
2. Interviews with crowdworkers (learning pathways,
learning and work goals, motives, aspirations)
3. Interviews with platform providers and employers
(to contextualise crowdworkers’ perspective)
4. Document analysis (training provisions, policies)

10. Hypotheses
• There are qualitative and quantitative
differences in learning practices
within microwork and online
freelancing
• Learning-intensity of microwork is
lower than that of online freelancing

11. SRLW Questionnaire
Fontana, P., Milligan, C., Littlejohn, A., & Margaryan, A. (2015). Measuring self-regulated learning in the
workplace. International Journal of Training and Development, 19(1), 32-52
Margaryan, A., Milligan, C., & Littlejohn, A. (2011). Validation of Davenport’s classification structure of
knowledge-intensive processes. Journal of Knowledge Management, 15(4), 568-581.
1. Demographic questions – 10 items
2. Workplace learning activity – 14
items
3. Self-regulated learning – 33 items
4. Typology of work tasks – 10 items

12. Pilot survey response
Microwork
(CrowdFlower)
–Total n=127
–Incomplete n=30
–Suspicious n=4
–Usable n=93
Online freelancing
(Upwork):
–Total n=30
–Incomplete n=8
–Suspicious n=2
–Usable n=20

13. Demographics
Upwork (n=20)
• Age range: 1949-1997
– 75% Millennials (1980+)
– 20% Generation X (1965-79)
– 5% Baby Boomers (1946-64)
• Gender:
– Male 40%
– Female 60%
• Countries:
– 25% USA
– 20% Philippines
– 10% each India, Romania,
Serbia
– 5% each Bosnia, Canada,
Pakistan, Thailand, UK
CrowdFlower (n=93)
• Age range: 1950-1996
– 61% Millennials (1980+)
– 32% Generation X (1965-79)
– 7% Baby Boomers (1946-64)
• Gender:
– Male 74%
– Female 26%
• Countries:
– 10% each India, Serbia, Bosnia
– Bulgaria, Venezuela, Greece,
Indonesia, Macedonia, Russia,
Turkey, Bangladesh, Canada, Italy,
Romania, Ukraine, Croatia, Spain,
Vietnam, Algeria, Brazil, Czech
Republic, Estonia, Mexico,
Portugal, Slovakia, Sri Lanka,
Uruguay, USA

14. Education
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Secondary School Undergrad Masters Professional
Qualifications
Doctorate
Upwork
CrowdFlower

15. Employment status
0%
10%
20%
30%
40%
50%
60%
70%
80%
Freelancer FT Employee PT Employee Retired Diasbled Student Homemaker
Upwork
CrowdFlower

16. Professional background: Online
freelancers

17. Professional background:
microworkers

18. Experience on platform
0%
10%
20%
30%
40%
50%
60%
70%
1 yr or less 2-3 yrs 4-10 yrs 11+ yrs
Upwork
ClowdFlower

19. WLA: Online freelancers
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acquiring new information
Working alone
Working with others
Following new developments in your field
Performing new tasks
Asking others for advice
Attending a F2F training course
Attending free online courses/tutorials
Using paid online tutorials
Self-studying professional literature
Replicating others’ strategies
Trial and error
Reflecting on your actions
Receiving feedback on your work
Never
Sometimes
Regularly

20. WLA: microworkers
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acquiring new information
Working alone
Working with others
Following new developments in your field
Performing new tasks
Asking others for advice
Attending a F2F training course
Attending free online courses/tutorials
Using paid online tutorials
Self-studying professional literature
Replicating others’ strategies
Trial and error
Reflecting on your actions
Receiving feedback on your work
Never
Sometimes
Regularly

21. My work is/involves…
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Mostly routine tasks
Highly reliant on formal rules/procedures
Limited/no freedom to decide what should be done
Mostly systematically repeatable
Reliant on formal processes/standards
Dependent on integration across disciplinary boundaries
Improvisational
Highly reliant on my deep expertise/judgment
Dependent on collaborating with others
Highly reliant on my individual expertise/experience
Upwork
CrowdFlower

22. Conclusions
• Considerable learning and professional
development appears to occur in both
microwork and online freelancing
• Crowdwork is not lonely work - considerable
amount of collaboration appears to take place
• Some quantitative differences in learning
activities within microwork and online
freelancing
• More analysis and data collection needed…

23. Next steps
• Full study (2016-2018)
– Survey with a larger sample and more comprehensive
analyses
– In-depth interviews with crowdworkers
– Interviews with platform providers and employers
– Study of platform documentation and policies
• Compare results to data from previous survey samples
in traditional occupations
– Financial analysts (WLBK project, 2012-2013)
– Researchers, scientists and consultants (SRL@W
project, 2014-2015)
– Medics and clinicians (Gates Foundation study, 2014-
2015)