2. 2
Before we start
][
2
amazon
web services
Daniel Gray, Principal Business Development
Amazon Mechanical Turk, Seattle, WA
djgray@amazon.com
T: 310-800-4832
4. 44
][Amazon.com’s three businesses
Consumer
Business
Tens of millions of
active customer
accounts
Eight countries:
US, UK, Germany,
Japan, France,
Canada, China, Italy
Seller
Business
Sell on Amazon
websites
Use Amazon
technology for
your own retail
website
Leverage Amazon’s
massive fulfillment
center network
IT Infrastructure
Business
Cloud computing
infrastructure for
hosting web-scale
solutions
Hundreds of
thousands of
registered
customers in over
190 countries
Marketplace
for Work
amazon
web services
5. 5
Our plan for today
][
5
Section I: Overview
1) Level-setting on ‘crowdsourcing’
2) What is Amazon Mechanical Turk
Section II: Workshop Intro
1) Best Practices
2) Workshop
10. 10
Common Use Cases[ ]
10
Data Management
_ Verification
_ Entry & Collection
_ De-dupe
_ Algorithm Training
Annotation
_ Tagging
_ Classification
Content Management
_ Moderation (Photos, Content)
_ Transcription
_ Writing/Editing
Analysis/Research
_ Sentiment
_ Relevance
_ Online Research
11. 1111
2. Amazon Mechanical Turk
More
Technical
More
Business Crowdsourcing
What is Amazon
Mechanical Turk?
Workshop Intro
Best Practices
12. 1212
[ Mechanical
Turk Ecosystem]
• Manual via WebUI
• Automate via API
How to Connect
• Self-serve
• Assisted
i. Partner network
ii. Workshops
How to Engage
• Documentation
• Best Practices
• Developer’s Resources
• Requestor UI Guide (account setup)
• Sandbox
How to get Support
13. 1313
$
Variable expense
Transform fixed human
capital costs into variable
expense
$
Economies of scale
A lower variable expense
than companies can
achieve themselves
Speed and agility
On-demand access to
scalable workforce &
higher productivity
Focus on business
Redeploy valuable
human resources to core
business
Global Reach
500K Workers across
190 countriesScalability
No need to guess
capacity requirements =
avoid waste
Fast Growth
Predictable peaks
On and Off
Variable peaks
Results’ Accuracy
Drive fast, accurate work
results at competitive
prices without long-term
contracts or minimum
commitments
17. 1717
[ Workshop Intro]
• Strategy
• Business issues & objectives: Establish priorities & goals for the core metrics (input & output) for
measurement and optimization.
• Planning & Design
• Establish blueprints for pilot, with scalability in mind, including:
• Visioning – Assessment of current workflow(s) and success metrics; disintegrating the stages into
Mechanical Turk workflow steps and ensuring they’re optimized for microtasking; integration with and
development of Mechanical Turk workflow(s).
• Scoping & Planning – API-connectivity, address HIT design, Worker instructions and layout,
adjudication strategies, Known Answers, Masters, workflow and best practices implementation.
• Implementation & Launch
• Implement the best practices and designs established in the planning phase; as Advisors, we help to
refine implementation according to the design.
• Continuous Improvement
• Adjust & optimize the prioritized metrics based on BI reporting; observing results, measuring against
goals, root causing gaps, and adjusting:
• Cost efficiency
• Results’ accuracy
• Productivity
18. 18
Thank you
][
18
Daniel Gray, Principal Business Development
Amazon Mechanical Turk, Seattle, WA
djgray@amazon.com
T: 310-800-4832
Notas del editor
To understand Amazon Mechanical Turk, firstit’s helpful to understand Amazon is a company of marketplaces - we have three macro and distinct businesses:1. our Consumer/Retail business (with tens of millions of active customers around the world)2. our Seller business (where third parties sell on Amazon websites, using Amazon technology for their own retail websites, leveraging Amazon’s massive fulfillment center network)3. our IT Infrastructure business (where cloud computing infrastructure for hosting web-scale solutions serves hundreds of thousands of registered customers around the world).Amazon Mechanical Turk is a marketplace for work, and is part of Amazon Web Services (AWS). Like our cloud computing infrastructure which offers computing power, Amazon Mechanical Turk is the infrastructure to access a global, scalable, on-demand workforce for human power.
Data management: This includes data or algorithm training…With Mechanical Turk you can quickly develop large sets of training data to train your algorithmsAnalysis/research: perform sentiment analysis, monitor customer feedbackContent management: moderate your collections for appropriateness/accuracy to market, improve search engine-level discoverability by adding descriptions to your collections in addition to tags/keywordsAnnotation: tag collections with keywords, improving classification/categorization. Again, discoverability is critical to image sales - Mechanical Turk Workers can quickly and accurately add attribute descriptions (or tags) to all types of content so that it can be easily indexed and searched. Example Tasks: Generate keywords for an image, advertisement, or website. Identify objects found in an image for easier searching and advertising targeting. Timestamp and tag sections of a video or audio clip based on content.
_ Mechanical Turk is a marketplace for work – think of it as the infrastructure layer upon which Partners like Tagasauris offer a technology solution to Requesters (those looking for workers) to access a scalable, global, on-demand Workforce _ Mechanical Turk provides access to the tool required to build workflows and control quality_ Mechanical Turk provides the payment administration (though Amazon Payments)_ …and lastly, it can handle millions of tasks, with web or API-connectivity options_The process includes breaking down a project into tasks, publishing the tasks via HIT’s (human intelligence tasks). Depending on the qualifications set for your tasks, either a defined or undefined group of Workers accept the assignments, submit their results for approval, and the process continues.What are the Key advantages that deliver business benefits: 1. VARIABLE EXPENSE: Lower Costs_ Transform fixed [human capital] costs into variable costs_ Offers a pay-as-you-go model2. GAINECONOMIES OF SCALE: as your collections continue to grow, scale more efficiently3. SPEED AND AGILITY_ On-demand (vs. on the bench) access to a scalable workforce_ Results-focus (vs. time focus) emphasizes productivity; Workers can work in parallel, around the globe, around the clock4. FOCUS ON YOUR BUSINESSWith increased efficiency, productivity, and lower costs—you have more time to focus on critical business objectives, increasing the pace of your innovation._ Experiment Often_ Fail quickly at a low cost= More Innovation5. GLOBAL REACH: 500k Workers across 190 countries6. ACCURACY: drive fast, accurate work results at competitive prices without long term contracts or minimum commitments7. SCALABILITY: avoid waste_ Burst up and down according to the capacity you need, when you need it_ 100% Efficiency_ Independent of your workload type, maintain the flexibility your business demandsThis is the ‘business’ buzz around Mechanical Turk It’s these business benefits that make Mechanical Turk a strategic initiative for companies like Comscore, Twitter, Linkedin…and the difference between the “disruptors” vs. the “disrupted.”