SlideShare una empresa de Scribd logo
1 de 27
Descargar para leer sin conexión
amazon
                                               web services



Crowdsourcing for
    improving
 Business Process



October, 2012 - Optimized for 1280x960   -   © Amazon Web Services
_ Presented by
John Hoskins, hoskins @ amazon.com
Mechanical Turk, Amazon




       [         Before we start
                                     ]
                           2
How does it
High     Crowdsourcing           work             Examples
Level

                                                         Questions



Detail           What is MTurk?         How is it used




         [   Our plan for today                          ]

                             3
[                    ]
What is Crowdsourcing?




          4
[   Crowd Labor Trends   ]
            5
1. Crowdsourcing



       6
[   Crowdsourcing:
    Key Advantages   ]

          7
[   1. Elastic Capacity   ]

             8
On and Off           Fast Growth




Variable peaks       Predictable peaks


   [  Staffing Usage Patterns      ]

                 9
On and Off        WASTE     Fast Growth



              Missed
Variable peaks Opportunity Predictable peaks


    [  Staffing Usage Patterns            ]

                       10
On and Off               Fast Growth




Variable peaks           Predictable peaks


   [         Usage Patterns:
             Crowdsourcing             ]

                    11
[   2. On Demand Availability   ]

                12
[                                    ]
    3. Pay as you go, for what you use




                    13
$                             $

    Lower costs               Capital efficiency




Improve productivity        Focus on your business



[       Business Benefits
        of Crowdsourcing                   ]

                       14
2. What is MTurk?



        15
Marketplace for Work
_ Access to global workforce, on demand
_ Quality controls and workflow “building blocks”
_ Pay as you go, only when satisfied with results
_ Programmatic Access (API)



   [      What is Mechanical Turk?                  ]

                              16
_ 500,000 Workers
_ 190+ Countries




       [      Global Workforce
                                 ]
                     17          (as of Apr 19th, 2012)
3. How it works



       18
[   How it works   ]
         19
Highest
Performing


                 Average
MASTERS
                           Unknown




  [          Mturk Workforce   ]
                     20
Highest
  Performing




                   Average
MASTERS                      Unknown
               Your Custom
                Workforce




    [      Custom Workforce      ]
                        21
Speed


      Price

                 Accuracy




[   Balances & Levers
                            ]

                  22
4. How is it used?



        23
Data Management                       Analysis/Research
_ Verification                        _ Sentiment
_ Entry & Collection                  _ Relevance
_ De-dupe                             _ Online Research
_ Algorithm Training




Content Management                    Annotation
_ Moderation (Photos, Content)        _ Tagging
_ Transcription                       _ Classification
_ Localization/Translation



          [       Common Use Cases                       ]
                                 24
How can we assess relevance of
tweets in real-time?
How can we research and categorize
merchants in our database?

How can we verify accuracy of low
confidence OCR scans?

How do we build a dictionary to
categorize ads on the internet?


[    Examples     ]
          25
Categorization


Web Research


Transcription


Categorization and DeDupe



[    Examples     ]
           26
amazon
                               web services



http://www.mturk.com
   John Hoskins, Mechanical Turk
      hoskins@amazon.com




                 27

Más contenido relacionado

Similar a Amazon mechanical turk intro to bpo's v3

ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...
ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...
ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...Sweetlets
 
Aws 101 cloud computing seminar (reference model included)
Aws 101 cloud computing seminar (reference model included)Aws 101 cloud computing seminar (reference model included)
Aws 101 cloud computing seminar (reference model included)Martin Yan
 
Mechanical Turk: Increase Image Sales (April 2013 Webinar)
Mechanical Turk: Increase Image Sales (April 2013 Webinar)Mechanical Turk: Increase Image Sales (April 2013 Webinar)
Mechanical Turk: Increase Image Sales (April 2013 Webinar)Daniel Gray
 
Finding the true value of cloud computing
Finding the true value of cloud computingFinding the true value of cloud computing
Finding the true value of cloud computingDavid Linthicum
 
SAP Product Lifecycle Management: Implementation Tip, Tricks and Lessons
SAP Product Lifecycle Management:  Implementation Tip, Tricks and LessonsSAP Product Lifecycle Management:  Implementation Tip, Tricks and Lessons
SAP Product Lifecycle Management: Implementation Tip, Tricks and LessonsEric Stajda
 
How varian medical_customized_sap_screens_in_hours
How varian medical_customized_sap_screens_in_hoursHow varian medical_customized_sap_screens_in_hours
How varian medical_customized_sap_screens_in_hoursPeter Spielvogel
 
Amazon Web Services - Media Use Cases
Amazon Web Services  - Media Use CasesAmazon Web Services  - Media Use Cases
Amazon Web Services - Media Use CasesSantanu Dutt
 
Mechanical turk increase image sales webinar 040213
Mechanical turk increase image sales webinar 040213Mechanical turk increase image sales webinar 040213
Mechanical turk increase image sales webinar 040213QuicoGray
 
1513 Driving Sap Value With It Bus Partnership
1513 Driving Sap Value With It Bus Partnership1513 Driving Sap Value With It Bus Partnership
1513 Driving Sap Value With It Bus PartnershipSuzanne Passante
 
Business process management center of excellence 2010
Business process management center of excellence 2010Business process management center of excellence 2010
Business process management center of excellence 2010Robert Levy
 
Leverage CRM & Enterprise Apps with Genesys
Leverage CRM & Enterprise Apps with GenesysLeverage CRM & Enterprise Apps with Genesys
Leverage CRM & Enterprise Apps with GenesysTom Leddy
 
Addwiser Presentation Pdf
Addwiser Presentation PdfAddwiser Presentation Pdf
Addwiser Presentation PdfPrince George
 
Addwiser Presentation
Addwiser PresentationAddwiser Presentation
Addwiser Presentationsmita.philip
 
Addwiser Presentation Pdf
Addwiser Presentation PdfAddwiser Presentation Pdf
Addwiser Presentation PdfMonika Singh
 
Webhomes what we do v1.3
Webhomes   what we do v1.3Webhomes   what we do v1.3
Webhomes what we do v1.3Peter Ashworth
 

Similar a Amazon mechanical turk intro to bpo's v3 (20)

ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...
ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...
ASUG 2010 - Efficient Usage Monitoring in a 160.000 User SAP Portal (by Frank...
 
Jz 101 b
Jz 101 bJz 101 b
Jz 101 b
 
Aws 101 cloud computing seminar (reference model included)
Aws 101 cloud computing seminar (reference model included)Aws 101 cloud computing seminar (reference model included)
Aws 101 cloud computing seminar (reference model included)
 
Mechanical Turk: Increase Image Sales (April 2013 Webinar)
Mechanical Turk: Increase Image Sales (April 2013 Webinar)Mechanical Turk: Increase Image Sales (April 2013 Webinar)
Mechanical Turk: Increase Image Sales (April 2013 Webinar)
 
Finding the true value of cloud computing
Finding the true value of cloud computingFinding the true value of cloud computing
Finding the true value of cloud computing
 
SAP Product Lifecycle Management: Implementation Tip, Tricks and Lessons
SAP Product Lifecycle Management:  Implementation Tip, Tricks and LessonsSAP Product Lifecycle Management:  Implementation Tip, Tricks and Lessons
SAP Product Lifecycle Management: Implementation Tip, Tricks and Lessons
 
How varian medical_customized_sap_screens_in_hours
How varian medical_customized_sap_screens_in_hoursHow varian medical_customized_sap_screens_in_hours
How varian medical_customized_sap_screens_in_hours
 
Amazon Web Services - Media Use Cases
Amazon Web Services  - Media Use CasesAmazon Web Services  - Media Use Cases
Amazon Web Services - Media Use Cases
 
Mechanical turk increase image sales webinar 040213
Mechanical turk increase image sales webinar 040213Mechanical turk increase image sales webinar 040213
Mechanical turk increase image sales webinar 040213
 
1513 Driving Sap Value With It Bus Partnership
1513 Driving Sap Value With It Bus Partnership1513 Driving Sap Value With It Bus Partnership
1513 Driving Sap Value With It Bus Partnership
 
Daybreak Pitch Deck.pdf
Daybreak Pitch Deck.pdfDaybreak Pitch Deck.pdf
Daybreak Pitch Deck.pdf
 
Business process management center of excellence 2010
Business process management center of excellence 2010Business process management center of excellence 2010
Business process management center of excellence 2010
 
Webhomes - What We Do
Webhomes - What We DoWebhomes - What We Do
Webhomes - What We Do
 
LMC
LMCLMC
LMC
 
Leverage CRM & Enterprise Apps with Genesys
Leverage CRM & Enterprise Apps with GenesysLeverage CRM & Enterprise Apps with Genesys
Leverage CRM & Enterprise Apps with Genesys
 
Addwiser Presentation Pdf
Addwiser Presentation PdfAddwiser Presentation Pdf
Addwiser Presentation Pdf
 
Addwiser Presentation
Addwiser PresentationAddwiser Presentation
Addwiser Presentation
 
Addwiser Presentation
Addwiser Presentation Addwiser Presentation
Addwiser Presentation
 
Addwiser Presentation Pdf
Addwiser Presentation PdfAddwiser Presentation Pdf
Addwiser Presentation Pdf
 
Webhomes what we do v1.3
Webhomes   what we do v1.3Webhomes   what we do v1.3
Webhomes what we do v1.3
 

Último

Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...allensay1
 
Falcon Invoice Discounting: The best investment platform in india for investors
Falcon Invoice Discounting: The best investment platform in india for investorsFalcon Invoice Discounting: The best investment platform in india for investors
Falcon Invoice Discounting: The best investment platform in india for investorsFalcon Invoice Discounting
 
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...Dave Litwiller
 
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...lizamodels9
 
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876dlhescort
 
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableCall Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableSeo
 
Value Proposition canvas- Customer needs and pains
Value Proposition canvas- Customer needs and painsValue Proposition canvas- Customer needs and pains
Value Proposition canvas- Customer needs and painsP&CO
 
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...amitlee9823
 
Organizational Transformation Lead with Culture
Organizational Transformation Lead with CultureOrganizational Transformation Lead with Culture
Organizational Transformation Lead with CultureSeta Wicaksana
 
Eluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort Service
Eluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort ServiceEluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort Service
Eluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort ServiceDamini Dixit
 
FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756dollysharma2066
 
Falcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to ProsperityFalcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to Prosperityhemanthkumar470700
 
Phases of Negotiation .pptx
 Phases of Negotiation .pptx Phases of Negotiation .pptx
Phases of Negotiation .pptxnandhinijagan9867
 
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...Sheetaleventcompany
 
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...daisycvs
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Neil Kimberley
 
Uneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration PresentationUneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration Presentationuneakwhite
 

Último (20)

Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
 
Falcon Invoice Discounting: The best investment platform in india for investors
Falcon Invoice Discounting: The best investment platform in india for investorsFalcon Invoice Discounting: The best investment platform in india for investors
Falcon Invoice Discounting: The best investment platform in india for investors
 
Falcon Invoice Discounting platform in india
Falcon Invoice Discounting platform in indiaFalcon Invoice Discounting platform in india
Falcon Invoice Discounting platform in india
 
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
 
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
 
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
 
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableCall Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
 
Value Proposition canvas- Customer needs and pains
Value Proposition canvas- Customer needs and painsValue Proposition canvas- Customer needs and pains
Value Proposition canvas- Customer needs and pains
 
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
 
Organizational Transformation Lead with Culture
Organizational Transformation Lead with CultureOrganizational Transformation Lead with Culture
Organizational Transformation Lead with Culture
 
Eluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort Service
Eluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort ServiceEluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort Service
Eluru Call Girls Service ☎ ️93326-06886 ❤️‍🔥 Enjoy 24/7 Escort Service
 
FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756
 
Falcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to ProsperityFalcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to Prosperity
 
Phases of Negotiation .pptx
 Phases of Negotiation .pptx Phases of Negotiation .pptx
Phases of Negotiation .pptx
 
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
 
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023
 
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabiunwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
 
VVVIP Call Girls In Greater Kailash ➡️ Delhi ➡️ 9999965857 🚀 No Advance 24HRS...
VVVIP Call Girls In Greater Kailash ➡️ Delhi ➡️ 9999965857 🚀 No Advance 24HRS...VVVIP Call Girls In Greater Kailash ➡️ Delhi ➡️ 9999965857 🚀 No Advance 24HRS...
VVVIP Call Girls In Greater Kailash ➡️ Delhi ➡️ 9999965857 🚀 No Advance 24HRS...
 
Uneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration PresentationUneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration Presentation
 

Amazon mechanical turk intro to bpo's v3

  • 1. amazon web services Crowdsourcing for improving Business Process October, 2012 - Optimized for 1280x960 - © Amazon Web Services
  • 2. _ Presented by John Hoskins, hoskins @ amazon.com Mechanical Turk, Amazon [ Before we start ] 2
  • 3. How does it High Crowdsourcing work Examples Level Questions Detail What is MTurk? How is it used [ Our plan for today ] 3
  • 4. [ ] What is Crowdsourcing? 4
  • 5. [ Crowd Labor Trends ] 5
  • 7. [ Crowdsourcing: Key Advantages ] 7
  • 8. [ 1. Elastic Capacity ] 8
  • 9. On and Off Fast Growth Variable peaks Predictable peaks [ Staffing Usage Patterns ] 9
  • 10. On and Off WASTE Fast Growth Missed Variable peaks Opportunity Predictable peaks [ Staffing Usage Patterns ] 10
  • 11. On and Off Fast Growth Variable peaks Predictable peaks [ Usage Patterns: Crowdsourcing ] 11
  • 12. [ 2. On Demand Availability ] 12
  • 13. [ ] 3. Pay as you go, for what you use 13
  • 14. $ $ Lower costs Capital efficiency Improve productivity Focus on your business [ Business Benefits of Crowdsourcing ] 14
  • 15. 2. What is MTurk? 15
  • 16. Marketplace for Work _ Access to global workforce, on demand _ Quality controls and workflow “building blocks” _ Pay as you go, only when satisfied with results _ Programmatic Access (API) [ What is Mechanical Turk? ] 16
  • 17. _ 500,000 Workers _ 190+ Countries [ Global Workforce ] 17 (as of Apr 19th, 2012)
  • 18. 3. How it works 18
  • 19. [ How it works ] 19
  • 20. Highest Performing Average MASTERS Unknown [ Mturk Workforce ] 20
  • 21. Highest Performing Average MASTERS Unknown Your Custom Workforce [ Custom Workforce ] 21
  • 22. Speed Price Accuracy [ Balances & Levers ] 22
  • 23. 4. How is it used? 23
  • 24. Data Management Analysis/Research _ Verification _ Sentiment _ Entry & Collection _ Relevance _ De-dupe _ Online Research _ Algorithm Training Content Management Annotation _ Moderation (Photos, Content) _ Tagging _ Transcription _ Classification _ Localization/Translation [ Common Use Cases ] 24
  • 25. How can we assess relevance of tweets in real-time? How can we research and categorize merchants in our database? How can we verify accuracy of low confidence OCR scans? How do we build a dictionary to categorize ads on the internet? [ Examples ] 25
  • 27. amazon web services http://www.mturk.com John Hoskins, Mechanical Turk hoskins@amazon.com 27