Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
RecSys Challenge 2016
http://recsyschallenge.com - @recsyschallenge
Martha, Róbert, András, Daniel, Fabian
RecSys, Boston,...
Agenda
Proceedings: titanpad.com/recsyschallenge2016
• 09:00-10:30 Welcome + Short presentations
• 10:30-11:00 Coffee brea...
Job recommendations
Job recommendations
Example: item (job posting)
5
RecSys Challenge
Given a user, the goal is to predict those job postings that the
user will interact with.
6
?
Scala Dev,
...
Datasets
1. Training data:
• User demographics (jobtitle, discipline, industry, career level, # CV entries,
country, regio...
Interaction Data
includes interactions that were not performed on recommendations
8
1"
10"
100"
1000"
10000"
100000"
10000...
Evaluation Measure
Mixture of…
- Precision@k (k = 2, 4, 6, 20)
= fraction of relevant items in the top k
- Recall@30 = fra...
Who participated?
• 119 teams participated (366 teams registered)
• Countries:
 USA (25%)
 Germany (11%)
 China (9%)
 ...
Top score over time
11
0"
100"
200"
300"
400"
500"
600"
700"
0"
500000"
1000000"
1500000"
2000000"
2500000"
0" 5" 10" 15" ...
Number of submissions per team
12
0"
100"
200"
300"
400"
500"
600"
0" 20" 40" 60" 80" 100" 120"
number'of'submissions'
ran...
Overlap with XING’s recommender
13
0"
2000"
4000"
6000"
8000"
10000"
12000"
0" 5" 10" 15" 20" 25" 30"
number'of'users'
num...
Outlook for 2017
• Current plan:
 Domain: again job recommendations
 Additional perspectives:
 is the user a good candi...
Thank you to PC!
• Alejandro Bellogín, Universidad Autónoma de Madrid, Spain
• Paolo Cremonesi, Politecnico di Milano, Ita...
16
Thank you to
RecSys Challenge
participants!
Agenda
• 09:00-10:30 Welcome + Short presentations
• 10:30-11:00 Coffee break
• 11:00-12:30 Full Papers
• 12:30-14:00 Lunc...
Thank you
@recsyschallenge
http://recsyschallenge.com
www.xing.com
Próxima SlideShare
Cargando en…5
×

RecSys Challenge 2016

The ACM RecSys Challenge 2016 was focussing on the problem of job recommendations: given a user, return a ranked list of jobs that the user is likely to be interested in. More than 100 teams actively participated and submitted solutions. All the winning teams used an ensemble of recommender strategies (e.g. learning to rank approaches, matrix factorization techniques, etc.). More details: http://2016.recsyschallenge.com/

  • Inicia sesión para ver los comentarios

RecSys Challenge 2016

  1. 1. RecSys Challenge 2016 http://recsyschallenge.com - @recsyschallenge Martha, Róbert, András, Daniel, Fabian RecSys, Boston, September 2016
  2. 2. Agenda Proceedings: titanpad.com/recsyschallenge2016 • 09:00-10:30 Welcome + Short presentations • 10:30-11:00 Coffee break • 11:00-12:30 Full Papers • 12:30-14:00 Lunch break • 14:00-15:30 Full Papers / Top 3 • 15:30-16:00 Coffee break • 16:00-17:30 Panel Discussion: RecSys Challenge ‘17 2
  3. 3. Job recommendations
  4. 4. Job recommendations
  5. 5. Example: item (job posting) 5
  6. 6. RecSys Challenge Given a user, the goal is to predict those job postings that the user will interact with. 6 ? Scala Dev, Hamburg job postings Scala Engineer 2 months of impressions & interactions click bookmark
  7. 7. Datasets 1. Training data: • User demographics (jobtitle, discipline, industry, career level, # CV entries, country, region) [1M] • Job postings (title, discipline, industry, career level, country region) [1M] • Interactions (user_id, item_id, interaction_type, timestamp) [10M, 2 months] • Impressions (user_id, item_id, week) [30M, 2 months] 2. Task files: • Users (= User IDs for whom recommendations should be computed) [150k] • Candidate items (= item IDs that are allowed to be recommended) [300k] 3. Solution (secret) • Interactions (user_id, item_id) [1M, 1 week] Anonymization (Strings  IDs; users and interactions are enriched with artitificial noise) 7
  8. 8. Interaction Data includes interactions that were not performed on recommendations 8 1" 10" 100" 1000" 10000" 100000" 1000000" 1" 10" 100" 1000" 10000" 100000" number'of'users/items'that'performed/ received'X'interac5ons' number'of'interac5ons' items"(train)" users"(train)" items"(test)" users"(test)" 81%$ 5%$ 2%$ 12%$ interac( on*types* clicks$ replies$ bookmarks$ deletes$
  9. 9. Evaluation Measure Mixture of… - Precision@k (k = 2, 4, 6, 20) = fraction of relevant items in the top k - Recall@30 = fraction of relevant items in the top k - Success@30 = probability that at least one relevant item was recommended in the top 30 9
  10. 10. Who participated? • 119 teams participated (366 teams registered) • Countries:  USA (25%)  Germany (11%)  China (9%)  France (7%)  Hungary (4%) • Type of organization:  academia (∼25%)  industry (∼75%)  most common industry: Internet & IT  larger companies such as Yandex, Alibaba, Microsoft or Amazon as well as start-ups 10
  11. 11. Top score over time 11 0" 100" 200" 300" 400" 500" 600" 700" 0" 500000" 1000000" 1500000" 2000000" 2500000" 0" 5" 10" 15" 20" Number'of'submissions'during'week'X' Top'score'at'the'end'of'week'X' Week' top"score"(full)" #submissions"
  12. 12. Number of submissions per team 12 0" 100" 200" 300" 400" 500" 600" 0" 20" 40" 60" 80" 100" 120" number'of'submissions' rank'of'team'
  13. 13. Overlap with XING’s recommender 13 0" 2000" 4000" 6000" 8000" 10000" 12000" 0" 5" 10" 15" 20" 25" 30" number'of'users' number'of'overlapping'recommenda3ons'
  14. 14. Outlook for 2017 • Current plan:  Domain: again job recommendations  Additional perspectives:  is the user a good candidate for the job?  Novelty (recommending new jobs)  New users (recommending jobs to new users)  Additional features (e.g. clicks from recruiters on profiles)  Additional tooling:  Proper API for submitting solutions  Advanced Baseline implementations (building up on this year’s solutions) • Goal: offline + online (!!) evaluation • More details: panel discussion in the afternoon 14
  15. 15. Thank you to PC! • Alejandro Bellogín, Universidad Autónoma de Madrid, Spain • Paolo Cremonesi, Politecnico di Milano, Italy • Simon Dooms, Trackuity, Belgium • Balasz Hidasi, Gravity R&D, Hungary • Levente Kocsis, Hungarian Academy of Sciences, Hungary • Andreas Lommatzsch, TU Berlin, Germany • Katja Niemann, XING AG, Germany • Alan Said, University of Skövde, Sweden • Yue Shi, Yahoo Labs, USA • Marko Tkalcic, Free University of Bozen-Bolzano, Italy 15
  16. 16. 16 Thank you to RecSys Challenge participants!
  17. 17. Agenda • 09:00-10:30 Welcome + Short presentations • 10:30-11:00 Coffee break • 11:00-12:30 Full Papers • 12:30-14:00 Lunch break • 14:00-15:30 Full Papers / Top 3 • 15:30-16:00 Coffee break • 16:00-17:30 Panel Discussion: RecSys Challenge ‘17 17
  18. 18. Thank you @recsyschallenge http://recsyschallenge.com www.xing.com

×