Data Modeling in IPTV and OTT Recommender Systems
•
2 recomendaciones•1,279 vistas
Data Modeling in IPTV and OTT Recommender Systems - Information flow from interaction to personalized recommendations -- Neo4j meetup (2016-02-19)
Recomendados
Más contenido relacionado
Similar a Data Modeling in IPTV and OTT Recommender Systems
Similar a Data Modeling in IPTV and OTT Recommender Systems (6)
Más de David Zibriczky
Más de David Zibriczky (10)
Último
Último (20)
Data Modeling in IPTV and OTT Recommender Systems
- 1. 1 DATA MODELING IN IPTV AND OTT RECOMMENDER SYSTEMS – INFORMATION FLOW FROM INTERACTION TO PERSONALIZED RECOMMENDATIONS Dávid Zibriczky, ImpressTV Neo4j Meetup 2016-02-18
- 3. 3 Who we are? • Acquisition of IPTV, OTT and media business line of Gravity R&D (July 2014) • Technical Centre in Budapest, sales and management from the UK What we are doing? • Providing personalized recommendations, data analytics, targeted advertising and audience measurement for corporate clients • Main domains: Video-On-Demand, Linear TV, OTT, Advertisements About ImpressTV
- 4. 4 • More time spent watching media contents than ever • Hundreds of channels, ten thousands of movies, millions of user uploaded videos • Heterogeneous mixture of devices and items • Massive consumption information • Cloudization and centralization Then and Now
- 6. 6 What is a Recommender System?
- 7. 7 • For the consumers › Content discovery (relevance, time, information filtering) › Exploring new preferences (habits, engagement) • For the business › Improving KPIs, balancing consumption (long tail contents) › Promotions, targeting, campaign, analytics, reporting • For the recommender system vendors › Data integration/modeling, insights, data science, optimization, research › Technology challenges, deployment, maintenance What does a Recommender System mean in TV business?
- 9. 9 Data sources • Interactions: › Sources: Remote controller + interface, touchpad, mouse, keyboard › Explicit / implicit feedbacks: Rating, like, (click) / buy, watch, record, subscribe, channel switch › Context: Time, date, device, duration, price • Item catalog: › Sources: TV provider, external sources (Freebase, DBPedia, IMDb, …) › Metadata: Descriptive (title, synopsis), technical (schedule, resolution, item type), commercial (price) • User catalog: › Sources: Internal, social network › Metadata: Gender, age group, location • Recommendation request: user id, item ids, device, location, time, session id
- 10. 10 Let’s build some graphs…!
- 12. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents
- 13. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items Inception (HD) VOD/SVOD Linear (EPG items) CatchUp
- 14. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD BBC HD Inception (HD) TV Channels
- 15. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD BBC HD Friends Series Movie Channels Products Inception (HD) Star Wars Package Inception TVOD ($11.99) ($9.99) ($2.99) VOD/SVOD/Channel products
- 16. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD Friends Series Movie Channels Products Inception (HD) Star Wars Package ($11.99) ($9.99) ($2.99) Inception TVOD Interface BBC HD Devices (remote controller, mouse, touch pad)
- 17. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD Friends Series Movie Channels Products Inception (HD) Star Wars Package ($11.99) ($9.99) ($2.99) Inception TVOD (19:25; 1136s) (19:00) (16:13; 3715s) (20:10; 1135s) Interface BBC HD (21:11; 3651s; 5*) Implicit/Explicit feedbacks + context
- 18. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD Friends Series Movie Channels Products Inception (HD) Star Wars Package ($11.99) ($9.99) ($2.99) Inception TVOD (19:25; 1136s) (19:00) (21:11; 3651s; 5*) (16:13; 3715s) (20:10; 1135s) Interface TV provider related space BBC HD
- 19. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD Friends Series Movie Channels Products Inception (HD) Star Wars Package ($11.99) ($9.99) ($2.99) Inception TVOD (19:25; 1136s) (19:00) (21:11; 3651s; 5*) (16:13; 3715s) (20:10; 1135s) Interface TV provider related space TopGear S01 Friends S01 Friends S02 Collections BBC HD
- 20. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD Friends Series Movie Channels Products Inception (HD) Star Wars Package ($11.99) ($9.99) ($2.99) Inception TVOD (19:25; 1136s) (19:00) (21:11; 3651s; 5*) (16:13; 3715s) (20:10; 1135s) Interface TV provider related space TopGear S01 Friends S01 Friends S02 Collections Movies Series Shows Domains Domain dependent contents BBC HD
- 21. Friends S01E01 Friends S02E05 Consumers Friends S01E02 News 2016-02- 18 07:00 (Mary) (John) (Mark) Contents Star Wars IV. VOD (SD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 News BBC HD 07:00 Items HBO SD HBO 2 SD Friends Series Movie Channels Products Inception (HD) Star Wars Package ($11.99) ($9.99) ($2.99) Inception TVOD (19:25; 1136s) (19:00) (21:11; 3651s; 5*) (16:13; 3715s) (20:10; 1135s) Interface TV provider related space TopGear S01 Friends S01 Friends S02 Collections Movies Series Shows Domains TV Domain dependent contents Musics Books BBC HD
- 22. Movies Series Shows Friends S01E01 Friends S02E05 Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Male Action Inception HBO 20:00 Domain dependent contents Collections Contents Items Products Interface Consumers TV Friends S01E02 Domains TV provider related space News 2016-02- 18 07:00 News News BBC HD 07:00 Harrison Ford Female Comedy HBO SD HBO 2 SD HD SD Friends Series Star Wars Package TopGear S01 Friends S01 Friends S02 Musics (19:25; 1136s) (19:00) (16:13; 3715s) ($11.99) Movie Channels ($9.99) (8880s; 20:00-22:28) (Mary) (John) (Mark) (20:10; 1135s) Inception TVOD ($2.99) (21:11; 3651s; 5*) BBC HD Meta data Books
- 23. Movies Series Shows Friends S01E01 Friends S02E05 Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Male Action Inception HBO 20:00 Domain dependent contents Collections Contents Items Products Interface Consumers TV Friends S01E02 Domains TV provider related space News 2016-02- 18 07:00 News News BBC HD 07:00 Harrison Ford Female Comedy HBO SD HBO 2 SD HD SD Friends Series Star Wars Package TopGear S01 Friends S01 Friends S02 Musics (19:25; 1136s) (19:00) (16:13; 3715s) ($11.99) Movie Channels ($9.99) (8880s; 20:00-22:28) (Mary) (John) (Mark) (20:10; 1135s) Inception TVOD ($2.99) (21:11; 3651s; 5*) BBC HD Meta data Books
- 24. 24 Graph representation Nodes • Entities: Users, items, contents, product, collections • Metadata: Genres, actors, category tree nodes, gender, age group • Node properties: Auxiliary meta data (e.g. descriptions) Edges • Interactions: Implicit/explicit feedbacks • Metadata containment, relations • Edge properties: Context, direction
- 25. 25 Unification challenges PVR LiveTV CatchUp SVOD TVOD ADs Devices Social Networks Metadata sources TV items (providers)
- 27. 27 Evolution of the Recommender Problem Source: http://www.slideshare.net/xamat/recsys-2014-tutorial-the-recommender-problem-revisited
- 28. 28
- 30. Movies Series Shows Friends S01E01 Friends S02E05 Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Male Action Inception HBO 20:00 Domain dependent contents Collections Contents Items Products Interface Consumers TV Friends S01E02 Domains TV provider related space News 2016-02- 18 07:00 News News BBC HD 07:00 Harrison Ford Female Comedy HBO SD HBO 2 SD HD SD Friends Series Star Wars Package TopGear S01 Friends S01 Friends S02 Musics (19:25; 1136s) (19:00) (16:13; 3715s) ($11.99) Movie Channels ($9.99) (8880s; 20:00-22:28) (Mary) (John) (Mark) (20:10; 1135s) Inception TVOD ($2.99) (21:11; 3651s; 5*) BBC HD Meta data Books
- 31. Movies Series Shows Friends S01E01 Friends S02E05 Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Male Action Inception HBO 20:00 Domain dependent contents Collections Contents Items Products Interface Consumers TV Friends S01E02 Domains TV provider related space News 2016-02- 18 07:00 News News BBC HD 07:00 Harrison Ford Female Comedy HBO SD HBO 2 SD HD SD Friends Series Star Wars Package TopGear S01 Friends S01 Friends S02 Musics (19:25; 1136s) (19:00) (16:13; 3715s) ($11.99) Movie Channels ($9.99) (8880s; 20:00-22:28) (Mary) (John) (Mark) (20:10; 1135s) Inception TVOD ($2.99) (21:11; 3651s; 5*) BBC HD Meta data Books
- 32. Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 Items Products Interface Consumers News BBC HD 07:00 HBO SD HBO 2 SD Friends Series Star Wars Package (19:25; 1136s) (19:00) (16:13; 3715s) ($11.99) Movie Channels ($9.99) (8880s; 20:00-22:28) (Mary) (John) (Mark) (20:10; 1135s) Inception TVOD ($2.99) (21:11; 3651s; 5*) BBC HD
- 33. Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Inception HBO 20:00 Items Consumers News BBC HD 07:00 (8880s; 20:00-22:28) (Mary) (John) (Mark)
- 34. 34 Collaborative Filtering / Graph representation The Matrix The Matrix 2 The Matrix 3 Twilight
- 35. 35 Collaborative Filtering / Matrix representation The Matrix The Matrix 2 The Matrix 3 Twilight R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 1 ? ? User 2 1 1 0 0 User 3 1 0 1 1 User 4 1 0 0 1 ? ?
- 36. 36 Collaborative Filtering / Binary preference matrix R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 1 ? ? User 2 1 1 0 0 User 3 1 0 1 1 User 4 1 0 0 1
- 37. 37 Collaborative Filtering / Implicit MF R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 User 2 𝒓 𝑢𝑖 User 3 User 4 Q 𝒒𝑖 PT 𝒑 𝑢 𝑇 𝑹 𝑵𝒙𝑴: preference matrix 𝑷 𝑵𝒙𝑲: user feature matrix 𝑸 𝑴𝒙𝑲: item feature matrix 𝑵: #users 𝑴: #items 𝑲: #features 𝑲 ≪ 𝑴 , 𝑲 ≪ 𝑵 𝑹~𝑷 𝐓 𝑸 𝑟 𝑢𝑖 = 𝒑 𝑢 T 𝒒𝑖
- 38. 38 Collaborative Filtering / Implicit MF / Weighted Sum of Squared Errors R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 1 ? ? User 2 1 1 0 0 User 3 1 0 1 1 User 4 1 0 0 1 C The Matrix The Matrix 2 Twilight The Matrix 3 User 1 c11 c12 1 1 User 2 c21 c22 1 1 User 3 c31 1 c33 c34 User 4 c41 1 1 1 𝒇 𝑷, 𝑸 = 𝑾𝑺𝑺𝑬 = (𝒖,𝒊) 𝒄 𝒖𝒊 𝒓 𝒖𝒊 − 𝒓 𝒖𝒊 𝟐• Objective Function:
- 39. 39 Collaborative Filtering / Implicit MF / Alternating Least Squares R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 User 2 User 3 User 4 Q 0.1 -0.4 0.8 0.3 0.6 0.7 -0.1 0.8 PT -0.2 0.6 0.6 0.4 0.7 -0.2 0.5 -0.2 𝑹~𝑷 𝐓 𝑸 𝑟 𝑢𝑖 = 𝒑 𝑢 T 𝒒𝑖 Ridge Regression • 𝑝 𝑢 = 𝑄𝐶 𝑢 𝑄T −1 𝑄𝐶 𝑢 𝑅 𝑟 𝑢 • 𝑞𝑖 = 𝑃𝐶 𝑖 𝑃T −1 𝑃𝐶 𝑖 𝑅 𝑐 𝑖
- 40. 40 Collaborative Filtering / Implicit MF / Alternating Least Squares R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 User 2 User 3 User 4 Q 0.3 -0.3 0.6 0.2 0.7 0.8 0.2 0.7 PT -0.2 0.6 0.6 0.4 0.7 -0.2 0.5 -0.2 Ridge Regression • 𝑝 𝑢 = 𝑄𝐶 𝑢 𝑄T −1 𝑄𝐶 𝑢 𝑅 𝑟 𝑢 • 𝑞𝑖 = 𝑃𝐶 𝑖 𝑃T −1 𝑃𝐶 𝑖 𝑅 𝑐 𝑖 𝑹~𝑷 𝐓 𝑸 𝑟 𝑢𝑖 = 𝒑 𝑢 T 𝒒𝑖
- 41. 41 Collaborative Filtering / Implicit MF / Alternating Least Squares R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 User 2 User 3 User 4 Q 0.3 -0.3 0.6 0.2 0.7 0.8 0.2 0.7 PT -0.2 0.7 0.6 0.5 0.8 -0.2 0.6 -0.2 Ridge Regression • 𝑝 𝑢 = 𝑄𝐶 𝑢 𝑄T −1 𝑄𝐶 𝑢 𝑅 𝑟 𝑢 • 𝑞𝑖 = 𝑃𝐶 𝑖 𝑃T −1 𝑃𝐶 𝑖 𝑅 𝑐 𝑖 𝑹~𝑷 𝐓 𝑸 𝑟 𝑢𝑖 = 𝒑 𝑢 T 𝒒𝑖
- 42. 42 Collaborative Filtering / Implicit MF / Alternating Least Squares R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 0.02 0.45 User 2 User 3 User 4 Q 0.3 -0.3 0.6 0.2 0.7 0.8 0.2 0.7 PT -0.2 0.7 0.6 0.5 0.8 -0.2 0.6 -0.2 𝑹~𝑷 𝐓 𝑸 𝑟 𝑢𝑖 = 𝒑 𝑢 T 𝒒𝑖
- 44. Movies Series Shows Friends S01E01 Friends S02E05 Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Male Action Inception HBO 20:00 Domain dependent contents Collections Contents Items Products Interface Consumers TV Friends S01E02 Domains TV provider related space News 2016-02- 18 07:00 News News BBC HD 07:00 Harrison Ford Female Comedy HBO SD HBO 2 SD HD SD Friends Series Star Wars Package TopGear S01 Friends S01 Friends S02 Musics (19:25; 1136s) (19:00) (16:13; 3715s) ($11.99) Movie Channels ($9.99) (8880s; 20:00-22:28) (Mary) (John) (Mark) (20:10; 1135s) Inception TVOD ($2.99) (21:11; 3651s; 5*) BBC HD Meta data Books
- 45. Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Action Inception HBO 20:00 Contents Items Products Interface Consumers News News BBC HD 07:00 Harrison Ford Comedy HBO SD HBO 2 SD HD SD Friends Series Star Wars Package (16:13; 3715s) Movie Channels (8880s; 20:00-22:28) (John) (20:10; 1135s) Inception TVOD BBC HD
- 46. Star Wars IV. VOD (SD) Inception (HD) Star Wars VII. VOD (HD) Inception CatchUp feb 17 Inception HBO2 22:00 Friends S01E01 SVOD Inception PVR 20:00 Action Inception HBO 20:00 Contents Items Products Consumers News News BBC HD 07:00 Harrison Ford Comedy HD SD (8880s; 20:00-22:28) (John)
- 47. 47 Content-based Filtering / Cosine Similarity (1/6) Sci-Fi RomanceAdventure The Matrix The Matrix 2 The Matrix 3 Twilight
- 48. 48 Content-based Filtering / Cosine Similarity (2/6) Sci-Fi RomanceAdventure The Matrix The Matrix 2 The Matrix 3 Twilight ? ? MT Sci-Fi Adventure Romance The Matrix 1 1 0 The Matrix 2 1 1 0 Twilight 0 1 1 The Matrix 3 1 1 0 R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 ? ?
- 49. 49 Content-based Filtering / Cosine Similarity (3/6) MT Sci-Fi Adventure Romance The Matrix 1 1 0 The Matrix 2 1 1 0 Twilight 0 1 1 The Matrix 3 1 1 0 R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 ? ?
- 50. 50 Content-based Filtering / Cosine Similarity (4/6) M’T Sci-Fi Adventure Romance The Matrix 1/6 1/8 0 The Matrix 2 1/6 1/8 0 Twilight 0 1/8 1/2 The Matrix 3 1/6 1/8 0 R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 ? ? 𝑚′𝑖𝑗 = 𝑚𝑖𝑗 𝑘 𝑚𝑖𝑘 𝑘 𝑚 𝑘𝑗
- 51. 51 Content-based Filtering / Cosine Similarity (5/6) M’T Sci-Fi Adventure Romance The Matrix 1/6 1/8 0 The Matrix 2 1/6 1/8 0 Twilight 0 1/8 1/2 The Matrix 3 1/6 1/8 0 R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 ? ? PT Sci-Fi Adventure Romance User 2/6 2/8 0 𝑃T = 𝑅𝑀′T
- 52. 52 Content-based Filtering / Cosine Similarity (6/6) M’T Sci-Fi Adventure Romance The Matrix 1/6 1/8 0 The Matrix 2 1/6 1/8 0 Twilight 0 1/8 1/2 The Matrix 3 1/6 1/8 0 R The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 ? ? PT Sci-Fi Adventure Romance User 2/6 2/8 0 𝑹 The Matrix The Matrix 2 Twilight The Matrix 3 User 1 1 0.268 1 𝑟𝑢𝑖 = 𝑘 𝑚′𝑖𝑘 𝑝 𝑢𝑘 𝑘 𝑚′𝑖𝑘 2 𝑘 𝑝 𝑢𝑘 2 𝑃T = 𝑅𝑀′T
- 54. 54 • Who is the front of the TV? (watching behavior pattern recognition) • Cold-start problem, opt-out users (metadata enrichment, hybrid filtering) • Noise filtering, lack of explicit data (filtering and weighting method, implicit feedbacks) • Context-dependency (device, time, mood) • Cross-domain recommendation, content centralization • Time-dependent recommendable set (model-based methods) • Popularity effect (diversification) • Upselling, subscription (Live VOD) • Offline vs. Online KPIs Data Science Challenges
- 55. 55 • Heterogeneous external data sources (data unification) • Big data (scalable algorithms, distributed databases) • Latency in data transfer, streaming service • Responsivity, load, SLA (distributed systems, load balancer) • Maintenance, service, follow-up • Open source vs. self-development • Distributed systems vs. single server • Software-as-a-service vs. on-site deployment Technology Challenges
- 56. 56 Presented by: Contact: THANK YOU! www.impresstv.com David Zibriczky Head of Data Science david.zibriczky@impresstv.com
- 57. 57 • Senior Software Engineers (Java, .NET) • Developer (Java, Python, Node.js) • Senior Data Mining Engineer • Pre-sales Technical Consultant • Data Scientist • Contact: hr@impresstv.com • Website: www.impresstv.com We are hiring!