Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

Recommendations with Neo4j (FOSDEM 2015)

Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Cargando en…3
×

Eche un vistazo a continuación

1 de 62 Anuncio
Anuncio

Más Contenido Relacionado

A los espectadores también les gustó (20)

Anuncio

Similares a Recommendations with Neo4j (FOSDEM 2015) (20)

Anuncio

Más reciente (20)

Recommendations with Neo4j (FOSDEM 2015)

  1. 1. GraphAware TM Michal Bachman graphaware.com @graph_aware Recommendations with Neo4j Building a high-performance recommendation engine
  2. 2. GraphAware TM Quick Intro Why Graphs? Business and Technical Challenges GraphAware Recommendation Engine About This Talk
  3. 3. GraphAware TM News you should read Books you should buy People you may know People you should date People you should market a product to … Recommendation Engines
  4. 4. GraphAware TM Content-based (features) Collaborative filtering (user <-> item relationships) Recommendation Engines
  5. 5. GraphAware TM Features as well as relationships can be naturally represented as a graph. Good News
  6. 6. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  7. 7. GraphAware TM Easy to understand Natural to model Flexible (schema-free) Fast to query Graphs (Neo4j)
  8. 8. GraphAware TM Great for a quick PoC Great for smaller data sets Great for relatively simple logic Cypher
  9. 9. GraphAware TM Cypher MATCH (u:User)-[:LIKED]->(m:Movie), (m)<-[:LIKED]-(another:User), (another)-[:LIKED]->(reco:Movie) WHERE NOT (u)-[:LIKED|DISLIKED]->(reco) RETURN reco;
  10. 10. GraphAware TM Requirements of real-world recommendation engines are often much more complex. The Reality
  11. 11. GraphAware TM Imagine you’re building the ”people you may know” feature on LinkedIn. Example
  12. 12. GraphAware TM After a brainstorming session, your team came up with the following ways of finding people one may know: Example
  13. 13. GraphAware TM Common contacts Facebook friends in common Email / mobile contacts in common Each others email / mobile contact Worked for the same company Studied at the same school Share the same interest Live in the same city People You May Know
  14. 14. GraphAware TM But that’s just the beginning! Let’s go back and re-visit. Example
  15. 15. GraphAware TM More contacts in common = better chance? Same city / school / company = does size matter? What about emails that don’t represent a person? What about people already connected? And pending… And rejected… And repeatedly ignored… People You May Know
  16. 16. GraphAware TM Finding things to recommend Serving the most relevant recommendations Measuring the quality of recommendations Time to market / cost of development Business Challenges
  17. 17. GraphAware TM Performance (real-time!) Simplicity Flexibility Technical Challenges
  18. 18. GraphAware TM So we came up with an open-source recommendation engine skeleton that will help you solve all the challenges. We’ve done it before
  19. 19. GraphAware TM plugin to Neo4j (uses GraphAware Framework) you have to use a JVM-language opinionated architecture very fast very flexible handles all the plumbing Recommendation Engine
  20. 20. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled into higher-level engines Design Decisions
  21. 21. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  22. 22. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled to higher-level engines Items discovered multiple times are more relevant Relevance depends on how was item discovered Design Decision
  23. 23. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  24. 24. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled to higher-level engines Items discovered multiple times are more relevant Relevance depends on how was item discovered Items not to be recommended: “cross-cutting” concern Design Decisions
  25. 25. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  26. 26. GraphAware TM Input -> Engine -> Recommendations Scores and Score Transformers Blacklists Filters Post-processors Context (how many, how fast,…?) Loggers Architecture
  27. 27. GraphAware TM In 5 minutes, we’ll build a simple engine that recommends who you should be friends with. Let’s Build Something
  28. 28. GraphAware TM 0) Model
  29. 29. GraphAware TM 1) Discover
  30. 30. GraphAware TM public class FriendsInCommon extends SomethingInCommon {
 
 @Override
 public String name() {
 return "friendsInCommon";
 }
 
 @Override
 protected RelationshipType getType() {
 return FRIEND_OF;
 }
 
 @Override
 protected Direction getDirection() {
 return BOTH;
 }
 }
  31. 31. GraphAware TM 2) Score
  32. 32. GraphAware TM public class FriendsInCommon extends SomethingInCommon {
 @Override
 protected ScoreTransformer scoreTransformer() {
 return new ParetoScoreTransformer(100, 10);
 }
 
 @Override
 public String name() {
 return "friendsInCommon";
 }
 
 @Override
 protected RelationshipType getType() {
 return FRIEND_OF;
 }
 
 @Override
 protected Direction getDirection() {
 return BOTH;
 }
 }
  33. 33. GraphAware TM 3) Post-Process
  34. 34. GraphAware TM public class RewardSameLocation extends RewardSomethingShared {
 
 @Override
 protected RelationshipType type() {
 return LIVES_IN;
 }
 
 @Override
 protected Direction direction() {
 return OUTGOING;
 }
 
 @Override
 protected float scoreValue(Node reco, Node in, Node shared) {
 return 10;
 }
 
 @Override
 protected String scoreName() {
 return "sameLocation";
 }
 }
  35. 35. GraphAware TM public class RewardSameLabels implements PostProcessor<Node, Node> {
 
 @Override
 public void postProcess(Recommendations<Node> out, Node in) {
 Label[] inLabels = toArray(in.getLabels());
 
 for (Recommendation<Node> reco : out.get()) {
 if (Arrays.equals(inLabels, toArray(reco.getItem().getLabels()))) {
 reco.add("sameGender", 10);
 }
 }
 }
 }
  36. 36. GraphAware TM 4) Filter
  37. 37. GraphAware TM public final class FriendsContextFactory extends Neo4jContextFactory {
 
 @Override
 protected List<BlacklistBuilder<Node, Node>> blacklistBuilders() {
 return asList(
 new ExcludeSelf(),
 new ExistingRelationshipBlacklistBuilder(FRIEND_OF, BOTH)
 );
 }
 
 @Override
 protected List<Filter<Node, Node>> filters() {
 return asList(
 new ExcludeSelf()
 );
 }
 }
  38. 38. GraphAware TM 5) Assemble
  39. 39. GraphAware TM public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsComputingEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new FriendsInCommon(),
 new RandomPeople()
 );
 }
 
 @Override
 protected List<PostProcessor<Node, Node>> postProcessors() {
 return asList(
 new RewardSameLabels(),
 new RewardSameLocation(),
 new PenalizeAgeDifference()
 );
 }
 }
  40. 40. GraphAware TM ?) Precompute
  41. 41. GraphAware TM public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsComputingEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new FriendsInCommon(),
 new RandomPeople()
 );
 }
 
 @Override
 protected List<PostProcessor<Node, Node>> postProcessors() {
 return asList(
 new RewardSameLabels(),
 new RewardSameLocation(),
 new PenalizeAgeDifference()
 );
 }
 
 @Override
 public ParticipationPolicy<Node, Node> participationPolicy(Context<Node, Node> context) {
 return ParticipationPolicy.IF_MORE_RESULTS_NEEDED;
 }
 }
  42. 42. GraphAware TM public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsRecommendationEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new Neo4jPrecomputedEngine(),
 new FriendsComputingEngine()
 );
 }
 }
  43. 43. GraphAware TM 6) Log
  44. 44. GraphAware TM public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsRecommendationEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new Neo4jPrecomputedEngine(),
 new FriendsComputingEngine()
 );
 }
 
 @Override
 protected List<Logger<Node, Node>> loggers() {
 return asList( 
 new Slf4jRecommendationLogger<Node, Node>(),
 new Slf4jStatisticsLogger<Node, Node>()
 );
 }
 }
  45. 45. GraphAware TM 7) Test
  46. 46. GraphAware TM List<Recommendation<Node>> reco =
 recommendationEngine.recommend(getPersonByName("Adam"), Mode.REAL_TIME, 2);
 
 String expected = 
 "(Vince {total:19.338144," +
 "ageDifference:-5.527864," +
 "friendsInCommon:14.866008," +
 "sameGender:10.0})," +
 
 "(Luanne {total:11.553411," +
 "ageDifference:-3.312597," +
 "friendsInCommon:14.866008})";
 
 assertEquals(expected, toString(reco));
  47. 47. GraphAware TM List<Recommendation<Node>> reco = recommendationEngine.recommend(getPersonByName("Luanne"), REAL_TIME, 4);
 
 assertEquals("Daniela", reco.get(0).getItem().getProperty("name"));
 assertEquals(22, reco.get(0).getScore().getTotalScore(), 0.5);
 
 assertEquals("Adam", reco.get(1).getItem().getProperty("name"));
 assertEquals(12, reco.get(1).getScore().getTotalScore(), 0.5);
 
 assertEquals("Vince", reco.get(2).getItem().getProperty("name"));
 assertEquals(8, reco.get(2).getScore().getTotalScore(), 0.5);
 
 assertEquals("Bob", reco.get(3).getItem().getProperty("name"));
 assertEquals(-9, reco.get(3).getScore().getTotalScore(), 0.5);
  48. 48. GraphAware TM Finding things to recommend Serving the most relevant recommendations Measuring the quality of recommendations Time to market / cost of development Business Challenges
  49. 49. GraphAware TM Performance (real-time!) Simplicity Flexibility Technical Challenges
  50. 50. GraphAware TM Getting Started <dependencies> ... <dependency>
 <groupId>com.graphaware.neo4j</groupId>
 <artifactId>recommendation-engine</artifactId> <version>2.1.6.27.2</version>
 </dependency> ... <dependencies>
  51. 51. GraphAware TM Built-in ability to pre-compute recommendations Other built-in base-classes But we need your help! https://github.com/graphaware/neo4j-reco There’s More!
  52. 52. GraphAware TM Built-in algorithms Time-based ParticipationPolicy Integration with compute engines Machine learning Future
  53. 53. GraphAware TM GraphAware Framework makes it easy to build, test, and deploy generic as well as domain- specific functionality for Neo4j. GraphAware Framework
  54. 54. GraphAware TM GraphUnit
 & RestTest RelCount WarmUp Schema (wip) Recommendation Engine GraphAware Framework ChangeFeed UUID TimeTree Algorithms NodeRank
  55. 55. GraphAware TM Open Source (GPL) Active Production Ready Github: github.com/graphaware Our Web: graphaware.com Maven Central GraphAware Framework
  56. 56. GraphAware TM Try it Give us feedback Contribute Build your own modules Get in touch for support / consultancy GraphAware Framework
  57. 57. GraphAware TM GraphAware Events 31
 Jan Recommendation Engines in Brussels (FOSDEM) 31
 Jan GraphGen in Brussels (FOSDEM) 5
 Feb Recommendation Engines Webinar 5
 Feb Meetup at GraphAware (build your own Recommendation Engine) 10
 Feb Neo4j Fundamentals in Manchester 10
 Feb Neo4j Meetup in Manchester 17
 Feb Neo4j Fundamentals in Edinburgh 17
 Feb Neo4j Meetup in Edinburgh
  58. 58. GraphAware TM GraphConnect Europe 2015 When: Where: Tickets: Call for Papers: Sponsors: Thursday, 7th May, 2015 - main Conference Day Wednesday, 6th May 2015 - Training Day Etc venues, 155 Bishopsgate, London (next to Liverpool Street Station) now available on www.graphconnect.com 199$ early bird plus 100$ for training 499$ full price plus 100$ for training open now till 29th January all Neo4j community members, customers or general graph enthusiasts are invited to submit their talk open now till 29th January, email: gceurope@neotechnology.com
  59. 59. graphaware.com @graph_aware Thank You! GraphAware TM

×