Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Recently uploaded
Recently uploaded (20)
BEKEE, Expert Knowledge Modeling with Bayesian Belief Networks
- 1. Bayesia Expert Knowledge Elicitation Plan Environment - BEKEE Modeling by An innovative Brainstorming Tool Brainstorming Bayesia Expert Dr. Lionel JOUFFE Knowledge Elicitation Environment February 2012 ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 1 1
- 2. Plan Modeling by Brainstorming MODELING BY BRAINSTORMING Bayesia Expert Knowledge MODELING BY BRAINSTORMING Elicitation Environment All models are wrong; the practical question is how wrong do they have to be to not be useful (Box&Draper 87) ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 2 2
- 3. Why? There is a clear need for Decision Support Systems Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment Every Decision Maker is faced to complex decisions Human Beings are not so good at taking rational decision under uncertainty ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 3 3
- 4. How? Data is not always available for automatically learning a Decision Support System with Data Mining algorithms Plan But experts have gathered invaluable Tacit Knowledge through Modeling by their Experience Brainstorming Bayesia Expert Knowledge Explicit Knowledge We need to convert this Tacit Elicitation Knowledge into Explicit Environment Knowledge and use it for building models Tacit Knowledge ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 4 4
- 5. What? Bayesian Belief Networks (BBNs) are ideal models for Expert Knowledge Modeling Plan Graphical Representation Powerful Probabilistic Engines Modeling by Brainstorming Drivers analysis What-if scenarios Bayesia Expert Knowledge Elicitation Environment 3,95 3,9 3,85 3,8 3,75 3,7 3,65 3,6 A priori Flowery Feminine Original Tenacious Fruity ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 5 5
- 6. BBNs are made of Two Distinct Parts Qualitative part: the Structure Directed Acyclic Graph (DAG), i.e. no directed loop Plan Nodes represent the variables Modeling by Each node has a set of exclusive states (e.g.: Poor, Good) Brainstorming Arcs represent the direct probabilistic relationships between the variables Bayesia Expert (possibly causal) Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 6 6
- 7. BBNs are made of Two Distinct Parts Quantitative part: the Parameters Probability tables are associated to each node Plan MARGINAL PROBABILITY DISTRIBUTION Half of the products are of Good quality Modeling by Brainstorming 40% of the Brand Images are Poor Bayesia Expert Knowledge Elicitation Environment The size of the Conditional Probability Tables grows exponentially with respect to the CONDITIONAL PROBABILITY DISTRIBUTION number of Parents There are 60% of chance that the Perceived Quality is Good for Poor Quality products with Good Brand Image ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 7 7
- 8. BBNs are Powerful Observational Inference Engines ... We get some evidence on the states of a subset of variables: Hard positive and negative evidence, Likelihood, Probability distributions, Mean values Plan We take these findings into account in a rigorous way to update our belief on the states of all the other variables Modeling by Probability distributions on their values Brainstorming Multi-Directional Inference (Simulation and/or Diagnosis) Bayesia Expert Knowledge Elicitation The evidence on Perceived Quality (a new Environment probability distribution) allows to Prior Distribution Posterior Distribution update the probability distribution of Brand Image (Diagnosis) and Satisfaction (Simulation) ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 8 8
- 9. ... and Powerful Causal Inference Engines We DO some state modification on a subset of variables: Hard positive and negative actions, Likelihood, Probability distributions, Mean values Plan We take these actions into account in a rigorous way to update our belief on the states of all the descendant variables Simulation of the effects of these actions Modeling by Brainstorming Probability distributions on their values Bayesia Expert Knowledge Elicitation Environment Prior Distribution Simulating a new Posterior Distribution population made of 85% of Good Perceived Quality products rather than focusing on a sub-population made of such products ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 9 9
- 10. BBN Modeling by Brainstorming Clear definition of the BBN’s objective(s) (e.g.: Improvement of the Product/Service Quality, improvement of the Purchase Intent, Plan improvement of the Company’s performance, ...) Modeling by Identification of the conceptual dimensions that are linked to these objectives Brainstorming (e.g.: Human resources, Management, Production, Marketing, ...) Bayesia Expert Knowledge Elicitation Environment Definition of the group of experts that will fully cover all the dimensions (and the different geographical zones), with a small redundancy for allowing fruitful expert debates Brain Storming Sessions with this group of Experts to manually build the BBN, conceptual dimension per conceptual dimension ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 10 10
- 11. The Structure The Qualitative Part One sub-network per Conceptual Dimension Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 11 11
- 12. Plan Modeling by BAYESIA Expert Brainstorming Knowledge Elicitation Environment Bayesia Expert Knowledge Elicitation Environment Batch ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission Interactive 12 12
- 13. The Parameters The Quantitative Part Each expert gives his/her belief on the probability distributions Plan Modeling by y, Brainstorming Probabilities do not have to be b ilit il a exact to be useful va Bayesia Expert l, A n tro Knowledge Gr Co ou Elicitation p( Emotional (Mood, Motivation) il ity, An sib Environment ch lau ori (P ng ,H ive erd g nit ring) ing ) Co cho An BIASES Facilitator (can be biased toward charismatic experts or toward the last expressed opinion) ☛ Bayesia Expert Knowledge Elicitation Environment for reducing ©2012 BAYESIA SAS these biases, improving traceability, gathering all the useful All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express knowledge, .... written permission 13 13
- 14. Interactive Sessions Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 14 14
- 15. Batch Sessions Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 15 15
- 16. Expert Management The Expert Editor allows defining: The Expert’s name, its Credibility (that will be use globally during the consensus computation), her/his Picture, a Comment to describe her/his area of expertise. The last field contains the Plan number of assessments realized by the expert on the current network Modeling by Brainstorming Group of experts can be Imported and Exported Bayesia Expert Knowledge Elicitation Environment Communication with the BEKEE web server* ©2012 BAYESIA SAS Allows generating a Bayesian network by using the All rights reserved. Forbidden reproduction in whole or part assessments of the selected experts only without the Bayesia’s express written permission * Available on subscription only 16 16
- 17. Posting a Question to the Server Selecting a cell in the probability Plan table activates the Assessment button for assessing the question corresponding to the selected line, i.e. what is the marginal probability distribution of Mobility over the 3 Modeling by defined states? Brainstorming Bayesia Expert Knowledge Elicitation Environment The Assessment Editor allows the Facilitator manually adding, deleting and modifying Experts’ Assessments. The Post Assessment button is used by the Facilitator to ©2012 BAYESIA SAS send the question to the BayesiaLab’s secured website for All rights reserved. Forbidden reproduction in whole or part an online assessment without the Bayesia’s express written permission 17 17
- 18. https://www.bayesialab.com/expertise2/ Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 18 18
- 19. Web Tool Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 19 19
- 20. Interactive Session Pressing Play allows participating to the interactive session Plan Modeling by Brainstorming Bayesia Expert Knowledge Waiting for a question send by the Facilitator Elicitation Environment Click the Lock to fix that probability Confidence level of the expert used for weighting the assessment ©2012 BAYESIA SAS Comment field for explaining, All rights reserved. Forbidden reproduction in whole or part detailing the assessment without the Bayesia’s express written permission 20 20
- 21. Interactive Session Node with Parents The context variables in the BBN Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 21 21
- 22. The Facilitator’s tool Once the Expert validates her/his assessment, this assessment is sent to the BayesiaLab’s server and the Facilitator’s listener is automatically updated Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation This listener allows following Environment the status of the Experts’ assessments Pressing OK makes BayesiaLab harvesting the assessments ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 22 22
- 23. The Facilitator’s tool The content of this editor is sortable by each column just by clicking on the corresponding It is sorted here in the header. ascending order on the Plan probabilities assessed for the state Weak Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment Sorting the assessments by state probabilities can be used for: - detecting Experts’ misunderstanding - Knowledge sharing, especially by making the 2 “extremes” Experts debate If some useful knowledge comes out from the debate, the Facilitator can post again the question for new Expert Assessments. Each Expert will then be allowed ©2012 BAYESIA SAS to update her/his assessment online (each Experts’ webpage is initialized with the All rights reserved. Forbidden information she/he set in the previous round) reproduction in whole or part without the Bayesia’s express written permission 23 23
- 24. The Consensus Once the assessments validated, a Mathematical consensus is computed by using the Experts’ credibility and their assessment’s confidence. This automatic consensus can be manually modified by the Facilitator to set a Behavioral consensus, i.e. Plan one issued after a fruitful expert debate A small icon is associated to each probability for graphically representing the consensus Modeling by That icon goes from full transparency, when all degree. Brainstorming the votes are identical, to no transparency at all, when the assessment range is 1 (one expert set Bayesia Expert 0% and another one set 100%) Knowledge Hovering over this icon returns the minimum Elicitation and the maximum assessments, and the Environment number of assessments A Consensus icon is also associated to the nodes for indicating the global consensus over all the distributions. The darker the icon is, the lower the global consensus is ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 24 24
- 25. The Consensus Pressing the “I” key while hovering over Plan the expert icon allows displaying the information panel below Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment This information panel contains: - the number of rows (probability distributions) that have Experts assessments - the total number of assessments that have been set in the probability table - the number of Experts that have assessed at least one probability distribution in the table - a measure of the global disagreement that takes into account the deviations from the mathematical consensus - the maximum disagreement corresponding to the greatest difference between two ©2012 BAYESIA SAS assessments in the probability table All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 25 25
- 26. The Assessment Report Right clicking on the Expert Icon in the lower left corner of the Graph window allows generating Plan an HTML report Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment This report first gives information on the Experts, then returns a sorted list of the nodes wrt the global disagreements, and another one wrt the maximal disagreements. Finally, for each node, a summary contains all the global information on the assessments of the (Conditional) Probability Table ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 26 26
- 27. The Graph Report The Graph report allows generating HTML Conditional Probability Tables. These tables comes with the consensual probability distributions and the maximum divergences. Plan Colors are associated to each cell Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment From green (0) to red (50) for the divergences From white (0) to blue (100) for the probabilities The information given by the Assessment and Graph reports is useful for the Model Verification. ©2012 BAYESIA SAS High divergences can be due to state inversion, fuzzy definitions of the variables and/or their All rights reserved. Forbidden states, different contexts reproduction in whole or part without the Bayesia’s express written permission 27 27
- 28. Batch Session In-person meetings are essential for building the qualitative part of the models. Probability elicitation is time consuming and that quantitative Plan part can be too long to allow the interactive elicitation of all the parameters during the meetings. Modeling by Batch sessions allow then each expert to remotely: Brainstorming Complete the parameter elicitation process Bayesia Expert Knowledge Verify the assessed probabilities Elicitation Environment The Facilitator can select the nodes for which the probability distributions have to be assessed and/ or verified Warning are generated for the distributions that are greater than 30% threshold ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 28 28
- 29. Web Tool Plan The Play button allows participating to the batch session Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment Nodes to assess or verify This expert has assessed 3 distributions out of 12 The pie chart represents that ©2012 BAYESIA SAS completion rate All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 29 29
- 30. Web Tool Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 30 30
- 31. Exportation of a Bayesian Network per Expert Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation This exportation tool allows the creation of one Bayesian Environment Belief Network per Expert. The parameters (probabilities) are those assessed by the Expert. For probabilities not assessed by the Expert, the model is based on the consensual probabilities, either the mathematical one, or the behavioral one entered by the Facilitator ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 31 31
- 32. Exportation of the Probability Assessments Generation of a CSV file with all the assessed probabilities, one line per cell/ probability Plan Context in terms of states Assessed Node Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 32 32
- 33. Exportation of the Expert Assessments Plan Generation of a CSV file with all the assessments of the Experts, one Modeling by line per cell/probability. Brainstorming Bayesia Expert 1/number of states of the Knowledge assessed variable Elicitation Environment ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 33 33
- 34. Analysis of the Expert Assessments The Expert Assessment file can be analyzed with the unsupervised learning algorithms of BayesiaLab for finding the direct probabilistic relationships that hold between the Experts’ assessments Plan Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Environment Each node represents the discretized probabilities assessed by the Expert ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 34 34
- 35. Automatic Segmentation of the Experts Based on the obtained network, Experts can be clustered into homogeneous groups by using the BayesiaLab’s Variable Clustering algorithm Plan Modeling by Brainstorming Dendrogram corresponding to that segmentation Each color corresponds to a Bayesia Expert cluster. Knowledge Elicitation Environment The real experts behind those anonymized experts have indeed 3 different profiles (functionally and geographically) Based on the obtained Expert Segments, one Bayesian network per segment can be generated (by using the Expert Editor). This can be useful for analyzing the sensibility of ©2012 BAYESIA SAS the model, but also to get specific networks (depending on the geographical localization All rights reserved. Forbidden for example) reproduction in whole or part without the Bayesia’s express written permission 35 35
- 36. Parameter Sensibility Analysis The Assessment Sensitivity Analysis tool allows measuring the uncertainty associated to the consensus Plan Generation of a set of networks by randomly selecting Experts’ assessments Modeling by Brainstorming Bayesia Expert Knowledge Elicitation Measurement of the uncertainty associated Environment to each probability distribution ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express The probability of Strong goes from 30% to 86% written permission 36 36
- 37. Contact Dr. Lionel JOUFFE President / CEO Plan Tel.: +33(0)243 49 75 58 Skype: +33(0)970 44 64 28 Mobile: +33(0)607 25 70 05 Modeling by Fax: +33(0)243 49 75 83 Brainstorming Bayesia Expert Knowledge Elicitation 6 rue Léonard de Vinci BP0119 Environment 53001 LAVAL Cedex FRANCE ©2012 BAYESIA SAS All rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express written permission 37 37