Svitlana Galeshchuk Development of Explainable NLP Models: "You show me the man and I will show you the rule"

Development of Explainable NLP
Models: "You show me the man and
I will show you the rule"
SVITLANA GALESHCHUK

About me
vData Scientist: 6 years
vIn NLP: 3 years
vFulbright Scholar in 2015-2016, USA
vVisiting Associate Prof. at University of Grenoble, France 2017
vData Scientist, Lecturer and Researcher at PSL/University of Paris
Dauphine, France, since 2017
vData Scientist at Starclay Consulting, France, since 2019
vEmail: svitlana.galeshchuk@gmail.com
Nov 5, 2020 UA Online Data Science Marathon

NLP — Natural Language “Processing” =
NLU — Natural Language “Understanding” (Sentiment Analysis, Topic
Classification, Entity Detection) +
NLG — Natural Language “Generation” (textual summaries, etc)
I. What is NLP ?

Word Embedding
NLP : Natural Language Processing
• 2001 : Neural language models: word embedding
> converting the words into vectors
Bengio, Y., Ducharme, R. & Vincent, P. A. Neural probabilistic language model.
Proc. Advances. Neural Information Processing Systems 13. 932–938 (2001)
• 2013 : Model Word2vec : Linguistic Contextualisation of words
> Predict the word based on the context
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. Distributed representations of words and phrases
and their compositionality. Advances in neural information processing systems. 3111-3119 (2013)

• 2018 : Le modèle révolutionnaire BERT de Google
> Bidirectional Encoder Representations from Transformers
[1] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. Attention is all you need.
Advances in neural information processing systems. 5998-6008. (2017)
[2] Devlin, Jacob, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint
arXiv:1810.04805 (2018)
BERT

ØText is a set of words;
ØWords are discrete values, hence the curse of dimensionality;
ØEmbedding (converting words into vectors) is the way to use text in
ML;
ØAutoregressive nature of natural language makes ML practitioners to
often use LSTM in NLP tasks;
ØBERT being a major breakthrough since 2017 is difficult to put into
production; it is good for texts less than 512 tokens.
To retain:

II. Explainable AI

Explainable AI Methods

Ribeiro, M. T., Singh, S., & Guestrin, C. " Why should i trust you?" Explaining the predictions of any classifier.
In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining.
1135-1144. (2016)
LIME Intuition
LIME

+ LIME delivers only
local explanations
LIME Disadvantages

Medical records
(14.000 patients)
Comments of the clinical history
of patient, lifestyle and the
symptomes
(features)
Motive of
hospitalisation
(features)
Principal diagnosis
(target)
Used Data
Use Case: Hospital Data

Ingelsson, E., Lundholm, C., Johansson, A. L., & Altman, D. Hysterectomy and risk of cardiovascular disease: a
population-based cohort study. European heart journal, 32(6), 745-750. (2011)
Laughlin-Tommaso, S. K., Khan, Z., Weaver, A. L., Smith, C. Y., Rocca, W. A., & Stewart, E. A. Cardiovascular and
metabolic morbidity after hysterectomy with ovarian conservation: a cohort study. Menopause (New York,
NY), 25(5), 483. (2018)
« Women who have had a hysterectomy, especially before the age of 35, have a higher
risk of having a stroke. About 70,000 hysterectomies are performed each year in France »
Stroke (863 patients) :

Coumbaras, M., A. Duval, P. Le Hir, N. Jomaah, L. Arrivé, and J. M. Tubiana. "Fibrolipome du filum terminal." J Radiol 84.
721-7222 (2003)
«When the lipoma is located in the thoracic region, it can be responsible for chronic back pain
and sometimes headaches»
Low back pain (1040 observations) :

« the Shapley value: It is the average of the marginal contributions across all
permutations »
« What Shapley does is quantifying the contribution that each player brings to the game.
What SHAP does is quantifying the contribution that each feature brings to the prediction
made by the model »
SHAP: both local and global explainability
Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." Advances
in neural information processing systems. 2017.
Shap Local Results
SHAP

LIME and SHAP into generating innocuous explanations which do not reflect the
underlying biases
Takes a long time to compute. For large datasets, it is computationally expensive
to use the entire dataset and we have to rely on approximations (e.g.,
subsample the data). This has implications for the accuracy of the explanation.
Original SHAP implementation has issues with visualization when more than 20
words are in the text:
Slack, Dylan, et al. "Fooling lime and shap: Adversarial attacks on post hoc explanation methods." Proceedings of the
AAAI/ACM Conference on AI, Ethics, and Society. 2020.
SHAP Disadvantages

Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic attribution for deep networks. arXiv preprint
arXiv:1703.01365.
Mudrakarta, Pramod Kaushik, et al. "Did the model understand the question?." arXiv preprint
arXiv:1805.05492 (2018).:
« As the input varies along the straight line path between the baseline and the
input at hand, the prediction moves along a trajectory from uncertainty to
certainty (the final prediction probability). At each point on this trajectory, one
can use the gradient with respect to the input features to attribute the change in
the prediction probability back to the input features. IG aggregates these
gradients along the trajectory using a path integral »
Øapt for all differentiable models;
Øeasy to implement;
Øcomputationally scalable to massive deep networks;
Ømuch faster than a naive Shapley-value-based method
INTEGRATED GRADIENTS

§ Not working with non-differentiable model types (random forest, etc);
§ Some counterintuitive explanations
IG Disadvantages

DeepLIFT proceeds in a backward fashion. Each unit is assigned an
attribution that represents the relative effect of the unit activated at the
original network input x compared to the activation at some reference
input. Reference values for all hidden units are determined running a
forward pass through the network, using the baseline as input, and
recording the activation of each unit
Pros: very fast
Cons: picking the baseline inputs
Gabriel Tseng
DeepLIFT

Integrated Gradients: Mukund Sundararajan, Ankur Taly, Qiqi
Yan, Axiomatic Attribution for Deep Networks, 2017
DeepLIFT: Avanti Shrikumar, Peyton Greenside, Anshul
Kundaje, Learning Important Features Through Propagating Activation
Differences, 2017
SHAP values: Scott M. Lundberg, Su-In Lee, A Unified Approach to
Interpreting Model Predictions, 2017
LIME: Ribeiro, M. T., Singh, S., & Guestrin, C. Why should i trust you?
Explaining the predictions of any classifier (2016)
Literature

•Explanation of the black-box models’ outputs is an important step towards
making the bridge between the model and its end-user;
•Explainable AI methods may deliver global or/and local interpretaions;
•Most of the current approaches are based on the cooperative game theory;
•Validation of interpretations is usually provided by field experts. Kullback-
Leibler divergence are sometimes used to assess the interpretations.
•Python implementation:
Shap: https://github.com/slundberg/shap
LIME: https://eli5.readthedocs.io/en/latest/overview.html
IG, DeepLIFT: https://captum.ai/
To retain:

Svitlana Galeshchuk Development of Explainable NLP Models: "You show me the man and I will show you the rule"

Recomendados

Recomendados

Más contenido relacionado

Similar a Svitlana Galeshchuk Development of Explainable NLP Models: "You show me the man and I will show you the rule"

Similar a Svitlana Galeshchuk Development of Explainable NLP Models: "You show me the man and I will show you the rule" (20)

Más de Lviv Startup Club

Más de Lviv Startup Club (20)

Último

Último (20)

Svitlana Galeshchuk Development of Explainable NLP Models: "You show me the man and I will show you the rule"