Impressive though Machine Learning (ML) often is, explaining how a ML model makes a specific prediction is nigh on impossible.
And if we can't explain it, how do we know if the basis for a model's prediction is reasonable?
Can this ML model be trusted?
Take a look at the photos of the huskies and wolves in the photset below.
One of these species is cute and cuddly, the other often a killer. Knowing which is which can literally be a matter of life and death.
Compare each of the ML's predictions with its correct (shown as 'True' state) description. It scores pretty highly, getting it correct 8 times out of 10. Seems like a pretty good model, right?
Well, yes if you don't question how it is making its prediction.
Look carefully; you will see that in every photo of a wolf it is accompanied by snow. In fact it is the snow that the ML model is using to predict whether it is a wolf or husky - which may result in a poor outcome if a hiker relies on this model during summer months!
Clearly, understanding the underlying reasons for a ML model's behaviour is key to its credibility and, most importantly, acceptance. For ML models that are used to advise on health-related outcomes such 'explainability' is likely to be essential.
Start Simple
Some ML models are relatively simple to explain. For example, those that have a linear relationship between input data and the predicted outcome. For a moving vehicle, if we assume its speed is constant, then time and distance travelled have a linear relationship. For example, if the Time spent travelling doubles, so does the Distance travelled.
Graphically, it looks like this:
Linear regression
Where a set of measurements have a similar 'linear' relationship to the related outcome, a technique known as linear regression can be used to predict the outcome for a set of measurements. Graphically, a simple linear regression will look similar to the following, where each y is the outcome and each 'dot' represents a set of measurements that result in that outcome:
As a concept, linear regression is relatively easy to understand and explain to others. We'll return to it shortly.
Machine Learning models that use neural networks
Imagine if you were trying to sell the predictions of an ML model. If the ML model predicted its outcomes using linear regression you might feel confident in explaining its how to others. But, what if the ML model used a neural network as its basis for predicting?
As we will see, an ML model that uses even a 'simple' neural network represents a far greater challenge when it comes to explaining how they reach a particular outcome.
Creating and interpreting an ML model for diabetes
We'll illustrate the challenges involved in explaining a neural network-based model by training on actual data from the National Institute of Diabetes and Digestive and Kidney Diseases.
Data on 768 subjects has been provided. Each of these subject records contain 8 diagnostic measurements and a related outcome; if the outcome is '1' it implies the subject was diabetic.
A sample of the data is shown below:
The data was 'cleaned' (ie non-sensible data modified) and split into 116 test records, with the remaining 652 records used for training the model. Then, to improve the effectiveness of the neural network, prior to training it, the training data was normalised by scaling each measurement appropriately,
The build and training' of the neural network only required the following 2 lines of code:
Note the '(8,8,8,8,8)' - this is the core of the neural network; it identifies 5 'hidden' layers, each with 8 neurons. This means that as the model iterates up to 1,000 times - in an attempt to minimise the total model error across all of the training data - it can tune more than 265 parameters during each iteration!
How accurate is the model's prediction?
To provide insight in to the model's precison, we calculate the associated confusion matrix; the graphical version is:
What does this show? In a nutshell, how well the model predicts the 'diabetes outcome' of the 116 test records.
The correct predictions are shown in the top left and bottom right of the confusion matrix. Hence, out of the 116 records of test data, 56+28, ie 84 were correctly predicted and 20+12, ie 32 were not. Or, put another way, overall, 72.4% of outcomes were accurately predicted.
If you are wondering why the model is better at predicting 'non-diabetics' than diabetics it is because the data set contains significantly more records with 'non-diabetic' outcomes, which improve model fine-tuning for that specific outcome.
So far, so good.
But how would we explain to a client how the ML model correctly predicts the outcome nearly 3 times out of 4? More specifically, for an individual set of readings, how do we know the key diagnostic(s) that drive its outcome?
Put simply, we don't - the neural network is in effect a black box.
Enter LIME...
The main reason that the internal workings of the neural network are so difficult to interpret is that the model tries to solve for all of the training data, minimising the total error as it iterates. However, in reality we are interested in examining one subject at a time, ie to understand the key drivers for reaching the predicted outcome for the subject under examination.
Now look at the diagram below.
Assume that the squiggly blue line represents the neural network's attempt to fit all of the training data. Clearly, it's not perfect, but it's pretty good. Let's not even try to explain it, but just use techniques like the confusion matrix to provide an insight to its accuracy.
But hold on right there!
Remember the huskies vs wolves example above? Being accurate isn't sufficient, the model needs to provide a credible basis for determining the outcome. One way to do this could be provide a sample of predictions, and their key drivers to clinical experts for review. If they give the model the thiumbs-up, we're a lot closer to having an interpretable and, most importantly, credible model.
This is where LIME can help us.
Local Interpretable Model-agnostic Explanations (no, I didn't make it up!)
LIME doesn't seek to explain the inner workings of a model, whether based on a neural network or otherwise. Rather it uses the model to generate a set of outcomes near (ie 'local') to a specific record that we're interested in (the area in red). It then uses linear regression to identify the few, most significant drivers of the outcome. LIME is model-agnostic, because it treats the model as a black box - it just needs to know what outcome is generated for a specific input.
Putting LIME to work
Let's see an example where LIME provides insight in to a predicted outcome. Look at the extract of normalised test data below:
[To convert any of the above data to its actual (underlying) value, we just multiply it by the relevant entry below:
For example, in record #46, the actual BMI (Body Mass Index) is: 0.552906 x 67.10 = 37.1 ]
Now, let's examine record #45 more closely using LIME. Firstly, what is the predicted outcome from the model?
This predicted outcome of '1' matches the outcome in the underlying test data, in the table immediately above.
So far, so good. But what is driving this outcome? LIME provides an explainer in graphical form:
This shows the relative importance of each diagnostic measurement; a 'green' bar means the measurement is positively correlated with the outcome; red negatively.
Hence, for record #45 we can deduce that the key model drivers for a positive diabetes outcome are: glucose levels (above the normalised) 0.7, followed by 'diabetes pedigree function' (ie how 'genetically close' the person is to relatives with diabetes).
LIME can also represent the prediction in more detail as:
Record #46 also shows a positive (ie diabetes present) outcome, but note that the LIME drivers are different.
Collating the results of the two records, BMI, Glucose, Age, Diabetes Pedigree function appear to be key drivers of diabetes..
A qualified medical practitioner can review these LIME results, and others, to determine if the ML model is basing its predictions of diabetes on known drivers.
As an aside... in the paper, Predicting Type 2 Diabetes Using Logistic Regression and Machine Learning Approaches Ram D. Joshi1,† and Chandra K. Dhakal2,*† Pedro Femia-Marzo, Academic Editor is the following:
Our analysis finds five main predictors of diabetes: glucose, pregnancy, body mass index, age, and diabetes pedigree function.
Which, is very similar to what we have found with our piece of LIME analysis.
But what about the wolves?
Finally, yes, LIME can be used to show that it is the snow that drives the 'identification' of the wolf in the introductory example, not the wolf's features itself.
Rob Sucher is currently a reviewer of 'Interpretable Machine Learning using Python, 2nd edition'
Comments