I think that this method makes a lot of sense for classification (images, text, …) and I’m looking forward to using it on image data when the R package will also cover this functionality.Exploring lime on the house prices datasetWeb Scraping with rvest: Exploring Sports Industry JobsBootstrap confidence intervals and confidence distritbutions – application on X-men data using ggdistI’m not yet sure if my interpretation of the output for regression problems is correct but if it is, then it will also be a useful method for regression. Here’s an example from the authors.It tells us that the 100th test value’s prediction is 21.16 with the “RAD=24” value providing the most positive valuation and the other features providing negative valuation in the prediction.To implement LIME, we need to get the categorical features from our data and then build an ‘explainer’. I thought it might be good to provide a quick run-through of how to use this library.
Since we are really most interested in looking at the LIME approach, we’ll move along and assume these are decent errors.
As opposed to lime_text.TextExplainer, tabular explainers need a training set. 126.96.36.199 Categorical features. He writes about utilizing python for data analytics at First, we’ll set up the RF Model and then create our training and test data using the train_test_split module from sklearn.
LIME is observing categorical features even though I am not passing anything.
Categorical features are handled more straight forward then numerical ones due to finite space. For this we can use partial dependence plots (PDPs) and individual conditional expectation (ICE) curves. To ensure that the explanation is interpretable, LIME distinguishes an interpretable representationfrom the original feature space that the model uses. The distance functions used are the cosine similarity measure for text data and the Euclidean distance for images and tabular data. The interpretable inputs map to the original inputs through a mapping function hʸ: X’→X, speci… For example, if we are trying to explain the prediction of a text classifier for the sentence “I hate this movie”, we will perturb the sentence and get predictions on sentences such as “I hate movie”, “I this movie”, “I movie”, “I hate”, etc. For classification it seems more intuitive to me. A big disadvantage is that the values of the features cannot be interepreted anymore easily, you would need to transform them back or look at the exact value of each sample if that is relevant.Now I’m curious what you think about the method. This demonstrates a binary classification problem (“Yes” vs. “No”) but the same process that you’ll observe can be used for a regression problem. Consider using consecutive integers starting from zero. It makes it much easier to ‘explain’ what the model is doing.Why Should I Trust You?”: Explaining the Predictions of Any ClassifierPandas Excel Tutorial: How to Read and Write Excel filesAs an example, if you are trying to classify plans as edible or poisonous, LIME’s explanation is much more useful. All values in categorical features should be less than int32 max value (2147483647). Let’s look into the code I wrote for my analysis (markdown report with code). Its a good library to add to your toolkit, especially if you are doing a lot of classification work. linear regression) models on perturbed input data to figure out which features are important. In other words, LIME relies on the assumption that every complex model is linear on a local scale.This equation cannot be directly solved, so they approximate the solution by first selecting
The parameters used by the function are: X_train = Training set; feature_names = Concatenated list of all feature names; class_names = Target values; categorical_features = List of categorical columns in the dataset; categorical_names = List of the categorical column names
lime: Local Interpretable Model-Agnostic Explanations. Have you heard about the paper before? The reason for this is because we compute statistics on each feature (column). Do you see a use case for local interpretable model-agnostic explanations?Most popular on Netflix.