Everything you need for AI Performance in one platform.
We decided that Docs should have prime location.
Fundamentals of ML observability
Metrics, feature importance and more
We’re excited ???? to share that Forbes has named Aporia a Next Billion-Dollar Company. This recognition comes on the heels of our recent $25 million Series A funding and is a huge testament that Aporia’s mission and the need for trust in AI are more relevant than ever. We are very proud to be listed […]
Introduction and agenda >
Explainable AI is the ability to understand the output of a model based on the data features, the algorithm used, and the environment of the model in question, in a human-comprehensible way. It makes it possible for humans to analyze and understand the results provided by ML models.
This illustration from the US agency DARPA shows the challenges addressed by XAI:
XAI is the solution to the problem of “black-box” models, which do not make it clear how they arrive at specific decisions. It is a set of methods and tools that allow humans to comprehend and trust the results of AI models and the output they create.
It’s important to point out that explainability isn’t just for machine learning engineers or data scientists – it’s for everyone. Any explanation of the model should be understandable for any stakeholder – regardless of whether they’re a data scientist, business owner, customer, or user. Therefore, it should be both simple and information-rich.
This is part of an extensive series of guides about AI technology.
The National Institute of Standards (NIST) is US agency that works as part of the Department of Commerce. NIST defines the following four principles of explainable AI:
These principles aim to define the output expected from an explainable AI system. However, they do not specify how the system should reach this output.
XAI is categorized into the following three types:
Currently, explainable data is the only category that drives XAI for neural networks. Explainable algorithms and predictions are still undergoing research and development.
Here are the two main explainability approaches:
XAI aims to enable users to understand the rationale behind a model’s decisions, but the techniques used to achieve explainability can severely limit the model. Common explainable techniques include Bayesian networks, decision trees, and sparse linear models.
Ongoing research is applied to find ways to make regular, unexplainable models more meaningful to users. For example, incorporating graph techniques like knowledge graphs to make the model more explainable.
Here are the main challenges of XAI:
Here are two methods that can potentially help overcome the challenges of XAI in offering a meaningful explanation:
There are various explainability methods such as Shap, Partial Dependence Plot, LIME, ELI5. These can be characterized into three types: global explainability, local explainability, and cohort explainability.
The Global Approach explains the model’s behavior holistically. Global explainability helps you know which features in the model contribute to the model’s overall predictions. During model training, global explainability provides information about what features the model used in making decisions to stakeholders. For example, the product teams looking at a recommendation model might want to know which features (relationships) motivate or engage customers most.
Local interpretation helps understand the model’s behavior in the local neighborhood, i.e. It gives an explanation about each feature in the data and how each feature individually contributes to the model’s prediction.
Local explainability helps in finding the root cause of a particular issue in production. It can also be used to help you discover which features are most impactful in making model decisions. This is important, especially in industries like finance and health where the individual features are almost as important as all features combined. For example, imagine your credit risk model rejected an applicant for a loan. With Local explainability, you can know why this decision was made and how to better advise the applicant. It also helps in understanding the suitability of the model for deployment.
Somewhere between global and local explainability lies Cohort (or Segment) explainability. This explains how segments or slices of data contribute to the model’s prediction. During model validation, Cohort explainability helps explain the differences in how a model is predicting between a cohort where the model is performing well versus a cohort where the model is performing poorly. It also assists when trying to explain outliers as outliers occur in a local neighborhood or data slice.
Note: Both Local and Cohort (Segment) explainability can be used to explain outliers.
Aporia’s full-stack ML observability solution gives data scientists and ML engineers the visibility, monitoring and automation, investigation tools, and explainability to understand why models predict what they do, how they perform in production over time, and where they can be improved.
To see how the explainability feature works in Aporia, login into Aporia with your email – or if you haven’t created an account yet, you can easily sign up here to book a demo with our ML experts.
Go to the demo model, and from there to the Data Points dashboard. Next, click the Explain button. For this model, you can see how features contributed to the model’s prediction. You can also get a business explanation to share with key stakeholders. And, you can also change any feature value by clicking “Re-Explain” and see how it affects the prediction. This lets you debug your model for specific predictions.
Aporia’s explainability feature lets you look under the hood of your model and better understand:
As machine learning models continue to be adopted across all industries and are fast becoming a standard critical component of the decision-making process for organizations, the idea that ML models are “black boxes” will be debunked. Model predictions can be explained with explainable AI tools like Aporia.
Aporia makes ML models explainable, helping data science and ML teams to better understand their models and leverage their machine learning in a more effective and responsible way.