What "Explainable AI" Actually Needs to Mean for an Underwriting Decision

Key Takeaways

• EIOPA, the IAIS and MAS all now require AI-driven insurance decisions to be "explainable" - but none of them define what a compliant explanation looks like in practice, and the industry is filling that gap with the wrong artefact.

• Most institutions are treating a SHAP or LIME feature-importance output - built for the model team - as if it were the explanation a customer, claims handler or ombudsman receives. It is not fit for that purpose, and regulators have said so explicitly.

• The 2023 US CFPB guidance on AI-driven credit decisions offers a working precedent BFSI leaders can borrow directly: a compliant reason must "relate to and accurately describe the factors actually considered," and model complexity is not a defence for vagueness.

• Every credible explanation for an underwriting decision needs to work as two distinct deliverables for two distinct audiences - a technical artefact for supervisors and model validators, and an operational one for the person who has to use it next. Building only the first is the most common and most expensive mistake.

• We propose a four-part standard - the Reason Code Bar - for judging whether an AI-generated underwriting explanation is actually usable by the humans who are accountable for the decision downstream.

Introduction

"Explainability" has become the word compliance teams reach for when they need an AI initiative to sound governed - appearing in board papers, vendor RFPs and model risk policies, usually followed by a checkbox rather than a specification. The problem is not that explainability is undefined in principle: EIOPA, the International Association of Insurance Supervisors (IAIS) and the Monetary Authority of Singapore (MAS) have each published substantive expectations on it in the past eighteen months. The problem is that none tell an insurer what the explanation has to look like when it lands on an underwriter's desk, in a claims file, or in front of an ombudsman.

That silence is doing real damage. Institutions are answering the explainability question with the artefact their data science team already has on hand - a feature-importance chart, a SHAP value table, a model card - because it is the fastest way to demonstrate that "explainability work" has happened. It has not. A chart that requires a data scientist in the room to interpret is documentation of a model, not an explanation of a decision. For underwriting specifically, those are different deliverables, built for different people, and regulators have started to say so in writing.

This piece sets out what the regulatory text actually requires, why the industry's default response misses it, and a concrete standard - the Reason Code Bar - for judging whether an AI-generated underwriting explanation would survive being handed to the person who actually has to act on it.

What the regulators actually require and where they stop short

EIOPA's Opinion on Artificial Intelligence Governance and Risk Management (EIOPA-BoS-25-360, August 2025) is the most direct statement APAC and Gulf-facing insurers with EU exposure will encounter. It requires undertakings to "ensure that the outcomes of AI systems can be meaningfully explained," and it explicitly separates the audience for that explanation. Supervisors are owed "a global and comprehensive explanation about the functioning of the AI system." Customers are owed something different: an explanation in "simple, clear and non-technical language," clarified further "upon the customer's request" where the AI system had "a material impact on them." EIOPA even extends the obligation to intermediaries, who must be told when a decision was AI-assisted "so that they can comply with their legal obligations towards customers." What EIOPA does not do is specify the form that second explanation takes - it instructs firms to "adapt the explanations to specific uses of AI systems," which is a governance principle, not a deliverable spec.

The IAIS's Application Paper on the Supervision of Artificial Intelligence (July 2025) goes one step further. It states plainly that "different stakeholders require different types of explanations, since not all stakeholders have the same technical knowledge." Supervisors get "comprehensive and technical information," including "data collection, processes and post-processing methodologies, feature importance," and "reasoning behind technical choices." Consumers get information that is "plain, simple and easy-to-understand," using "visual aids and layman's terms" and avoiding "excessive technical language." The paper even offers a worked underwriting example: "Your premium was influenced primarily by your driving history (40% impact), vehicle type (30% impact) and location (20% impact)." That example is more instructive about the industry's confusion than the paper probably intends - a percentage-weighted SHAP output translated into a sentence is still a SHAP output. It reads as plainer language than a raw model chart, but it is not yet a reason a claims handler could act on, and the IAIS's own text elsewhere concedes SHAP and LIME "still have relevant limitations that need to be duly considered."

Singapore's position rests on older ground. MAS's 2018 FEAT Principles already required firms to "offer its customers - and the general public - clear explanations of how they utilise AIDA and the impact doing so may have," citing the example of "explaining how it may use AIDA to determine a customer's insurance premium." In March 2026, MAS and a 24-firm industry consortium - including regional insurers AIA and Prudential - published the AI Risk Management Toolkit under Project MindForge, structured around four pillars: scope and oversight, AI risk management, AI lifecycle management, and enablers. Notably absent, based on the toolkit's public structure: a prescribed format for the customer-facing explanation FEAT promised eight years earlier. The transparency obligation is well established; the deliverable that satisfies it still is not.

The US offers the closest thing to a resolved answer, and it did not come from insurance regulation. The CFPB's September 2023 guidance on AI-driven credit decisions under the Equal Credit Opportunity Act states that adverse action reasons must be "specific" and must "relate to and accurately describe the factors actually considered or scored by a creditor" - and rejects complexity as an excuse: creditors "cannot justify noncompliance with ECOA because their own technology is too complicated, opaque, or novel." That is a credit rule, not an insurance one, and the legal basis does not transfer. But the operational discipline it demands - a reason tied to the actual drivers of this case, not a generic bucket - is exactly the standard insurance regulators are gesturing at without naming.

The operational translation problem

Put a Chief Underwriting Officer and a Head of Compliance in a room with "the model must be explainable" as their shared instruction, and they will build two different things. Compliance is trying to satisfy a supervisory expectation and survive an audit. The CUO is trying to make sure that when a customer or their broker asks "why was I rated this way," someone in the business can answer without escalating to the data science team. Both are legitimate readings of "explainable." Neither, on its own, produces the artefact the other needs.

What typically gets built is the compliance answer, because it is easier to specify and easier to demonstrate to an examiner: a model card, a SHAP summary, a validation report. It genuinely satisfies the IAIS's "comprehensive and technical information" bar. It does not satisfy the plain-language bar the same paper sets three paragraphs later, and it is not what a claims handler can put in front of a policyholder or a complaints officer can put in front of an ombudsman. The result is an institution that can pass a model risk review and still fail a real complaint, because the complaint doesn't ask "is the model documented" - it asks "why did this happen to me," and the honest answer on file is a percentage-weighted list of variables nobody downstream was equipped to translate.

This is not a data science failure - the SHAP output is doing exactly what it was built to do. The failure is organisational: nobody owns the translation layer between the model's technical explanation and a reason a human outside the model team can use. It is not a modelling problem, so data science does not own it; not a policy problem, so compliance does not own it; and it looks technical, so underwriting does not own it either. It falls into the gap between three functions that each assume someone else is closing it.

What most institutions get wrong

Three patterns show up repeatedly when institutions attempt this without a defined standard.

Treating the technical artefact as the deliverable. A SHAP chart or model card gets filed as "the explanation" because it exists and is defensible in an audit - never tested against whether the person handing it to a customer could actually use it.

A generic reason taxonomy that doesn't reflect the actual case. The pattern the CFPB called out directly for credit decisions is at least as common in insurance: a preset list of rating factors gets attached to every declined or repriced case regardless of what the model weighted for that individual, because a static list is easier to maintain than a case-specific mapping. It satisfies the letter of "a reason was given" while failing the substance of "the factors actually considered."

No stability testing. A reason code generated today needs to be the same one generated eighteen months from now if the case resurfaces in an ombudsman file or regulatory examination. Few institutions test whether their explanation output is reproducible under model retraining or data drift - so the explanation on file may not match what the current model would give, a gap that tends to surface during a dispute rather than before one.

A standard for the deliverable: The Reason Code Bar

If "explainable" is going to mean something operational rather than aspirational, it needs a test that can be applied to an actual output - not a policy, a principle, or a chart, but the specific sentence or notice a real case produces. We propose four clearances an AI-generated underwriting explanation should have to pass before an institution can call it operationally explainable, as distinct from technically documented.

1. Recipient clearance. The explanation names a real downstream reader - an underwriter, a claims handler, a complaints officer, an ombudsman panel - and that reader can use it unaided, without a data scientist present to interpret it. If the explanation only works with someone from the model team in the room, it has not cleared this bar, regardless of how accurate it is.

2. Fidelity clearance. The stated reason reflects the factors the model actually weighted most heavily for this specific case, not a static category from a preset list. This is the CFPB's standard applied to insurance: the explanation has to be true of this decision, not merely plausible in general.

3. Action clearance. The recipient can do something with it - dispute a specific fact, supply a specific document, explain a specific basis to a customer or a regulator. "Your risk profile was assessed as elevated" fails this test. "Your premium reflects three prior water-damage claims at this address in the past five years" passes it.

4. Reproducibility clearance. The same case, run again - next month, after a model refresh, during a dispute eighteen months later - produces the same reason, or the change is itself logged and explainable. An explanation that can't be reproduced can't be relied on in a file review.

The Reason Code Bar is not a replacement for the technical layer EIOPA and the IAIS require for supervisors and model validators - it sits on top of it. The two deliverables need to be traceable to each other: a claims handler's plain-language reason should trace back, on demand, to the SHAP or LIME output and full model documentation that satisfies the technical audience. Institutions that build only the operational layer will fail a model risk review as surely as institutions that build only the technical one will fail a complaint.

Business and technology implications

Producing a genuine reason code - rather than a translated feature-importance chart - is a design problem, not a documentation exercise, and it does not happen by asking the data science team to "write it in plain English" case by case. It requires four things, jointly owned by underwriting, compliance and model risk:

- A finite, case-relevant reason taxonomy, defined by underwriting and compliance together - not emergent from whatever variables the model happens to weight - so every output maps to a category a claims handler recognises and a customer can be told.

- A deterministic mapping layer between the model's raw attribution output (SHAP, LIME, or an ante-hoc interpretable model where viable - Deloitte's distinction between "ante-hoc" transparent-by-design models and "post-hoc" explanation techniques is relevant here) and that taxonomy, so the same inputs reliably produce the same category.

- Template-bound natural-language generation anchored to the case's actual dominant factors, not a generic paragraph - what turns "vehicle type: 30% impact" into a sentence a policyholder or ombudsman can read unassisted.

- A defined exception path for cases where full disclosure is deliberately withheld - fraud referral being the clearest example the IAIS flags. The lower-disclosure reason still needs to be logged and available to fraud, compliance and a supervisor on request - a designed exception, not the default for every inconvenient case.

None of this is exotic. Counterfactual explanations and surrogate models that "emulate [a complex model's] behaviour with easier-to-follow reasoning," in Deloitte's phrasing, are established techniques for this exact problem. What is missing in most institutions is not the technique; it is the governance decision to own the translation layer as a deliverable, with named accountability, rather than an afterthought to the model.

A fair counterargument

It would be a mistake to read any of this as "plain-language reason codes instead of technical documentation." They are not substitutes, and several sources above are explicit that both are required. The IAIS wants "comprehensive and technical information" for supervisors specifically - a plain-language reason code would not satisfy an examiner asking how the model was validated. The NAIC's Model Bulletin (2023) requires documentation of model development and assessments of "interpretability, repeatability, robustness" for regulatory examination - a technical obligation not discharged by a customer-facing notice. The Bank for International Settlements' Financial Stability Institute makes the underlying point directly: existing model risk guidance largely treats explainability as implicit "in the provisions relating to governance, model development, documentation, validation, deployment, monitoring and independent review" - baked into technical governance, not written as a consumer deliverable at all. And regulators will sometimes need to accept a genuine trade-off between explainability and predictive performance, provided - as EIOPA specifies - the institution documents the justification and adds compensating controls.

The honest position is that BFSI institutions need two properly built deliverables, not one improved one. The mistake is not building the technical layer - it is stopping there and assuming it discharges an obligation it was never designed to meet.

What leaders should do next

Start with an inventory, not a policy. Pull the actual explanation artefact on file for five recent real underwriting decisions - not demo cases - and run each against the four clearances above. Most institutions doing this for the first time find their artefact clears fidelity and reproducibility reasonably well, since those sit close to existing model governance discipline, and fails recipient and action clearance badly, because nobody was ever asked to build for that reader.

Second, assign ownership explicitly. Model risk and data science own the technical layer. Underwriting and compliance jointly own the reason-code taxonomy and translation layer, with a named accountable owner rather than a shared assumption someone has it covered - in most institutions we see, no one currently does.

Third, design the exception path before the taxonomy goes live: decide, in writing, which case categories get a lower-disclosure reason and why, rather than improvising under pressure from a complaints team.

Fourth, test reproducibility on a schedule, not just at launch. Re-run last year's declined and re-rated cases through the current model and confirm the reason code holds - and document it when it doesn't, before that surfaces in a dispute file rather than during one.

The sourceCode’s perspective

We build the systems that sit behind underwriting and claims decisions for BFSI clients across APAC and the Gulf, and the pattern above shows up on the delivery side as often as the compliance side: institutions commission "explainability" as a single feature to switch on, when it is two deliverables with two owners, two audiences and, done properly, two build tracks that stay traceable to each other.


The technical layer usually gets funded, because a model risk function knows how to specify it. The operational layer - the reason code a claims handler actually hands to a person - is usually improvised late, in response to a real complaint rather than in anticipation of one. Our view: the operational layer should be evaluated like any other production deliverable, against named users, defined acceptance criteria and a reproducibility standard - not treated as a compliance narrative written after the model is already live.

If you want a second opinion on where your current explainability output would hold and where it wouldn't, we'll read a de-identified sample against the Reason Code Bar above and tell you plainly. Contact us here!

Conclusion

EIOPA, the IAIS and MAS have all done the hard part of establishing that explainability is a live, enforceable expectation for AI-driven insurance decisions. None has done - or should be expected to do - the more specific work of telling an individual institution what a compliant explanation looks like for its own underwriting book. That work sits with the institution, and it will not get done by asking a data science team to caption a SHAP chart. It gets done by treating the reason a customer, a claims handler or an ombudsman receives as a deliverable in its own right - one that names its reader, reflects the real drivers of the case, gives that reader something to act on, and holds up if someone pulls the file again next year.


That is what "explainable" needs to mean for an underwriting decision. Everything short of it is documentation wearing explainability's name.

Frequently Asked Questions

Does EIOPA require insurers to publish the algorithms behind underwriting decisions? No. EIOPA's August 2025 Opinion requires that AI outcomes "can be meaningfully explained" and that customers receive an explanation in "simple, clear and non-technical language" on request where the decision has a material impact on them. It does not require disclosure of source code, model weights, or proprietary methodology, and it explicitly allows for complementary risk management measures where full transparency isn't achievable for highly complex systems.

Is a SHAP or feature-importance output sufficient to meet insurance explainability requirements? It satisfies the technical, supervisor-facing half of the requirement - the IAIS explicitly expects "feature importance" data for supervisors and auditors. It does not, on its own, satisfy the plain-language, consumer-facing half both EIOPA and the IAIS separately require, because a percentage-weighted attribution list is not the same thing as a reason a non-technical recipient can act on.

Do EIOPA, the IAIS and MAS require the same explainability standard? No, and this is a genuine complexity for cross-border BFSI groups. All three require some form of adapted, audience-appropriate explanation, but none prescribes an identical format, and MAS's foundational FEAT transparency principle (2018) predates and is structured differently from EIOPA's 2025 Opinion and the IAIS's 2025 Application Paper. An institution operating across the EU, Singapore and the wider APAC/Gulf region needs an internal standard that satisfies the strictest reasonable reading of each, rather than three parallel compliance tracks.

Does explainability conflict with fraud detection or model IP protection? Sometimes, and regulators acknowledge this. The IAIS notes that for certain use cases such as fraud detection, insurers "may not be able to disclose detailed information to consumers." The answer is a defined, documented exception path with a lower-disclosure reason and an internal audit trail - not blanket vagueness applied to every case an institution would rather not explain in detail.

Can model complexity be used as a defence for a vague or generic explanation? Not credibly, based on the pattern across regulators. The clearest statement comes from the CFPB's 2023 guidance on AI-driven credit decisions, which states that a creditor "cannot justify noncompliance with ECOA because their own technology is too complicated, opaque, or novel." While that is a lending rather than an insurance rule, the same operational logic is implicit in EIOPA's and the IAIS's insistence that explanations be tailored to the actual decision, not a generic disclosure.

What's the fastest way to test whether our current underwriting explanations are actually usable? Pull five real recent cases - not demo cases - and check whether the explanation on file names a specific downstream reader, reflects the actual dominant factors in that case, gives the recipient something concrete to act on, and would reproduce if the case were run again today. That is the Reason Code Bar described above, and it can be run as a same-week internal exercise before any new tooling is commissioned.

Reference List

Bank for International Settlements, Financial Stability Institute (n.d.) How regulators can address AI explainability, FSI Insights. Basel: BIS. Available at: https://www.bis.org/fsi/fsipapers24.pdf (Accessed: 28 September 2026).

Cleary Gottlieb (2018) Meet FEAT: Singapore's New AI and Data Analytics Principles for the Financial Sector, Cleary FinTech Update. Available at: https://www.clearyfintechupdate.com/2018/11/meet-feat-singapores-new-ai-data-analytics-principles-financial-sector/ (Accessed: 28 September 2026).

Deloitte (2025) Unleashing the power of machine learning models in banking through explainable artificial intelligence (XAI). Available at: https://www.deloitte.com/us/en/insights/industry/financial-services/explainable-ai-in-banking.html (Accessed: 28 September 2026).

EIOPA (2025) Opinion on Artificial Intelligence Governance and Risk Management, EIOPA-BoS-25-360. Frankfurt: European Insurance and Occupational Pensions Authority. Available at: https://www.eiopa.europa.eu/document/download/88342342-a17f-4f88-842f-bf62c93012d6_en (Accessed: 28 September 2026).

IAIS (2025) Application Paper on the Supervision of Artificial Intelligence. Basel: International Association of Insurance Supervisors. Available at: https://www.iais.org/uploads/2025/07/Application-Paper-on-the-supervision-of-artificial-intelligence.pdf (Accessed: 28 September 2026).

insuranceNEWS.com.au (2026) Ombudsman 'will never use AI' for dispute rulings. Available at: https://www.insurancenews.com.au/daily/ombudsman-will-never-use-ai-for-dispute-rulings (Accessed: 28 September 2026).

Monetary Authority of Singapore (2018) Principles to Promote Fairness, Ethics, Accountability and Transparency (FEAT) in the Use of Artificial Intelligence and Data Analytics in Singapore's Financial Sector. Singapore: MAS. Available at: https://www.mas.gov.sg/publications/monographs-or-information-paper/2018/feat (Accessed: 28 September 2026).

Monetary Authority of Singapore (2026) MAS Partners Industry to Develop AI Risk Management Toolkit for the Financial Sector. Singapore: MAS. Available at: https://www.mas.gov.sg/news/media-releases/2026/mas-partners-industry-to-develop-ai-risk-management-toolkit-for-the-financial-sector (Accessed: 28 September 2026).

Morrison Foerster (2023) CFPB Issues Guidance on AI Use in Credit Decisions. Available at: https://www.mofo.com/resources/insights/231002-cfpb-issues-guidance-on-ai-use-in-credit-decisions (Accessed: 28 September 2026).

National Association of Insurance Commissioners (2023) Model Bulletin: Use of Artificial Intelligence Systems by Insurers. Kansas City: NAIC. Available at: https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf (Accessed: 28 September 2026).

Allen & Gledhill (2026) MAS launches AI Risk Management Toolkit for financial services sector. Available at: https://www.allenandgledhill.com/sg/publication/articles/32846/mas-launches-ai-risk-management-toolkit-for-financial-services-sector (Accessed: 28 September 2026).

Compliance Corylated (2026) Singapore launches AI risk management toolkit. Available at: https://www.compliancecorylated.com/news/singapore-launches-ai-risk-management-toolkit/ (Accessed: 28 September 2026).