Ex Machina: How AI Threatens EU Financial Stability and What Regulators Are Doing

The European Systemic Risk Board's "Ex machina" report on artificial intelligence and financial stability, published alongside similar assessments from the European Securities and Markets Authority and the European Banking Authority, marks a turning point in how EU regulators conceptualise AI risk. The concern is no longer simply that individual AI systems might fail or discriminate. It is that widespread adoption of similar AI tools by interconnected financial institutions creates systemic vulnerabilities that no single firm's risk management framework can address.

This article examines the key findings of the EU financial stability AI assessment and the regulatory obligations that financial institutions must now satisfy.

Herding Behaviour From Correlated AI-Driven Trading

The most acute systemic risk identified in the "Ex machina" report is herding. When a large number of financial institutions use AI models trained on similar historical data, using similar architectures, and optimised against similar performance benchmarks, those models will generate correlated outputs in response to market signals. Where human traders might bring divergent interpretations and risk appetites to the same market event, AI trading systems may simultaneously move in the same direction, amplifying price movements rather than absorbing them.

The September 2022 UK gilt market crisis — triggered partly by liability-driven investment strategies executing simultaneously — was cited in subsequent EU analysis as a non-AI analogue for how model homogeneity creates fragility. AI-driven trading at scale presents a structurally similar but potentially faster-moving version of the same problem.

ESMA has examined this concern through the lens of MiFID II Article 17, which requires investment firms using algorithmic trading to have effective systems and risk controls. Existing MiFID II requirements — annual self-assessment, kill switch availability, pre-trade controls — were designed for deterministic algorithmic strategies. AI systems with adaptive behaviour and complex feedback loops may not be adequately captured by these frameworks, and ESMA's guidance has begun addressing this gap.

Model Risk and Concentration Risk

The concentration dimension of AI risk in financial services is distinct from herding. Concentration risk arises when many institutions rely on the same or functionally identical underlying models — often provided by a small number of cloud AI vendors or foundation model providers.

The "Ex machina" report documents significant concentration in the AI tooling used by European banks: a handful of cloud providers supply the computational infrastructure, a small number of AI platform vendors supply the model libraries, and foundation models from two or three providers underlie a significant proportion of deployed financial AI applications. If a vulnerability is discovered in a widely used model, or if a key provider experiences an operational incident, the impact propagates across institutions simultaneously.

DORA — the Digital Operational Resilience Act, Regulation (EU) 2022/2554 — directly addresses this through its ICT third-party risk management requirements. DORA Article 4 requires financial entities to implement a comprehensive ICT risk management framework that includes identification and documentation of ICT dependencies, including on AI model providers. Under DORA Articles 28 through 44, contracts with critical ICT third-party service providers must include provisions on concentration risk, business continuity, and audit rights.

For AI specifically, the interaction with DORA requires financial institutions to treat their AI model providers as ICT third-party service providers subject to DORA's enhanced due diligence requirements. Where an AI model provider is designated as a critical third party by the ESAs, additional supervisory oversight applies.

Cybersecurity Vulnerabilities

AI systems in financial services introduce cybersecurity attack surfaces that traditional IT risk frameworks do not fully address. The "Ex machina" report identifies three categories of concern.

First, adversarial attacks: inputs deliberately crafted to cause AI models to misclassify or behave incorrectly. In a fraud detection context, adversarial inputs could be designed to evade detection. In a credit scoring context, adversarial inputs could be used to manipulate scores. Second, data poisoning: attacks that corrupt training data to introduce systematic biases or backdoors into deployed models. Third, model extraction: techniques that allow adversaries to reconstruct a financial institution's proprietary model through careful querying, enabling competitive theft or targeted circumvention.

DORA Article 4 requires ICT risk management frameworks to address the full lifecycle of threats, including novel attack vectors. The European Banking Authority's guidelines on ICT and security risk management, which DORA supersedes but which inform DORA's interpretive guidance, require financial institutions to assess AI-specific cybersecurity risks as part of their operational resilience programmes.

EU AI Act Article 15 requires high-risk AI systems to be developed with cybersecurity in mind, including robustness against adversarial attacks. For AI systems used in credit scoring and insurance risk assessment — Annex III point 5 — this requirement applies to both providers and, through Article 26 deployer obligations, to the financial institutions using them.

Explainability Failures in Credit Decisions

Credit decisions based substantially on AI model outputs raise both prudential and legal concerns. On the prudential side, supervisors have found that financial institutions using complex AI models for credit scoring sometimes cannot adequately explain to their own risk managers or boards why particular decisions were made, making model validation and governance challenging. The EBA guidelines on loan origination and monitoring, GL/2020/06, require institutions to be able to explain the factors driving credit assessments to enable meaningful model governance.

On the legal side, GDPR Article 22 provides individuals with the right not to be subject to decisions based solely on automated processing that produce significant effects, including the right to an explanation of the logic involved. For AI-driven credit decisions, this requires financial institutions to maintain explainability capabilities sufficient to respond meaningfully to individual access and explanation requests. "The model said so" is not adequate.

EU AI Act Article 13 requires high-risk AI systems to be designed with sufficient transparency that deployers can understand the system's outputs and fulfil their human oversight obligations. For credit scoring systems in Annex III point 5, this means providers must supply documentation enabling deployers to understand the key factors influencing outputs — documentation that feeds directly into GDPR Article 22 compliance.

EBA Internal Governance Requirements

The EBA guidelines on internal governance under the Capital Requirements Directive require management bodies to have sufficient understanding of material risks, including model risk. As AI models become material to credit, market, and operational risk decisions, board-level AI literacy and oversight have become governance requirements, not merely best practices.

The guidelines specifically require institutions to maintain model inventories, conduct model validation, and assign clear accountability for model performance. For AI models, model validation must address data quality, performance stability, and the potential for unexpected behaviour under distributional shift — market conditions that differ materially from training data.

Practical Compliance Steps for Financial Institutions

The regulatory framework for AI in EU financial services is now substantially complete. DORA applies from January 2025. The EU AI Act's high-risk obligations apply from August 2026. Financial institutions should treat these as a single integrated compliance programme rather than separate workstreams.

Concretely: conduct an AI inventory mapping all AI systems against Annex III point 5 (credit, insurance, and essential services) and point 8 (administration of justice and democratic processes) to determine high-risk classification. For each high-risk system, verify that conformity assessment documentation exists or is in progress. Map all AI model providers to DORA's ICT third-party risk management framework. Update model governance policies to address AI-specific risks including adversarial robustness, distributional shift, and concentration. Review GDPR Article 22 compliance for all AI systems making or materially influencing credit, insurance, and employment decisions.

Frequently Asked Questions

Does DORA apply to AI model providers directly? DORA applies to financial entities and to their ICT third-party service providers designated as critical by the ESAs. AI model providers that are designated critical third parties are subject to direct supervisory oversight. Non-designated AI vendors are subject to DORA indirectly through the contractual and due diligence requirements placed on financial institutions.

What is the relationship between EU AI Act Article 9 and EBA model validation requirements? Article 9 requires high-risk AI providers to implement risk management processes throughout the lifecycle. EBA model validation requirements apply to financial institutions as deployers. The two frameworks are complementary: Article 9 addresses provider obligations, while EBA guidelines address deployer governance. A financial institution that is also the developer of its own credit scoring AI must satisfy both.

How does MiFID II Article 17 interact with EU AI Act obligations for algorithmic trading AI? MiFID II Article 17 governs operational controls for algorithmic trading; the EU AI Act governs AI system conformity. Trading AI is not automatically Annex III high-risk under the current text — the high-risk categories focus on critical infrastructure, biometrics, employment, essential services, law enforcement, migration, and justice. However, AI Act general obligations on transparency and human oversight apply to all AI systems regardless of Annex III classification.

Sources

• European Systemic Risk Board, "Artificial intelligence and financial stability," ESRB Report 2024 ("Ex machina" report)
• Regulation (EU) 2022/2554 (DORA), Articles 4, 28–44
• Regulation (EU) 2024/1689 (EU AI Act), Articles 9, 13, 15, 26; Annex III point 5
• Regulation (EU) 2016/679 (GDPR), Article 22
• Directive 2014/65/EU (MiFID II), Article 17
• European Banking Authority, Guidelines on loan origination and monitoring, EBA/GL/2020/06
• European Banking Authority, Guidelines on internal governance under CRD, EBA/GL/2021/05
• European Securities and Markets Authority, Report on the use of AI in financial services, ESMA50-164-6229