At a private all-hands in early 2025, a US regional bank’s chief risk officer asked her newly hired AI lead a simple question, namely how the bank could deploy a large language model to summarize loan committee memos without tripping the same model risk rules that govern its credit scorecards. The answer that came back ran 26 pages and pointed straight at Federal Reserve guidance from 2011. That is the working reality of ai decisioning finance in 2026, where the models are new but the regulatory plumbing is not. The Federal Reserve and OCC’s joint SR 11-7 guidance on model risk management still sets the bar for every machine learning system that touches a banking decision.
The model stack inside US financial institutions
A typical large US bank now runs four distinct model layers. The first is classical machine learning, gradient boosted trees and logistic regression, used in credit underwriting, fraud scoring, and anti-money laundering alert ranking. The second is deep learning, applied to document understanding, voice biometrics, and image recognition for check deposits. The third is large language models, used for customer service chat, internal knowledge retrieval, and summarization of regulatory text. The fourth is reinforcement learning, used in narrow execution and pricing problems where the action space is small and the feedback loop is fast.
Each layer has its own training pipeline, monitoring dashboard, and validation team. The bank-wide model inventory at a top-five US bank now lists between 2,000 and 4,000 active models, of which several hundred touch customer decisions directly. Every one of those models is logged, owned by a named team, and re-validated on a schedule that ranges from quarterly to annual depending on materiality. The inventory is itself a deliverable for the model risk function and is reviewed by US federal examiners during the annual supervisory cycle.
Model risk under SR 11-7 and the OCC bulletin
The Federal Reserve’s SR 11-7, issued jointly with the Office of the Comptroller of the Currency as OCC Bulletin 2011-12, is still the operating manual. It defines a model as any quantitative method that processes inputs into outputs, sets expectations for development, implementation, and use, and requires independent validation. Every US bank holding company with more than $10 billion in assets is examined against the guidance, and the same standard now applies to AI systems that meet the definition.
The practical impact is that an AI model going into production needs documentation covering data lineage, training methodology, performance metrics, limitations, and a fallback plan. The validation team, organizationally separate from the developers, has to be able to reproduce the result and challenge the assumptions. For a credit scoring model that is a familiar exercise. For a 70-billion parameter language model the validation team has had to invent new methods, including counterfactual prompts, red team testing, and bias audits against US protected classes.
The OCC’s 2024 examination letters to several large US banks have made clear that fine-tuned open source language models fall inside the same framework. Banks that wanted to skip the validation queue by buying a third-party model still have to validate the third party, under the OCC’s separate guidance on third-party risk management.
NIST AI RMF, governance, and the audit trail
Outside the bank examiners, the National Institute of Standards and Technology’s AI Risk Management Framework has become the de facto governance standard for US financial firms. The framework, released in January 2023 and updated with a generative AI profile in 2024, sits at the NIST AI RMF resource hub. It defines four functions, namely govern, map, measure, and manage, and US banks have adopted it to fill the gaps in their existing model risk programs.
The govern function covers policy, accountability, and culture. The map function asks who is affected and what the model is supposed to do. The measure function defines metrics for fairness, performance, drift, and security. The manage function decides what to do when one of the metrics breaches a threshold. US banks have built dashboards that tie these four functions to their existing model inventory, and several have appointed a chief AI officer reporting to either the chief risk officer or the chief technology officer.
The audit trail behind all of this is the most engineering-heavy piece. Every prompt to a production language model, every response, every retrieved document, every user identity, and every downstream action is logged. The volumes are large, often 50 million events per day at a single large US bank, and the logs themselves become regulated records under the Securities Exchange Act and the Gramm-Leach-Bliley Act.
Prompt engineering and retrieval augmented generation in compliance
The technical work that distinguishes a finance-grade language model deployment from a generic one happens in three places. The first is the system prompt, which sets the rules the model has to follow on every turn. A US compliance team will write hundreds of lines covering tone, refusal patterns, escalation triggers, and the exact disclosure language required by FINRA Rule 2210 for any communication that touches investments. The second is the few-shot example set, which shows the model how to answer the questions it will see in production, including the awkward ones like a customer asking for advice the bank is not allowed to give.
The third, and the one that has consumed the most engineering hours in US banking through 2025 and 2026, is retrieval augmented generation. Instead of asking the language model to recall a fact, the system retrieves the source document from an indexed knowledge base, passes it to the model as context, and instructs the model to answer only from the passed text. The architecture cuts hallucination rates by roughly an order of magnitude in published US bank benchmarks, and the source document attached to every answer satisfies the examiner’s question of where the information came from.
For compliance specifically, retrieval augmented generation reads from the bank’s policy library, the relevant regulator’s rulebook, and the firm’s internal interpretations. The model summarizes the rule for the front-office user, cites the section, and refuses to answer if the retrieval returns nothing relevant. McKinsey’s financial services insights series tracks the productivity gains across US back offices that have rolled this pattern out at scale.
What US institutions are building next
Three engineering bets are shaping the next two years of AI decisioning in US finance. The first is agentic workflows, where a chain of model calls handles a multi-step task such as opening a small-business account, gathering the required documents, running KYC, and producing the file for human review. Capital One, JPMorgan, and BNY Mellon have all publicly described pilots. The hard part is not the model. It is the deterministic checks that wrap every model call so the workflow can be audited end to end.
The second bet is small specialist models. Rather than running a single 70-billion parameter general model for every task, US banks are fine-tuning 7-billion to 13-billion parameter models for specific functions, including loan memo summarization, suspicious activity report drafting, and call center quality monitoring. The smaller models cost a fraction to run, fit inside the bank’s own data center under existing GLBA and SOX controls, and validate more cleanly under SR 11-7. TechBullion’s AI in financial services hub and the regtech compliance overview track the rollouts as they happen.
The third bet is human-in-the-loop design for any decision that affects a customer’s money, credit, or insurance. US regulators have signaled that fully autonomous AI lending decisions will draw closer scrutiny, and several states have already enacted disclosure laws that require a human review path. The TechBullion fintech news desk continues to track state-level rulemaking. The next round of US Federal Reserve examinations, scheduled to land through 2026 and into 2027, will set the precedent for how aggressively US institutions can push AI into the parts of finance that still require a banker’s signature.
