Daniel Trinder, Executive Director, Strategy and Design.
Generative Artificial Intelligence (Gen AI) is the latest technological breakthrough that regulators must grapple with. Gen AI refers to large learning models (LLMs) that can both comprehend and generate human language text – text, audio, code, graphics – based on user-defined queries of the large quantities of data on which the model was trained.
As Gen AI technologies are still maturing, their application is nascent. However, this “technology cycle” is not being driven by technology upstarts, but by the world's biggest technology firms with established adoption across the financial sector in New Zealand which gives potential to transform financial services.
Industry engagement will be essential for the FMA to understand how Gen AI is achieving positive consumer and market outcomes, as well as how risks are being managed by firms.
Potential benefits include better consumer outcomes and personalised customer experience. For instance, through faster response times and claims pay-outs along with efficiency gains to insurance firms deploying Gen AI sensibly. Advancements in transaction monitoring can result through improvements in detecting suspicious activity. These tools should also improve fraud detection and risk management.
However, with potential benefits come potential risks. New Zealand banks have said recently that Gen AI is a “double-edged sword”. View the FIPS Banks: Review OF 2023 report from KPMG. Where there are opportunities, there's also potential for cyber threats and fraud to be AI-weaponised by criminals and bad actors.
There may be the need for some firms to reassess their approach to governance to ensure adequate oversight of AI tools. The incorrect adoption of Gen AI applications can lead to undesired consequences and reputational damage or may not contribute to the business value of the organisation. Appropriate governance of such applications mitigates the associated risks, holds the appropriate people responsible, and can increase longevity and security of applications.
Firms need to focus on implications for their own operational resilience, including any services that they outsource to third parties. Most Gen AI models are run on the cloud and powered by large data, creating digital infrastructure and third party-related stability risks that must be considered. Firms are also at risk from AI-powered cyber-attacks.
One of the defining features of Gen AI is its ability to process large volumes of data, and to detect and exploit patterns in those data. Model risk management, and the risks that Gen AI exacerbates with sample bias, and model drift are important considerations for firms.
There are also a broader set of issues regarding data usage, accountability and protection.
AI accountability is the ability to explain, justify, and take responsibility for the outcomes and impacts of an AI system. Governance and assurance over data privacy and ownership will also be paramount.
Last, but not least, are the heightened risks to consumers. AI scams are already on the rise and the deepfake technology raises risks to consumers from scams or fraudulent activities.
Despite potential risks, the FMA believes that firms can mitigate these risks through appropriate governance structures, risk management, remaining vigilant over operational resilience implications and addressing data risks. All this should ensure firms utilising Gen AI remain focused on achieving positive and safe outcomes for consumers and markets. That said, it is not the FMA’s job to tell firms how to establish the guardrails they need.
Overtime we will assess whether our regulatory framework needs strengthening to support better deployment of Gen AI. We welcome ongoing dialogue with all firms on their experiences of using Gen AI, and similar technologies, to enhance our understanding of the benefits and understand if there are risks materialising to consumers through the incorrect adoption of Gen AI.
The FMA is technology-neutral and pro-innovation. We are keen to see firms harness the opportunities and to ensure firms utilising Gen AI remain focused on achieving outcomes that will benefit and enhance consumer’s experience of financial markets and services.
