Tell me in 2 minutes
KWM and Sapere have recently published a report, commissioned by the Australian Finance Industry Association (AFIA), on the impact of artificial intelligence (AI) on the Australian finance industry (Report).
Despite a slow and cautious start, uptake of Generative AI (GenAI) in the finance industry is set to more than double over the next three years, improving employee productivity and business processes. Under the Report’s medium adoption scenario, GenAI is projected to provide a value add to the finance industry of $15.9 billion and $48.9 billion to GDP by 2035.
However, there are unique risks associated with GenAI, relating to accuracy, transparency and misinformation and there is no intentional consensus on the best approach to regulate these risks. In Australia, there is no economy-wide, mandatory, AI-specific regulation, but AI is already heavily regulated in the finance industry by various rules and regulations relating to consumers, privacy and discrimination.
Given the potential productivity and economic benefits to Australia from the adoption of GenAI, the Report suggests that the imposition of unnecessary ex-ante regulation is likely to slow the adoption of Generative AI, and reduce the scale and timing of those benefits.
Context
In 2024, the AFIA commissioned KWM and Sapere to research the impact of AI on the Australian finance industry. Between July and October 2024, KWM and Sapere surveyed and interviewed participants in the Australian finance industry on their use and adoption of AI, including several of the Big 4 banks, finance companies and product providers.
Narrow AI
Narrow AI focuses on machine learning models that use predetermined algorithms and rules to analyse and make predictions, recommendations or decisions. Narrow AI often relies on an organisation’s existing dataset, providing a ‘point’ solution for a particular use case and cannot be used more generally or for solving novel scenarios.
Examples include traditional fraud detection tools or sales forecasting tools like Salesforce Einstein.
Generative AI
GenAI, the focus of the Report, also uses machine learning models, but generates new content based on data contained in its training datasets. GenAI primarily relies on external third-party datasets to train the underlying AI model. As such, outputs are probabilistic and risk inaccuracy, bias or discrimination.
Examples include OpenAI’s ChatGPT, Microsoft’s Co-Pilot or Google Gemini.
The Report, linked at the end of this article, uses the answers from those interviews and surveys, supplemented by economic modelling and a review of the current literature and market commentary, to analyse the impact of AI on the Australian finance industry, now and into the future. Annexure 3 of the Report contains further detail on the Report’s methodology.
How does the finance industry currently use AI and what’s next?
The Report found that Narrow AI is well-established within the Australian finance industry. Use cases include fraud detection, cybersecurity threat detection, risk management and document processing. In contrast, the uptake of GenAI has been slow and cautious. According to respondents, this cautious approach is due to concerns with privacy and data security, the regulatory environment, integration issues, skills shortages and limited expertise.
Nevertheless, GenAI is emerging in the Australian finance industry and adoption is projected to more than double in the next three years. Respondents suggest that GenAI will help reduce risk, increase productivity and improve customer experience. Currently, prominent use cases primarily relate to employee productivity (eg internal chatbots or process automation) and business processes (eg automating quality assurance or preparing/reviewing internal documentation).
Chapter 3 of the Report further explores the current use cases of both Narrow AI and GenAI.
The next step in GenAI development will be more customer-facing use cases, enabling the finance industry to provide more tailored and efficient services. This may include faster more accurate loan approvals or 24/7 direct customer support through AI-powered chatbots.
Those businesses that invest and deploy AI at scale, AI Leaders, are likely to benefit from significant competitive advantages, through improved productivity and enhanced customer experience. For CommBank, as an example, AI and technology has delivered improved customer outcomes including, a 50% reduction in customer scam losses, a 30% reduction in customer-reported frauds, and a 40% reduction in call centre wait time.[1]
Widespread economic benefits
The Report triangulated forecast impacts by the Tech Council of Australia and McKinsey to develop three scenarios regarding the direct productivity savings generated depending on the level of GenAI adoption. Irrespective of the low, medium or high adoption scenario, the economic benefits of GenAI will be begin modest, but significantly increase from 2031. In net present value terms, the finance industry’s investment in GenAI is projected to increase the industry’s revenue by an accumulative total of $15.9 billion from 2025 to 2035 (under the medium adoption scenario). For businesses operating in the finance industry, savings will primarily stem from a reduction in wages and employment, unlocking capacity for employees to concentrate on greater value tasks.
Commonwealth Bank, ‘Customer Safety, Convenience and Recognition Boosted by Early Implementation of Gen AI’ (Media Release, 28 November 2024).
This economic benefit will be shared with the nation and consumers, with a projected cumulative value add[2] of $48.9 billion, in net present value, to GDP, and $35.2 billion to household consumption[3], by 2035.
Chapter 4 of the Report contains further information and visual representations of the economic impact that GenAI is likely to have on the Australian finance industry.
What about the risks of GenAI?
While GenAI proposes to increase productivity and positively impact the economy, there are still risks involved, and identifying and mitigating those risks will be critical to effective adoption.
Broadly speaking, there are three kinds of risks that may arise from GenAI:
- Performance Risks: GenAI systems create harm by failing to perform as intended, eg biased system performance or security failures.
- Compliance Risks: GenAI systems create harm by performing as intended but failing to comply with existing laws, contractual requirements or societal expectations, eg failure to comply with privacy and Intellectual Property requirements when training AI systems.
- Use Risks: Malicious, misleading, reckless or inappropriate use where GenAI systems are used in a way which creates or amplifies a risk of harm, eg provision of misleading advice or fraudulent and unlawful use.
These risks are amplified when using GenAI. GenAI has a higher risk of inaccurate or misleading outputs given the use of probabilistic algorithms, hallucinations, and complex or opaque outputs. GenAI systems are also developed by a limited number of international developers, creating a lack of control over the AI model, exacerbation of herd behaviour and issues with appropriateness for the Australian market. Chapter 5 of the Report explores these risks in more detail.
While there can be no single solution to removing the risks of GenAI, risks can be reduced and managed through the development and deployment of GenAI in a responsible, ethical and legal manner, within a suitable risk and governance framework. Despite no consistent approach, many respondents have implemented some form of AI governance. Whatever form an AI governance framework may take, it is critical that it is implemented before GenAI is used in the organisation. This is not only best practice but expected by ASIC:
We expect licensees to carefully consider their readiness to deploy AI safely and responsibly. Decisions that licensees make now about how they will govern their AI use will determine whether they establish solid foundations on which to deliver the expected benefits and manage risks to themselves and their customers….[4]
No international harmonisation on regulation
International approaches to regulating AI have not reached a consensus and are highly fragmented.
In Europe, the European Union AI Act mandates prescriptive obligations on AI providers, deployers, importers and distributors, depending on the level of risk posed by an AI system.
President Trump has repealed President Biden’s executive order ‘Safe, Secure and Trustworthy Artificial Intelligence’ which established federal regulatory principles and encouraged private entities to safely use AI. Leaving states to regulate AI in their own unique ways, like Utah’s AI Policy Act enacting disclosure requirements on entities using GenAI with customers and Colorado’s AI Act imposing duties on developers and deployers of high-risk AI system.
In the Asia-Pacific region, Singapore has implemented government led voluntary principles and frameworks, including a voluntary Model AI Governance Framework. Hong Kong focuses on existing legislation and regulator guidance, with the Government releasing an Ethical Artificial Intelligence Framework and a Policy Statement on Responsible Application of Artificial Intelligence in Financial Markets.
See Annexure 5 of the Report for more detail on the international approach to AI regulation.
Australia has no mandatory, economy-wide AI specific regulation
Australia does not currently have economy-wide, mandatory AI specific regulation. In September 2024, taking inspiration from Europe’s AI Act, the Australian Government released the Proposed Mandatory Guardrails, which would see the mandatory imposition of 10 Guardrails on AI developers and deployers in high-risk settings. As of May 2025, the Proposed Mandatory Guardrails have not been implemented by the Australian Government. Although ASIC has indicated its approval of the Mandatory Guardrails in principle,[5] several regulators, including APRA, have already stated that existing regulation suitably addresses the risks of GenAI:
…we believe our prudential framework already has adequate regulations in place to deal with generative AI for the time being. Our prudential standards may not specifically refer to AI but nor do they need to at the moment. They have intentionally been designed to be high-level, principles-based and technology neutral.[6]
In support of this comment, the Report found that AI is already heavily regulated in the Australian finance industry. Annexure 6 of the Report outlines the legislation and regulations likely to apply to GenAI in the finance industry:
Value add is the difference between the value of the goods and services and the costs of the inputs to produce them.
Household consumption directly reflects the goods and services that households consume to meet their needs.
Australian Securities & Investments Commission, Beware the Gap: Governance Arrangements in the Face of AI Innovation (Report No 798, October 2024) 3: ‘[w]e support the Australian Government’s Voluntary AI Safety Standard and intention to introduce mandatory guardrails ensuring testing, transparency and accountability for AI in high-risk settings.’ Also see Introducing mandatory guardrails for AI in high-risk settings: Submission by the Australian Securities and Investments Commission (October 2024) 3: ‘ASIC supports the introduction of ex ante regulatory measures to mandate guardrails for the use AI in high-risk settings’.
Therese McCarthy (Executive Board Member, APRA), AFIA Risk Summit 2024.
Directors must ensure there is effective risk management and compliance systems for companies that use AI.[7] This includes understanding the regulations, implementing appropriate AI governance, considering the risks of AI use and ensuring ongoing compliance.
Corporations Act 2001 (Cth) s 180.
The Privacy Act 1988 (Cth) and the Security of Critical Infrastructure Act 2018 (Cth) regulate cyber risk and data protection/privacy.
Incorrect outputs risk breaching consumer guarantees and/or product liability laws, and misleading information produced by GenAI may be false, misleading, deceptive or unconscionable conduct.
Amongst other finance regulations listed in Annexure 6 of the Report, the APRA Prudential Standards consider risk management and operational resilience, and the Australian Security and Investments Act 2001 (Cth) focus on governance, market integrity and conduct, and competition.
Outputs of GenAI may not be protected by copyright under the Copyright Act 1968 (Cth) and using copyrighted data to train or input into GenAI may breach user rights.
AML/CTF Act 2006 (Cth)[8] covers outsourcing and third-party risk, and the potential for discriminatory outputs from GenAI may be covered by the various discrimination acts.[9]
Anti-Money Laundering and Counter Terrorism Financing Act 2006 (Cth)
Age Discrimination Act 2004 (Cth); Disability Discrimination Act 1992 (Cth); Racial Discrimination Act 1975 (Cth); Sex Discrimination Act 1984 (Cth); and State and Territory Discrimination laws.
What now?
As GenAI continues to develop and broaden its use cases, potential regulation must balance protecting against harm and encouraging significant investment in AI to leverage the substantial productivity and economic benefits on offer.
While this is a complex balancing act, to realise the potential productivity and economic gains it is important that the Government does not over-regulate AI and provide industry with regulatory certainty as soon as possible.
knee-jerk approaches to regulating AI threaten to stifle uptake and squander potential benefits. While there are clearly risks from AI adoption, government should take a considered approach to regulation that also keeps the benefits of AI in view.[10]
For more detail, download the report below.
Productivity Commission submission to the Senate Select Committee on Adopting Artificial Intelligence (May 2024).
Download
4.14MB, 56 Pages
