As the adoption of AI in service delivery increases, government needs to decide what to require of its suppliers in relation to their use of AI. The DTA has released AI Model Clauses that Commonwealth Government Agencies can use when procuring services where the supplier may be using AI. In this sixth alert in our AI in Government series, we take a closer look at the model clauses and when and why you might use them.
Scope of the provisions
The model AI clauses aim to give Agencies guidance in two situations:
- an Agency is procuring services where the Seller may be using AI systems in their delivery of the services, and
- an Agency is engaging a vendor to develop a bespoke AI tool for the Agency to use.
This alert addresses some things for Agencies to consider when using the provisions for the first situation. A future alert will dive into the model clauses for the second situation.
The DTA is also developing further clauses for Agencies to use when they are procuring software products that have embedded or integrated AI capabilities. Watch this space for when those are developed.
The model clauses
Reports suggest that AI is now widely used in business, so Agencies can safely assume that many suppliers integrate it into the delivery of their services. If your contract doesn’t prevent the use of AI, then this may well be happening already, without your knowledge. The model clauses are clearly aimed at ensuring government knows when AI is being used in service delivery, and they can help Agencies navigate some of the accountability and privacy concerns that can arise with AI use. However, we think care should be taken in adopting these model clauses. Agencies need to think practically about how to apply the clauses rather than copying them into their service contracts and hoping for the best.
These are our thoughts on using the clauses.
|
What it requires
|
But think about …
|
Example
uses 2
|
|
|
AI System approval (clause 1.1)
|
Sellers must obtain the Agency’s written approval prior to using an AI system for the provision of the goods and services. |
The purpose of this clause is to make sure suppliers aren’t using AI in the delivery of services to you without your knowledge. It’s broad – any AI use “for the provision” of goods or services. So if your consultant is using AI for background research to supply services, or your software provider is using it to help with coding, this will require consent. Consider if you want the supplier to get consent for every different use case or whether you will approve use of an AI system for various use cases. If you want to approve individual use cases, then you will need to turn your mind to the burden this could impose. Having to constantly review and approve AI use in a service with considerable AI use on a large project could delay things – so factor that in. |
|
|
Accuracy and reliability of outputs (clause 1.2)
|
The Seller must conduct quality assurance checks on the AI system to ensure its outputs are accurate and reliable. |
It is important to have oversight mechanisms in place when dealing with AI (eg having a human check AI outputs). That’s a given. But accuracy and reliability are inherent issues with AI systems that can rarely be resolved by just conducting quality assurance processes. Your supplier probably won’t be able to guarantee the accuracy and reliability of the AI outputs, but what they can do is ensure there is oversight and checking of the AI system output to ensure that the product or service actually delivered to you is accurate and reliable. |
|
|
Record keeping requirement (clause 1.3)
|
Sellers must keep details of:
|
Depending on the service or product you are acquiring, this record keeping requirement could impose costs on your supplier that they may want to pass through to you. Consider what this would practically mean for your supplier in the context of what you are buying (eg the volume of data that your supplier is working with) before deciding to include this clause (or whether it needs amendment). These clauses also require that the supplier provides these records to the Agency on request. However, the obligation will only apply while they are using AI to deliver the services to you. Consider whether you want there to be an ongoing obligation on the supplier to provide the records (even after they have stopped using the AI to deliver the service to you). |
|
|
Banned AI Systems (clause 1.4)
|
Sellers must not use any banned AI systems. Sellers must also ensure that a banned AI system is not used in any part of the supply chain that is used to provide the products and services. The Seller’s supply chain includes any subcontractor, agent, supplier or entity that they are in a direct or indirect relationship with. |
A banned system is one that is listed as banned in the Statement of Requirement/Specification. According to a Protective Security Direction published by the Department of Home Affairs, Agencies need to prevent the use or installation of DeepSeek. So if you are using this clause, your Statement of Requirement/Specification should cross refer to the Home Affairs list (as it is updated from time to time) to make sure you are aligned with the latest banned AI systems. This clause requires sellers to monitor their own supply chain to the extent that it is relevant to the goods and services delivered to government. Government contracts tend to have audit rights, so suppliers are going to need to consider how they will demonstrate that they have complied with this (which they may not currently do in a way that would be auditable). |
|
What should my Agency do now
The model clauses are optional and designed to be tailored to the specific AI requirements and risk profile of each procurement contract.
We think it is a good idea for Agencies to be including at least the clauses in relation to the notification of the use of AI in all goods and services contracts that they enter into given the prevalence of AI, but consider how granular you need these notifications to be (eg all use cases for a particular AI system, or individual use cases to be approved separately). Agencies should also consider whether they want to include the additional clauses having regard to the comments above and where they are appropriate given the size, value and subject matter of the contract.
Stay tuned for our next alert on the model clauses covering the procurement of a bespoke AI system.
