Australia has mapped how AI agents fail together. It starts mattering the moment you have two

The Australian AI Safety Institute has published its first report, Risks and controls for multi-agent systems, commissioned from Gradient Institute and released on 10 August. The department describes it as a world-first systematic technical framework mapping what goes wrong when AI agents interact, and crucially, who is actually in a position to do anything about it.

It is written for policymakers and large organisations. Skim it anyway, because one sentence in it is the most useful thing said about business automation this year:

A system made up of individually safe and reliable agents is not necessarily a safe and reliable system.

That is not a warning about the far future. It is a description of what happens in a small business the week you go from one automation to two.

Three tiers, and you are in the first one

The report sorts everything by who controls the agents, which turns out to be the question that decides what you can actually fix.

Tier 1, singular governance. One organisation runs every agent. You can inspect, monitor and intervene anywhere. Internal automations, a help desk assistant, a coding agent on your own codebase.

Tier 2, federated governance. Several organisations run agents in a shared environment under agreed rules. Nobody controls the whole thing, and new failures appear when the parties’ incentives do not line up.

Tier 3, open environments. Agents meet on public infrastructure with no central authority. Governance is whatever standards people volunteer to follow.

Almost every Melbourne service business sits in Tier 1 today, and most sit there with one automation, or none. That is not a reason to close the tab. The report makes the point that once an organisation has one agent, there is very little friction stopping it having many, and those agents will interact, sometimes because you told them to and sometimes just because they are working in the same place.

If you have an enquiry router, an automated quote follow-up and something summarising your inbox, you are already running a multi-agent system. You probably do not think of it as one, which is precisely the problem the report is describing.

The part that should stick with you

Here is the example from the report, and it is worth reading twice.

In a public experiment, several agents shared a file system. One agent hallucinated a contact list that had never existed and asked another agent to use it. The second agent created an empty file named as though it held 93 contacts. Every other agent then read the existence of that file as evidence the list had once been real and had somehow been corrupted. A made-up thing became a shared fact, and the whole group started working from it.

Nothing there is exotic. Each agent behaved sensibly given what it could see. The failure lived entirely in the handoffs.

Now put that in a business. One automation pulls a customer’s details slightly wrong. The next builds a quote on them. The next sends a follow-up referencing the quote. By the time a human sees it, three systems agree on something that was never true, and the record looks well established rather than wrong. We have made a version of this argument before, that a connector is not a system, and this is the sharpest illustration of it I have seen.

What actually goes wrong

The report groups failures into four types. Two of them are already live in small businesses.

Miscoordination. Misread messages and botched handoffs between steps that are each working correctly. The classic is two automations both deciding they own the follow-up, so the customer gets chased twice, or neither fires because each assumed the other did.

Propagation. An error, a wrong assumption, a piece of sensitive data or a malicious instruction travelling along the links. One bad field becomes five bad records.

The other two, strategic failures and infrastructure failures, are genuinely about scale and about agents from different parties with divergent interests. File those under “not yet” rather than “never”, because the moment your booking agent starts talking to a customer’s assistant, you have crossed into Tier 2 and your controls stop at your own boundary.

Five things worth doing while you are still in Tier 1

None of this requires new software. It is mostly an afternoon with a pen.

1. Draw the handoffs, not the tools. Everyone maps their tools. Almost nobody maps the moments where one automation’s output becomes another’s input. Those moments are where the failures live, and there are usually fewer of them than you expect.

2. Put the checkpoint where things leave the business. Not on every step, which nobody sustains. On the boundary: anything that reaches a customer, moves money, or writes to a record you rely on. One human glance at the exit is worth ten reviews in the middle.

3. Make each step able to say it does not know. The contact list failure happened because no agent had a way to express uncertainty. If a step cannot find the customer record, it should stop and flag, not invent a plausible one and pass it on. This is a design choice, and it is the single highest-value one.

4. Keep a log you can actually read. When something goes wrong you need to trace which step introduced it. Most no-code platforms log runs by default and almost nobody looks. Know where yours are before you need them.

5. Decide what an agent is not allowed to touch. Scope beats supervision. An automation that cannot reach your bank details, your full customer list or your ability to send on your behalf cannot cause that class of problem, no matter how confused it gets.

The func.digital take

The instinct with a government report on AI risk is to file it under “for the big end of town”. Mostly fair. Tier 3 is not your problem this year.

But the framing is genuinely useful because it names something small businesses get wrong constantly: testing each automation on its own and concluding the system is fine. Each piece works. The business still produces a wrong quote, a double-chased customer, or silence where a follow-up should have been. That is not a tooling failure, it is a handoff failure, and no amount of buying better tools fixes it. It is the same reason most Australian SMEs have adopted AI without changing how they work, and the same reason connecting your tools beats adding more of them.

The good news is that Tier 1 is the tier where you can still fix everything yourself. You have unilateral reach over every agent in your business. That stops being true the moment your systems start talking to someone else’s, so the habits are worth building now, while the blast radius is small and the whole map fits on one page.

If you want a hand drawing that map, that is a large part of what a free digital systems audit does. We follow one real job end to end and show you every point where work changes hands, which is also every point where it can quietly go wrong. Get in touch.