Switching Automation On Is Easy. Keeping It Running Is the Trick

What is most often sold in AI and automation is the go-live. This is when the investment, time and effort all land, and business owners and staff can see the impact and the benefits.

But there is one thing that often doesn't get discussed: maintenance and support. Now it is very much in the interests of the service provider to deliver a solution that is resilient and handles errors and exceptions well.

However this only goes so far. Automations are fragile and break when the tools around them change, AI models drift and lose accuracy on their own, and the failures are sometimes silent so you don't notice them until you hear about them from an annoyed customer rather than a dashboard.

For a small business the maintenance is the difference between automation that earns its keep and one more subscription you cancel next year. Price the upkeep before you buy, or you have not seen the real cost.

The sale is most often priced and pitched around the build. The true cost of a small business's AI shows up somewhere else entirely, in keeping it alive after the person who set it up has moved on.

So what does the research say? Gartner expected at least 30% of generative AI projects to be abandoned after the proof of concept by the end of 2025, with escalating cost and unclear value among the reasons. In Australia, Deloitte's maturity work found only 5% of small and medium businesses are fully set up to get value from AI, even though about two thirds are using it. Too often the pilot works but the upkeep doesn't get done and the tool dies.

The build is the cheap part

The quote you sign covers build and set-up. A support cost may get quoted and it's important to understand this cost and also annualise it. It's worth mentioning that the new solution becomes a core part of your business, so an outage or break in that solution will have direct impacts on the wider business. On traditional automation the split is plain once you measure it. Analyst firm HfS Research found 70 to 75% of the total cost of a Robotic Process Automation (RPA) programme goes to implementation, maintenance and support, and only 25 to 30% to the software licence. Other independent research (Xenoss and Glean) puts ongoing maintenance, monitoring and retraining at 15 to 30% of the system cost every year, on top of the build, and that repeats for as long as you run the tool. It is worth reading this alongside what AI automation actually costs in Australia, because the build number is only half the picture.

Automations are more fragile than they look

An automation is a chain of separate tools that never agreed to work together. Your CRM, your inbox, a lender portal, a spreadsheet, a form. Any one of them changes and the chain snaps. An API gets changed, a data schema changes, a new type of data is received, or a system fails in an unexpected way. The solution wasn't designed or built badly. It's just that some of its underlying design assumptions can and often do change.

This is not a new phenomenon, it's the normal failure mode. A 2016 EY report put the failure rate of initial RPA projects at 30 to 50%, in its own words "we have seen as many as 30 to 50% of initial RPA projects fail." Bots break the moment a field name, a button or an interface behind them shifts. Source: EY, Get ready for robots, 2016. One automation vendor describes a fifty-bot setup losing more than 250 hours a week to diagnosing and fixing breaks, with weekly breakage normal at that scale. Treat the exact figure as a vendor example rather than research, but the shape of it is real (source: duvo). Agentic systems can help here as they can handle higher levels of uncertainty, but they can be overly complex to build, run, govern and maintain.

Fragility is not a fault you can buy your way out of with a better tool. It comes with gluing systems together, so the real question is not whether it breaks but who notices, how fast, and who fixes it.

One potential solution is to focus on a single vendor. For many businesses this can be viable. However it can introduce additional limitations in being locked in to how the vendor solution works and its roadmap. The data changes and may not be as accessible in the same way, and onboarding and setup can be difficult. Not necessarily wrong, but worth considering carefully, and worth reading next to what happens when a vendor goes dark (there be dragons).

Your AI gets worse while sitting still

Ordinary software stays the same until you change it. An AI model does the opposite and this is called AI drift. It is a gradual degradation of an artificial intelligence model's performance, accuracy, or behavioural alignment over time, caused by shifting real-world data, evolving user contexts, or extended interaction loops. It often starts slowly but then can accelerate.

To keep an AI model accurate means monitoring, retraining and revalidation, and one cost analysis puts that at up to an extra 15 to 25% of compute overhead. While this is lower for SMBs given their lower levels of complexity, it is worth considering and factoring in to plans. Selecting the right use case (scope) and the right initial model for this helps to start from the right place.

When it fails, does it fail loudly or quietly?

A cheap automation can fail silently and carry on, handing out confident, wrong output, or it can just stop working altogether. A well built one stops, flags the problem, and handles it in an appropriate way, for example by handing it to a person. Engineers who run these systems in production treat error handling as the actual work, not a nicety. Retries, fallbacks, alerting, and a human checkpoint on the edge cases are standard practice for anything left to run on its own.

Gartner named inadequate risk controls as one of the reasons generative AI projects get abandoned. A tool with no handling for its own mistakes is a tool nobody can trust to leave alone, so it never gets left alone, and the time saving never arrives.

With inadequate error handling you are trusting an unwatched system to be right every time and reading its silence as success. Silence usually means nobody is looking.

AI brings in some new problems and techniques. There are plenty of great sources of information around (Airia.com).

What a build that is meant to last looks like

None of this is a reason to avoid AI. Every problem above has a known fix, and the fixes are what separate a proper build from a demo.

• Monitoring and alerting: Monitoring and alerting sit on the automation, so a problem shows up on a dashboard the day it starts, not in a customer complaint three weeks later. Silence is never assumed to mean success.
• It fails safely: When something goes wrong the automation stops, flags it, and handles it in the most appropriate way, including stopping altogether and handing it to a human for resolution.
• It gets checked for drift on a schedule. Someone looks at whether the AI is still accurate on a set timetable, so decay is caught on purpose instead of by accident. This is the maintenance job people forget exists.
• Documented and portable: The workflow is written down and the data stays in your hands, so the automation is not trapped in one contractor's head or one platform, and it survives the person who built it leaving.
• Someone owns it: One named person is responsible for it, because an automation nobody owns is an automation nobody maintains, and that is the one that dies first.
• Budgeting: Maintenance needs to be budgeted from day one. Put aside 15 to 30% of the build cost a year for maintenance, and start with one workflow kept properly before adding the next. A small automation you can keep beats a big one you cannot.

None of this is exotic and all of it costs money and effort.

The questions to ask before you sign

You don't need to become an engineer to protect yourself. You need five questions that will tell you most of what you need to know.

• Who fixes it when it breaks? How fast? And at what cost?
• What monitoring and alerting is included? You want to learn about a problem from a dashboard, not a phone call.
• What happens to a failed item? Does it stop and flag for a human, or sail through wrong?
• How will I know the AI is still accurate in six months, and who is responsible for checking?
• What do I keep if I leave? Your data and a written version of the workflow should be yours, full stop.

If you want a hand working out the real running cost of an automation before you commit to it, or why one you already have keeps breaking, email me at pete@autocognition.com.au.