Timeline that continues past day 90: the plant makes decisions the agent never sees
Case & method

After AI Agent Implementation: What to Monitor

Rodolfo Manzur8 min read

In short: After implementation, an AI agent updates with the data in your systems, but not with the decisions made in your plant. If a tolerance changes, a customer agreement changes or a spec sheet never gets uploaded, it keeps deciding by the old rule; and the model provider updates it on its own schedule too. That's why, before signing, the question isn't who implements it, but who reviews its results.

We know this from the inside. Neurya grew out of an industrial group with 53 years of history, and the first agent we had to keep up to date was the one in our own plant.

Gartner forecasts worldwide AI spending will reach $2.7 trillion in 2026, up 49.5% from the year before, according to its September 16 press release. That money buys starts: models, infrastructure, implementations. Our read is that what decides whether it pays off happens later, from day 91 on, when the implementation team has left and the plant keeps changing.

In why AI pilots don't reach production we looked at what separates a pilot from a system in production. This article starts where that one ends: the agent is already in production, already delivering results, and now those results have to hold.

Day 91: implementation ends, the business doesn't

What happens after day 90 decides whether the project was worth it or only looked that way for three months. The risk isn't that the agent goes down, because that gets noticed and someone fixes it. The risk is that results drift slowly away from the agreed number, nobody reviews them, and one day someone stops using the agent without anyone having decided to.

What you lose there isn't the initial investment: it's the return that was already coming in. An illustrative example: if an agent recovers 400 thousand pesos a month and its results drift 25% without anyone noticing for two quarters, that's 600 thousand pesos left on the table before anyone asks the question. And you lose something more expensive: the internal confidence to approve the next project, because it will be debated in the shadow of the one that got switched off.

Why an agent drifts even though it updates

Because it updates with data, not with decisions. A well-connected agent reads the current bill of materials, the supplier catalog and the day's orders: if engineering adds a new part number to the system, the agent uses it from that moment on. What it doesn't read is what your plant decides outside the system. Quality accepts a different tolerance from a new supplier and agrees to it by email. Your main customer starts accepting partial shipments and says so in a meeting. A raw material spec sheet arrives as a PDF and nobody uploads it. The agent keeps deciding by the old rule, because nobody told it the rule changed.

There's a second change that doesn't depend on your plant: the AI model provider updates the model. A new model is usually better, but it can resolve the same case differently, and that doesn't show up in any alert in your system. In neither case did the agent fail: it does what it can with what it knows. Changes that go into the system get noticed; the ones that stay in an email, a meeting or a model version are the ones nobody sees.

What to monitor: results against the baseline, not whether it's on

An agent can be on 100% of the time and still be making the wrong calls. That's why system availability, the share of time it's up, says nothing about the business. What you monitor is whether the number that justified the project, correct material deliveries, rejects or cycle days, is drifting away from the day 0 baseline.

There's a second, earlier signal: how often the person corrects the agent. If the supervisor reviewing its proposals starts correcting it more often, something changed in the plant before the number shows it. That figure gets reviewed every week; results against the baseline, every month, with whoever is accountable for them.

Who keeps it running from the inside

Someone in your plant has to review every month whether what the agent decides is still delivering the money that was agreed. If that review is left only to the provider, the project dies the day the contract changes. The provider can help measure, but it can't be the one deciding whether the result is still worth what it costs.

That person isn't the one who corrects the agent day to day. In AI adoption with your employees we explain why that role belongs to whoever works the process. The monthly review is something else: it's done by whoever answers to leadership for the number, and they're the one who decides when a change in the plant means those rules need adjusting.

What an AI specialist does inside your plant

After implementation, an agent needs someone who understands the plant, not just the system. Traditional software support is built for the second: a help desk that responds when something goes down, with a ticket and a response time. That works if the problem is technical. But what throws an agent off, as we saw, doesn't look like a failure: it's the tolerance agreed by email or the new deal with a customer, and finding it means being in the plant, talking to the supervisor and knowing how that process makes money. That's where the deployed specialist comes from: someone from the provider who works inside the client's company, not from a distance.

At Neurya that person is an AI specialist dedicated to your company, working shoulder to shoulder with your plant team: they understand the process and the business, and can change how the agent decides, not just open a ticket. The role is known as a *forward deployed engineer* (FDE), and it's how we provide support for AI agents in production.

You can see the difference with the tolerance agreed by email. With a help desk, someone reports that the agent is getting things wrong, a ticket is opened and everyone waits. With a specialist inside the plant, the correction rate goes up, the specialist sits down with the supervisor, finds the agreement that never made it into the system and adjusts the agent's rules that same week. Every quarter they review results with leadership and propose what to adjust or which agent to add; the decision stays with your company. The goal isn't to stay forever: it's for your people to learn to make that adjustment, so the result doesn't depend on us.

Not every plant needs one. If your processes change little and you already have someone who reviews results every month and knows how to make those adjustments, that review is enough. It's justified when changes arrive faster than your team can keep up with.

What to get in writing before you sign

Four things, in the contract and not in the presentation: who reviews the results, how often, against what number and what happens when the business changes a rule. The fourth is the one worth spelling out: who adjusts the agent's rules, how fast and whether that adjustment costs extra, because the first supplier change will arrive before the first renewal. And there's a fifth that protects your investment: the agent's rules, its logs and the baseline should be yours and documented. If you change providers tomorrow, what already works stays with you.

What to do on Monday

  1. Check whether any system already running is deciding today against a rule that changed. Start with last quarter's agreements that were made by email or in a meeting and never made it into the system.
  2. Name one person in the plant as accountable for the results of each system in production. If the only name that comes up belongs to the provider or to IT, that's the finding.
  3. Set how often those results get compared against the day 0 baseline. Without a date, it gets pushed back every time the plant has an urgent issue.
  4. Put the four things in the contract for your next project, before you sign it. After signing, each one gets negotiated from a weaker position.

Before you sign your next project

We hold our own agents to the same standard. The purchasing review running on our own Odoo is still measured every week after handover, and today it covers every receiving line against standard cost.

If you already have an agent in production and want its results not to depend on anyone else, we go straight to ongoing support. If you're not sure where to start yet, consulting starts at 120 thousand pesos plus VAT. Whichever path you take, who keeps the results on track is written down from day one.

**Start with 45 minutes at no cost with our leadership. There we'll see whether to go straight to the solution or whether consulting makes more sense →**

Go deeper:

Frequently asked questions

Do I need my own AI department to keep agents running?

Not to start. One person accountable for each result is enough, plus an AI specialist alongside them while your team learns to adjust the agent.

Can I change providers without losing what already works?

Yes, if the agent's rules, logs and baseline are yours and documented. That's why it's worth getting it in writing from the first contract.

How long does the support last?

Until your team knows how to adjust the agent and the results hold without outside help. It's reviewed every quarter; it isn't an open-ended contract.

Let's talk

Have a question about this for your company?

Leave it and we'll reply by email. No commitment.

Prefer to explore on your own? Take the self-assessment (5 min)