Your AI Pilot Saved Time. So Why Didn't the P&L Move?

Last month I co-hosted a webinar with Pann Koutsogkilas, founder of Griffinshire Capital, called The True ROI of AI: What to fund, what to cut, and how to prove it. Pann has spent close to two decades in finance, working his way from analyst to CFO. I've spent 28 years in the technology industry and while we come at this problem from opposite sides of the table, we keep landing in the same place.

A day doesn't go by that we don't see another headline about AI investments not yielding results. What I see far less often is a credible line from that investment to the bottom line.

PwC's 29th Global CEO Survey, published in January, put some numbers behind that feeling. More than 4,400 CEOs across 95 countries weighed in, and 56% said AI had produced no financial benefit at all, whether in revenue, cost, or margin. Only 12% reported gains on both the revenue and the cost side.

So what's going on? My view is that most organizations don't have a technology problem. They have a measurement problem, and underneath that, a discipline problem. The tools are working and people are using them. What's missing is the step that converts an operational improvement into economic value. With 2027 budgets being built right now, that gap is worth closing before the next round of renewals gets signed.

Time Saved Is Not ROI

Early on, I watched a lot of organizations run some version of the same math. A Copilot license costs $30 a month. Sally in accounting makes $30 an hour. If the tool saves her two hours a month, it more than pays for itself. On paper, that looks like a win.

The problem is that the math only looks at time savings in isolation. Most of us have seen how that story plays out. The pilot saves ten hours a week, users like it, adoption is high, and the license gets renewed. Then finance goes looking for the impact and can't find it anywhere on the P&L. During our webinar, Pann called out hours saved on their own "vanity metrics": nice to report, but not something the business can act on.

Here's the distinction I think matters most. Saving 1,000 hours is not ROI. What you do with those hours determines whether ROI exists.

Time saved creates capacity, and capacity only becomes economic value in one of two ways. Either you reduce cost, meaning actual labor expense goes down, or you deliberately redeploy that capacity into something you can measure, like revenue, service levels, quality, or throughput. If neither happens, the hours simply get absorbed.

A simple example makes the point. Twenty employees each save two hours a week, which adds up to 40 hours of capacity. If nothing else changes, you haven't removed any cost, necessarily increased throughput, or generated new revenue. Now suppose you redirect those 40 hours into prospect outreach, track the activity and the conversion, and compare the incremental margin against the total AI investment. You still don't know the answer yet, but for the first time you have something you can actually test.

Pann added a caveat worth underlining. The redeployed capacity has to go toward work that produces results. Retasking people into busy work just means you keep spending money without creating anything new.

Evaluate AI Like Any Other Business Investment

Pann offered an analogy that I think every finance leader will recognize. If you run a manufacturing business, you don't buy a new machine without first asking whether it will produce enough additional widgets to justify the cost. AI deserves the same scrutiny. Before you implement it, you should be able to say whether it's expected to save money, grow revenue, reduce risk or errors, or improve customer satisfaction enough to bring people back.

In practice, that comes down to three disciplines. Start with the value by defining the business objective and what success looks like before choosing a solution. Prove the economics by showing how operational gains translate into cost, revenue, risk, service, or throughput. Then validate over time by tracking what actually happened against the baseline and the full investment.

There's a bigger mindset shift underneath this, too. Most organizations today are adding AI into processes that already exist. That's a reasonable way to start, but I'm not convinced it's where the real opportunity lives. The more useful question is what new organizational capabilities AI makes available to us, and what we would do differently if we had them.

Business development is a good illustration. The incremental version helps reps research prospects a little faster. The new capability version uses agents to monitor hundreds or thousands of market signals, such as leadership changes, press releases, hiring activity, and financials, and then synthesizes them. The rep starts the day with a dashboard showing the ten prospects most worth contacting, why each one matters, and a first draft of the outreach. The function is the same, but the way it operates is fundamentally different.

Pann compared this moment to the early days of spreadsheets. While people were still experimenting with them, spreadsheets were an interesting novelty. Once they became part of how work actually got done, their value became measurable. I expect AI to follow a similar path. Curiosity is a good starting point, but experiments on their own rarely produce a return.

From Problem to Proof: A Five-Step Framework

None of this is complicated, but it is sequential. Each step produces the evidence you need for the next one, and skipping ahead is where most of the trouble starts.

Step 1: Define the needle you intend to move. The first questions are straightforward. What process are we improving? Who owns the outcome? What metric should change, and what happens if we do nothing? Then comes the question that often gets skipped: is AI actually necessary? Many of our conversations begin with AI and end with a workflow automation or a rules change that solves the problem more simply. If the objective is "implement AI," the initiative doesn't have a business case yet.

Pann raised a point here that deserves more attention than it gets. Market positioning matters. Some customers pay a premium specifically for human interaction, and introducing AI into that relationship could weaken the very thing they value. That isn't true for most companies, but for the ones it applies to, it changes the calculation.

Step 2: Establish a defensible baseline. Candidly, this is the hardest part for most organizations. When we ask how long a process takes today and what it costs, the answers are often vague. The full cost of a current process includes more than direct labor. It includes errors and rework, delays and handoffs, risk and compliance exposure, and opportunity cost. If sales proposals take five days to get out the door and that pace keeps you from sending twice as many, that constraint has a real price.

A practical way to get started is to bring the right stakeholders together for 90 minutes, record and transcribe the session in Teams, and walk through how the work gets done today, where the friction is, and where the group wants it to go. That transcript gives you a documented workflow and a starting point for your numbers. It also tends to reveal something useful: people in the same department often don't agree on how the work actually happens. Getting everyone aligned on the current state is valuable in its own right.

The timing matters here. If you wait until after implementation to define ROI, you may have already lost the baseline.

Step 3: Describe the improved process before the AI. What should the work look like when it's better? Manual intake might become structured intake. Multiple handoffs might become automated routing. Inconsistent review might become AI-assisted review, with a person handling the exceptions. A five-day cycle might become a two-day cycle. The key question is what observable operational change will tell us the initiative worked, and where human judgment needs to stay in the loop.

Step 4: Translate the improvement into financial value. This is where you connect what the technology enables to how the work is redesigned, which operational metric moves, and where the value ultimately shows up. Does it appear in margin, revenue, or cost reduction? If you can't answer that, the business case isn't finished.

Pann's advice on assumptions is especially useful at this stage. If you expect a 10% revenue lift, break it into the variables behind it. Are you reaching more clients? How many more? Is conversion improving, and to what rate? You won't be completely confident in those numbers, and that's fine, because nobody can predict the future. Writing the assumptions down is what makes them testable.

Step 5: Measure and validate. Once the initiative is live, compare what actually happened with what you assumed. That requires the capacity and the framework to track results over a quarter, a year, or longer. The goal is to learn quickly enough to adjust the plan, the rollout, or the way the tool is being used.

Two Companies That Connected the Dots

It helps to see what this looks like when it works.

UPS: the textbook case. At UPS's scale, small route inefficiencies compound into significant fuel, labor, and maintenance costs. The company knew it had a problem and built ORION to optimize delivery-route sequencing using its own operational data. Public use case reporting credits the system with more than 100 million fewer miles driven each year and roughly $300 to $400 million in annual savings. The chain is clean: better routes lead to fewer miles, fewer miles mean less fuel and time, and lower operating costs show up in the financials. As Pann put it, this one is extremely easy to measure. Most initiatives won't be this tidy, but it's the shape worth aiming for.

IKEA: capacity turned into a new revenue channel. This is one of my favorite examples, partly because one of employees' biggest fears is that AI will take their jobs. IKEA noticed that many customer-service contacts involved routine questions, such as order status and delivery timing. It introduced Billie, an AI bot, to handle those inquiries, and over two years Billie took on 47% of call-center queries. The more interesting move came from listening to the service team, who said customers were frequently asking for interior design help. IKEA trained 8,500 call-center workers as interior design advisers, and that remote design channel generated €1.3 billion in fiscal 2022, about 3.3% of the company's revenue.

What made IKEA's time savings count is that the capacity was deliberately redeployed into something measurable. And as Pann noted, those employees were upskilled instead of replaced, which likely left them more financially secure.

Counting the Full Investment

ROI falls apart when the denominator is incomplete, and in my experience the license or subscription is never the total cost of an AI initiative.

The real investment includes software and licensing, model and token consumption, implementation and integration, data preparation, internal business and technical resources, outside expertise, training and adoption, change management, security and governance, and ongoing support. A few of these are consistently underestimated.

Data preparation is at the top of that list. As individuals, we're digital pack rats, and an organization is the sum of all of its employees. Getting data into a well-defined, curated state takes real time and effort, and it's usually more than anyone planned for.

Security and governance are about to become much bigger issues. We're moving beyond individual chat tools toward agentic AI, where an organization may eventually have hundreds, thousands, or even tens of thousands of agents at work. Each one needs access limited to a defined set of systems and data, along with clear limits on which other agents it can interact with. Many of us remember VM sprawl. Agent sprawl is coming, and it will need to be managed across its full lifecycle.

Ongoing support is the cost I most want organizational leaders to take seriously. Models and capabilities change constantly. We're also still in a honeymoon period on pricing, with much of today's cost subsidized by the venture capital flowing into AI companies. I expect costs to rise and usage to grow, which means today's numbers won't tell the whole story for long.

Pann offered a helpful counterpoint. Some of these costs shrink as you scale. Building the governance baseline and the data foundation is expensive the first time, but each additional initiative built on that foundation adds only a fraction of the cost.

Every Initiative Deserves a Decision

Adoption tells you the engine is running. Business metrics tell you where it's taking you. Active users and frequency of use are reasonable leading indicators, but an ROI conversation also needs operational indicators like cycle time, throughput, error rates, rework, and backlog. Ultimately, it needs financial ones: cost reduction, cost avoidance, margin improvement, and incremental revenue. Pann's point was that vanity metrics sit at the front of that chain, and the work is following them through until they show up on the P&L.

Once you're measuring, every initiative in your portfolio should land in one of four places. Fund it when there's clear value, a clear owner, a baseline, and a way to measure. Optimize it when the value is there but adoption, cost, or the workflow needs work. Delay it when the potential is real but the organization isn't ready or the right controls aren't in place. As Pann pointed out, that call can come before implementation, when the assumptions simply don't support the desired outcome yet. In that case, set it aside and revisit the assumptions periodically.

The fourth option is to stop, and it's the hardest one for most organizations. You expected one outcome, got another, made adjustments, and the business case still isn't there. Pann's advice is direct: accept the loss, stop putting money into it, and move on to the next initiative, ideally having learned something along the way. Leaving it in indefinite pilot status is still a decision. It just isn't a deliberate one.

Don't Forget the People Doing the Work

One of the most valuable moments in the session came from an attendee who has spent months running AI discovery sessions inside his own organization.

His point was simple. Ultimately, the goal is to improve earnings per share. But if you want frontline employees, the people closest to the work, to engage honestly in those sessions, you can't lead with that. They're hearing every day that AI is coming for their jobs. The message that resonates is about making work faster, better, and cheaper so they can do more for customers.

He also shared where his team is actually finding value. A few wins have come from bringing vendor work in-house, saving $30,000 to $50,000 a year each. The most significant gains, though, have come from productivity improvements that allowed teams to absorb natural attrition without letting anyone go. Those are real savings.

Pann took the idea a step further. Be upfront about what AI will mean for people, and if it produces savings, share that with them. Telling someone they can now do more work for the same pay isn't much of a selling point. If leaders assume every dollar of savings flows straight to the bottom line, with nothing in it for the people making it happen, adoption is likely to stall.

This is a delicate balance. The ROI framework is the right conversation for finance and leadership, but it isn't the conversation you lead with on the front lines. Both conversations matter, and leaders have to be willing to have them at the same time.

Where This Leaves You

As you head into budget season, I'd suggest asking eight questions of every AI investment on the table:

1. What business outcome are we funding?
2. Who owns it?
3. What is the baseline?
4. What operational metric should change?
5. How does that change create value?
6. What is the total investment?
7. When will we review the result?
8. What triggers us to scale, redesign, or stop?

If you can't answer one of them, mark it as unknown and assign someone to own it. The temptation is to smooth over the gaps, and that's exactly how weak business cases end up funded.

That leads to a harder question, and I think it's the right one to sit with: which AI initiative in your organization would fail this test today? If one comes to mind, that isn't a failure. It's a starting point.

To help, we've released the 2027 AI Budget Planning Guide in early October. It's a cross-functional workbook built on this framework, designed for organizations with fewer than 1,000 employees, to help leadership teams decide what to fund, optimize, delay, or stop. 

If you're weighing a specific initiative, renewal, or 2027 budget decision, our team is also happy to spend 30 minutes working through it with you: clarifying the outcome, identifying the missing baseline, and testing whether there's a credible path to economic value.

The goal of AI was never automation for its own sake. The goal is measurable economic value, and getting there starts with being honest about what we're measuring today.