What’s holding companies back from realizing the ROI of AI?

If it feels like tech leaders are simultaneously stepping on the gas and slamming the brakes, you’re not imagining things. In Apptio’slatest surveyof 1,510 technology leaders, 74% said their IT budgets are up—but a whopping 90% said they are struggling to measure ROI on their investments.As companies move from one-off pilots to scaling AI throughout their organizations, the tension between optimism and reality is growing. “It’s hard to get the ROI, and people don’t have the right tools,” said Ajay Patel, one of the report’s authors, in an interview withIBM Think.

Why isROI so hard? Poor data quality for one. The Apptio report found that distrust of data anddata silosare among the top reasons tech leaders struggle to justify the spend. Governance and understanding AI costs are also big blockers, said Mihai Criveti, a Distinguished Engineer for Agentic AI at IBM, on arecent episode ofMixture of Experts.Criveti meets regularly with CTOs, CIOs and other executives. Over the past six months, he said, the conversation has shifted from “‘How do I use it?’ to ‘How do I actually implement and get value?’” And to understand value, you need to know “who’s responsible for AI within your organization,” Criveti said. The reality at most companies, he said, is that too many people are in charge of too many AI projects. “There’s not a client that doesn’t have at least 60 random acts of AI.” Fragmented projects also make it harder to track and control costs, he added.

What’s a tech leader to do? Picking the right-sized pilot to scale increases your chances of accurately tracking the ROI and generating real growth, said Aaron “Ronnie” Chatterji, OpenAI’s Chief Economist, at theCharter Leading with AI Summitin New York last week. He said he advises OpenAI clients to choose “Goldilocks” AI projects—that is, medium-sized projects that are small enough to actually reach fruition, but big enough to actually yield real value.When IBM Senior Vice President of Consulting Neil Dhar works with clients, he uses a finance mindset to frame ROI. “For every dollar invested, we should be aiming for two and a half to three dollars in return, at a minimum,” hewrote in a LinkedIn post, adding that “this ROI-driven approach resonates deeply with business leaders.” For Dhar, productivity and innovation together serve as the bridge to ROI, wherein productivity yields cost savings and innovation translates to revenue growth.Rethinking the business model of frontier AI labs may be another approach to solving the AI ROI puzzle, according to IBM Fellow Kush Varshney. OnMixture of Experts, Varshney proposed that instead of charging per token, companies could charge per outcome. “As a customer, why should I pay for a token rather than pay for the value that I’m getting from it?” he asked. “Charging per outcome—that’s when everything changes,” Criveti replied.