Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
In the race to harness the transformative power of generative AI, companies are betting big - but are they flying blind? As billions pour into gen AI initiatives, a stark reality emerges: enthusiasm outpaces understanding. A recent KPMG survey reveals a staggering 78% of C-suite leaders are confident in gen AI's ROI. However, confidence alone is hardly an investment thesis. Most companies are still struggling with what gen AI can even do, much less being able to quantify it.
"There's a profound disconnect between gen AI's potential and our ability to measure it," warns Matt Wallace, CTO of Kamiwaza, a startup building generative AI platforms for enterprises. "We're seeing companies achieve incredible results, but struggling to quantify them. It's like we've invented teleportation, but we're still measuring its value in miles per gallon."
This disconnect is not merely an academic concern. It's a critical challenge for leaders tasked with justifying large gen AI investments to their boards. Yet, the unique nature of this technology can often defy conventional measurement approaches.
Why measuring gen AI's impact is so challenging
Unlike traditional IT investments with predictable returns, gen AI's impact often unfolds over months or years. This delayed realization of benefits can make it difficult to justify AI investments in the short term, even when the long-term potential is significant.
At the heart of the problem lies a glaring absence of standardization. "It's like we're trying to measure distance in a world where everyone uses different units," explains Wallace. "One company's "productivity boost"' might be another's "cost savings". This lack of universally accepted metrics for measuring AI ROI makes it difficult to benchmark performance or draw meaningful comparisons across industries or even within organizations.
Compounding this issue is the complexity of attribution. In today's interconnected business environments, isolating the impact of AI from other factors - market fluctuations, concurrent tech upgrades, or even changes in workforce dynamics - is akin to untangling a Gordian knot. "When you implement gen AI, you're not just adding a tool, you're often transforming entire processes," explains Wallace.
Further, some of the most significant benefits of gen AI resist traditional quantification. Improved decision-making, enhanced customer experiences, and accelerated innovation don't always translate neatly into dollars and cents. These indirect and intangible benefits, while potentially transformative, are notoriously difficult to capture in conventional ROI calculations.
The pressure to demonstrate ROI on gen AI investments continues to mount. As Wallace puts it, "We're not just measuring returns anymore. We're redefining what 'return' means in the age of AI." This shift is forcing technical leaders to rethink not just how they measure AI's impact, but how they conceptualize value creation in the digital age.
The question then becomes not just how to measure ROI, but how to develop a new framework for understanding and quantifying the multifaceted impact of AI on business operations, innovation, and competitive positioning. The answer to this question may well redefine not just how we value AI, but how we understand business value itself in the age of artificial intelligence.
Summary table: Challenges in measuring gen AI ROI
Key performance indicators for gen AI ROI
To better navigate these challenges, organizations need a blend of quantitative and qualitative metrics that reflect both the direct and indirect impact of gen AI initiatives. "Traditional KPIs won't cut it," says Wallace. "You have to look beyond the obvious numbers."
Among the essential KPIs for gen AI are productivity gains, cost savings and time reductions -- metrics that provide tangible evidence to satisfy boardrooms. Yet, focusing only on these metrics can obscure the real value gen AI creates. For example, reduced error rates may not show immediate financial returns, but they prevent future losses, while higher customer satisfaction signals long-term brand loyalty.
The true value of gen AI goes beyond numbers, and companies must balance financial metrics with qualitative assessments. Improved decision-making, accelerated innovation and enhanced customer experiences often play a crucial role in determining the success of gen AI initiatives -- yet these benefits don't easily fit into traditional ROI models.
Some companies are also tracking a more nuanced metric: Return on Data. This measures how effectively gen AI converts existing data into actionable insights. "Companies sit on massive amounts of data," Wallace notes. "The ability to turn that data into value is often where gen AI makes the biggest impact."
A balanced scorecard approach helps address this gap by giving equal weight to both financial and non-financial metrics. In cases where direct measurement isn't possible, companies can develop proxy metrics -- for instance, using employee engagement as an indicator of improved processes. The key is alignment: every metric, whether quantitative or qualitative, must tie back to the company's strategic objectives.
"This isn't just about tracking dollars," Wallace adds. "It's about understanding how gen AI drives value in ways that matter to the business." Regular feedback from stakeholders ensures that ROI frameworks reflect the realities of day-to-day operations. As gen AI initiatives mature, organizations must remain flexible, fine-tuning their assessments over time. "Gen AI isn't static," Wallace notes. "Neither should the way we measure its value."
Industry-specific approaches to gen AI ROI
Not all industries leverage gen AI in the same way, and this variation means that ROI measurement strategies must be tailored accordingly. Insights from the KPMG survey highlight key differences across sectors:
These findings underscore the importance of building ROI frameworks that align with the specific use cases and strategic goals of each industry. "You can't force-fit gen AI into existing measurement models," Wallace warns. "It's about meeting the use case where it lives, not where you want it to be."
Example: How Drip Capital measured gen AI ROI
Drip Capital, a fintech startup specializing in cross-border trade finance, provides a concrete example of how businesses can apply a structured approach to measuring the ROI of gen AI initiatives.
The company's use of large language models (LLMs) has led to a 70% productivity increase by automating document processing and enhancing risk assessment. Rather than building proprietary models, Drip Capital focused on optimizing existing AI tools through prompt engineering and a hybrid human-in-the-loop system to address challenges like hallucinations.
Their journey aligns closely with key elements of the 12-step framework, offering insights into the practicalities of quantifying AI's impact.
To assess the success of their gen AI implementation, Drip Capital uses both quantitative metrics and qualitative assessments:
They also monitor operational capacity increases, ensuring no additional staffing is required to handle larger volumes.
This metric ensures accuracy improvements while validating the effectiveness of their human-in-the-loop verification layer.
This metric reflects both increased throughput and improved customer satisfaction.
They also track the number of accurately flagged high-risk accounts as a key measure of gen AI's predictive power.
Higher scores directly correlate with gen AI-driven improvements in service delivery.
This metric ensures that Drip Capital maximizes the value of its accumulated data assets through AI optimization.
A comprehensive 12-step framework for measuring gen AI ROI
Through our conversations with industry experts across multiple sectors -- technology, healthcare, finance, retail and manufacturing -- we identified patterns in what works, what doesn't and the blind spots most organizations encounter. Drawing from these insights, we've created a 12-step framework to help organizations evaluate their gen AI initiatives holistically.
The idea is to provide IT leaders with a roadmap for measuring, optimizing, and communicating the impact of gen AI initiatives. Rather than relying on outdated ROI models, this framework offers a more nuanced approach, balancing immediate financial metrics with strategic, qualitative benefits.
This 12-step approach balances quantitative metrics like cost savings and revenue generation with qualitative benefits such as improved customer experience and enhanced decision-making. It guides organizations through every phase of the process, from aligning gen AI investments with strategic goals to scaling successful pilots across the enterprise.
This framework ensures that companies capture both financial and non-financial outcomes while maintaining flexibility to adjust as the technology and business landscape evolve:
1. Strategic alignment and objective setting
The success of any gen AI initiative depends on its alignment with broader business objectives. Before diving into implementation, organizations must ensure that the use cases they pursue are linked to strategic priorities, such as revenue growth, operational efficiency, or customer satisfaction. This alignment prevents AI investments from becoming siloed projects disconnected from the core business mission.
Key Actions:
2. Baseline assessment
Establishing a clear performance baseline is essential to measure progress accurately. This involves collecting data on current processes, outcomes, and key metrics before deploying gen AI solutions. The baseline serves as a reference point for assessing post-implementation impact.
Key Actions:
3. Use case identification and prioritization
Not all AI initiatives deliver the same value, so it's critical to identify and prioritize high-impact use cases. Decision-makers should focus on projects with a clear path to ROI, strong strategic alignment, and measurable outcomes.
Key Actions:
4. Cost modeling
Effective gen AI deployment requires a detailed cost model that goes beyond upfront investments. Organizations need to capture ongoing operational expenses, including infrastructure, maintenance, and staffing.
Key Actions:
5. Benefit projection
Forecasting potential benefits provides a roadmap for expected outcomes. In addition to financial returns, organizations should project intangible benefits like improved employee satisfaction, decision-making, or customer engagement.
Key Actions:
6. Risk assessment and mitigation
Every gen AI project carries risks, from technical challenges to ethical considerations. Identifying these risks early and developing mitigation strategies ensures smoother implementation.
Key Actions:
7. ROI calculation
Standard ROI formulas may not capture the complexity of gen AI's impact. Organizations should tailor their ROI models to include direct, indirect, and strategic returns, balancing immediate financial gains with long-term value creation.
Key Actions:
8. Qualitative impact assessment
Many of gen AI's most valuable contributions -- such as improved customer experience or enhanced innovation -- resist traditional quantification. Organizations need qualitative assessments to capture these impacts effectively.
Key Actions:
9. Implementation and monitoring
Implementation must include a robust monitoring system to track progress against benchmarks. Real-time data collection allows organizations to course-correct as needed and ensures that benefits materialize as planned.
Key Actions:
10. Continuous improvement and optimization
Gen AI initiatives require constant fine-tuning to maximize impact. Regular evaluation and iteration allow organizations to identify opportunities for improvement and adapt to changing needs.
Key Actions:
11. Scalability and enterprise-wide impact assessment
Once a gen AI solution proves successful in a limited context, organizations must evaluate its potential for broader deployment. Assessing scalability ensures that AI investments deliver value across the enterprise.
Key Actions:
12. Stakeholder Communication and Reporting
Clear communication with stakeholders is essential to maintain alignment and support. Regular reports that capture both financial and non-financial outcomes keep stakeholders informed and engaged.
Key Actions:
Summary Table: 12-Step framework for measuring gen AI ROI
Practical Strategies for Achieving ROI early with gen AI
From our conversations with experts across industries, a clear theme emerged: achieving measurable ROI with gen AI requires more than enthusiasm -- it demands a deliberate, strategic approach. Many companies dive into ambitious AI projects, only to encounter challenges in translating initial excitement into meaningful outcomes. The key to success isn't launching large, complex systems right away but focusing on manageable, high-impact use cases that demonstrate value early.
Below are a few practical takeaways from these expert discussions, designed to help organizations move from gen AI exploration to execution and ROI measurement. These strategies serve as a bridge from planning to sustained value creation, laying the groundwork for effective implementation and continuous ROI growth.
1. Start with focused use cases
Begin with smaller, high-impact use cases: Start with something that offers immediate value without being overwhelming. The trick is to target processes that are both measurable and impactful. This approach avoids the complexity of large-scale rollouts and ensures early wins.
2. Select the right infrastructure
Many companies struggle with infrastructure decisions. Prototype with cloud tools first, then refine as you go. The key is to remain flexible -- hybrid or on-prem setups might make sense later, depending on your data compliance needs.
3. Set realistic expectations on returns
Don't expect miracles out of the gate. The first phase is experimental, and that's okay. Plan for iterative learning cycles, where teams refine prompts and processes over time to maximize ROI.
4. Maintain human oversight
Keep people in the loop, especially in areas like finance or legal, the AI's output needs verification. Combining automation with human expertise ensures both efficiency and reliability.
5. Leverage existing data
Organizations sitting on years of data can turn it into a goldmine by refining AI prompts and validating outcomes. Well-curated datasets lead to better, more consistent returns.
Redefining business value in the age of gen AI
In the race to harness the transformative power of gen AI, enthusiasm alone won't generate returns. As companies confront the complexities of measuring impact, they must move beyond traditional metrics to embrace a more nuanced understanding of value -- one that accounts for both tangible and intangible outcomes. The path to success lies not in grand, sweeping implementations but in focused, high-impact initiatives that align with business objectives and evolve over time.
The challenges are clear: a lack of standardization, complexities in attribution, and benefits that often resist easy quantification. Yet, as the experiences of companies like Drip Capital show, a pragmatic, iterative approach -- anchored by clear objectives, human oversight, and data-driven insights -- can unlock gen AI's potential. Organizations that treat ROI as a continuous process, refining their strategies and metrics as they go, will be best positioned to turn AI investments into measurable impact.
The true value of gen AI goes beyond cost savings and efficiency gains -- it lies in its ability to transform processes, spark innovation, and empower better decision-making. In this evolving landscape, those who succeed will be the ones who reimagine ROI, balancing measurable financial outcomes with strategic, long-term contributions.