Umesh Sachdev is CEO & cofounder of Uniphore, a large AI-native, multimodal enterprise-class SaaS company. He’s based in Palo Alto, CA.
AI is the darling of Fortune 500 earnings calls, yet a devil in disguise in the boardroom.
We’ve seen an explosion of AI application companies in the past 24 months, yet despite the craze and hefty AI budgets, just a mere 5% of generative AI (Gen AI) deployments are yielding a meaningful share of EBIT improvements, according to McKinsey.
The culprit? A staggering 28% of enterprise decision-makers admit to struggling with Gen AI integration. As companies rush headlong into the AI gold rush, many are finding that their proof-of-concept demos are colliding hard with the realities of practical implementation.
Lessons From Early Adopters: Balancing Experimentation And Sustained Value Creation
In the initial months following the launch of ChatGPT and, hence, mainstream AI, the initial adoption of Gen AI was characterized by rapid experimentation, with companies eager to explore the technology’s potential. However, as the dust settles, I’ve seen leaders shift their focus to metrics that indicate scalability and long-term value, offering valuable lessons and driving the next wave of digital transformation in these companies.
Today, I believe companies will find the most success when they think of their AI infrastructure as a “four-layer cake.”
1. The Data Lake: This is the first layer of an effective AI system, where the data sits. This is often handled by data storage companies like Databricks or Snowflake who store and organize a company’s raw proprietary data.
2. Knowledge-As-A-Service: This is the layer where technologies like RAG databases can help turn data into knowledge and reduce the chance of hallucinations by double-checking LLM results against a high-quality set of proprietary data.
3. Model-As-A-Service: This is where an enterprise will integrate with LLMs, either foundation models (OpenAI, Anthropic or Google), smaller open-source models (Llama, Falcon, Mixtral and more) or, most likely, a combination of different models. Given the rapid changes in models, it’s helpful to design infrastructure to easily change models or work with multiple models.
4. AI Agents And Applications: At the top of the cake sits the actual applications, chatbots, data tools, analytics and other applications that end users will use to make use of the AI output. This is where AI agents sit.
With this AI architecture in place, companies can move to implement AI projects into production and, more importantly, adjust individual components more quickly, which is critical considering the pace of AI innovation.
Change management has emerged as a critical factor in successful AI implementation. While it’s tempting to view Gen AI solely through the lens of its chat interface, tech leaders are learning to prioritize user experience and work backwards from there.
A leading North American telecommunications provider we worked with reimagined its customer care function by integrating Gen AI into existing workflows. Instead of creating a standalone AI chatbot, they embedded AI capabilities directly into their customer service systems, reducing call volume by 20%-25%, deflecting another 20%-30% of calls and overall reducing calls by 50% by integrating AI into the standard workflow.
Cost-performance optimization has also become a focal point for early adopters. Many organizations initially experimented with foundational large language models (LLMs) through existing cloud service provider relationships. However, leaders are now seizing opportunities to reduce costs and improve performance by fine-tuning open source small language models that they can manage directly.
Lastly, the rapidly evolving landscape of foundation models has prompted CIOs to adopt a model-agnostic infrastructure, given that models are changing quickly, but best practices can be applied to multiple AI models. This strategy allows organizations to maximize work done in prompt engineering, optimization of retrieval augmented generation systems, model training, evaluation and orchestration across different models. Such flexibility ensures that investments in AI remain relevant even as the technology continues to advance at a breakneck pace.
A Timeless Recipe To Cross The Proof Of Concept Chasm
As organizations strive to move beyond proof of concepts, these best practices can guide this transition. This approach emphasizes strategic planning, readiness assessment and iterative implementation.
1. Align AI With Business Objectives: Focus on where AI can drive tangible value, not just chase trends. Analyze processes, customer interactions and inefficiencies to create a business-driven AI roadmap.
2. Assess Readiness: Evaluate your data quality, governance and team capabilities. Identify gaps in skills or data that could hinder implementation.
3. Develop A Tech Blueprint: Plan how AI will integrate with existing systems. Aim for seamless incorporation rather than isolated applications. Consider data flow, model training and scalability.
4. Prepare Your Team: Create an operating model with clear roles and responsibilities. Form cross-functional teams and invest in upskilling to ensure your workforce can leverage AI effectively.
5. Set Clear, Measurable Goals: Define specific, achievable objectives tied to business outcomes for your first production-ready AI project. This provides focus and a benchmark for success.
Throughout this process, maintain a focus on integration rather than addition. The goal isn’t to create new, standalone AI systems that employees must learn to use. Instead, strive to enhance existing workflows and tools with AI capabilities, making the technology an invisible yet powerful assistant in day-to-day operations.
Navigating the Gen AI landscape requires more than enthusiasm—it demands strategy and patience. Today, AI is becoming a central infrastructure priority, along with cloud and cybersecurity. The most successful organizations won’t be those that adopt every new AI tool, but those that thoughtfully, yet swiftly, integrate AI into their existing processes.
By taking a measured approach—pausing to assess, aligning AI initiatives with business goals and focusing on seamless workflow integration—companies can unlock AI’s true potential and move to realize the benefits more quickly. The key is to prioritize user experience and tangible business outcomes over technological novelty. As we move beyond the initial hype, the organizations that will thrive are those that view AI not as a standalone solution, but as a powerful enhancement to their core business strengths and a gateway to new opportunities.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
