Bob Venero is the CEO & Founder of Future Tech Enterprise, Inc., an award-winning, global IT solutions provider.
As artificial intelligence (AI) has largely gone from experimentation to everyday business use, cost has become a central topic. Companies are identifying use cases that work and beginning to think about what it will take to scale them across their organizations.
That’s when the economics can start to look very different.
A company may begin with a handful of people using a cloud-based model and see a relatively manageable expense. As more employees begin using it, more use cases move into production and AI becomes part of day-to-day operations, token consumption grows along with it.
That growth raises important questions about the architecture supporting AI. Where models run, where data resides and how frequently those resources are accessed can have a significant impact on cost. They can also affect speed and availability, along with how easily an organization can scale the use cases that are creating real business value.
For companies planning to expand their use of AI, these are decisions worth making early.
Token Costs Add Up Quickly
Tokenization is a big part of the conversation right now. Every time an organization uses a cloud-based AI model, tokens are being consumed, and those costs become much more significant as usage grows.
For companies planning to use AI broadly across the enterprise, this deserves attention from the beginning. The cost of the infrastructure required to run AI on-prem can be a fraction of the token expense associated with running large volumes of AI workloads in the cloud. At enough scale, those ongoing costs can affect how much AI the organization can realistically use and how quickly it can expand successful applications.
This is also why I spend so much time talking with customers about business outcomes. A use case that saves a company $100 has a very different value than one that saves $10 million, even if the investment required to support them is similar. Companies need to understand which use cases will have the greatest impact and put their resources behind those opportunities.
That same thinking should carry into architecture. The organization should understand what it expects AI to accomplish, what it will cost to deliver that outcome and what the economics will look like as adoption grows.
Availability Becomes More Important At Scale
The architecture decision also affects access to the technology and the data behind it. Think about an organization running AI entirely through the cloud. As more teams begin depending on those tools, the AI environment becomes part of how people work every day. An internet connectivity issue can suddenly affect access across the organization. The tools may still exist, but employees cannot use them if they cannot reach them.
Keeping certain models and data within the organization’s environment gives companies another level of control over how and when those resources are available. That becomes increasingly important when AI is supporting business processes that people rely on throughout the day.
Speed factors into this decision as well. It’s probably the question I hear most often from customers: How quickly can we stand something up? How quickly can we start seeing actual business outcomes? Where models and data reside can have a direct impact on those answers, which is why speed and cost both need to be part of the architecture conversation from the beginning.
Put AI Where The Use Case Needs It
We’re seeing more options emerge for running AI closer to where the work happens. Edge devices can keep models and data on the device itself, giving organizations the ability to process information locally and get results very quickly.
The right architecture will look different from one organization to the next. Healthcare organizations may be looking at AI for radiology, where speed and accuracy have a direct connection to the outcome. Financial organizations may be evaluating AI for modeling. Federal environments have their own requirements around model development and security. Each environment has use cases that need to be identified, vetted and tied to the outcomes that matter to that organization.
This is where companies can start thinking about AI architecture at the workload level. Some workloads may make sense in the cloud. Others may belong on-prem or at the edge. The use case, data, security requirements, economics and desired business outcome all help determine where that workload should live.
Prove The Business Case, Then Scale
Companies also have an opportunity to start smaller. We’re working with environments where organizations can get models running quickly, begin testing use cases and understand whether the business case is there. Once they know the application delivers value, they can scale into a more robust environment.
That gives leaders a much better foundation for making the next investment. Four factors tend to shape these decisions: speed, price, business outcomes and use cases. Architecture connects all four.
As AI becomes more widely used across the business, those decisions will matter even more. Companies need the flexibility to put workloads where they make the most sense and scale the applications that deliver the greatest value. Understanding the full cost of cloud-based AI is part of making that decision, and it gives organizations a much clearer path for building AI environments they can continue to grow.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?