Start Asking How Your AI System Is Designed

Not A Single Model: AI Is A System

One of our clearest lessons came from a document intelligence project, extracting information from technical documents, manuals, reports and specialized PDFs.​ To see whether the model was holding us back, we ran a simple experiment: same prompt, same use case, different LLMs. We expected a noticeable jump in extraction quality, but it didn’t happen.​

The difference was surprisingly small. The real challenges were elsewhere: document layout, tables and OCR, the context, and how we validated results. So we stopped optimizing the model and fixed everything around it: better preprocessing, cleaner context and validation to catch errors before they reached the client.​

The result was a far more reliable system. Not a better model, but a better system around the one we had.​

Today’s frontier models can reason, summarize, write code and make decisions. But no LLM automatically knows how to process a 200-page PDF, find the pages that matter or verify that extracted data is correct.

Seven Layers Around The Model

​I’ve come to think of the LLM as the brain. It provides the reasoning, but a brain can’t do much in isolation. What makes the LLM useful is the environment around it: the data, tools, context and workflows that give it something meaningful to act on.​

1. Context management decides what the model sees at each moment: the right information in, the irrelevant out. As agents take on longer tasks, this context engineering discipline has become one of the most important skills in AI development.

2. Memory retains information across interactions, past conversations, preferences and project history. Without it, every interaction starts from zero.

3. Tools and integrations let the model act on the world: search a database, open a PDF, query an API and run code.

4. Orchestration manages the workflow: act, observe and decide what’s next until the task is done.

5. Business data grounds the agent in your company’s knowledge: your pricing rules, policies and customer records that no general-purpose model knows.

6. Verification checks the work instead of trusting the first answer, the way a coding agent tests, fixes and tests again.

7. Security and permissions define what the agent may do and what requires human approval. Reading a document is very different from deleting a file.

That is the agent harness. The model provides reasoning, and the harness turns it into useful action.​

Why Agent Harness Matters More Than Model Choice

The biggest advantage of a strong agent harness is flexibility.​ Models will keep changing. Claude evolved into the Fable 5 generation, built for long-horizon autonomous work. Gemini reached 3.7 Flash, a high-efficiency workhorse with adjustable reasoning levels and a 1-million-token context window. DeepSeek grew from a research curiosity into a full open-weight family: V4 Pro, Flash and a vision variant. Each cycle brought models that were faster, cheaper or better at specific tasks.​

If your application is tightly coupled to one model, every one of those cycles may become an expensive reengineering project. With a well-designed harness, your business logic, data pipelines, integrations and evaluations can stay intact while the model underneath changes.​​

I’ve seen this firsthand in my company’s document intelligence work. Once the system was designed properly, the LLM became a component we could swap, not the foundation everything depended on.​

So don’t build your competitive advantage around access to a particular model. Build it around how effectively your system uses whatever models are available.​​

The Decisions To Make Before Selecting A Model

The real question isn’t “Which model should we use?” It’s: “Have we built a system that lets any model perform well?”​

Before getting deep into model selection, make a few fundamental decisions first:​

1. Start with the business process. What problem are we solving, and where does human judgment still matter?​

2. Connect AI to the systems where work happens. What data and applications does it need to complete the job, not just generate an answer?​

3. Give it the right knowledge and context. What should it know, remember and never see?​

4. Design for verification. Define what “good” looks like and the checks required before an output can be trusted.​

5. Build for failure. When a tool fails or information is missing, the system needs to retry, recover or escalate, not simply break.​

6. Define where humans stay in the loop. Identify the decisions that require human review or approval.​

7. Measure business outcomes, not just model performance. Track processing time, cost, throughput, adoption and the quality of decisions, not just benchmark accuracy.

This is ultimately a different way of thinking about AI. Don’t build your strategy around a model. Build a system that can leverage whichever model is best for the job.

Conclusion

Model selection will always matter. Whatever comes next will bring different strengths and trade-offs, and I still expect that question in every client meeting.​ But my answer has changed. After years of building these systems, I’ve stopped seeing models as foundations and started seeing them as components. The lasting value comes from the system around the model: the agent harness.​

So the next time someone pulls up a benchmark comparison, here’s what I’ll say: The competitive advantage won’t come from having the best model. It will come from building the best system to put that intelligence to work.​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?