I've watched more than one AI programme move from a polished demonstration to an awkward conversation about operating cost — and nobody in the room could quite say when the critical design choices had been made.
There is a decision buried inside almost every AI system a business deploys that nobody on the business side is likely to know has been made.
It concerns something as unglamorous as how the system remembers and reuses information between requests — what engineers often shorthand as caching, memory or context reuse.
Get it right, and an AI service can become faster and cheaper as usage grows.
Get it wrong, and the opposite can happen: costs rise, response times deteriorate, and what looked impressive in a demonstration becomes expensive to operate at scale.
There can also be a more serious consequence. If information is stored or reused incorrectly, questions about privacy, data separation and regulatory compliance can arise.
To an engineer, this may look like a technical implementation detail.
To a business, it is a decision about cost, customer experience, scalability, risk and compliance.
And yet, the business may never have been asked.
That is the bigger issue. Caching is only the example. The pattern is default technical decisions with business consequences.
The technical decision that isn't just technical
This example illustrates something I increasingly see in AI adoption.
A business decides it wants an AI assistant.
The technology team, or an implementation partner, is asked to build it.
A technically competent solution is produced.
But along the way, dozens of decisions are made about how the system will work, where information will be stored, how long it will be retained, how it will scale, how different customers will be separated, what happens when information changes, and how much the system will cost to operate.
These decisions may look technical.
Many of them aren't.
They have business consequences.
Consider three simple questions:
How much should this system cost as usage grows tenfold?
That is not just a technology question. It affects the economics of the proposition.
How current does the information need to be before an answer becomes unacceptable?
That is not just a data question. It may affect customer service, operational risk or regulatory exposure.
What happens to one customer's information when the same AI platform is serving thousands of customers?
That is not simply an architecture question. It is a question about privacy, trust and accountability.
The technical team needs to design the answer.
But the business should own the decision about what outcome is acceptable.
That distinction is important.
Conceptual schematic
An end-to-end mental model
Caching and reuse sit in one layer. The pattern is wider: each step carries choices about cost, freshness, isolation, autonomy and accountability. The business should own what “good” looks like; engineering designs how to deliver it.
1
Request
Customer or employee asks for help — or a process triggers the service.
2
Context & reuse
What is remembered or reused between requests — the “cache” example: speed, cost, freshness, privacy.
3
Model & tools
What the system can understand, generate and call — and within what boundaries.
4
Business systems
Records, quotes, bookings, payments — what the AI is allowed to see and do.
5
Response or action
An answer, a recommendation, or a change in the real world.
6
Human judgement
Where accountability, exceptions, relationships or risk require a person — not bolted on at the end.
Underneath the flow: business outcomes and engineering design stay aligned through an Owner's Engineer — especially when a vendor is involved. The business decides what is acceptable; engineers determine how to achieve it.
This isn't about engineers making bad decisions
It is tempting to frame this as a communication problem between “the business” and “technology”.
I don't think that is quite right.
Good engineers generally make good technical decisions.
The problem is that the technically sensible decision is not necessarily the right business decision.
An engineer may quite reasonably choose the architecture that is easiest to build, easiest to maintain and easiest to demonstrate.
A business leader may reasonably prefer an architecture that costs more initially but scales better, provides stronger controls, or creates less regulatory risk.
Both positions can be perfectly rational.
The problem occurs when nobody explicitly makes the trade-off.
Instead, the decision gets made by default.
And default decisions have a habit of becoming expensive decisions later.
Why AI makes this more important
This matters with any significant technology programme.
AI makes it more important because the technology is developing so quickly, the capabilities are changing so rapidly, and the consequences of getting adoption wrong can extend well beyond the technology itself.
A business might start with a simple question: “Can we use AI to improve customer service?”
Within months, that can become: what information can the AI access? What decisions can it make? When should a human intervene? How do we know its answers are reliable? Who is accountable when it gets something wrong? How does the system behave at ten times the volume? What does it cost? What happens to customer data? Can we demonstrate to a regulator that we have appropriate controls?
These are not questions that should be answered one at a time as problems emerge.
They should influence the design from the beginning.
And importantly, the business needs to understand the implications without needing to become an AI engineer.
The missing role — bridge, not referee
This is where I believe the Owner's Engineer has a particularly useful role to play.
The concept is not new. On a major construction or infrastructure project, the Owner's Engineer represents the interests of the organisation commissioning the work. They are independent of the contractor building it. They understand enough of the engineering to challenge the design, test the assumptions and assess whether what is being built will actually deliver what the owner needs.
I believe the same principle is increasingly valuable in AI.
The Owner's Engineer does not replace the engineering team. Nor does the role exist to second-guess every technical decision.
The job is to make sure that important technical decisions are connected to the outcomes the business is actually trying to achieve.
That means asking questions such as:
What happens to the economics when usage increases significantly?
What assumptions are we making about the quality and freshness of the information the AI uses?
Where does customer information go, who can access it, and how long is it retained?
What happens when the AI is wrong — and which decisions require human judgement?
Are we building something the business can control, or becoming dependent on a particular vendor?
And when a supplier says, “That's just how the technology works,” do we understand whether that is a hard constraint — or one option among others?
These questions don't require the CEO to become an AI architect. They require someone who understands enough of both worlds to make sure the right conversation happens.
This is not about putting business people in charge of technical architecture. Nor is it about putting technology teams under suspicion.
It is about recognising that modern technology decisions increasingly sit at the intersection of technology, commercial value, risk and strategy. Someone needs to connect those four things.
The business should decide what matters. The engineers should determine how to achieve it. And an independent Owner's Engineer can help make sure the two remain aligned.
That becomes particularly important when an external vendor or implementation partner is involved. A vendor may genuinely recommend the solution they believe is best. But they also have their own economics, product roadmap and commercial interests. That does not make the relationship dishonest. It simply means the customer needs someone whose primary responsibility is to the customer's outcome.
The cache is only the beginning
The irony is that the business leader probably never needs to know anything about the particular caching mechanism described at the beginning of this article.
They do need to know that a decision was made.
They need to know what that decision means for cost, performance, scalability, privacy and risk.
And they need confidence that someone has challenged the assumptions behind it.
That is the real lesson.
As AI moves from experiments and demonstrations into the core operations of businesses, more and more decisions that look technical will actually determine commercial outcomes.
The question is not whether the business should start making engineering decisions.
It shouldn't.
The question is whether someone is making sure that the engineering decisions are being made in the context of the business's objectives, constraints and accountability.
Because by the time those decisions become visible to the board, the customer, or the regulator, they may already be very expensive to change.
Someone needs to be in the room on the business's behalf.
That's the Owner's Engineer role — and why unglamorous choices like caching deserve a place in the boardroom conversation, even if the mechanism itself never does.