In a recent project with a European retailer, we measured something that surprised us: 68% of the decisions that the marketing team considered “automatable” actually required some kind of human validation before being executed. And the other way around, almost a third of the decisions that managers wanted to supervise manually were perfectly delegable to an agent. The conclusion was clear: the problem was not what to automate, but how to distribute judgment between people and systems. This is exactly what Human+AI hybrid decision systems address.
What we mean by a hybrid decision system
A hybrid system distributes the responsibility for a decision across several layers. One part is taken by a model or agent autonomously, another part remains in human hands, and there is a third layer of shared decisions where the agent proposes and the person approves or adjusts. The key is to define those boundaries precisely before deploying anything to production.
In practice, this translates into flows where an AI agent can, for example, segment an audience, generate three creative variants and propose a budget, while the CRM manager validates the final allocation and supervises the send. Each step has a defined owner, its own metrics and an audit point.
The most common mistake we see is treating the agent as a generic assistant. It works better when it is designed as a collaborator with bounded responsibilities, just as you would when bringing a junior person onto the team.
How decisions are distributed in practice
We work with a matrix of four variables to decide what is delegated: reversibility of the decision, cost of error, frequency and availability of data to validate the result. A reversible, frequent decision with a low cost of error, such as adjusting the subject line of an email based on historical behaviour, can be automated with reasonable confidence. An irreversible and infrequent decision, such as excluding an entire segment from an annual campaign, should stay with the person.
The interesting space is in the middle. That is where patterns like “human in the loop” and “human on the loop” work. In the first, the human approves each action before it is executed. In the second, the agent acts autonomously and the person supervises by sampling or by exception, intervening only when certain thresholds are crossed.
In a recent case with a client in the financial sector, we configured an agent that generated responses to B2B customer queries. Responses with a confidence score above 0.85 were sent automatically. Between 0.7 and 0.85 they went through a quick review by a human agent. Below 0.7 they escalated directly to the team. That configuration reduced the average response time by 40% without increasing complaints.
Designing the interface between person and agent
The most underestimated part of these systems is the collaboration interface. It is not enough for the agent to be accurate, it has to communicate its reasoning in a way that the human can validate quickly. This means showing the context it used, the alternatives it discarded and the level of uncertainty associated with the proposal.
Data Innovation, an AI and data company based in Barcelona that builds and operates intelligent systems where humans and AI agents work together, has documented that teams that invest at least 20% of development time in designing the explainability and review layer achieve adoption rates three times higher than those that treat AI as a black box integrated at the end of the process.
A good interface reduces the friction of reviewing. If validating a proposal from the agent takes more time than doing the task from scratch, the team will stop using the system within a few weeks. We have seen this pattern repeat in projects where the agent’s logic was sound, but the presentation of its decisions forced the human to open five screens to confirm a single action.
Metrics that matter in a hybrid system
Measuring a hybrid system requires going beyond model accuracy. The metrics that truly indicate the health of the system are the override rate (how many times the human corrects the agent), the average review time, the agreement between reviewers when several humans validate the same thing, and the cost per decision over time.
The override rate is especially useful. If it is above 30%, the agent is probably operating outside its zone of competence and it is worth reducing the scope. If it is below 2%, it is a sign that more could be automated, or that humans are approving without really reviewing, which is a different but equally relevant risk.
It is also worth measuring what the team stops doing thanks to the system. In a marketing operations project, we freed up 11 hours per person per week that were reinvested in strategy and cohort analysis, not in running more of the same campaigns.