Execution policy instead of model accumulation

What we were testing ?

The original idea was straightforward : Since different models have different strengths and may serve different purposes, why not route each task to the most appropriate LLM?

We implemented that approach progressively through:

• a Smart Router;
• a sequential multi-model pipeline;
• a Comparison Mode allowing two models to answer the same request.

This gave us a practical environment to test different ways of combining models, but it also exposed the limitations of thinking only in terms of model selection.

However, several questions kept appearing:

1. Should a difficult task use a stronger model or a pipeline?
2. Should a large context trigger deeper reasoning or better retrieval?
3. Should economy mode select a cheaper model or simply execute fewer calls?
4. Should every task use an LLM at all?
5. How do we explain why a particular execution path was chosen?

We found out we were facing more of an execution problem than a routing problem.

What we observed ?

Model choice was only one variable as six signals proved more useful for describing what an execution decision actually needed to consider:

1- Task type
2- Reasoning complexity
3- Decision criticality
4- Context pressure
5_ Expected output class
6- Cost mode

These signals affect more than model selection. They can affect the whole execution strategy:

deterministic
cache
single model
pipeline
evaluator + critic
parallel extraction
map/reduce
web research

A model should only become relevant when enough is known about how the task should be executed.

1 - Large context does not necessarily mean difficult reasoning

A context-heavy request may need retrieval, compression or staged processing rather than additional model reasoning.

2 - Economy does not mean cheapest model

A critical decision can establish a minimum quality floor, while cost optimisation should happen inside that boundary.

3 - Better quality does not always mean more model calls

A sequential or parallel pipeline can improve one task and create unnecessary latency and cost for another. However, sometimes one stronger model is enough, and sometimes a deterministic calculation is preferable.

4 - Model diversity is not an execution policy

Adding more available models increases options, but it does not tell the system when those options should be used.

What we changed

We focused on an execution-policy engine implementation that emerged from the limits we encountered while operating the model combinator.

The main concrete output was a different decision contract. Instead of asking the router to return only:

task → model

the target decision focused on:

task
→ complexity
→ criticality
→ context pressure
→ cost intent
→ execution strategy
→ capability required
→ reasoning depth
→ token envelope
→ model/provider

The resulting routing decision was designed to explicitly carry:

task type
complexity
criticality
context pressure
expected output class
cost mode
execution strategy
capability class
selected model
thinking level
token budget
fallback model
reason for the route
policy version

This is important because the policy decision can then exist independently from the model currently attached to it.

For example:

critical analysis
→ strong reasoning capability required
→ pipeline execution

becomes a policy decision, while the provider or model currently fulfilling strong reasoning remains a separate implementation choice.

The same applies to:

weighted calculation
→ deterministic
→ no LLM call

or:

large source corpus + moderate reasoning
→ retrieval / staged processing
→ standard reasoning capability

What we retained ?

The implemented combinator proved that Smart Routing, pipelines and model comparison are useful execution techniques, but none of them should become the default simply because they are available.

What we needed instead was something explicit enough to explain and test the execution decision itself, including:

• Why was this strategy selected?
• Which requirement forced this quality level?
• Why was a pipeline used instead of a single call?
• Why was a model not used at all?
• What would change if the cost or criticality changed?

Only after these decisions can be observed should adaptive routing become relevant.

Takeaway

The real issue was not about building a better model router, as it was more about defining an execution policy capable of deciding how much AI a task needs, how it should be applied, and when using less AI is actually the better decision.