Cortex Router · tau2-benchTask-level routing across the GPT-5.6 model family
A retrospective 100-task pilot of one routing decision for the full agent conversation, measured on task success and recorded model cost.
Key takeaways
- The operating point: On the 100-task common cohort, Cortex reaches 66% task success at $0.625 per completed conversation.
- Point comparison: Against GPT-5.6 Sol, it adds 1 percentage point of success and uses 52% less mean cost.
- Route mix: 35 GPT-5.6 Luna / 34 GPT-5.6 Terra / 31 GPT-5.6 Sol across the 100-task pilot cohort.
66% success at $0.625 per task
Against GPT-5.6 Sol, the task-level router adds 1 percentage point of task success and reduces mean agent cost by 52% on the 100-task common cohort. This is a point comparison, not an iso-cost result.
Every point is a complete agent run
Task success is the tau2-bench reward. Cost includes the recorded model usage for the full conversation. The table shows each arm's available valid rows and makes differing denominators explicit.
Mean cost vs. task success
Provider logos identify every model and the Cortex router · scroll chart →
| Configuration | Passed | Success | Cost / task |
|---|---|---|---|
 GPT-5.6 Luna | 46/100 | 46% | $0.277 |
| GPT-5.6 Terra | 57/100 | 57% | $0.731 |
| GPT-5.6 Sol | 65/100 | 65% | $1.306 |
| Cortex task-level router | 66/100 | 66% | $0.625 |
Per-task distributions, not cumulative curves
Stacked histograms show raw binned task counts across cost, turns, and token use. No smoothing. Hover a legend item or bar segment to isolate a series.
Agent cost
USD per task →
Stacked binned task counts by agent cost (x truncated at the 98th percentile). No smoothing. Each bar is a raw bin total.
- GPT-5.6 Luna
- GPT-5.6 Terra
- GPT-5.6 Sol
- Cortex task-level router
Agent turns
Assistant turns per task →
Stacked binned task counts by assistant turns (x truncated at the 98th percentile). No smoothing. Each bar is a raw bin total.
- GPT-5.6 Luna
- GPT-5.6 Terra
- GPT-5.6 Sol
- Cortex task-level router
Input tokens
Tokens per task →
Stacked binned task counts by input tokens (x truncated at the 98th percentile). No smoothing. Each bar is a raw bin total.
- GPT-5.6 Luna
- GPT-5.6 Terra
- GPT-5.6 Sol
- Cortex task-level router
Output tokens
Tokens per task →
Stacked binned task counts by output tokens (x truncated at the 98th percentile). No smoothing. Each bar is a raw bin total.
- GPT-5.6 Luna
- GPT-5.6 Terra
- GPT-5.6 Sol
- Cortex task-level router
Where routing changes the answer
The same 100 valid task IDs are compared between the router and GPT-5.6 Sol.
One decision for the full conversation
Cortex scores the initial task prompt, chooses one model, and keeps that model for every turn. This avoids mid-conversation model changes and preserves prefix-cache locality.
Retrospective, paired, and recomputable
This is a 100-task pilot from the tau2-bench base split. Cortex scores the initial task prompt, chooses one model, and keeps that model for every turn. Infrastructure failures are excluded from an arm's descriptive metric; headline comparisons use only task IDs valid in both the router and baseline arms.
Recompute every number
The export includes task ID, domain, arm, pass/fail, cost, turns, token counts, selected route, infrastructure status, and a claim map.