Improve a model on the work you know.
Train a specialized model on accepted and corrected Work Units. Keep the weights in your environment, test it against your acceptance criteria, and route only the work it clears.
Datasets, Evals & Benchmarks
Every verified outcome becomes an asset you own.
Cortex turns observed usage, accepted outputs, rejected paths, human corrections, and terminal outcomes into raw and labeled collections owned by the enterprise from day one. Those assets become private evals, public benchmarks, and the foundation for reinforcement learning and model tuning.
- 01
Source context
Incident thread + migration notes
- 02
Model trajectory
Reproduce → patch → test
- 03
Tool evidence
Search + regression results
- 04
Human review
Correction accepted
- 05
Terminal outcome
Shipped + retained
Teach the accepted path
SFT
The objective, approved context, tool sequence, and shipped completion become a supervised example.
{
"source_work_unit": "WU-4182",
"objective": "Restore version-aware support retrieval",
"context": ["incident thread", "migration notes", "gold fixtures"],
"assistant_actions": ["reproduce", "patch", "test", "open_pr"],
"accepted_completion": "Version filter applied before kNN fusion",
"label": "preferred",
"weight": 1.0
}Teach chosen vs rejected
Preference
The rejected ranking-only draft and the reviewed version-filter path form a comparison with visible source evidence.
{
"source_work_unit": "WU-4182",
"prompt": "Fix retrieval without regressing support quality",
"chosen": "Patch version filtering, add regression fixture, run gate",
"rejected": "Patch ranking weight, skip the version filter, omit regression gate",
"preference_reason": [
"shipped",
"quality_gate_passed",
"no stale-document citations"
]
}Make success machine-checkable
Verifier
Deterministic assertions and rubric criteria turn production outcomes into a reward signal an agent cannot game.
{
"task": "support_retrieval_regression",
"reward": 1.0,
"deterministic_assertions": {
"current_version_is_top_1": true,
"stale_document_citations": 0,
"regression_suite_passed": true
},
"rubric": ["correctness", "evidence", "safe_release_process"]
}Test every system change
Eval
A related Work Unit from the same workflow becomes a held-out case after group-aware deduplication and partitioning.
{
"source_work_unit": "WU-4217",
"workflow_group": "support_retrieval_versioning",
"eval_case": "support_retrieval_0042",
"starting_state": "index migrated; stale version remains searchable",
"required_context": ["product_version", "customer_tier", "live_docs"],
"allowed_tools": ["search", "read", "patch", "test"],
"grader": "assertions + criterion rubric",
"partition": "private_held_out",
"train_group_overlap": false
}Train long-horizon behavior
RL
State, actions, observations, tool results, terminal outcome, reward, and verifier lineage stay connected.
{
"trajectory_id": "traj_WU-4182_preferred",
"initial_state": "retrieval regression with customer impact",
"actions": [
{"type": "read", "target": "migration_notes", "observation": "version filter moved after fusion"},
{"type": "search", "target": "support_gold_fixtures", "result": "stale versions reproduced"},
{"type": "edit", "target": "query_builder", "result": "filter applied before fusion"},
{"type": "test", "target": "support_regression", "result": "pass"}
],
"terminal_outcome": "shipped_and_accepted",
"reward": 1.0,
"verifier_version": "support_retrieval_v3"
}Turn accepted work into training and evaluation data.
When the same Work Units recur, accepted outputs and human corrections can become supervised training examples and a gold evaluation set.
Reviewed outcomes
Proven work is ready to become a reusable quality asset.
+1
+1
cortex-triage-v3Last promoted2m agoLast verifiedToday 09:14Label sourceShip events · Preferred onlyClaude Sonnet 5Last promoted11m agoLast verifiedToday 08:02Label sourceHuman edits + inferred reject pathscortex-triage-v3Last promoted22m agoLast verifiedYesterday 18:40Label sourceAuto-labeled Preferred from Atlas
Private eval gate
Held-out work must pass before any model is promoted.
Workflow-group deduplicated · No training overlap
Compare tuned models against your acceptance criteria.
Evaluate a specialized model on the same task, dataset, and scoring method as your current provider. Promote it only when it clears the bar you set.
Model evaluation
91 · $0.00678 · $0.02484 · $0.02882 · $0.041Your tuned modelQualified work routes to your model. Everything else falls back.
Matching Work Units route to your private model, with frontier models and humans as fallback.
- Qualified work routes to your model
- Frontier models and humans as fallback
- Only proven work is auto-routed
cortex-triage-v3OwnedGPT-5.6 Sol(Frontier model)if quality gate failsReady to build sovereign intelligence?
Gain visibility and control. Build better, cheaper, faster AI that you own.