Routing
The router selects the best supplier for each request by optimizing cost, latency, and reliability in real time.
Supplier Selection
Requests are mapped to a pool of compatible suppliers (model, context length, region, and policy), then ranked by score.
Latency-Aware
The router monitors p50/p95 latency and error rates. High variance suppliers are deprioritized to prevent tail-latency spikes.
Cost Optimization
Suppliers publish price tiers. The router can trade off cost vs latency under user policy (e.g. “cheapest within 100ms”).
Reliability Scoring
Historical performance and verification outcomes feed a reliability score that compounds over time, rewarding stable operators.