Documentation
User scenarios and sizing paths
ConfigIQ helps you size LLM inference. This page covers which tool to use for which ask, the scenarios we test sizing against, and how to size multiple workloads. Cost and pricing are covered in Costs.
What ConfigIQ is
ConfigIQ is a web application for LLM inference sizing. It answers three questions:
- What performance will I get? Predict TTFT, TPOT, and throughput for a model on a given GPU configuration.
- What should I buy? Recommend GPU counts and parallelism from latency and throughput targets.
- Will the model fit? Estimate KV cache memory and headroom for a model configuration.
The webapp is powered by the AISimulators REST API and is also available as MCP tools, so the same sizing is reachable from the UI, from scripts, and from AI assistants such as Claude. You can also compare GPUs side by side in the GPU Explorer.
Which tool to use when
Three customer asks, each with a webapp path, REST endpoint, and MCP tool:
| Customer input | Ask | Webapp | REST | MCP tool | Output |
|---|---|---|---|---|---|
| Existing GPU count and type, plus a model | What performance will I get? | Performance (/predict) |
POST /predict |
predict |
Expected TTFT, TPOT, throughput, and memory |
| Models plus latency and throughput targets | What should I buy? | Recommend Sizing (/recommend) |
POST /recommend |
recommend |
GPU count, TP/PP, and replicas |
| A model and a workload shape | Will it fit, and how many requests? | KV Cache Calculator (/kv-cache) |
POST /memory |
memory |
Memory breakdown and KV headroom |
Predict and Recommend answer different questions. Predict takes a configuration you already have and tells you how fast it will be. Recommend starts from targets and searches for configurations. Neither substitutes for the other, and the numbers can differ because they run different searches.
The KV Cache Calculator needs explicit workload parameters (sequence lengths and concurrency) today. It does not infer a workload from a use-case description.
Scenarios to test against
- Predict performance for an existing fleet. Given a model, a GPU system,
and a parallelism configuration, expect TTFT, TPOT, and throughput. Use
/predict(or the Performance page). - Recommend sizing from SLOs. Given a model plus latency and throughput
targets, expect GPU count, TP/PP, and replicas. Use
/recommend. - KV cache sizing for a known workload shape. Given a model and workload
parameters, expect a memory breakdown and KV headroom. Use
/memory. - Support lookups. Which models, GPU systems, serving backends, network
fabrics, and simulators are supported. Use the catalog endpoints:
GET /models,GET /systems,GET /backends,GET /networks,GET /simulators. - Single workload catalog shapes. Size one workload at a time from the workload catalog shapes: Chat, Coding, RAG, Voice, Deep Research, Batch Summarization, Synthetic Data Gen, and Deep Thinking. Reasoning is selected per row with an explicit output budget. Keep Coding distinct from Deep Research, and Batch Summarization distinct from Synthetic Data Gen.
Shared prefixes and repeated requests
A shared prefix is input every request starts with, usually a system prompt. It is part of the input length, not tokens added on top: 1,000 shared tokens at a 4,000-token input leave 3,000 uncached tokens. It cannot exceed the input length.
The Performance page has two related controls:
- Shared prefix tokens says how much of the input is shared. The simulator computes that part once and reuses it, which is what prefix caching reflects.
- Requests per user says how many requests each concurrent user sends. A run simulates concurrent users × requests per user requests, and every request after the first can reuse the prefix. Leave the field empty to simulate one request in total. That is not the same as one request per user, and it is not the same as setting the field to 1.
POST /predict takes both prefix and requests_per_load_unit.
POST /recommend takes prefix and credits it when sizing capacity, but not a
request count: it sizes from demand rather than replaying a fixed number of
requests. /memory takes neither.
Sizing multiple workloads
- Size one workload per call. There is no mixed-workload endpoint. Run
/recommend(or/memory) once per workload and record each result separately. - Do not sum standalone GPU counts for a shared deployment. That is not validated shared-pool sizing.
- GPU counts must come from recorded runs, not extrapolation.
Sharing a sized view
Sizing is usually a conversation, and a link is how you hand someone the view you are looking at instead of describing it.
- Send the view, not a screenshot. Copy link records every value that differs from the page's starting state, along with the simulator, backend, and backend version that produced it. The recipient opens the same form, not the same answer: they press Calculate themselves.
- A partial link is safe to send. A field the link leaves out keeps the recipient's own value, so a link never resets a control you did not touch.
- Nothing is switched on their behalf. If they use a different simulator, the page names both and offers a one-click switch for that view only. Their saved settings are untouched, and an unsupported combination is reported rather than silently replaced.
- Disaggregated views travel intact. Each pool's parallelism is carried separately, and an aggregated link carries none of it, so the recipient sees the same controls you did.
See Settings and simulators for the exact provenance rules.
What customers get
- Predict: expected TTFT, TPOT, and throughput for a given configuration.
- Recommend: a configuration with GPU count, TP/PP, and replicas that meets the stated targets.
- KV Cache Calculator: a memory breakdown and KV headroom for candidate configurations.
Related
- MCP server: connect Claude and other MCP clients
- REST API: endpoints and examples
- Settings and simulators: how a shared link behaves
- Tested models: what "validated" means