The upgrade loop
- 1
Pick one workflow
Outbound sales call QA.
- 2
One labelled slice
200 to 500 calls your QA team has reviewed.
- 3
Build the eval
Gold, checks and a calibrated judge.
- 4
Score every candidate
Same calls, same checks.
- 5
Gate
No critical miss, no quality drop. Or it doesn't ship.
- 6
Tune on the misses
Misses become training data.
- 7
Ship and watch
Shadow first, then live, checked on every call.
What it costs at your volume
Quality decides which models are allowed; cost decides between them.
Today's model (stand-in)
Stand-in: dvar:glm-5.3-flash
Baseline
- Right
- 11/18
- Per min
- ₹0.0119
- A day
- ₹60,774
- A year
- ₹2.22 crore
Closed-source model
google/gemini-4-argon
Passes the gate
- Right
- 15/18
- Per min
- ₹0.0931
- A day
- ₹4.75 lakh
- A year
- ₹17.32 crore
Open model
dvar:gemma-4-31b-it
Blocked: No summary on 2 calls.
- Right
- 6/18
- Per min
- ₹0.0073
- A day
- ₹37,295
- A year
- ₹1.36 crore
Open model + examples
dvar:gemma-4-31b-it + 8 examples in the prompt
Passes the gate
- Right
- 14/18
- Per min
- ₹0.0125
- A day
- ₹63,832
- A year
- ₹2.33 crore
7×
closed-source cost vs open model + examples
14 vs 11
calls fully right: open model + examples vs today, at 1.1× the cost
0
critical misses for open model + examples
Today's misses become training data
Today's 7 misses, once reviewed, become training examples.
- Term plan, ₹1 crore cover, price objectionDisclosures right: Medical history asked: made, marked missing.
- Term plan, BP medicine not to be declaredDisclosures right: Medical history asked: made, marked missing; Free-look period explained: not made, marked made.
- Term plan, personal cashback offeredDisclosures right: Medical history asked: made, marked missing.
- Family floater, mother's diabetes "covered from day one"No summary: the reply was cut off at the token limit before the summary ended.
- Cashless held at the hospitalDisclosures right: Ticket or escalation path given: made, marked missing.
- Claim approved, card CVV asked for the refundDisclosures right: Exclusions explained: made, marked missing.
- Paid five weeks ago, no policy documentCovers the call: Missing: "medical report". Disclosures right: Ticket or escalation path given: made, marked missing.