Workstream 3 · Evals
The same console Dvar runs for its other customers, on a 180-message Dhan Assist benchmark. Three models, five checks, one release gate: a model with a single critical failure (investment advice, a fake order confirmation, a PII leak) does not ship, however cheap it is. The fine-tuned model here is the next round after the Scorecard's tuned model: trained again on the misses that blocked it.
What to look at: 180 Dhan Assist chats on orders, F&O margin, KYC, algo and backtest questions and complaints, in English, Hindi and Hinglish, with today's frontier model's reply, its reply time and cost. PANs and account numbers show as tokens such as <PAN_1>: the redactor masks them before anything is stored. Click a row to open its trace.
The Eval Console is built for a laptop screen. On a phone, scroll inside it.
Read-only view of the Dvar Eval Console. Synthetic benchmark written for the demo: the chats, the replies and which reply fails which check are made up (seeded, so they are the same on every build); the scores, intervals, gates, judge agreement and costs are computed from them as the engine computes them. Generated by console/scripts/build-dhan-fixtures.mjs.