Audit Discovery Calls Like RevOps: Turn Playbook Criteria Into Deterministic Scoring and Coaching Gaps

Most teams don’t have a discovery call quality problem. They have an evaluation problem: they cannot measure buyer understanding against their playbook, so coaching runs off noise. The rep got positive feedback because the call felt energetic. The deal stalled because the buyer never articulated the consequence of their problem. Those two facts were never connected because nobody had a way to connect them.

Auditing discovery calls like RevOps means fixing the measurement layer first. That means turning playbook criteria into structured evaluation Bricks, scoring every call against them, and routing the results into coaching workflows where the evidence is already attached. Reviewing a sample of calls and coaching from impressions produces coaching that is inconsistent, hard to scale, and disconnected from the deal outcomes it was supposed to improve.

The evaluation problem, not the quality problem

Discovery call quality is not a coaching volume problem. Most teams already review calls. The problem is what gets measured during the review. Subjective impressions of rep delivery, notes on whether the call felt like a good conversation, and summaries of what was discussed are not the same as a consistent assessment of whether the buyer understood the qualification logic your playbook requires.

The test of a good discovery evaluation is whether two separate reviewers assessing the same transcript reach the same conclusion. On most teams they do not, because the criteria are implicit and the evidence is not tracked. When the criteria are implicit, coaching reflects whoever did the review that week, not a stable standard. And when coaching is inconsistent, the only thing that changes is the volume of feedback, not the outcomes.

Why freeform AI scorecards repeat the same mistake

Freeform AI scorecards built on a general prompt, "assess whether this was a good discovery call," inherit the same problem as subjective human review. The criteria are still implicit. The model interprets what "good" means and returns a score, but the interpretation varies across runs, across model versions, and across calls the prompt author never anticipated. The score looks authoritative because it came from an AI, but it reflects the same unconstrained judgment that subjective review does.

The evaluation contract post covers this in detail. AI scorecards disagree not because the model is unreliable but because the prompt never specified what evidence would confirm the criterion. A scorecard built on a vague prompt produces vague outputs regardless of the model running it, and coaching built on those outputs produces inconsistent results regardless of how frequently it happens.

The reframe: buyer understanding, not rep behaviour

The unit of improvement in discovery is not rep behaviour. It is whether the buyer understood the qualification and next-step logic your playbook requires, evidenced in the transcript. A rep who delivered a polished, energetic call without getting the buyer to articulate a business consequence did not run a good discovery call. A rep who spent the first fifteen minutes on background and then got the buyer to name a timeline, a consequence, and a decision-maker ran a better one. The transcript evidence is different, and the score should be different.

This reframe changes what you are evaluating. Rather than assessing whether the rep asked the right questions, you are assessing whether the buyer produced the right evidence. The distinction matters because it is the buyer side of the transcript that determines whether a deal has genuine qualification, not the rep side. Buyer-side evidence is measurable, consistently, from the transcript, without inference about the rep’s intent or effort.

Build evaluation Bricks from your playbook

Turning a playbook into a scoring system starts with identifying the observable, evidence-grounded criteria that confirm each qualification element. For each criterion, the design question is: what would a buyer explicitly say in the transcript that confirms this element is present, and what would a reliable null look like when it is absent? A criterion that has a clear answer to both questions can become a Brick. A criterion that requires the evaluator to interpret intent or infer quality cannot be scored consistently and should be redesigned before it is added to the schema.

Concrete examples of criteria that work well as Bricks include: whether the buyer named a specific business consequence of their current problem, whether they identified a timeline for resolution, whether the economic buyer was named on the call, and whether a specific next step was committed to with a date. Each has clear transcript evidence when present and a reliable null when absent, and a typed output, boolean or extracted string, that can be written to a CRM field without transformation. More interpretive criteria, such as whether the rep demonstrated value or whether the buyer seemed engaged, are not discoverable from transcript evidence in a way that two reviewers would agree on, so they produce unreliable Bricks and should be excluded from the scoring schema.

Once the Bricks are defined, they are grouped into a Kit that represents the discovery evaluation schema. Every call scored against the same Kit version produces an output with the same fields, the same types, and the same possible values, which is what makes the results comparable across reps, managers, and time periods.

Score every call and route gaps into coaching

With the discovery Kit in place, every call produces a structured output: a JSON object with one scored field per Brick. That output can be written to CRM opportunity fields, pushed to a reporting table, or routed to a coaching workflow. The workflow logic can be simple: if a deal has had two discovery calls and the economic buyer field remains null on both, that is a gap. If a rep’s discovery calls have a consequence field fill rate below the team average, that is a coaching signal. Neither observation requires a manager to review calls manually, because the evidence is already extracted and stored.

Rep-level gaps and team-level gaps serve different purposes. A rep-level gap, a specific criterion consistently absent across one rep’s calls, drives a one-to-one coaching conversation with specific transcript evidence attached. A team-level gap, a criterion missing on a high percentage of calls across the whole team, drives a change to the playbook, to training, or to the enablement materials that introduce the criterion. The behavioural change post covers how structured call outputs connect to the kind of coaching that actually changes what reps do next week.

Semarize turns discovery call transcripts into typed, evidence-grounded scores against the playbook criteria you define, ready for CRM enrichment, coaching workflows, and rep and team gap analysis.

Start building →

Common questions