AI Sales Coaching Platform

Every sales call transcribed, scored against a fixed rubric, and written back to Salesforce within minutes.

Private · write-up

Calls scored: 100%
Feedback latency: minutes

The problem

Sales coaching only works if someone actually listens to the calls. In practice a manager samples under 10% of them, picks the ones they already have a hunch about, and the rest are never reviewed at all. The feedback that does land arrives days later, detached from the call it refers to.

Architecture

Aircall webhook drops a call recording onto a Celery queue; transcription and scoring run as separate tasks so a slow LLM call never blocks ingestion.
Every transcript is scored against one fixed rubric rather than a free-form prompt, so two calls a month apart are comparable.
Scores and the generated coaching notes are written back to the Salesforce record, so reps see feedback where they already work instead of in a second tool.

What I chose, and what I rejected

The interesting constraint here was not the model. It was that coaching feedback is worthless if it is inconsistent.

Why the rubric is fixed, and lives in code

Scoring calls with a free-form prompt produces feedback that is individually plausible and collectively useless. Two calls a month apart get judged against subtly different standards, because the prompt drifted, or the model version changed, or the same prompt simply landed differently. A rep cannot improve against a moving target, and a manager cannot compare two reps at all.

So the rubric is fixed and lives in code rather than in a prompt body. That sounds like a detail and it is the difference between a tool and a toy: changing what “good” means becomes a reviewable diff with a date on it, not an untracked edit to a string. When scores shift, you can tell whether the reps changed or the ruler did.

The same call scored twice should not drift. That is a testable property, and it is the one I would want to be asked about, because it is the property that makes every number downstream mean anything.

Transcription and scoring are separate tasks on purpose

The two halves of this pipeline have nothing in common operationally.

Transcription is slow, I/O-bound and roughly proportional to call length. It either succeeds or fails on audio it can reach. Scoring is a bounded model call against text that either exists or does not.

Coupling them means one retry policy for two failure modes, and any failure retries both — so a transient scoring error re-transcribes a forty-minute call for nothing. Splitting them into separate Celery tasks means each retries on its own terms, transcripts are durable once produced, and a scoring change can be replayed across the whole history without touching the audio.

That last property is the sleeper. Because transcripts are a persisted intermediate rather than a step inside one job, improving the rubric means re-scoring everything you already have — the historical data is not frozen at whatever the rubric was on the day the call happened.

Coverage is a different activity, not a better one

The headline number is that call review went from a manager sampling under 10% of calls to every call being scored. It is worth being precise about why that is not a quality improvement but a change in kind.

A 10% sample is not random. A manager listens to the calls they already have a hunch about — the deal that went sideways, the rep they are worried about. That is a reasonable use of scarce attention, and it means the sample is selected on the outcome you are trying to study. Anything you learn from it tells you about calls that already looked interesting.

At full coverage the population is the population. Patterns that were invisible at 10% become measurable: an objection that consistently precedes a lost deal, a phase of the call where good and bad reps diverge, a script change that helped in one segment and hurt in another. None of those are findable in a biased sample of one in ten, no matter how carefully you listen.

Writing back to Salesforce, rather than building a dashboard

The feedback lands on the Salesforce record rather than in a separate coaching tool.

A second tool is a second login, a second set of permissions and a second place people forget to look. Coaching feedback that lives where the rep already works gets read; coaching feedback behind another URL gets read in the first week. The integration is unglamorous and it is most of why the thing gets used — a scoring pipeline nobody reads is an expensive no-op.

Latency matters for the same reason. Feedback that arrives within minutes attaches to a call the rep still remembers. The same feedback days later is an audit, and people receive audits differently from coaching.

The limit worth naming

Everything here rests on the rubric being a good one. The system makes scoring consistent, comparable and complete — it does not make it correct. If the rubric rewards the wrong behaviour, this pipeline will now reward that behaviour uniformly across every call, at speed.

That is an argument for keeping the rubric in code where it can be reviewed and versioned, and against treating any of these scores as ground truth rather than as a consistent measurement of a chosen standard.