Home › Blog › Build an AI Cold Email Reply Classifier with 10 Intent Categories
Build an AI Cold Email Reply Classifier with 10 Intent Categories
Positive or negative is not a classification system, it is a coin with labels. Real reply traffic contains meeting requests, referrals to colleagues, future-timing interest, questions, soft nos, hard nos, out-of-office robots, and legal threats, and each one deserves a different action. Here is the full classifier: categories, prompt design, confidence rules, and the referral detection most systems miss.
Quick Answer: 10 Categories, Strict JSON, Confidence Gate
Classify every reply into 10 intents (meeting, interested now, interested later, question, referral, soft no, hard no, OOO, automated, hostile) with a cheap fast LLM, feeding it the reply plus the original email and prospect context, demanding strict JSON with a confidence score. Above 0.8 the automation acts; below it, a human review queue. Referrals get structured extraction (who, role, email) because "talk to Sarah" is the warmest lead cold email produces.
The 10 Categories and What Each One Triggers
1 meeting_request → page rep, propose times, CRM task
2 interested_now → page rep, 1h SLA
3 interested_later → extract timing, nurture segment,
scheduled re-engage task
4 question → route to rep with context, answer
personally, never a template
5 referral → extract person/role/email, create
contact, referral sequence, thank
the referrer
6 soft_no → stop sequence, keep nurturable
7 hard_no → suppress everywhere (sync post),
stop company sequences
8 out_of_office → parse return date, pause + resume
(OOO playbook)
9 automated_other → discard from stats
10 hostile_legal → suppress domain, human review,
never auto-respondThe categories earn their keep in the actions, not the labels. Interested-later with an extracted date becomes a scheduled task instead of a lost maybe. Hard no fans out through your suppression sync to every tool. OOO routes into the pause-and-resume flow with the return date parsed. And automated noise stops polluting your reply rate, which quietly fixes your metrics too.
Prompt and Output Contract
- •Input: reply text, the outbound email it answers, prospect name, company, sequence step number. Context is half the accuracy.
- •Output, strict JSON only: category, confidence (0-1), one_line_summary, and extracted fields: referral_name, referral_email, timing, unsubscribe_request (boolean). Reject and retry any non-JSON response; automations downstream depend on the shape.
- •Few-shot the weird ones. Include 6-10 real examples in the prompt, especially the ambiguous cases your review queue surfaces: the polite no that reads positive, the "yes" that answers the wrong question, the referral hidden inside a rejection.
- •Never let it write replies. Classification and extraction only. The moment the same pipeline drafts responses, hallucination risk enters the one channel where a bad message costs a warm deal.
Measuring It Honestly
Sample 50 classified replies weekly and score them yourself for the first month. You are watching two numbers: overall agreement (should sit around 90-95%) and the override pattern in the review queue. Overrides clustering on one category means the prompt needs another example, not a bigger model. Once the numbers hold for a few weeks, drop to spot checks, the system is now more consistent than a tired human at 6pm on a Friday, which is exactly who it replaced.
Frequently Asked Questions
What intent categories should a cold email reply classifier use?
Ten covers real traffic: (1) meeting request, (2) interested now, (3) interested later with timing, (4) question or objection, (5) referral to a colleague, (6) soft no, (7) hard no or remove me, (8) out of office, (9) automated or irrelevant (receipts, tickets, newsletters), (10) hostile or legal threat. Each maps to a distinct action, from paging a rep instantly (1-2) to permanent suppression plus human review (10). Positive/negative loses exactly the distinctions that drive different actions.
How do I detect referral replies like 'talk to Sarah' automatically?
Make referral a first-class category and have the model extract structured fields: referred person's name, their role or email if stated, and whether the referrer wants to stay copied. 'Speak with Sarah in procurement' then triggers a flow: create the new contact, find their email (enrichment waterfall if not given), start a referral sequence that references the colleague by name, and thank the referrer. Referral replies convert at multiples of cold outreach and most systems file them as neutral, which is throwing away the warmest leads cold email produces.
Which model should I use for reply classification?
A fast, cheap model handles this well, classification of short business emails is not a frontier problem. Claude Haiku-class or GPT-mini-class models deliver strong accuracy at fractions of a cent per reply. Spend your effort on the prompt and the output contract instead: full message context in, strict JSON out with category, confidence, extracted fields, and a one-line summary. Upgrade the model only if your human-review queue shows systematic misses.
How do I handle replies the AI is unsure about?
Require a confidence score in the output and set a threshold, typically 0.8. Above it, the automation acts. Below it, the reply lands in a human review queue with the model's best guess pre-selected, so review takes seconds. Track the disagreement rate: when humans keep overriding a specific category, that is prompt feedback, add the edge case to the prompt's examples and the queue shrinks. This human-in-the-loop design is what makes it safe to run unattended.
What context does the classifier need beyond the reply text?
Send it the whole picture: the reply, the original outbound message, the prospect's name and company, and which touch of the sequence this was. 'Yes, sounds good' is unintelligible alone but obvious under the question it answers. Full-context classification is where accuracy jumps, and it also lets the model extract timing ('reach out in Q4') and sentiment shifts across a thread rather than judging one message in a vacuum.
Get FREE .com Domains
Prewarmed Inboxes So You Can Send Cold Emails Today
Get Prewarmed & Fresh InboxesPrewarmed & Fresh Inboxes
Google, Microsoft 365 and Azure inboxes with free prewarmed domains and DNS configured before you log in. Prewarmed inboxes send the day they land; every order is delivered within 48 to 72 hours.
Get Prewarmed & Fresh Inboxes