Handling sales objections with an AI voice agent: build an objection library

The best approach to handling sales objections with an AI voice agent is a small library of short response patterns, organised by the type of objection, plus a clear rule for when to stop. The agent acknowledges first, answers once from facts you supplied, then either moves on or lets the person go.

This is different from the old model of memorising rebuttals. On a phone call an agent that argues sounds worse than one that listens, and the aim is to find out whether there is a real fit, not to win. If you want the rest of the prompt structure first, read how to write an AI voice agent prompt.

What an objection actually is

Most objections are one of four things in disguise:

• A brush-off. "Send me an email." "Not now." The person wants the call to end and has no strong view about you.
• A genuine information gap. "How much is it?" "Do you cover my area?" They are interested and need a fact.
• A real mismatch. "We already use something." "We're too small." Sometimes that means no fit, sometimes it opens a useful question.
• A trust problem. "Who are you?" "How did you get my number?" "Is this a robot?" These come first and have to be dealt with before anything else.

Treating all of them with the same rebuttal is the usual mistake. Sorting them is the first step in building a library.

Building the library

Collect the real objections

Do not invent them. Pull them from three sources: your best salespeople, your CRM notes on lost leads, and the first batch of call transcripts after launch. Write each one the way people actually say it, in their words.

Sort them into four groups

Using the four types above gives you structure. For each objection, write a response pattern, not a script. A pattern has three parts:

1. Acknowledge in a few words so the person feels heard.
2. Give one useful fact or ask one clarifying question.
3. Return to the goal or exit.

Here is how that looks for common cases (illustrative wording you would adapt to your offer):

Objection	Type	Response pattern
"I'm busy right now."	Brush-off	Acknowledge, offer a specific callback time, confirm it, end.
"Just send me some information."	Brush-off or interest	Agree, ask what they most want to know so the message is relevant, then offer a short call as an option.
"How much does it cost?"	Information gap	Give the range or starting price from the knowledge base. If none is given, say a person will confirm and offer to book that conversation.
"We already have a supplier."	Mismatch	Acknowledge, ask one question about what would make them look at alternatives, accept the answer.
"Is this a robot?"	Trust	Say yes, you are an AI assistant calling for the company, offer to continue or arrange a person to call.
"How did you get my number?"	Trust	State the source honestly (they enquired, they are on a list you were given), offer to remove them if they prefer.
"Not interested."	Brush-off or mismatch	Accept it. Thank them and end the call.

Keep responses short

Voice responses should be one to three sentences. If an answer needs a paragraph, it belongs in a follow-up message or a human conversation. Long rebuttals on the phone sound like pressure.

Where the library lives

Split the work between the prompt and the knowledge base:

• In the prompt: the rules of engagement. Acknowledge before answering. Make at most one attempt to move the conversation forward after an objection. Never argue. Treat "stop calling" as an instruction, not an objection.
• In the knowledge base: each objection and its response pattern, written as a short entry with the objection phrase and a few variants people use.

This keeps the prompt readable and lets you add or edit objection responses without risking the core behaviour. It also makes it easy to see gaps: if you cannot find an entry for something a person just said, you know what to add. For guidance on structuring the content itself, see the general advice in improve your AI agent with call transcripts.

Examples of good handling

Price early. The caller asks about cost before the agent has said anything about value.

"Fair question. Plans start from around forty pounds a month depending on size. Would it help if I asked two quick questions so I can say which one would apply to you?"

This answers honestly, avoids dodging, and moves toward qualification. If the agent had no price to share, it would say so and offer a person.

Already using a competitor. Do not run down the other option.

"That makes sense, a lot of people do. Is there anything about it you would change if you could?"

If the person names something, the agent has a genuine opening. If they say no, it accepts and ends the call. The question is useful because the answer decides what happens next.

Trust first.

"Good question, I'm an AI assistant calling on behalf of Harbour Dental. You asked about a quote on our website this morning. Would you like me to carry on, or would you prefer a person to call you?"

Honest, short and gives control back to the person. More on disclosure in disclose AI on calls.

Knowing when to stop

An agent should not try to convert every objection. A good rule set:

• After one response to an objection, if the person repeats it, accept it.
• If the person says no, or uses clear language like "not interested" twice, end the call.
• If the person asks to be removed or not called again, confirm, apologise briefly, end, and make sure the number is excluded from future calls. This is a compliance matter as well as a courtesy; see do not call lists and dialer hygiene.
• If the person becomes upset, or the objection is complicated (contract terms, a complaint, a negotiation), hand off to a human, as described in warm transfer from AI to human.

The compliance material in this article is general information, not legal advice. Check the current rules where you call.

Maintaining the library

An objection library is never finished. Review transcripts weekly in the first month, then monthly. Look for:

• Objections that appear often and have no entry.
• Entries where the agent's answer led to a hang-up.
• Objections that really signal a problem upstream, such as people saying "I never asked for this", which is a lead source problem, not a script problem.

Remove entries that never come up. A small library that matches reality beats a long one nobody maintains.

Try it

You can set up an objection list as part of an agent's knowledge base and test it by calling yourself and raising each objection in turn. Purple Voice has a Free plan with no card needed, so you can build a first version and see how the agent handles it. Start at voice.purplerocket.co.