Scenario 01 · Customer Support Agent

~14 min read · interactive · 3‑question self‑check

Prompt vs Hook The 100‑request experiment.

A refund agent that verifies identity ninety‑two times out of a hundred is not a refund agent. It is a compliance incident waiting for an audit.

Prompts are probabilistic. Production guarantees are not.

§ 01 · Setup

The constraint.

A customer‑facing support agent handles refund requests. Two tools are registered on the agent: get_customer(email, order_id), which returns a customer and order record if identity can be matched, and process_refund(order_id, amount), which issues the refund.

The rule is unambiguous. process_refund may only be called after get_customer has returned a verified match on the active conversation. No verified match, no refund. The rule is not advisory.

The reason is also unambiguous. Unverified refunds are fraud exposure, PCI drift, and a chargeback curve the business cannot absorb. A single refund issued to the wrong person costs materially more than several dozen correctly‑denied legitimate ones. That asymmetry is why the constraint has to hold one hundred percent of the time — not ninety‑five, not ninety‑nine.

The tempting framing — treat identity verification as an instruction the model will come to obey through clear enough prompting — reads correctly and fails the exam. The exam rewards answers that treat probabilistic behaviour as probabilistic, and route around it.

§ 02 · Evidence

One hundred requests. Twice.

We simulated one hundred refund requests against two agent architectures. Both use the same model. Both receive the same system prompt, with identical instructions to verify identity before issuing a refund. The only structural difference is that the hook‑based agent has one PreToolUse callback registered on process_refund.

Press run. Each grid fills left‑to‑right, one request per cell — green for verified refund, red for refund issued without verification.

Agent A

Prompt only

Verification rule in system instruction.

— / 100 unverified refunds

Agent B

Hook‑enforced

Same prompt + PreToolUse block.

— / 100 unverified refunds

Verified refund Unverified refund (slipped) Run 0

The prompt agent’s miss rate is not a model failure. It is the architecture operating exactly as designed — probabilistic inference responding to probabilistic signals. The hook agent is not smarter. It simply cannot reach process_refund without the verification having passed.

Run the experiment again. The prompt column’s tally changes. The hook column’s does not. That invariance is the point.

§ 03 · Why prompts leak

Soft constraints have a tail.

An LLM is a conditional probability engine over token sequences. A system instruction — you must call get_customer before process_refund — is one signal among many thousands the model integrates at each step of inference. The signal is strong. It is not a hard constraint.

Over a given population of requests you can narrow the miss rate with prompt work. Emphatic language, role framing, and priority tags help. Few‑shot examples of correctly‑verified refunds help. Structured reasoning scaffolds — before responding, list the tools you must call in order — help too.

None of them reach zero. The distribution has a tail. At ten thousand requests you will see misses. At a million, you will see them predictably. That tail is the engineering surface the business is actually exposed to, and the exam writer knows it.

A production log shows the agent skipped verification in 8% of refund requests. The tempting answers — add emphasis, add examples, retry with stronger instructions — all address the average case. The exam wants the answer that addresses the tail. Programmatic enforcement is the only intervention that closes it.

§ 04 · The pattern

What a hook actually is.

Hooks in the Claude Agent SDK are callbacks that fire at fixed lifecycle points. PreToolUse fires before a tool is invoked. PostToolUse fires after. They execute in your code, not the model’s — deterministic, debuggable, testable.

A hook receives the same context the model used to decide, plus the tool’s input (for PreToolUse) or its output (for PostToolUse). It returns a structured response: continue, block, or mutate. For the refund case, thirteen lines are enough.

// PreToolUse hook — fires before every tool invocation
const verificationHook = async ({ toolName, toolInput, session }) => {
    if (toolName !== 'process_refund') {
        return { continue: true };
    }

    const verified = session.metadata.verifiedCustomers?.includes(
        toolInput.customer_id
    );

    if (!verified) {
        return {
            continue: false,
            reason: 'Identity not verified. Call get_customer first.',
        };
    }

    return { continue: true };
};

agent.hooks.PreToolUse.push(verificationHook);

The hook runs every time the model decides to call process_refund, no exceptions. If verification hasn’t happened, the tool call is blocked and a structured reason is returned to the model. The model sees the block, adjusts, calls get_customer instead, succeeds, and retries the refund. From outside the system the agent appears to always verify first. From inside, the guarantee lives in a small, testable function.

Identify the decision point. Write the rule in code. Place the rule on the lifecycle event that fires before the decision takes effect. Prompt engineering, few‑shot examples, and reasoning scaffolds are all secondary reinforcement — they bias the model toward the right path but do not enforce it.

§ 05 · Pattern reach

A compliance layer, not a verification trick.

Hooks generalise. Anything that must be true at a decision boundary belongs in a hook, not in the prompt. Three examples, briefly.

Example 01

PreToolUse on process_refund

Refund thresholds

Block any amount above £500 unless an approval flag is set on the session. The model can ask for the refund; the business rule enforces the cap regardless.

Example 02

PostToolUse on final response

Policy gates

Scan the final response for excluded content — prescriptive medical advice, competitor names, internal identifiers. Mutate or force regeneration. The model never needs to know the policy exists.

Example 03

PostToolUse on get_customer

Data normalisation

Standardise phone numbers, strip soft‑deleted fields, flatten nested addresses before the model sees them. A whole class of downstream agent errors disappears at the source.

In every case the hook is doing work the model is bad at by construction: enforcing an invariant, not producing content. The division of labour is the point.

§ 06 · Handoff

When the hook can’t enforce.

Not every decision reduces to a rule. A customer escalates angrily over a billing dispute where no clear policy applies, or a request falls outside the agent’s permission scope. The agent has to hand the conversation to a human. The question is when, and the common wrong answer shows up on every version of this exam.

Sentiment‑based escalation — if sentiment score below −0.6, escalate — looks plausible and is a distractor on every version of this question. Sentiment classifiers are noisy, adversarially gameable, biased across demographics, and produce false positives on any customer who writes assertively. Model‑confidence‑based escalation is a similar trap: self‑reported confidence is not calibrated. Time‑based escalation punishes slow, legitimate work.

The triggers that hold up:

Explicit customer request. “I want to speak to a human.” “Transfer me.” “Escalate this.” Unambiguous, deterministic, no inference required.
Policy boundary hit. The requested action is outside the agent’s permission scope — closing an account, overriding a protected rule, accessing data from another tenant.
Retry loop exhausted. A required tool has failed the configured number of attempts. The agent cannot progress; a human must.
Critical error category. A security‑relevant tool failure, a data integrity violation, a permission error. These hand off unconditionally.

The escalation itself returns a structured response, not a transcript. Include: customer context snapshot, conversation summary, resolution path attempted, escalation category. The receiving human needs state they can act on — not a wall of chat history.

§ 07 · Self‑check

Three questions. Scroll after answering.

Progress

Use keys ABCD

Question 01 · Enforcement

A production log shows the agent skips identity verification in 8% of refund requests. Which intervention closes the gap?

Why B

Move the guarantee into deterministic code.

The model will always have a miss rate on rules it’s asked to enforce on itself. Prompt work narrows that rate but never closes it. A PreToolUse hook moves the guarantee into code: the tool call is blocked before it completes if verification state isn’t present on the session, and the block returns a structured reason the model uses to self‑correct.

APrompt emphasis narrows the average case; the tail stays.

CFew‑shot examples bias toward compliance but don’t enforce it.

DA retry loop runs the same probabilistic decision again.

Question 02 · Escalation

Which of the following is a valid trigger for handoff to a human support agent?

Why A

Only deterministic, unambiguous signals survive exam scrutiny.

An explicit customer request is a deterministic signal that requires no inference. The other three rely on classifiers or heuristics — all of which produce false positives, demographic bias, or outright gaming. The exam treats sentiment‑based, confidence‑based, and purely time‑based triggers as anti‑patterns.

BSentiment is noisy and demographically biased.

CModel self‑reported confidence is not calibrated.

DTime thresholds punish slow legitimate work.

Question 03 · Subagent contract

A subagent fails because of a transient timeout against a backend it depends on. What does a well‑designed subagent return to the orchestrator?

Why C

The return envelope is the subagent’s contract.

A subagent is an encapsulated unit called by an orchestrator. Its contract with the orchestrator is the return envelope. That envelope must carry enough state for the orchestrator to decide intelligently: the category of the failure, whether retry is useful, and any partial results. Strings, exceptions, and silent empties all destroy information the orchestrator needs.

AA generic string tells the orchestrator nothing.

BExceptions break the orchestrator’s control flow.

DSilence destroys the distinction between empty‑legitimate and failure.

Scenario 01 · Result

0/3

The takeaway

Prompts shape behaviour.
Hooks guarantee it.