How to Use Claude to Flag Phishing Attempts in Business Email

The short answer: Claude can read a suspicious email the way a trained analyst would — authentication results, sender history, the writing itself, and where the links actually resolve — and return a structured verdict with its reasoning in seconds. The right place to run it is a "report suspicious" mailbox that triages what staff flag. It is a supplement to a mail security gateway, never a replacement: no sandbox, no quarantine, no post-delivery clawback.

Most small and mid-sized companies have exactly one line of defence against phishing: whatever Microsoft 365 or Google Workspace filters out by default, plus the judgement of whoever happens to be reading the message. That works against bulk spam and known-bad domains. It works considerably less well against the message that costs real money — a short, plausible, payload-free email from someone the recipient recognises, asking for something that sounds like a Tuesday. This article is about a narrow, genuinely useful thing you can build with the Claude API in an afternoon, and about the much larger set of things it cannot do.

What Actually Gets Past a Standard Spam Filter

Business email compromise has nothing to detect. No attachment, no malicious link, no malformed header — just text. "Are you at your desk? I need you to handle a supplier payment before the cut-off." A filter tuned to find malware finds nothing, because there is nothing there except intent.

Display-name spoofing survives authentication. SPF, DKIM and DMARC verify the sending domain, not the human name shown in a mail client. An attacker sending from a domain they legitimately own, with all three checks passing, can still put your finance director's full name in the From field. On a phone, the domain is often not shown at all.

Compromised legitimate accounts pass every check. When a real supplier's mailbox is taken over, the mail arrives from the real domain, authenticated, often inside a real existing thread. Reputation-based filtering is structurally blind to this — the reputation is genuine.

Lookalike domains are registered fresh and used once. A domain registered forty-eight hours ago has no bad reputation yet because it has no reputation at all. By the time it is blocklisted, the campaign is over.

Thread hijacking borrows your own context. The attacker replies into a copied real conversation, quoting real earlier messages. Everything that would normally reassure a recipient — the subject line, the history below, the familiar names — has been supplied by the attack itself.

What all five share is that the signal is contextual rather than technical. A language model is not a better spam filter, but it is genuinely good at exactly the kind of judgement those five require.

How an AI Phishing Check Actually Works

What to Feed the Model — Headers, Sender History, Body, and Link Destinations

The verdict is only as good as the evidence you assemble before the API call, and much of that evidence should be gathered deterministically rather than judged by the model. Parse the Authentication-Results header yourself and pass the SPF, DKIM and DMARC outcomes as plain facts. Extract the true From domain and compare it, in code, against your directory and your contact history — "this display name matches an internal executive, but the domain has never sent us mail before" is a fact worth more than any amount of stylistic analysis.

Resolve every link. Take the href, not the anchor text, follow redirects without executing anything, and pass the final destination plus the registrable domain and its registration age if you have a lookup available. Give the model the plain-text body, the subject, the recipient's role, and whether the message is a reply to a thread that genuinely exists in the mailbox. Then ask for structured output — a JSON verdict with a risk rating, the specific indicators found, the ones explicitly checked and absent, and a one-paragraph explanation a non-technical colleague can read. Instruct it to mark anything it cannot determine as unknown rather than guessing; a confident wrong verdict is worse than an honest gap.

Build Options: A Reporting-Mailbox Triage Bot vs an Inline Gateway

There are two shapes this can take, and only one of them is a sensible do-it-yourself project. The inline version sits in the mail flow, inspects everything before delivery, and blocks or quarantines. Building that yourself means owning message delivery for the whole company — every timeout, rate limit, model outage and false positive becomes lost or delayed business mail, and you now need spooling, failover, and a way to release quarantined messages at two in the morning. That is a product, not a script, and mature ones already exist.

The triage version sits beside the mail flow. Staff forward or report a suspicious message; the bot analyses it and answers. Nothing is blocked, nothing is delayed, and the worst failure mode is a slow reply. It also targets precisely the messages that got through everything else, which is where the residual risk lives. Start here, and stay here unless you have a specific reason not to.

A Practical Setup — The "Report Suspicious" Mailbox Triage Workflow

Give people one obvious action. A shared mailbox such as phishing@yourcompany.com, plus the built-in report button in Outlook or Gmail if you use it, and a single instruction: if it feels wrong, report it, and you will get an answer. Reporting has to be easier than deciding.

Capture the original, not a forward. A plain forward destroys the headers you need. Ask staff to report as attachment, or use the platform's own reporting mechanism, or pull the message via the Microsoft Graph or Gmail API using the message ID so the raw source is intact.

Enrich before you ask. Run the deterministic checks — authentication results, first-contact detection against your sent-mail history, link resolution, domain age, whether the display name collides with a directory entry — and hand Claude a structured evidence package rather than a wall of text.

Answer the reporter in minutes, in plain language. "This is very likely a payment-redirection attempt. The display name matches your finance director, but the domain yourc0mpany-finance.com was registered nine days ago and has never contacted us. Do not reply. IT has been notified." That reply is what makes staff report the next one.

Route by severity, and keep a human on the high end. Low-risk verdicts can close themselves with an explanation. Anything scored high should page IT, and the model's job there is triage and evidence-gathering, not decision-making.

Log everything and grade it weekly. Store the evidence package, the verdict, and what turned out to be true. Twenty minutes a week reviewing disagreements will improve the prompt faster than any amount of upfront design, and it gives you a real false-positive and false-negative rate instead of a feeling.

Feed confirmed attacks back into your controls. A confirmed malicious domain should end up in the tenant block list and, if others were targeted, trigger a search across mailboxes. The analysis is only worth what you do with it.

AI-Assisted Triage vs a Managed Email Security Gateway

• Catching BEC and social-engineering text — AI triage is genuinely strong here, and it is the gap most gateways were historically weakest at. Modern gateways have added their own AI-based impersonation detection, so this is a narrowing advantage rather than a permanent one.
• Blocking before delivery — Only a gateway does this. A triage bot answers after the message is already in the inbox, which means the risk window is however long it takes an employee to decide to report it.
• Attachments and malicious links — The gateway wins outright. Detonating a file in a sandbox and rewriting URLs so they are re-checked at click time are engineering capabilities, not reasoning capabilities. A model reading a filename learns nothing about what the file does.
• Post-delivery removal — Only a gateway or the mail platform can reach into every mailbox and pull a message that turned out to be malicious after delivery. Nothing you build with an API can do that.
• Explaining the verdict to a non-technical user — AI triage wins clearly. Gateways produce quarantine notices; a model produces a paragraph the recipient actually understands, which is worth more for staff awareness than most training modules.
• Cost and effort — A triage bot is a few days of integration work and cents per analysis. A managed gateway is a recurring subscription and a supplier relationship. They price differently because they are not the same control.
• What happens when it fails — A triage bot that breaks is silent and annoying. A self-built inline filter that breaks stops company mail. This asymmetry is the whole argument for the triage shape.

Why This Is a Supplement, Not a Replacement

There is no sandbox. The model cannot open an attachment in an isolated environment and watch what it does. If the answer requires detonation, the answer is not available.

There is no clawback. When a message is confirmed malicious an hour after delivery, removing it from forty mailboxes is a platform function. Your bot can recommend it; it cannot perform it.

There is no queue. When your integration is down, nothing is protected and nothing is spooled. A gateway is built to hold mail; a script is not.

The model can be socially engineered too. Prompt-injection text inside an email body — instructions addressed to the analysing system rather than the human — is a real and obvious attack against this design. Treat email content as untrusted input, keep it clearly delimited from your instructions, never let the model's output trigger an action on its own, and assume someone will eventually try.

A confident false negative is worse than no tool. If staff learn that the bot says "looks fine," they will stop reading carefully. Make uncertainty visible in the reply, and never let a low-risk verdict read as an all-clear guarantee.

Getting This Right — Mail Data Governance, API Keys, and When to Bring in IT

Reported mail is some of the most sensitive data in the business. A phishing report often arrives with a real invoice, a real contract, or a real conversation attached, and forwarding it to a third-party API is a processing decision with contractual and regulatory weight — under Hong Kong's PDPO, Singapore's PDPA, or the PRC's PIPL, depending on where your people and data sit. Decide deliberately what leaves the tenant, whether you are using zero-retention API terms, and write the answer down before a client's security questionnaire asks for it.

Scope the credentials properly. The mailbox integration needs read access to one shared mailbox, not organisation-wide mail read permission, and application permissions in Microsoft Graph are broad by default — use an application access policy to restrict them to that mailbox specifically. Keep the Anthropic API key in a secrets manager rather than in the automation's configuration, rotate it on a schedule, and alert on spend anomalies: an unexpected bill is often the first visible sign that something is looping or has been abused.

This is the point where a managed partner earns its fee. Brocent's managed email security service covers the gateway layer this workflow sits beside — AI-assisted impersonation and BEC detection, attachment sandboxing, and the post-delivery actions no in-house script can perform. Our AI+ support practice builds and tunes the triage integration itself, and managed IT support owns the credentials, monitoring and incident response once it is running. If your gap is staff behaviour rather than tooling, our guide to phishing simulation programmes in Asia covers the training side of the same problem. Brocent has run managed IT and security engagements across Asia since our founding in Beijing in 2007, with headquarters in Singapore and a Hong Kong office since 2016.

Frequently Asked Questions

Can AI replace our email security gateway?

No, and it is worth being blunt about why. A gateway blocks before delivery, detonates attachments in a sandbox, rewrites URLs so they are re-checked at click time, and can remove a message from every mailbox after the fact. An API-based analyser does none of those things. It adds contextual judgement on top of a stack — it does not become the stack.

Does forwarding suspicious email to an AI create its own risk?

It creates a data-processing question you have to answer deliberately. The content is often sensitive, and it is leaving your tenant. Check your provider's data-retention and training terms, keep a record of the decision, and if you operate under PDPO, PDPA or PIPL, make sure the transfer is covered by whatever basis you rely on for other cloud processing.

How do we handle false negatives?

Assume they will happen and design the reply so that a low-risk verdict never reads as a guarantee. Log every analysis with its outcome, review disagreements weekly, and track the false-negative rate as a real number. If staff treat "looks fine" as permission to click, the tool has made things worse rather than better.

What about attachments and malicious links?

This is the clearest limit. The model can read a filename and the visible text of a link, and that tells it very little. Resolve link destinations in code before the analysis, and leave attachment inspection to a gateway with a real sandbox. Never let the model's assessment of an attachment stand in for scanning it.

Can an email attack the AI that is reading it?

Yes — prompt injection is the obvious attack on this design. An attacker can write instructions into the body aimed at your analyser. Delimit untrusted content clearly, never let model output trigger an action without a human or a deterministic rule in between, and test your own prompt against injection attempts before you trust it.

Which mailbox should it watch?

One shared reporting mailbox, and only that one. It keeps the permission scope small, it keeps the data footprint small, and it targets the messages that already got past every other control — which is exactly the population where extra judgement is worth paying for.

Where to Start

Build the read-only version first: pull reported messages, run the deterministic enrichment, get a verdict, and post it into a private IT channel where nobody outside the team sees it. Run that for two weeks against real reports and grade every one. You will learn more about your actual phishing exposure in those two weeks than from any vendor report, and you will know whether the replies are good enough to send to staff. Only then turn on the automatic reply. If you would rather have the gateway, the triage layer and the incident process built as one thing, get in touch.