Function calling (or tool calling) is how a language model turns a request into an action: it outputs a structured call — a function name and arguments — and your code executes it. LLM function calling security comes down to one principle: a tool call produced by a model is untrusted input. It may be wrong, manipulated, or perfectly valid, and your code has to tell the difference before anything runs.
Why function calls need validation
A model choosing to call issue_refund(order="A-2210", amount=500) tells you what it decided — not whether it should happen. The arguments might be hallucinated, the order might belong to another customer, or the call might come from prompt injection hidden in content the model read. Structured outputs guarantee the shape of a call. They say nothing about whether it's allowed.
Four layers of LLM function calling security
1. Schema validation
Check types, required fields and formats. Reject unknown fields. This catches malformed calls and is the minimum every tool should do — but it's only the start.
2. Semantic validation
Check that arguments make sense: the order exists, belongs to the customer in the conversation, and the amount doesn't exceed what was paid. Never let the model supply identifiers it shouldn't control, such as another user's ID.
3. Policy checks
Check whether this action is permitted right now: under the agent's limit, within daily totals, in the right environment, for the right customer tier. These rules belong outside your tool code, where process owners can read them — see AI agent permissions.
4. Approval for high-impact calls
For actions that are irreversible or above a threshold, the right answer isn't yes or no — it's "ask a person". See human in the loop for AI agents.
Patterns that help
- Bind identity server-side. Take the user and customer IDs from your session, not from model arguments.
- Prefer narrow functions.
refund_current_order(amount)is safer thanrefund(order_id, amount, account). - Return clear denials. When a call is denied, return a structured reason so the agent can explain or choose another path, instead of retrying blindly.
- Make writes idempotent. Retries shouldn't double a payment.
- Log the decision, not just the call. Record what was requested, what was decided and why — your AI agent audit trail.
Centralise the checks
When every tool wrapper implements its own limits, nobody knows which version is live, and changing a threshold means a deploy. A single policy checkpoint in front of all tool calls gives you one place to define, review, test and audit those rules. The same applies to tools reached through MCP servers.
Where Gaten fits
Gaten is that checkpoint: your tool wrapper (or a gateway) sends each call for a decision before executing it, and gets back allow, deny or require approval with a reason. See how it works.
Frequently asked questions
Is function calling safe?
Function calling itself is a format. Safety depends on how you validate and authorise each call before executing it.
Do structured outputs prevent malicious tool calls?
No. They ensure a call is well-formed, not that it's appropriate. You still need semantic checks and policy.
Where should tool-call validation live?
Schema and semantic checks belong with each tool. Permission rules are best centralised in a policy layer so they can be reviewed, versioned and audited in one place.

