Llama 4 Scout and Maverick have been public since April 2025, but “tool calling” does not mean that downloading weights produces a safe agent. The model generates a candidate call. A chat template and inference runtime encode and parse it. An application gateway checks authority. Only a registered function creates an effect. Local deployment changes where these layers run; it does not make any layer optional.

Select a model the machine can actually serve

Meta describes Scout as having 17 billion active parameters and 16 experts, while Maverick has 17 billion active parameters and 128 experts. Total parameters, weight precision, KV-cache size, and context length all contribute to memory. Active parameters alone are not a hardware requirement.

The official single-H100 claims refer to specific model, host, and quantization configurations. They are not a promise that an ordinary consumer GPU reproduces the same setup. Measure loaded weights, peak KV cache, time to first token, throughput, and the context length your tasks really use. Treat the published maximum context as a capability boundary, not a production default.

Record the model-card revision, license terms, quantization source, runtime version, and chat template. If hardware requires a shorter context or heavier quantization, evaluate the resulting quality rather than hiding the change under the same model label.

Keep four layers explicit

Layer one is the model plus its official chat template. Layer two is an inference runtime that correctly implements that template. Layer three is a harness that parses structured proposals and maps them to registered functions. Layer four is an execution gateway holding credentials and authorization policy.

Never feed model output to eval, a shell, raw SQL, or an unrestricted HTTP client. A tool registry should contain a fixed name, a strict input schema, and a local handler. Reject unknown names. Return a structured error on parse failure. Constrain string lengths and formats. Resolve paths beneath an allowed workspace and verify the final path. Allowlist outbound hosts and parameterize database queries.

This separation makes failures diagnosable. If a model emits malformed syntax, that is not an executor error. If the parser accepts an argument but tenant ownership fails, that is a policy denial, not a reason to retry the model until it slips through.

Isolate permissions from model context

The model never needs an API token. The executor acquires short-lived, least-privilege credentials for the current user and task. Split reads from writes. Deletion, publication, purchase, and permission changes wait for human confirmation after the exact arguments are known.

An approval record contains call id, tool, redacted argument summary, approver, and expiration. Changing a destination or amount invalidates approval. A reviewed plan does not authorize altered runtime parameters.

Local files can contain prompt injection just like web pages. Mark retrieved text as untrusted data and prevent it from changing the tool set, system policy, or approval rules. A model proposal remains a proposal even when it cites a local “instruction” file.

Build a recoverable loop

Persist model id, quantization, template version, prompt hash, call ids, and results. A transient read may retry within a small budget. A write uses a domain idempotency key and reconciles external state before another attempt. Stop after an identical failure repeats. Bound steps, tokens, time, and tool calls, and expose cancellation.

Map exceptions to a small vocabulary such as invalid_arguments, permission_denied, temporary_failure, and conflict. Return only fields needed for the next decision. Internal stack traces, credentials, and full records do not belong in model context.

For deterministic testing, the executor can run against mock handlers. Those handlers should implement the same validation and idempotency contract as production, including simulated timeout and partial success. This tests orchestration without granting real authority or consuming a hosted API.

Evaluate trajectories, not chat impressions

An offline suite should cover normal reads, missing arguments, cross-tenant objects, path traversal, prompt injection, tool timeout, duplicate writes, restart, and cancellation. It is useful to score whether the model chose the right tool. It is essential to prove the gateway remains safe when the model chooses the wrong one.

When comparing Scout, Maverick, quantizations, or runtimes, freeze the template, tools, sampling settings, and budgets. Report task success and policy violations separately. A higher completion rate cannot compensate for unauthorized effects.

The advantage of local tool calling is control over data movement and execution, not automatic trust. Treat Llama 4, the inference runtime, the parser, and the permission gateway as replaceable modules with explicit contracts. Then one layer can be upgraded without silently changing the others. The real success criterion is not that the model can call tools; it is that the system stays within authority when the model calls the wrong tool.