Skip to content

Prompt and Context Engineering

A prompt defines a model task; context engineering selects the instructions, evidence, tools, state, and examples available for that task. Treat both as versioned application code whose behavior is measured, not as magic wording.

A Clear Prompt

State:

  1. objective and intended user;
  2. authoritative context and delimiters;
  3. constraints and policy;
  4. required output contract;
  5. what to do when information is missing;
  6. a few representative examples only when they clarify ambiguity.
Task: classify the support request.
Allowed labels: billing, access, bug, other.
Return JSON matching {"label": string, "reason": string}.
Use only the request inside <request>. If uncertain, choose other.
<request>{{untrusted_request}}</request>

Delimiters clarify structure but do not secure untrusted input.

Context Selection

More context can increase latency, cost, distraction, and attack surface. Include the smallest authoritative material needed for the current decision. Prefer structured state and source excerpts over entire conversation history. Put stable shared prefixes together when provider caching behavior makes that useful, but measure it.

Never rely on context for durable authorization or workflow state. Store those in application code and databases.

Outputs

Use schema-constrained output when the API supports it, then validate types, bounds, enums, relationships, and authorization. A valid JSON object can still contain an unsafe action or fabricated value.

Separate generated display text from machine decisions. Use deterministic code for arithmetic, identifiers, access checks, and irreversible state transitions.

Sampling

Temperature and related sampling controls affect diversity. They do not provide a universal “creativity” scale or confidence measure. Tune them with task evaluation; use multiple samples only when the quality gain justifies added cost and selection bias.

Prompt Injection

Prompt injection occurs when untrusted content influences model instructions. No prompt phrase reliably prevents it.

Defend with architecture:

  • minimize tools and permissions;
  • separate trusted instructions from untrusted data;
  • validate actions with deterministic policy;
  • require approval for high-impact mutations;
  • filter and bound retrieved content;
  • isolate secrets from model-visible context;
  • test adversarial inputs and tool outputs;
  • log attempted policy violations safely.

Evaluation

Keep a representative dataset with expected outcomes, important slices, adversarial cases, and prior failures. Measure task correctness, constraint following, abstention, safety, latency, and cost. Compare prompt and model versions on the same cases before rollout.

Human review is needed where quality is subjective or consequential. Model-based judges can assist but may share biases with the system under test; calibrate them and retain independent checks.

Production

Version prompt templates, model configuration, tools, schemas, and retrieval settings. Bound time, tokens, retries, and spend. Retry only transient API failures; a different generated answer is not a safe retry for a side effect. Cache only when authorization, freshness, privacy, and nondeterminism semantics allow it.

Checklist

  • Is the task and missing-information behavior explicit?
  • Is context minimal, authoritative, and correctly permissioned?
  • Is output schema and domain logic validated?
  • Are security controls outside the prompt?
  • Does a stable evaluation set catch regressions?
  • Are version, latency, tokens, cost, and failures observable?