Large Language Models

Large language models (LLMs) predict tokens conditioned on context. Training across large corpora produces useful language and reasoning behavior, but generation remains probabilistic and does not guarantee truth, consistency, or intent.

Transformer Model

The original Transformer paper replaced recurrence with attention. Modern decoder-only LLMs repeatedly apply token embeddings, positional information, self-attention, feed-forward transformations, residual connections, and normalization, then produce a probability distribution for the next token.

Scaled dot-product attention is:

Attention(Q, K, V) = softmax(QKᵀ / √d) V

Multiple heads learn different projections. Causal masking prevents a token from attending to later tokens during autoregressive generation. Attention cost and memory generally grow strongly with sequence length, though implementations use many optimizations.

Tokens and Context

Tokenizers map text to integer units; boundaries vary by language, whitespace, and code. Token counts affect cost and available context. A context window is working input, not durable memory, and long context does not imply equal attention to every detail.

Structure context, put authoritative data near the task, remove irrelevant history, and measure performance at realistic lengths.

Training and Adaptation

Pretraining learns next-token or related objectives. Instruction tuning teaches task-following examples; preference optimization shifts behavior toward evaluated responses. Fine-tuning changes weights and is useful for stable behavior or domain patterns, not for frequently changing factual knowledge.

Parameter-efficient tuning updates a small subset or added parameters. It reduces training resources but still needs representative data, evaluation, provenance, and serving support.

Inference

Temperature and sampling change output diversity, not factual reliability. Greedy or low-temperature decoding can still be wrong. KV caching avoids recomputing prior attention state; batching improves accelerator use but adds queueing; quantization reduces memory and may change quality.

Measure end-to-end time to first output, generation rate, total latency, throughput, memory, failures, quality, and cost. Vendor benchmark numbers rarely match your prompts and concurrency.

Limitations

• plausible fabrication and unsupported claims;
• sensitivity to wording, ordering, and irrelevant context;
• finite context and imperfect recall;
• inconsistent counting, calculation, and constraint following;
• training-data bias and unknown provenance;
• prompt injection when untrusted text shares the instruction channel;
• non-determinism and behavior changes across model versions.

Ground claims in sources, use deterministic tools for calculations and validation, constrain outputs with schemas, and keep high-impact decisions under an appropriate human or rule-based control.

Model Selection

Evaluate candidate models on the same representative set for task quality, safety, latency, context needs, structured-output reliability, tool use, supported modalities, deployment constraints, data policy, version stability, and total cost. Route between models only when the added operational complexity has measured value.

Checklist

• Does evaluation match real prompts, languages, and edge cases?
• Are factual claims grounded or independently checked?
• Are context, output, latency, and spend bounded?
• Are model and prompt versions recorded?
• Can provider or model changes be regression-tested and rolled back?