Performance Engineering¶

Performance engineering makes a system meet measurable workload objectives with acceptable cost. Optimize only after measuring a representative end-to-end path; faster code in a non-bottleneck does not improve the user outcome.

Define the Objective¶

Specify:

operation or user journey;
workload mix, arrival pattern, concurrency, and data shape;
latency distribution, throughput, and error threshold;
resource and cost limit;
environment and measurement point;
steady-state and recovery requirements.

Example: “At 400 requests/s with the production read/write mix, 99% of eligible responses complete within 300 ms, errors stay below 0.1%, and no resource exceeds its safe saturation limit for ten minutes.”

Latency, Throughput, and Utilization¶

Averages hide tails. Report percentiles, distributions, and worst bounded intervals. State whether latency includes queueing and client/network time.

Little's Law relates stable averages:

concurrency = throughput × time in system

At 200 requests/s and 0.25 seconds average time, about 50 requests are in the system. As utilization approaches a constrained resource's capacity, queueing delay can rise sharply.

Measurement Method¶

reproduce the important workload;
establish a baseline and variance;
identify the limiting resource or wait;
form one causal hypothesis;
change one relevant factor;
compare under the same conditions;
verify correctness and cost;
retain the regression check if valuable.

Warm-up, caches, compilers, garbage collectors, power management, co-tenancy, network location, dataset size, and background work can distort results. Record them.

For web clients, standard timing APIs such as Resource Timing help separate network phases from application observations.

Profiling¶

Use the profiler matching the suspected resource:

CPU sampling: where execution time is spent;
allocation/heap: what allocates and retains memory;
off-CPU/blocking: locks, I/O, scheduler, and waits;
database plans and wait events: query and contention cost;
distributed traces: latency across service boundaries;
continuous profiles: resource changes across versions and traffic.

Profiles are evidence of a specific workload. A hot function may be necessary work, and percentage samples can rise when unrelated work becomes faster.

Python includes adequate first tools:

python -m cProfile -s cumulative app.py
python -m timeit -s 'data=list(range(1000))' 'sum(data)'

Microbenchmarks isolate an operation but omit system effects. Prevent dead-code elimination where relevant and confirm results in the real path.

Load Tests¶

baseline tests establish comparison;
load tests validate expected demand;
stress tests find saturation and failure shape;
spike tests test sudden demand and scaling lag;
soak tests reveal leaks and degradation;
capacity tests estimate safe operating limits.

Generate load from enough independent capacity, use realistic connection reuse and think time, and confirm the load generator is not saturated. Validate response correctness; a server returning fast errors is not succeeding.

Track offered versus completed rate, latency, errors, queue depth, oldest work age, CPU, memory, I/O, locks, connections, downstream limits, and recovery after load stops.

Bottleneck Patterns¶

Reduce Work¶

Remove unused queries, fields, serialization, copies, logs, polling, and repeated computation. Algorithm and data-model improvements usually beat low-level tuning.

Batch¶

Batching amortizes round trips and fixed costs but increases buffering delay, memory, and retry scope. Bound batch size and wait time.

Cache¶

Cache repeated expensive reads only with a defined key, source of truth, freshness, invalidation, size, stampede behavior, and failure mode. Measure hit ratio and end-to-end latency; cache misses may become slower.

Concurrency¶

Concurrency hides independent waiting; parallelism uses multiple compute resources. Bound both. More workers can increase context switching, lock contention, memory, and downstream overload.

Connection Pools¶

Pools amortize connection setup and cap concurrency. Size them from downstream capacity and request behavior, not instance CPU count alone. Measure wait time separately from execution.

Data Access¶

Use query plans and actual cardinality. Avoid N+1 access, unnecessary columns, large offsets, missing or unused indexes, and long transactions. Partitioning and replicas introduce new routing and consistency costs.

Compression¶

Compression trades CPU for network or storage. Choose by payload, link, cacheability, and client cost; precompress stable assets when useful. Do not compress tiny or already compressed data blindly.

Overload and Back-Pressure¶

Bound queues, request bodies, concurrent work, retries, memory, and connections. Reject excess work early, protect higher-priority operations, and propagate back-pressure. Autoscaling reacts after delay and cannot repair a serial dependency.

Retry storms amplify overload. Use deadlines, limited attempts, jitter, idempotency, and one retrying layer.

Performance and Correctness¶

Optimizations can change ordering, consistency, precision, security, accessibility, and failure behavior. Keep correctness tests and compare outputs. A relaxed guarantee must be an explicit product decision.

Checklist¶

Is the target tied to a representative workload and user outcome?
Are tails, errors, saturation, recovery, and cost measured?
Is the bottleneck demonstrated rather than guessed?
Is load generation itself healthy?
Are queues and concurrency bounded?
Did the change preserve correctness and improve the end-to-end path?
Is the improvement guarded against regression where worthwhile?