Network fundamentals¶
Networking lets processes exchange bytes across machines and failure boundaries. The useful model is the path a request follows, the guarantees each layer provides, and where to observe failure.
Use Module 4: Networks, APIs, and storage for a guided request trace.
A practical layer model¶
| Layer | Responsibility | Examples |
|---|---|---|
| Application | message meaning and behavior | HTTP, DNS, application protocols |
| Secure transport | peer authentication, confidentiality, integrity | TLS |
| Transport | communication between processes | TCP, UDP, QUIC |
| Internet | addressing and routing across networks | IPv4, IPv6, ICMP |
| Link | delivery on one local network | Ethernet, Wi-Fi |
| Physical | signals over a medium | copper, fibre, radio |
The OSI model divides some responsibilities more finely. Use either model to locate a problem; packets do not change behavior because a diagram has seven boxes instead of five.
Encapsulation adds protocol metadata as data moves down the stack. The receiver validates and removes corresponding metadata while moving upward.
Trace one request¶
URL
→ resolve host name to addresses
→ choose address and route
→ establish transport connection
→ establish TLS when HTTPS is used
→ send HTTP request
→ server accepts, processes, and responds
→ client parses response
Each arrow can fail or consume the timeout budget. “The API is down” is a symptom, not a layer diagnosis.
IP addressing and routing¶
An IP address identifies an interface in an IP network. Routers forward packets according to destination prefixes.
CIDR notation combines an address and prefix length, such as 192.0.2.0/24. The prefix identifies the network portion; remaining bits identify addresses within that prefix. IPv4 uses 32-bit addresses and IPv6 uses 128-bit addresses.
Important categories:
- loopback: reaches the local host;
- link-local: valid only on a local link;
- private/internal: not globally routed under the relevant address convention;
- public/global: potentially routable beyond the local network;
- unspecified or wildcard: used for binding or route semantics, not as an ordinary peer address.
An internal or private address is not an authorization control. Network position can reduce exposure but does not establish identity or permission.
Routing decides the next hop. It does not guarantee delivery, latency, bandwidth, or that the destination application is listening.
DNS¶
DNS maps names to typed records. A typical lookup may involve:
- application and OS caches;
- a configured recursive resolver;
- referrals through authoritative DNS hierarchy;
- one or more returned records cached for their TTL.
A name may resolve to multiple IPv4 and IPv6 addresses. Clients may try addresses in an order influenced by OS and network policy.
DNS commonly uses UDP for ordinary queries and TCP when required by response size, truncation, transfer, or policy. Modern encrypted DNS transports also exist; inspect the actual client and environment.
Common failure distinctions:
- name does not exist;
- resolver cannot be reached;
- response is stale in a cache;
- record exists but points to an unavailable service;
- one address family works while another does not.
DNS success proves only that a record was resolved, not that the application is healthy.
Transport protocols¶
TCP¶
TCP provides a reliable, ordered byte stream between endpoints.
- connection establishment confirms a reachable listening transport endpoint;
- retransmission handles some packet loss;
- flow control protects the receiver;
- congestion control adapts sending to network conditions;
- a write boundary is not preserved as a read boundary.
Applications must define message framing. A successful write can still be followed by connection loss before the peer processes the data.
UDP¶
UDP sends independent datagrams without connection-level delivery, ordering, retransmission, or congestion behavior supplied by TCP. Datagrams may be lost, duplicated, or reordered.
Use it when the application protocol provides the needed recovery or when late data is less valuable than missing data. “Lower overhead” is not enough; the application inherits more responsibility.
QUIC¶
QUIC is a secure transport built over UDP. It integrates TLS and supports multiple streams without imposing TCP's ordering across all streams. HTTP/3 uses QUIC.
Clients and servers still need application timeouts, retry rules, and observability. The transport does not make requests exactly once.
Ports and sockets¶
A transport endpoint combines protocol, address, and port. A listening socket accepts communication addressed to that endpoint; accepted TCP connections are separate sockets.
Binding to loopback accepts only local traffic. Binding to a wildcard address accepts traffic on matching host interfaces, subject to firewall and routing policy.
A socket is an OS communication abstraction. A WebSocket is an application protocol carried over an HTTP-established connection; the terms are not interchangeable.
TLS¶
TLS can provide:
- confidentiality against observers on the path;
- integrity against undetected modification;
- authentication of the server name through certificate validation;
- optional client authentication.
A client validates the certificate chain, validity period, requested host name, and applicable policy. Encryption without correct peer verification can protect a connection to the wrong party.
TLS does not define application authorization, make a compromised endpoint safe, or hide all connection metadata. Use maintained TLS libraries and current platform configuration rather than selecting protocol versions or cipher suites from static notes.
For current deployment guidance, consult the OWASP Transport Layer Security Cheat Sheet and your server/runtime documentation.
HTTP¶
HTTP is a request-response application protocol. Its semantics are independent of whether a particular connection uses HTTP/1.1, HTTP/2, or HTTP/3.
A request contains a method, target, headers, and optional content. A response contains a status, headers, and optional content.
Method properties¶
| Method | Intended use | Safe | Idempotent by HTTP semantics |
|---|---|---|---|
GET |
retrieve a representation | yes | yes |
HEAD |
retrieve headers corresponding to GET | yes | yes |
POST |
process or create under server semantics | no | no |
PUT |
create or replace the selected resource | no | yes |
PATCH |
apply a partial modification | no | depends on patch semantics |
DELETE |
remove the selected resource | no | yes |
Idempotent means repeated identical requests have the same intended effect, not that responses are identical or retries are always harmless. Authentication expiry, concurrent updates, and rate limits can change outcomes.
Status classes¶
2xx: request succeeded under the method's semantics;3xx: further action or a different representation location is involved;4xx: request cannot be fulfilled as sent under the current client context;5xx: server failed to fulfil an apparently valid request.
Choose specific codes as part of the API contract. Do not return success with an error hidden only in the body.
Headers and content¶
Headers carry representation metadata, caching rules, conditional requests, authentication credentials, and connection-independent protocol controls. Content-Type describes the representation sent; Accept describes what the client can process.
Set body, header, and time limits. Treat request data as untrusted even after TLS.
Connections, timeouts, and retries¶
A client timeout should cover explicit phases or one bounded total deadline:
- name resolution;
- connect;
- TLS handshake;
- sending request content;
- waiting for response headers;
- reading response content.
An expired client deadline does not prove the server stopped. Retrying a non-idempotent operation can duplicate effects. Use operation-specific idempotency keys or client-generated identities where required.
Retry only failures likely to be transient, with bounded attempts, backoff, jitter, and an overall deadline. Honor server retry guidance when appropriate. A retry storm can turn a partial failure into an outage.
Connection pooling avoids repeated setup but introduces lifecycle limits, stale connections, and per-destination capacity. Measure before tuning pool sizes.
Intermediaries¶
Firewalls and NAT¶
A firewall permits or denies traffic according to network and transport attributes, connection state, and sometimes application information. It reduces reachability; it is not a substitute for application authentication and authorization.
Network address translation rewrites address or port information and maintains mapping state. NAT changes reachability and complicates inbound connections; it is not inherently a security boundary.
Proxies and load balancers¶
- a forward proxy acts on behalf of clients;
- a reverse proxy acts in front of servers;
- a load balancer chooses among eligible backends;
- a gateway may terminate protocols or apply application policy.
When an intermediary terminates TLS, trust and identity propagation cross a new boundary. Accept forwarded client information only from explicitly trusted intermediaries and with a defined overwrite policy.
Health checks must reflect whether a backend can serve real requests. Load balancing cannot repair shared database failure or incompatible application state.
Caches and CDNs¶
HTTP caches reuse responses according to cache keys and freshness/validation rules. A CDN places shared caching and network services near clients.
Cache only data whose authorization and variation rules are represented in the cache key or policy. Incorrect caching can leak one user's response to another. Invalidation and stale behavior are part of correctness.
Browser boundaries¶
The browser same-origin policy restricts scripts from reading responses across origins. CORS headers let a server opt into selected cross-origin browser access.
CORS is enforced by browsers; it does not authenticate callers or prevent direct HTTP requests. Avoid reflecting arbitrary origins while allowing credentials.
Cookie-based authentication introduces CSRF considerations because browsers attach cookies automatically. JavaScript-accessible bearer tokens introduce different theft risks, including XSS. Choose the browser session architecture deliberately.
Streaming and server-initiated communication¶
| Mechanism | Direction | Good fit |
|---|---|---|
| Long polling | server response delayed, then client reconnects | compatibility and low event volume |
| Server-sent events | server to browser over HTTP stream | ordered text events and reconnect behavior |
| WebSocket | bidirectional message protocol | interactive two-way communication |
| Webhook | server makes HTTP request to another server | asynchronous event notification |
All require authentication, authorization, size limits, timeouts, backpressure, and reconnect/duplicate semantics. A persistent connection does not guarantee message delivery.
Troubleshooting by layer¶
| Question | Useful observation |
|---|---|
| Does the name resolve? | resolver query and returned addresses |
| Is there a route? | local route table and path diagnostics |
| Is the service listening? | socket/listener inspection on server |
| Can transport connect? | connection attempt with timing |
| Does TLS validate? | handshake and certificate diagnostics |
| What HTTP was exchanged? | verbose client output and server request log |
| Is loss or retransmission involved? | packet capture where authorized |
| Is the server slow after receipt? | request tracing, application metrics, query timing |
Start with curl -v, name-resolution tools, and listener inspection. Use packet capture only on systems and traffic you are authorized to inspect; encrypted payloads will not be visible without endpoint keys or instrumentation.
Record timestamp, client and server addresses, protocol, command, environment, and timeout. A successful ping does not prove TCP, TLS, HTTP, or application health.