A.A.
System Design

Networking Essentials

Networking Essentials

At its core, networking is about connecting devices and enabling them to communicate. Networks are built on a layered architecture (the so-called OSI model).

OSI (Open System Interconnection)

A conceptual framework. It serves as a guide for how network protocols and devices are designed and developed.

OSI is a 7-layer theoretical diagram/model that explains how networking works in general. It does not contain any executable code.

  1. Physical Layer
  2. Data Link Layer
  3. Network Layer
    • At this layer is IP, the protocol that handles routing and addressing. It is responsible for breaking data into packets, forwarding packets between networks, and providing best-effort delivery to any destination IP address on the network.
    • There are other protocols at this layer (like InfiniBand, which is used extensively for massive ML training workloads).
  4. Transport Layer
    • At this layer we have TCP, QUIC, and UDP, which provide end-to-end communication services.
    • Think of them as a layer that adds reliability, ordering, and flow control on top of the network layer.
  5. Session Layer
  6. Presentation Layer
  7. Application Layer
    • At the final layer are application protocols like DNS, HTTP, WebSockets, and WebRTC.
    • These build on top of TCP (or UDP, in the case of WebRTC) to provide a layer of abstraction for the kinds of data typically associated with web applications.

TCP/IP (Transmission Control Protocol / Internet Protocol)

TCP/IP is both a model and a functioning suite of actual protocols that make internet communication possible in the real world.

| TCP/IP layer | OSI layers | Examples | | --- | --- | --- | | Network Access / Link | 1, 2 | Ethernet, Wi-Fi | | Internet | 3 | IP, ICMP (diagnostics), ARP (address resolution) | | Transport | 4 | TCP, UDP | | Application | 5, 6, 7 | HTTP, HTTPS, DNS, FTP |

A Simple Web Request Lifecycle

  • DNS Resolution: Converts a human-readable domain name (e.g., hellointerview.com) to an IP address.
  • TCP Handshake: A 3-way handshake initiates the connection (SYN → SYN-ACK → ACK).
  • HTTP Request: The client sends the request (e.g., GET).
  • Server Processing: The server retrieves data and prepares a response.
  • HTTP Response: The server sends the requested content.
  • TCP Teardown: A 4-way handshake closes the connection (FIN → ACK → FIN → ACK).

Network Layer Protocols (Layer 3)

  • IP Protocol: Handles routing and addressing.
  • IPs are typically assigned by a DHCP server.
  • Public IPs are routable internet addresses allocated by an RIR (Regional Internet Registry).

Transport Layer Protocols (Layer 4)

UDP (User Datagram Protocol)

  • "Spray and pray" approach. Connectionless and fast, with low overhead.
  • No guarantees: Packets may be lost, duplicated, or arrive out of order.
  • Use cases: Speed is preferred over reliability (live video streaming, online gaming, VoIP, DNS lookups).

TCP (Transmission Control Protocol)

  • The workhorse of the internet. Connection-oriented (stateful stream).
  • Reliable: Guarantees ordered, error-checked delivery of data. Includes flow and congestion control.
  • Use cases: Anything where data integrity is critical (the default choice for most system designs).

QUIC

A modern protocol that provides TCP's reliability with modernized performance benefits. It is the basis for HTTP/3.

| Protocol | Connection | Guarantees | Typical fit | | --- | --- | --- | --- | | UDP | Connectionless | None (loss, duplication, reordering possible) | Streaming, gaming, VoIP, DNS | | TCP | Connection-oriented | Ordered, error-checked delivery + flow/congestion control | Most system designs | | QUIC | Connection-oriented (over UDP) | TCP-like reliability with faster handshake / HTTP/3 | Modern web traffic |

Application Layer Protocols (Layer 7)

HTTP / HTTPS

  • De-facto web standard. A stateless, request-response protocol.
  • Methods: GET, POST, PUT, PATCH, DELETE.
  • HTTPS: Adds TLS/SSL encryption for security in transit.

REST (Representational State Transfer)

API paradigm that performs operations on resources using standard HTTP methods. Simple, flexible, and the default for most public APIs.

GraphQL

Allows clients to request exactly the data they need. Solves under-fetching and over-fetching. Great for complex frontends, but adds backend complexity.

gRPC

High-performance RPC framework using HTTP/2 and Protocol Buffers (binary format). Strongly typed and highly efficient.

  • Use cases: Internal service-to-service communication.

SSE (Server-Sent Events)

Allows servers to push data to clients over a single, long-lived HTTP connection (unidirectional).

  • Use cases: Live notifications, auction prices.

WebSockets

Persistent, TCP-style connection for real-time, bidirectional communication. Initiated via an HTTP "upgrade".

  • Use cases: Chat apps, multiplayer games. Resource-heavy — use only when necessary.

WebRTC

Direct peer-to-peer communication between browsers (uses UDP). Uses STUN/TURN servers for NAT traversal.

  • Use cases: Audio/video calling.

Load Balancing

Vertical scaling (bigger servers) vs. horizontal scaling (more servers).

Client-Side Load Balancing

The client decides which server to contact by querying a service registry or directory.

  • Examples: Redis Cluster (gossip protocol), DNS.
  • Great for internal microservices, or when you control the clients.

Dedicated Load Balancers

  • Layer 4 (Transport): Routes based on IP/port without inspecting packets. Maintains persistent TCP connections. Very fast. Best for WebSockets.
  • Layer 7 (Application): Routes based on request content (URL, headers, cookies). Terminates incoming connections and creates new ones to the server. CPU-intensive but highly flexible. Best for HTTP/REST.

Health Checks

Load balancers monitor backend health (e.g., HTTP 200 OK) to automatically fail over and route around crashed servers.

Algorithms

Round Robin, Random, Least Connections (good for persistent connections), Least Response Time, IP Hash.

Common Deep Dives and Challenges

Regionalization and Latency

The physical distance between the client and server adds unavoidable network latency (constrained by the speed of light).

  • CDNs (Content Delivery Networks): Cache static assets (images, videos) at "edge locations" geographically close to users to minimize latency.
  • Regional Partitioning: Group data by geography (e.g., Uber hosting local city data on regional servers) so queries are handled close to the user.

Handling Failures

  • Timeouts and Retries: Reattempt a failed request. Must be paired with exponential backoff and jitter (randomness) to prevent a "thundering herd" from crushing a recovering server.
  • Idempotency: Ensure that retrying a request does not cause duplicated actions (like double-charging a credit card). Achieved with naturally idempotent methods (GET, PUT) or by passing an idempotency key.
  • Circuit Breakers: Prevent cascading failures.
    • Monitor external calls; if failures exceed a threshold, the breaker trips "Open" (instantly failing new requests to protect the struggling system).
    • Transitions to "Half-Open" after a timeout to test recovery. If the test passes, it closes and resumes traffic.