Load Balancing and Network Planning Flashcards

Question 1

What is load balancing in a networked service?

Accepted Answer

Load balancing is the practice of distributing incoming requests or traffic across a pool of servers, links, or devices so that no single resource is overloaded. It improves capacity, responsiveness, and availability, because work continues on the remaining resources if one fails. Clients usually reach the pool through a single virtual address.

Question 2

Why do organizations use load balancers instead of one large server?

Accepted Answer

Scaling up a single server has practical limits and creates a single point of failure. A load balancer lets operators add or remove smaller servers as demand changes, and it routes around any machine that fails. Maintenance can be performed on one backend at a time without taking the whole service offline.

Question 3

What is a backend server in a load-balanced pool?

Accepted Answer

A backend server is one of the machines that actually processes requests after the load balancer selects it. Backends are usually identical copies of an application, so any of them can answer any request. The load balancer keeps a list of these members and tracks which ones are currently eligible to receive traffic.

Question 4

What does a virtual IP address represent in load balancing?

Accepted Answer

A virtual IP address is a single public or internal address that clients use to reach a service, while the load balancer maps it to the actual addresses of backend servers. Because the virtual address stays constant, backends can be added, removed, or replaced without changing the address clients were given.

Question 5

What is the difference between horizontal and vertical scaling?

Accepted Answer

Vertical scaling adds more resources, such as CPU, memory, or faster storage, to one existing machine. Horizontal scaling adds more machines that share the work. Load balancing is what makes horizontal scaling practical, because it spreads requests across the new machines. Vertical scaling is simpler but is capped by the largest available hardware.

Question 6

What is a single point of failure in a network?

Accepted Answer

A single point of failure is any component whose failure would stop an entire service. A lone load balancer or lone server can be one. Redundancy addresses this by providing duplicate components, so that when one fails, another takes over. Pairing two load balancers in an active standby arrangement is a common example.

Question 7

What is the difference between throughput and latency?

Accepted Answer

Throughput is the amount of work a system completes over a period, such as requests or bits per second. Latency is the time a single request takes from start to finish. A balanced pool can raise throughput by serving more requests in parallel, while latency depends on how quickly each request is handled.

Question 8

What is equal-cost multipath routing and why is it used?

Accepted Answer

Equal-cost multipath routing lets a router forward traffic toward one destination across several paths that have the same routing metric. It increases usable bandwidth and spreads load across links instead of leaving spare links idle. Flows are usually hashed onto paths so that packets from one connection stay on one path and arrive in order.

Question 9

What is round robin load balancing and how does it work?

Accepted Answer

Round robin sends each new request to the next backend in a rotating order, starting over after the last server. It is simple and works well when backends are identical and requests take similar time. Its weakness is that it ignores how busy each server is, so slow servers still receive equal shares.

Question 10

What is weighted round robin and when is it useful?

Accepted Answer

Weighted round robin assigns each backend a weight that reflects its capacity. A server with weight three receives three requests for every one sent to a server with weight one. It is useful when the pool mixes machines of different sizes, so stronger servers take a larger share of traffic.

Question 11

How does least connections balancing choose a server?

Accepted Answer

Least connections balancing sends each new request to the backend that currently has the fewest active connections. It adapts to uneven request durations, so a server stuck on long-running requests receives fewer new ones. It works best when connections are long-lived or varied in cost, but it requires tracking open sessions on each backend.

Question 12

How does source IP hash balancing work, and why is it used?

Accepted Answer

Source IP hash balancing applies a hash function to the client address and uses the result to pick a backend. Repeat requests from one client therefore tend to reach the same server. This aids session continuity without extra state, but clients behind one shared address land on one backend, which can skew the distribution.

Question 13

What is consistent hashing and why is it used?

Accepted Answer

Consistent hashing maps both servers and keys onto a ring, so each request goes to the next server clockwise from its hash position. When a server is added or removed, only the requests that fall in its segment move to a different backend. It suits caches, where remapping everything would cause heavy misses.

Question 14

What is the difference between static and dynamic balancing algorithms?

Accepted Answer

Static algorithms make decisions from fixed rules and configuration, such as round robin or source hashing, without reading the current state of servers. Dynamic algorithms use live information, such as active connections or response times, to decide where each request goes. Dynamic methods handle uneven workloads better but require more monitoring and add complexity.

Question 15

How does the power of two choices method pick a backend?

Accepted Answer

The power of two choices method picks two backends at random and sends the request to the less loaded one. Comparing just two candidates is far cheaper than checking every server, yet it avoids most of the imbalance that pure random selection produces. It suits large pools where checking every backend would be costly.

Question 16

What is a hot spot in a load-balanced system and how does it arise?

Accepted Answer

A hot spot is a backend, or a key range, that receives a disproportionate share of traffic while others sit underused. It can arise from skewed client addresses, a popular item that many requests target, or a balancing method that ignores real load. Adjusting weights, rehashing keys, or changing algorithms can fix them.

Question 17

What is a health check in load balancing?

Accepted Answer

A health check is a periodic test that the load balancer runs against each backend to decide whether it can receive traffic. Common tests include opening a TCP connection or requesting a specific URL. A backend that fails several consecutive checks is marked down and removed from rotation until it passes again.

Question 18

What is the difference between a passive and an active health check?

Accepted Answer

An active health check is initiated by the load balancer, which probes backends on a schedule. A passive health check observes client traffic, such as errors or timeouts, and marks a backend unhealthy when responses fail. Active checks detect failures sooner, while passive checks add no probe traffic but react once users are affected.

Question 19

What is session persistence and why is it needed?

Accepted Answer

Session persistence, also called stickiness, sends all requests from one client to the same backend. It is needed when application state, such as a shopping cart or login, lives in one server's memory instead of shared storage. Without it, a user could be sent to a server with no record of their session.

Question 20

How does cookie-based session persistence work at the balancer?

Accepted Answer

With cookie-based persistence, the load balancer inserts or reads a cookie that identifies the chosen backend. On a client's first request it selects a server and returns the cookie in the response. On later requests, the client sends the cookie back, and the balancer routes to the same server if it is still healthy.

Question 21

What happens to persistent sessions when their backend fails?

Accepted Answer

When the backend holding a persistent session fails, the balancer stops sending that client to it and routes the next request to a healthy server. Its session data is lost unless it was replicated or stored in shared storage. Users may need to log in again, so externalized session state is often preferred.

Question 22

What is connection draining during a backend removal?

Accepted Answer

Connection draining lets a backend finish its in-progress requests after the balancer stops sending it new ones. Existing connections are allowed to finish within a timeout before the machine is removed or taken down for maintenance. This avoids cutting off users in the middle of an action.

Question 23

Why should health checks test more than whether a port is open?

Accepted Answer

A port can accept connections while the application behind it is stuck, overloaded, or returning errors. A deeper check requests a real endpoint and verifies the response status and content, so the balancer removes servers that cannot serve users, not just ones that accept connections. Health endpoints should stay lightweight so monitoring stays cheap.

Question 24

Why might a newly added backend need a slow start period?

Accepted Answer

A newly added backend may have cold caches, empty connection pools, or other warm-up costs, so sending it a full share of traffic immediately can cause errors or slowdowns. Slow start gradually increases the traffic weight of the new server over a warm-up period. This lets it reach steady performance before it carries its full load.

Question 25

What is the difference between layer 4 and layer 7 load balancing?

Accepted Answer

Layer 4 load balancing makes decisions from network and transport details, such as addresses and ports, without reading application content. Layer 7 load balancing inspects application data, such as HTTP headers, URLs, cookies, or hostnames, before choosing a backend. Layer 7 is more flexible, while layer 4 is faster with less overhead.

Question 26

What is TLS termination at a load balancer?

Accepted Answer

TLS termination means the load balancer completes the encrypted connection with the client, decrypts the traffic, and then forwards it to backends, often over the internal network. This centralizes certificate management and frees backend servers from cryptographic work. Re-encrypting traffic to the backends is an option when internal traffic must also be protected.

Question 27

Why might a layer 7 balancer route requests by URL path?

Accepted Answer

Routing by URL path lets one public address serve several distinct applications, for example sending requests under one path prefix to an image service and others to an API. Each group of backends can then be scaled, deployed, and monitored separately. This approach depends on reading the request, so it requires layer 7 inspection.

Question 28

What is a host-based routing rule on a load balancer?

Accepted Answer

A host-based routing rule selects a backend pool according to the hostname in the request, such as the Host header in HTTP. Requests for one domain can reach one pool while requests for another domain reach a different pool. It is a common way to host several websites on shared infrastructure.

Question 29

What is direct server return and when is it used?

Accepted Answer

Direct server return lets backends send responses straight to the client, bypassing the load balancer on the return path. Only incoming traffic passes through the balancer, which reduces its load for workloads with large responses, such as media streaming. It requires backends to accept traffic addressed to the virtual IP and careful network configuration.

Question 30

How does a reverse proxy relate to a layer 7 load balancer?

Accepted Answer

A reverse proxy accepts client requests on behalf of backend servers and forwards them onward, hiding the backends from clients. Many layer 7 load balancers are built as reverse proxies, adding balancing across the pool to this forwarding role. The terms overlap, so one product may be described as either, depending on emphasis.

Question 31

What is TLS passthrough and what trade-off does it involve?

Accepted Answer

TLS passthrough forwards the encrypted stream to a backend without decrypting it at the balancer, so the backend holds the certificate. The trade-off is that the balancer cannot read HTTP content, so layer 7 routing and header inspection are unavailable. It is common when end-to-end encryption or client certificate checks must reach the application.

Question 32

How can a layer 7 balancer use HTTP headers to route traffic?

Accepted Answer

A layer 7 balancer can read request headers such as a version tag, device type, or language preference and choose a backend pool that matches. This supports canary releases, where a small share of users is sent to a new version, and A/B testing. Header values come from clients, so trust them carefully.

Question 33

What is network capacity planning and why does it matter?

Accepted Answer

Network capacity planning is the process of estimating future traffic and sizing links, devices, and paths so that performance targets are met. It matters because underprovisioned networks produce congestion, packet loss, and slow applications, while overprovisioning wastes money. Good planning balances current measurements against growth forecasts and business requirements.

Question 34

What is link utilization and why is it watched?

Accepted Answer

Link utilization is the percentage of a link's capacity in use over a given interval. Planners watch it because high sustained utilization leaves little headroom for bursts, and queues build quickly as a link approaches its limit. Many teams set threshold alerts well below full capacity to trigger upgrades before users notice delays.

Question 35

What is a traffic matrix and how is it used in planning?

Accepted Answer

A traffic matrix records the volume of traffic flowing between each pair of source and destination sites. Planners use it to see where demand concentrates and to model how traffic will shift when links fail or routes change. It is the foundation for estimating link loads in both normal and failure conditions.

Question 36

What does N plus one redundancy mean for capacity planning?

Accepted Answer

N plus one redundancy means providing one extra unit of capacity beyond what normal operation requires, so service continues if any single unit fails. For example, if four load balancers can carry peak load, a fifth is the spare. The approach protects against a single failure but not against several failing at once.

Question 37

How does peak busy-hour traffic affect capacity sizing?

Accepted Answer

Capacity is usually sized for the busy hour, the highest average traffic in a typical day, rather than for the average over all hours. Designing for averages would leave links congested during peak periods. Planners often add headroom above the busy-hour figure to absorb bursts and growth between upgrades.

Question 38

What is the bandwidth-delay product and why does it matter?

Accepted Answer

The bandwidth-delay product is the link capacity multiplied by the round-trip time, giving the amount of data that can be in flight on the path. It matters because TCP throughput is limited when buffers and window sizes cannot cover this volume, so a fast link over a long distance can still perform poorly. Planners use it when sizing buffers and assessing wide area paths.

Question 39

How should planners account for growth over time?

Accepted Answer

Planners typically combine historical traffic trends with business forecasts, such as expected user counts or new services, to project demand. They then model several scenarios, including a high-growth case, and estimate when each link reaches its threshold. Lead times for hardware and circuits must be included, so orders are placed before capacity runs short.

Question 40

What is failure-mode analysis in network capacity planning?

Accepted Answer

Failure-mode analysis tests whether the network can carry expected traffic after a link, device, or site fails. Planners simulate each likely failure, reroute traffic over remaining paths, and check whether any link exceeds its safe utilization. The results show where extra capacity or alternate routes are needed, rather than relying on normal-day measurements alone.

Load Balancing and Network Planning Flashcards

1Load Balancing Fundamentals

2Balancing Algorithms

3Health Checks and Persistence

4Layer 4 and Layer 7 Balancing

5Network Capacity Planning

Learning Resources