Unit
- Definition of caching: Caching is the process of storing copies of frequently accessed data in a temporary storage layer so that future requests can be served faster. The cache acts as a high-speed intermediary between the application and the primary data store, reducing access latency and read load on the backend.
- How caching improves application performance: By retrieving data from a fast cache instead of a slower database or external service, response times drop dramatically. This translates to a smoother user experience and the ability to handle higher concurrency without scaling the database.
- Common caching scenarios: Web applications cache HTTP responses, rendered pages, and database query results. APIs cache response payloads. Microservices cache service-to-service call results. Mobile apps cache data locally to work offline.
- Trade-offs of caching: Caches consume memory and introduce complexity. Stale data is a risk if invalidation is not handled correctly. Developers must balance performance gains against consistency requirements and operational overhead.
- Cache layers in a typical architecture: Caching can occur at multiple layers: browser cache (local HTTP cache), CDN cache (geographically distributed), reverse proxy cache (e.g., Nginx, Varnish), application cache (in-memory like Redis), and database cache (query result cache). Each layer serves a different purpose and has distinct characteristics.
- Performance metrics: Key metrics to evaluate caching effectiveness include cache hit ratio (percentage of requests served from cache), cache miss penalty (time to fetch from origin), and cache latency (time to serve from cache). These help identify whether the cache is beneficial.
- Real-world impact: A well-implemented caching strategy can reduce database read load by 80-90% in read-heavy applications. For example, a social media feed serving cached posts can achieve sub-10ms response times versus 100ms+ from database queries.
- When not to cache: Data that changes rapidly or is highly dynamic may not be suitable for caching. Applications requiring strong consistency for every read (e.g., financial transactions) should avoid caching or use write-through invalidation.
- Cache eviction policies: Common policies include Least Recently Used (LRU), which discards the least recently accessed items, and Time To Live (TTL), which removes entries after a fixed duration. Choosing the right policy depends on access patterns and memory constraints. LRU works well for workloads with temporal locality, while TTL is simpler but may cause premature eviction or stale data if set incorrectly.
- Cache invalidation strategies: Invalidation ensures stale data is removed or updated. Approaches include write-through (update cache and database simultaneously), write-behind (asynchronous update), and expiration-based (TTL). Write-through guarantees consistency but adds write latency. Write-behind improves write performance but risks data loss if the cache fails before persistence.
- Cache warming: Preloading cache with expected data before handling user requests reduces cold start latency. Common techniques include seeding the cache during deployment or running a warmup script that queries the most accessed keys. Without warming, the first users after a restart experience slow responses until the cache fills naturally.
Code Example
Python
import time
class SimpleCache:
def __init__(self):
self.store = {}
def get(self, key):
return self.store.get(key)
def set(self, key, value, ttl=None):
self.store[key] = value
if ttl:
# schedule expiry (simplified)
pass
cache = SimpleCache()
cache.set('user:1001', 'Alice')
print(cache.get('user:1001')) # Output: AliceThis Python class demonstrates a basic in-memory cache. The get and set methods form the core interface. In production, libraries like functools.lru_cache or external stores like Redis provide advanced features.
Quick Revision
- Caching stores copies of data in a fast-access layer to reduce latency.
- It improves performance but introduces complexity and potential for stale data.
- Cache layers exist at the browser, CDN, proxy, application, and database levels.
- Key metric: cache hit ratio.